< gmaxwell> TD-Linux: the link below the comment luke adds.
< TD-Linux> oh duh
< bitcoin-git> [bitcoin] askmike opened pull request #9994: [trivial/doc] clearify current blockchain size (master...update-doc) https://github.com/bitcoin/bitcoin/pull/9994
< bitcoin-git> [bitcoin] askmike closed pull request #9994: [trivial/doc] clarify current blockchain size (master...update-doc) https://github.com/bitcoin/bitcoin/pull/9994
< bitcoin-git> [bitcoin] askmike opened pull request #9995: [trvial/doc] clarify current blockchain size (master...update-doc-2) https://github.com/bitcoin/bitcoin/pull/9995
< NicolasDorier> afk11_: next time you need a block you can use https://api.qbit.ninja/blocks/[block]?format=raw where [block] can be height, hash, or tip. They can be combined with an offset like https://api.qbit.ninja/blocks/tip-2?format=raw to the block at (tip-1) height
< NicolasDorier> sorry tapi for testnet
< NicolasDorier> does anyone successfully ran bitcoind on a network share (datadir on cifs) ? it think it always worked for me but somehow it does not anymore
< NicolasDorier> I don't think it has to do with a recent commit either
< warren> gmaxwell: oh damn. You're right. I somehow failed to look at it that way. This is even more clever than I thought it was!
< gmaxwell> warren: woot. another person enlightenend.
< warren> Makes a lot more sense why 4x isn't arbitrary!
< wumpus> NicolasDorier: what kind of network share?
< NicolasDorier> Windows network share cifs I am trying to debug into leveldb right now
< wumpus> NicolasDorier: some of the databases have problems with network shares of some kinds
< NicolasDorier> yeah the strange thing is that up till now it was working fine
< wumpus> I think I worked around that with symlinking just the blocks directory there a few times, but that's not something that will work on windows
< NicolasDorier> I tried to do that
< NicolasDorier> for some reason with rpc-test it does not work
< NicolasDorier> if I run bitcoind with datadir in my sym link it works fine
< NicolasDorier> somehow if I run the tests it does not
< wumpus> anyhow, improving leveldb to work with those FSes would make sense
< NicolasDorier> I am debugging it now
< wumpus> the tests don't use your datadirectory they create their own
< NicolasDorier> problem in PosixWriteableFile::SyncDirIfManifest
< NicolasDorier> I know, in /qa/cache
< wumpus> PosixWritableFile shouldn't be used on windows at all
< NicolasDorier> I know
< NicolasDorier> I am compiling on linux
< NicolasDorier> and running tests on linux
< wumpus> there's a Win32 environment that we use for windows
< NicolasDorier> except the code is on a share network drive
< NicolasDorier> I do that to code with visual studio
< wumpus> ohh
< wumpus> don't out-of-tree builds help there?
< wumpus> maybe the test framework needs a way where to put its cra^D^D^D cache
< NicolasDorier> out-of-tree builds ?
< wumpus> yes. calling configure from outside the source tree to put the build output there
< wumpus> it's designed for cases where the source is on say, a network share, and the build artifacts should go to some big fast local drive
< NicolasDorier> oh I did not know.
< NicolasDorier> I will first try to pinpoint why it fails in leveldb
< NicolasDorier> if it fails I will do that
< wumpus> well up until a few versions ago it didn't work with bitcoin core due to how we integrated leveldb. It works for 0.13/0.14 at least though
< NicolasDorier> wumpus: it is fsync() which fail
< NicolasDorier> with errno == 22
< NicolasDorier> will find out why oO
< NicolasDorier> wumpus: http://man7.org/linux/man-pages/man2/fsync.2.html EINVAL error
< NicolasDorier> it makes sense
< wumpus> ah our good friend EINVAL
< NicolasDorier> but this blow mymind that it worked before
< wumpus> a "special file" which does not support synchronization
< wumpus> I'd read that as "device" or "pipe", not so much any random file on a network OS. But yes it's clear, network filesystems cannot give that guarantee either.
< NicolasDorier> wondering if it is not my docker which got sneaky upgraded and use a network share for file sharing with the container instead of something else... But even then, I remember running bitcoin core on a network share just fine in the passt
< wumpus> maybe cifs mounts have some mount settings that specify how to handle syncs?
< wumpus> could also be a server setting but I'd think that less likely
< NicolasDorier> searching... when I do mount : // on /home/bitcoin type cifs
< NicolasDorier> (rw,relatime,vers=3.02,sec=ntlmssp,cache=strict,username=NicolasDorier,domain=DESKTOP-M7N69JH,uid=0,noforceuid,gid=0,noforcegid,addr=,file_mode=0755,dir_mode=0755,iocharset=utf8,nounix,serverino,mapposix,nobrl,mfsymlinks,noperm,rsize=1048576,wsize=1048576,echo_interval=60,actimeo=1)
< wumpus> nfs did have some extra daemon that needed to be installed to handle that at some point, I vaguely remember or probably misremember
< wumpus> in any case you can make a minimal reproduction now - a small program that just opens a file on the network FS and tries to fsync() it
< wumpus> a lot easier for testing than launching the whole bitcoind monilith
< NicolasDorier> yep
< wumpus> googling fsync cifs gives a lot of results about similar issues with other database products
< NicolasDorier> yes this one seems relevant https://github.com/docker/for-win/issues/445
< NicolasDorier> but still wondering why it worked before
< wumpus> most common recommendation is "ignore EINVAL for fsync". I guess it's something that should log a warning once, then just continue
< NicolasDorier> I can do that at leveldb level yes
< NicolasDorier> is there a way to print leveldb warning in the console ?
< wumpus> it is a data coherency issue and in the case of crashes not being able to guarantee synced-ness is a bad thing, but it's not a reason to crash out
< NicolasDorier> I saw leveldb has its own log stuff
< wumpus> it'd have to go to the bitcoind log some way, not the console
< NicolasDorier> yes
< wumpus> maybe leveldb logging can be redirected there
< NicolasDorier> this is not done currently right ?
< wumpus> no, I don't think so. It probably should be done though
< wumpus> for all we know leveldb has been spewing other useful warnings and errors and we've been ignoring them
< NicolasDorier> ok I will work on a PR trying to fix all this stuff
< wumpus> we do do that for libevent, for example, we register handlers for their log messages and redirect them to our log
< wumpus> but yes it is weird that this just happened and worked before
< wumpus> some version upgrade maybe changed the behavior
< wumpus> Add new mount
< wumpus> parameter to allow user to disable sending the (slow) SMB flush on
< wumpus> fsync if desired (fsync still flushes all cached write data to the server).
< wumpus> it doesn't say what that parameter is, though
< wumpus> but this implies that fsync should simply work?
< wumpus> even with disabling it it still flushes the data to the server,which is what it should do
< NicolasDorier> I have not updated my environment I think
< wumpus> it doesn't guarantee that the server will have written the data to disk, but it does guarantee that it is written to the server
< wumpus> so maybe it is worth doing another pass over local settings and configuration before resorting to leveldb changes, I don't know
< NicolasDorier> so it should support it in theory
< wumpus> yes
< NicolasDorier> might it be a SMB error caused by a windows update
< NicolasDorier> I am quite sure my linux environemnt have not moved, I always work on fresh container of always the same image
< NicolasDorier> ok I will just fix it in leveldb for now
< wumpus> all the results I find deal with fsync on directories, which is apparently not always supported
< wumpus> I don't think leveldb does that, or does it?
< wumpus> oh it does!
< wumpus> that's what "SyncDirIfManifest" is about, it syncs the dir
< wumpus> NicolasDorier: ^^ okay yes patching leveldb is the way forward
< NicolasDorier> ok. I will do that and try to add log support
< bitcoin-git> [bitcoin] kobake opened pull request #9998: tests: Remove unused function dumpKeyInfo (master...remove-unused-dump-function) https://github.com/bitcoin/bitcoin/pull/9998
< bitcoin-git> [bitcoin] NicolasDorier opened pull request #9999: [LevelDB] Plug leveldb logs to bitcoin logs (master...leveldblog) https://github.com/bitcoin/bitcoin/pull/9999
< Victorsueca> quick! someone make another PR so you're the PR #10000!
< gribble> https://github.com/bitcoin/bitcoin/issues/10000 | HTTP Error 404: Not Found
< Victorsueca> :v
< bitcoin-git> [bitcoin] NicolasDorier opened pull request #10000: [LevelDB] Do no crash if filesystem can't fsync (master...fsync) https://github.com/bitcoin/bitcoin/pull/10000
< Victorsueca> \o/
< Victorsueca> Happy 10000 PRs!
< NicolasDorier> OH YEAH
< wumpus> whoa PR 10000 party
< rabidus> \o/
< luke-jr> why are LevelDB stuff being submitted to Core's git?
< wumpus> for visibility
< luke-jr> makes sense I guess. (maybe should have a label "DO NOT MERGE"?)
< wumpus> NicolasDorier: any idea where leveldb uses logging, if the logger is not generally passed around?
< wumpus> NicolasDorier: uhm.
< wumpus> NicolasDorier: I never mind, I was confused, thought I saw a global log function in leveldb
< wumpus> looks like it is used for some low-level logging in db_impl.cc
< wumpus> so it's more like a debug log than an error/warning log
< wumpus> void DBImpl::MaybeIgnoreError(Status* s) const also uses it (but it's never triggered as with paranoid checks it doesn't ignore any errors)
< wumpus> I guess it can't hurt to bridge the leveldb logger to our debug log (as you did, as a separate logging category) in any case, but just wondering what messages we've been missing
< NicolasDorier> wumpus: it seems there use only options_.info_log
< NicolasDorier> I don't find the debug logs you are telling me
< NicolasDorier> wumpus: this is a joke.... I came back home from my coffee shop and now I do not have fsync error.... there is a phase-of-the-moon bug somewhere
< NicolasDorier> ha maybe I did not recompiled
< NicolasDorier> ha no it is crashing... I thought being crazy for a while
< bitcoin-git> [bitcoin] greenaddress opened pull request #10002: fix gitian doc example script typo (master...gitian_typos) https://github.com/bitcoin/bitcoin/pull/10002
< bitcoin-git> [bitcoin] kobake closed pull request #9998: tests: Remove unused function dumpKeyInfo (master...remove-unused-dump-function) https://github.com/bitcoin/bitcoin/pull/9998