< dcousens> Is there any difference to minReasonableRelayFee and minRelayTxFee ?
< dcousens> I'm not sure why there is two, but it seems the minReasonableRelayFee is not kept in sync with minRelayTxFee, any one know why?
< dcousens> aye, the `-minrelaytxfee` seems to have no effect at all on `estimatefee` or the mempool in general
< dcousens> Is that intended?
< dcousens> (Or am I missing something)
< dcousens> Also, is there a reason estimatefee returns `-1` if `nblocks` is > 25?
< GitHub157> [bitcoin] laanwj opened pull request #6969: doc: there is no libboost-base-dev, add missing sudo in release notes (master...2015_11_docfix) https://github.com/bitcoin/bitcoin/pull/6969
< wumpus> fingers itch to merge #6954
< Luke-Jr> wumpus: Is it possible to generate/use a cookie file in addition to the rpcuser/pass?
< wumpus> no, only one authentication method at a time is supported
< Luke-Jr> k
< Luke-Jr> I guess -rpcpassword= does it for debugging purposes
< Luke-Jr> hrm, any way to access strDataDir QSettings without Qt? :/
< Luke-Jr> thoughts on a bitcoin-cli -printdatadir option or something?
< Luke-Jr> although it's unlikely to be in PATH on Windows.. so maybe that won't work
< Luke-Jr> wait, is that a bug for Core? that bitcoin-cli won't find -qt's cookie file?
< dcousens> Luke-Jr: the datadir will be in the same line you're executing?
< Luke-Jr> dcousens: bitcoin-qt supports overriding the data dir from a QSettings value
< wumpus> qt has its own settings, bitcoind/bitcoin-cli are not able to access those
< dcousens> ah
< wumpus> that's not a bug
< Luke-Jr> wumpus: what's the point of the cookie then?
< wumpus> to let bitcoind and bitcoin-cli communicate given just the datadir
< Luke-Jr> …
< Luke-Jr> awful lot considering bitcoin-cli is just a testing tool, and bitcoind users should be capable of configuring bitcoin.conf anyway :x
< wumpus> pass the right datadir to bitcoin-cli and it should work
< wumpus> well other software could use it too, but sure it needs to find out what the datadir is, or let the user point them at it
< Luke-Jr> I thought the whole point of the cookie file was so Bitcoin-Qt users didn't need to configure RPC explicitly
< wumpus> yes, it is
< Luke-Jr> but there's no reasonable way to get to it
< wumpus> any software that knows the bitcoin data dir can communicate with bitcoin{d,qt} if server=1
< wumpus> if it doesn't it needs to be pointed at it
< Luke-Jr> would it be terrible to have Bitcoin-Qt get strDataDir from a special QSettings that explicitly uses <default datadir>/datadir.ini or something?
< Luke-Jr> or better yet, ignore strDataDir for cookie purposes?
< wumpus> sure, another option would be to ignore the datadir completely
< wumpus> e.g. tor drops the cookie file in /var/... by default
< wumpus> then again, how to handle multiple instances in that case
< wumpus> I think putting it in the datadir makes sense
< wumpus> you can override the cookie path to be something else
< Luke-Jr> multiple Qt instances can't use the QSettings for datadir anyway
< Luke-Jr> if the user has to override anything, it defeats the purpose of being zero-configuration <.<
< wumpus> -rpccookiefile=/var/run/bitcoindcookie
< wumpus> just look in the registry, find the data dir, and use that
< Luke-Jr> that's a lot of code, considering Mac and Linux do it differently
< wumpus> normally you can assume that the cookie is in the data directory along with the other files, that's a feature not a bug
< dcousens> Any ideas on the question I had before RE minReasonableRelayFee being out of sync with minRelayTxFee, is that intentional? In terms of the estimator policy. All good if it is, but, it'd be nice to maybe just add a comment there to reassure its not just a missing reference
< wumpus> you can also use QSettings if your application is qt-based
< Luke-Jr> it's not even C++
< wumpus> yes it's some code, maybe we can make a libfindbitcoindatadir ...
< Luke-Jr> ugh
< gmaxwell> wait why isn't bitcoin-qt and bitcoin-cli finding the same datadir?!
< wumpus> although I think it's a bad idea, as it wouldn't handle multiple instances, you need a way to choose a data directory anyway
< wumpus> bitcoin-qt allows choosing a data directory on first start in the GUI, this is stored inthe qt settings
< wumpus> bitcoin-cli and bitcoind don't care about that
< Luke-Jr> gmaxwell: bitcoin-qt gets datadir from Windows registry, Mac registry-equivalent, or an INI file on Linux; based on user input at first run
< Luke-Jr> wumpus: multiple instances can't mix with QSettings datadir period, unless I am missing something?
< wumpus> Luke-Jr: that's true. The datadir from QSettings is seen as a default, it's still possible to override it from the command line
< Luke-Jr> wumpus: so what if cookie file uses the same logic as bitcoin-cli for datadir? this seems to work in all cases..?
< Luke-Jr> if it's overridden via bitcoin.conf or cmdline, it uses that; otherwise the platform default
< wumpus> dcousens: -1 for estimatefee means 'unknown'
< dcousens> wumpus: why not just give back an error
< wumpus> dcousens: don't know about minReasonableRelayFee/minRelayTxFee
< wumpus> dcousens: because
< Luke-Jr> …
< dcousens> (because ...?)
< wumpus> dcousens: yes I don't know, the interface was written in this way not that way
< wumpus> it's just a choice that was made at some point that made sense
< wumpus> or something
< dcousens> ok
< wumpus> if you go change it around now then people will complain about that
< wumpus> the values it can return are documented in the RPC help
< dcousens> that wasn't my main question, just something I noticed that was odd why playing with -minrelaytxfee and noticing it had no effect on the estimation algorithm
< wumpus> that's everything of an interface definition that we have, asking why it doesn't behave otherwise is quite pointless
< Luke-Jr> wumpus: if I made cookies ignore QSettings, will you accept it?
< wumpus> Luke-Jr: I think it's strange to make something ignore qsettings
< wumpus> Luke-Jr: it's a valid concern and maybe reason to re-think where the cookie file gets placed, but 'let's ignore qsettings' sounds like a terrible hack
< wumpus> then it will make *two* data directories
< gmaxwell> using something QT specific for handling finding the datadir on windows was probably what should have been avoided, since bitcoind and bitcoin-cli also need to find the data directory.
< Luke-Jr> it's tempting to change this to write to bitcoin.conf instead of QSettings then
< wumpus> NO NO NO NO
< Luke-Jr> wouldn't an append-only be safe?
< wumpus> don't write to bitcoin.conf
< wumpus> goddamnit
< Luke-Jr> sigh
< gmaxwell> then you have the "how do you find bitcoin.conf" problem.
< wumpus> no it's not safe, software should not write to its configuration file, it can't even be asssumed to be writable
< dgenr8> writing to bitcoin.conf should be controlled by a setting in bitcoin.conf
< Luke-Jr> gmaxwell: that's at least possible to solve
< wumpus> that's why I went with the cookie solution in the first place
< gmaxwell> Luke-Jr: it reduces to the same problem, finding the data directory.
< wumpus> otherwise I'd just have made it write a rpcuser and rpcpassword to bitcoin.conf
< Luke-Jr> gmaxwell: minus QSettings
< wumpus> bitcoin.conf is usually in the same directory as the cookie so it won't avoid *any* problems
< Luke-Jr> if the data directory cannot be overridden by QSettings, the problem is solved.
< wumpus> yes, let's rip that feature out again
< wumpus> why would people want a user friendly dialog to choose the datadir
< wumpus> what a stupid thing
< Luke-Jr> …
< gmaxwell> Luke-Jr: yes if you make the datadir not actually configurable then the issue of finding it goes away...
< wumpus> maybe 1% of users actually use bitcoin-qt with bitcoin-cli, 0.1% uses it with a changed datadir through qt
< wumpus> most people that want to use RPC with bitcoin-qt will use the debug console
< gmaxwell> wumpus: it is a bit broken that changing your datadir breaks bitcoind / bitcoin-cli from finding it.
< Luke-Jr> wumpus: my main concern is BFGMiner, not bitcoin-cli
< gmaxwell> wumpus: e.g. joinmarket and bfgminer and such need to find the rpc stuff too.
< wumpus> gmaxwell: qsettings is the qt equivalent of adding command line arguments, you can pass a datadir to bitcoin-cli and it will work
< gmaxwell> wumpus: datadir isn't a qt specific thing. Finding the default datadir via the registery is the reasonable thing to do on windows, sure. requiring QT for that is less so.
< wumpus> yes because we really want to write our own registry/everyOSsettingsmechanism wrappers...
< Luke-Jr> I'm sure there's some way to avoid Qt for it, but that means implementing each platform's stuff
< wumpus> Qt is not strictly required though, other applications can read bitcon's datadir from the registry without qt
< gmaxwell> wumpus: sometimes we have to... there are dozens of windows ifdefs in the code. Unless you think linking bitcoind/bitcoin-cli to QT is a better plan. :)
< wumpus> but we use qt because that's available anyway
< wumpus> and we don't want piles of extra codes to maintain
< wumpus> sigh
< wumpus> there's tons of windows ifdefs in the code so let's add even more
< gmaxwell> wumpus: take a step back, ignore how its implemented. The behavior of our own compoents looking in different places due to internal architecture reasons and to no benefit to the user is surprising and suboptimal.
< wumpus> I'd prefer to reduce the amount of platform dependent code
< wumpus> well at least it's working for most people now, it's one of the things that is hardly ever complained about
< gmaxwell> I prefer to reduce the amount of platform dependant code too. But that doesn't mean that we should be disfunctional in some cases where a bit is needed to prevent disfunction.
< wumpus> there are tons of things *really* broken, this is not one of them!
< wumpus> no, it's not disfunctional... never mind
< wumpus> go ahead, implement it some way else, just make really sure it's backward compatible
< gmaxwell> Luke-Jr: I think we could probably just add code that reads the registry in the same way RandAddSeedPerfmon and looks to the same place QT looks, and call that from bitcoind/bitcoin-cli on windows, and document the location so other programs can look. No?
< Luke-Jr> gmaxwell: wouldn't make implementing it externally any easier than how it currently is.
< Luke-Jr> I'm sure QSettings has some well-defined location/format so things are backward compatible.
< gmaxwell> Luke-Jr: well externally is going to have to deal with the OS specific way of finding things, regardless. But that can be made arbritarily easier through better documentation.
< * wumpus> really likes 'better documentation' instead of 'adding piles of code'
< Luke-Jr> supporting cookies was already going to be a major chore, so I'm inclined to just not do so at this point
< gmaxwell> Luke-Jr: this has nothing to do with cookies though.
< gmaxwell> You'd fail to find the config location.
< gmaxwell> wumpus: I'd still suggest adding (hopefully) 1/2 LOC: code to use RegQuery to find the same location in bitcoind/bitcoin-cli.
< wumpus> indeed it's the same for finding bitcoin.conf
< Luke-Jr> does bitcoin-qt use strDataDir for the config location too? :/
< wumpus> yes
< Luke-Jr> gmaxwell: you're forgetting Mac and Linux..
< wumpus> it uses it to set -datadir
< wumpus> (which generally, unless you override it with -conf, specifies the configuration file location as well)
< Luke-Jr> bleh
< wumpus> bitcoin, by design, only has one datadir. If you move it, everything moves with it
< gmaxwell> Luke-Jr: nothing is special on linux. There is no magical registry path location there. I'm not well enough informed about the mac parallel.
< Luke-Jr> gmaxwell: hehehe.. yes it is
< Luke-Jr> ~/.config/Bitcoin/Bitcoin-Qt.conf
< wumpus> in linux it's just a file
< gmaxwell> what do we have an installer just setting a gconf thing for it on linux too?
< Luke-Jr> gmaxwell: it's not part of the installer
< wumpus> however you turn it, finding the datadir was always going to be an OS-specific affair
< Luke-Jr> wumpus: hm, that's a point.. I guess I must already have some ifdefs going on
< gmaxwell> even if it wasn't configurable, the defaults are different.
< GitHub126> [bitcoin] laanwj opened pull request #6970: Fix crash in validateaddress with -disablewallet (master...2015_11_rpc_validateaddress_crash) https://github.com/bitcoin/bitcoin/pull/6970
< Luke-Jr> FWIW, a KDE developer in #qt told me if it needs to be accessed from other software, don't use QSettings and write our own :/
< Luke-Jr> (I asked a while ago when it was ongoing discussion)
< wumpus> won't 'write your own' effectively end up reimplementing qsettings? you need to do something for every OS
< wumpus> not saying that is not the way forward to 'find' the same datadir in bitcoin-cli and bitcoind, as well as other software, but it doesn't quite make it easier
< wumpus> oops accidentally had 'debian desktop environment' on while installing a VM, was wondering why it was pulling 2300+ packages in debian 8 :-)
< wumpus> executables for 0.11.2rc1 are live: https://bitcoin.org/bin/bitcoin-core-0.11.2/test/ sorry for the delay
< GitHub133> [bitcoin] laanwj closed pull request #6968: [Docs] First-draft release notes for 0.11.2RC1 (0.11...release-notes-0.11.2) https://github.com/bitcoin/bitcoin/pull/6968
< GitHub138> [bitcoin] laanwj pushed 4 new commits to 0.11: https://github.com/bitcoin/bitcoin/compare/984587ac5d3e...3dcb390fe9e2
< GitHub138> bitcoin/0.11 40941d9 David A. Harding: [Docs] First-draft release notes for 0.11.2RC1
< GitHub138> bitcoin/0.11 929b2c7 David A. Harding: [docs] Minor revisions to 0.11.2RC1 release notes...
< GitHub138> bitcoin/0.11 9149589 David A. Harding: [docs] 0.11.2 release notes: add sipa graphs & leveldb note...
< wumpus> it should be forbidden to use C and its cursed offspring for anything such as string processing
< gmaxwell> wumpus: yea, there are a bunch of multimedia file formats where there are subtle incompatiblities in every implementation because of number in "ascii" headers.
< gmaxwell> You can change your locale in the program, but doing so will change it in other threads.. not exactly nice to do in libraries.
< wumpus> yes that solves one problem but causes many others. We tried forcing the locale to C in bitcoin for a while but this messed with file name encodings on windows.
< wumpus> solve one subtle problem, create another
< phantomcircuit> just pause world, change locale to c, call strtol, change back to whatever, and unpause world
< phantomcircuit> no problem
< * phantomcircuit> runs
< gmaxwell> Well for example, you could change it just around the single point(s) you need C locale-- at least for the specific problem of file format compatiblity; but ... threads break that. so to fix this elsewhere I've had to just rewrite those C parser functions.
< wumpus> phantomcircuit: travel back in time and ... try to convince Dennis Ritchie to not make this awful mistake
< gmaxwell> wumpus: thats really a subcase of my maxium about strings being doom. :) Other languages have their own string awfulness. :(
< wumpus> yes, typical C, inthe end you have to roll everything yourself, because the library support sucks
< gmaxwell> (often around unicode)
< wumpus> and if it happens to not suck on your platform, be prepared to do that when making your application portable
< wumpus> well other languages also have issues, but usually not as dangerous
< gmaxwell> yea, then you end up somewhere where someone thought 'UCS-2' was a good idea... and then you want to carry some chinese characters that aren't part of the BMP. :)
< wumpus> yes, typical C, inthe end you have to roll everything yourself, because the library support sucks <- and of course while doing that you write it half assed, so it walks out of a buffer somewhere... happiness, more fun
< * midnightmagic> likes C
< wumpus> gmaxwell: LOl yeah
< wumpus> I like C to, but not for anything involving string processing or dynamic structures
< gmaxwell> I am pretty sure that every cases where I've used a string where something else would have been a reasonable choice, I regretted the string. :)
< gmaxwell> But this may reflect too much time spent in C. OTOH, I think in terms of sheer hours lost, I've lost more to string hell in python. (probably because I avoid strings like the plague in C).
< wumpus> or anything that can result in buffers that overflow, for that matter
< midnightmagic> +1 python string hell, plus incompatibility with unicode handling across versions
< wumpus> almost the entire infosec industry can thank C for their jobs
< wumpus> I've never had any problems with strings in python
< gmaxwell> wumpus: python string hell mostly needs the help of unicode.
< wumpus> what's so bad about unicode? just use UTF-8 everywhere
< gmaxwell> esp less common unicode. Like chinese.
< midnightmagic> former employer wrote unicode handling in python 2.x, and as a result, had a *dual* python installation requirement for about a year for a flagship product while transitioning to python 3
< * midnightmagic> claws own eyes out
< gmaxwell> wumpus: because random python library code catches fire inconsistently with invalid, or worse --valid but unusual-- unicode.
< gmaxwell> I believe its improved a lot in the last couple years at least.
< wumpus> anyhow, I don't have any reason to defend python. I just think C for anything involving strings should die. And this reflects spending too much time in C (and related languages) solving stupid string handling, truncation, buffer overflow issues and crashes and security holes
< gmaxwell> wumpus: not just C's fault, we can also blame microarchitecutrs for not making tagged/bounds checked pointers cheap in hardware. (Intel tried once but their product was premature and a commercial failure...)
< wumpus> gmaxwell: well, sure, that would be perfect. But I think C could have been less devious and not suffer anything in performance.
< wumpus> sometimes it seems to have been designed to make software as insecure and buggy as possible
< wumpus> (and mind you,that's mostly the library, not the langauge at fault!)
< gmaxwell> I think part of the problem is also that "parsing" code ends up having much higher branch-complexity than almost any other code, so it is very hard to test completely... so thats where the bugs endup being, and often its strings being parsed. In libsecp256k1 the der parsers are ~25% of the conditional statements, but only about 9% of the lines of code... and they required very clever tests to ev
< gmaxwell> en execute all their branches.
< wumpus> sure, but if straightforward functions like strtol, which they teach new people to use, simply did parse according to a clear specification instead of 'leave it to the locale', that doesn't make parsing code any easier to write
< wumpus> s/doesn't/would/
< gmaxwell> things like strtol did originally not to locale stuff... that was a later enhancement that fit poorly with the architecutre of the functions.
< wumpus> bleh sometimes I think programming is no longer for me
< wumpus> get too angered about stuff lately
< gmaxwell> One must come to terms with everything being broken as the natural state of the world. On the plus side, you will never run out of things to fix. :)
< dcousens> gmaxwell: I have a weekly break-down about that on Fridays
< wumpus> ah yes 'enhancement' :-) just like there was strcpy, which was plain stupid, then they tried to fix it using strncpy, which had other warts, and then there came platform-specific ones like strlcpy ...
< gmaxwell> in software we makes things 10,000x (or more) more complex than other more mature domains of engineering would even dare; and with much smaller budgets and compressed timelines.
< wumpus> which of course are still not supported on all platforms, so even if there is a better function you still end up implementing it yourself, or wrapping something, of course introducing more bugs
< * midnightmagic> is not surprised at all that deRaadt seems deflated and fatalistic these days
< wumpus> well given how much time we spend working around warts in our tools that's amazing
< dcousens> wumpus: interestingly, in 6 years of doing C++
< dcousens> I think I can count strcpy uses on my hand
< wumpus> dcousens: oh yes, but you'll encounter it in other people's code that you are using / relying on! that's half the fun
< gmaxwell> C++ allows you to swap out your choice of the warts in C with different and more exciting warts. :)
< wumpus> midnightmagic: it's hard to not be fatalistic about these things, this is quite a hole we've digged ourself in
< dcousens> wumpus: simple, don't just don't rely on other peoples code
< dcousens> (/s)
< wumpus> dcousens: hehe
< gmaxwell> The space shuttle had about 2 million moving parts. Firefox has something like 10 million 'moving parts'. If the space shuttle has a problem a half dozen people have a very bad day, if firefox has an issue a few hundred million people may have a (less, I hope) bad day.
< wumpus> dcousens: that's surprisingly hard, even if you'd write your own OS and compiler, there is a whole castle of shit (depending on the platform, bootrom, UEFI, ACPI, etc) that runs before you can and can subtly mess things up!
< wumpus> no surprise we're not going to the moon anymore :-)
< midnightmagic> wumpus: people yelling at you all the time probably doesn't help much. I dunno how deRaadt's made it this far without melting down and just disappearing off into the bush with a crossbow..
< gmaxwell> In any case, there is something worse than (all) things being broken. And thats being stuck working on software where things are broken and _no one cares_. Here we do really care, even if we're sometimes helpless because there is so much debt and the problems are so hard.
< midnightmagic> (I mention deRaadt only because you said something about strlcpy)
< dcousens> haha, gmaxwell here here
< dcousens> wumpus: I know, you can only chase the rabbit whole so far until you realise hardware is completely retarded as well
< midnightmagic> gmaxwell: Or, worse than no-one caring, but everyone getting *angry* at you for pointing out all these problems that indicate the software will fail in production!
< gmaxwell> but yea, the specifics around C/posix where common mistakes reliably result in exploitable vulnerabilities is why @blockstream we're trying rust for some things.
< wumpus> gmaxwell: very true
< wumpus> gmaxwell: yes, I've also started using rust for some private tools, can say it's pretty nice
< gmaxwell> midnightmagic: yea no kidding. It's really bad in much of the software writing world. You're the bad guy for pointing out things are dangerously broken.
< midnightmagic> As in.. they *want* to not care about broken things, but by pointing broken things out to them, they are forced to care, and then they get angry at you and make you present at conferences things that kill you a little inside.
< dcousens> midnightmagic: haha
< gmaxwell> wumpus: I heard from andytoshi that you've been keeping him honest on rust-bitcoin. :)
< wumpus> midnightmagic: well the messenger of bad news and al
< wumpus> midnightmagic: or rather, people *know* how broken things are, but feel helpless, and get homicidal when they are reminded :)
< wumpus> like we're all on the same boat, and it's sinking, but there's no use in telling people that because they already know, but there's no way to stop it
< * midnightmagic> mutters about Mongol messengers getting a bit more respect after the first few times they were decapitated..
< gmaxwell> some of this is about variance, the tools we have for software are so amazingly powerful-- but they mostly reduce the average case complexity of software engineering. The worst case complexity of software engineering is still little better than if were wirewraping out programs, except in wirewrap form they would be the size of the planet.
< midnightmagic> loll
< gmaxwell> s/out programs/our programs/
< dcousens> wumpus: its like the boat is perpetually sinking though
< dcousens> but you keep inflating life boats to hold up the hull
< gmaxwell> (I really didn't appriciate how awesome the tools we have for software were, until I had a dream once that I was working on a car, and had GDB like tools. "set a watch on that air molecule and single step; ah it escapes the intake at the throttle body, but only under boost" :) Slim consolation, we use these tools to make software much more complex than we'd dare attempt otherwise.
< gmaxwell> )
< dcousens> haha
< wumpus> gmaxwell: one problem is that it's become so easy to make something that *seems* to do what you want :)
< gmaxwell> In 15 seconds you can write a program with 20 moving parts (e.g. branches or what not); if you built a simple machine with that complexity you'd probably expect to spend days twiddling with it to get it working right.
< dcousens> gmaxwell: thats why I try to just avoid moving parts as much as possible :P
< dcousens> functionality purity is great
< dcousens> (a different problem, but, similar prospect)
< wumpus> gmaxwell: hehe, yes, some of the tools are pretty awesome. When they work.
< gmaxwell> Still, even the most pure functional code has its own 'moving part complexity' (even though avoiding mutable state really does simplify analysis of many kinds)
< wumpus> (and once you've learned how to use them, which, by the time you fully grasp them, they're deprecated and you need to learn something new :p)
< jonasschnelli> hah
< gmaxwell> gdb is still gdb. :) Part of why I'm so happy that RR works with GDB so well now.
< wumpus> and gdb, while great with C, works so bad with C++ :-( the many times I've seen (optimized out) just for the variable I was trying to inspect... even when compiling without optimizations. Or inspecting e.g. stl containers, which have a maze of templated objects inside...
< wumpus> anyhow, enough ranting for today, sorry
< gmaxwell> A rant here or there is good for everyone.
< gmaxwell> :) Always good to confirm that you're not alone.
< wumpus> yes, thanks :)
< * jonasschnelli> likes lldb
< wumpus> haven't tried lldb yet
< GitHub67> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/6176e9bf3d55...f24880b13213
< GitHub67> bitcoin/master c53d48a Jorge Timón: BIP70: Chainparams: DRY: Make qt/guiutil.cpp fit BIP70 chain name strings...
< GitHub67> bitcoin/master f24880b Wladimir J. van der Laan: Merge pull request #6908...
< GitHub130> [bitcoin] laanwj closed pull request #6908: Chainparams: DRY: Make qt/guiutil.cpp fit BIP70 chain name strings (master...chainparams-bip70-0.12.99) https://github.com/bitcoin/bitcoin/pull/6908
< GitHub173> [bitcoin] laanwj closed pull request #6953: Backport bugfixes to 0.10 (2015-10-22 / f2c869a) (0.10...backport-bugfixes-to-0.10-20151014) https://github.com/bitcoin/bitcoin/pull/6953
< GitHub98> [bitcoin] laanwj pushed 15 new commits to 0.10: https://github.com/bitcoin/bitcoin/compare/cbc4e3bd37da...3b89bf643896
< GitHub98> bitcoin/0.10 9c81005 Diego Viola: Fix spelling of Qt
< GitHub98> bitcoin/0.10 3ad96bd Alex Morcos: Fix locking in GetTransaction....
< GitHub98> bitcoin/0.10 612efe8 MarcoFalke: [Qt] Raise debug window when requested...
< GitHub120> [bitcoin] ptschip opened pull request #6973: Zlib Block Compression for block relay (master...compress) https://github.com/bitcoin/bitcoin/pull/6973
< GitHub72> [bitcoin] laanwj opened pull request #6974: Always allow getheaders from whitelisted peers (master...2015_11_whitelisted_allow_headers) https://github.com/bitcoin/bitcoin/pull/6974
< GitHub127> [bitcoin] laanwj pushed 1 new commit to 0.10: https://github.com/bitcoin/bitcoin/commit/c2e7baf2bdb2e2c163bcadaebe68ee00c9a27e7c
< GitHub127> bitcoin/0.10 c2e7baf Wladimir J. van der Laan: Bump version to 0.10.4, add release notes
< wumpus> * [new tag] v0.10.4rc1 -> v0.10.4rc1
< helo> another couple days, another abort on assert(onlyMaybeDeadlock)... am i the only one seeing these?
< wumpus> haven't seen it yet
< helo> i think it's only started since i upgraded to ubuntu 15.10
< wumpus> so you're building with -DDEBUG_LOCKORDER?
< wumpus> that became the default for --enable-debug some time ago, that's probably when it started
< helo> i've been using --enable-debug for quite a while now
< helo> i've seen plenty of the potential deadlock warnings, but never aborting until recently
< wumpus> the commit that enables it is Mon Jul 13 14:28:03 2015
< GitHub135> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f24880b13213...503ff6e1ae69
< GitHub135> bitcoin/master 9ea7762 Matt Corallo: Use Pieter's signing subkey instead of his primary key...
< GitHub135> bitcoin/master 503ff6e Wladimir J. van der Laan: Merge pull request #6967...
< GitHub148> [bitcoin] laanwj closed pull request #6967: Use Pieter's signing subkey instead of his primary key (master...verify-commits-fixes) https://github.com/bitcoin/bitcoin/pull/6967
< jonasschnelli> on my libsecp256k verification node, my active tip is at 382748 (~-10 blocks), headers are up to 382758...
< GitHub75> [bitcoin] harding opened pull request #6975: [doc] 0.11.2 release notes: use original pull numbers (0.11...note-0.11.2-orig-prs) https://github.com/bitcoin/bitcoin/pull/6975
< jonasschnelli> 20049 transaction stuck in the mempool
< jonasschnelli> 286MB dynamic usage
< jonasschnelli> On of my local nodes (non libsecp256k1 verification) tells me i'm on 382758 with ~5000tx in mempool.
< wumpus> jonasschnelli: why is the node stuck? did it reject a block?
< jonasschnelli> wumpus: can't see anything in the debug.log (have no -debug flag activated)
< wumpus> you don't need a debug flag to log block rejection, just for tx rejection
< jonasschnelli> let me check again...
< jonasschnelli> i can see the update tip to: 382748
< jonasschnelli> after that, only "boring" stuff like "receive version message:"
< jonasschnelli> no reject
< jonasschnelli> last 1000 lines of my debug log: https://bitcoin.jonasschnelli.ch/secp/stuck_debug.log
< bsm1175321> jonasschnelli: that seems like an awful lot of version messages to me. Why so many disconnects/reconnects?
< bsm1175321> My node has been running since ~Thursday and is on peer=14, yours is at peer=2715.
< helo> i get insane disconnects/reconnects from bitnodes
< bsm1175321> Mine is firewalled though, no incoming connections.
< jonasschnelli> hmm... no idea. getpeerinfo reports a huge list.
< jonasschnelli> node is up since: 2015-11-08 19:17:23
< bsm1175321> Maybe your node has fallen behind and is getting banned?
< bsm1175321> jonasschnelli: my node is up since 2015-11-05.
< jonasschnelli> bsm1175321: hmm.. don't think so.. getpeerinfo reports many peers with reasonable connect time
< * jonasschnelli> needs to attach the debugger for more info...
< jonasschnelli> i assume -dbcache=6000 should not be related to this issue?
< bsm1175321> jonasschnelli: In between blocks 382748 and a peer notifying blocks=382749 is: 2015-11-09 13:39:07 socket recv error Connection reset by peer (104)
< bsm1175321> Could that have been the 382749 block download?
< jonasschnelli> theres a second headers-only chain which i accidentally cut of in the comment: now re-added: https://github.com/bitcoin/bitcoin/pull/6954#issuecomment-155093019
< wumpus> ouch, I shut down a VM where a bitcoind w/ -dbcache=8000 was running, without waiting for bitcoind shutdown to complete and flush. A learning moment. Lost a lot of state :-) But no leveldb corruption at least...
< jonasschnelli> wumpus: hah. yeah. 8GB are some amount of blocks... :) happy catch up...
< jonasschnelli> as said, im stuck on 382748, and i have two headers-only chains, one at the most recent height of 382761, the other at 382649 (=stuck block +1)
< bsm1175321> jonasschnelli: Might be worth checking out the code path if a connection gets closed during block download.
< jonasschnelli> i could restart with -debug=net, but i guess i ruin the error-picture we have here...
< sipa> jonasschnelli: strange
< sipa> jonasschnelli: does getpeerinfo list any of those blocks as in flight?
< jonasschnelli> sipa: let me check...
< sipa> or anything about failed validation in debug.log
< jonasschnelli> sipa: nope. All inflight array empty.
< sipa> very strange
< jonasschnelli> sipa: nothing in the debug log: check here if you wan't: https://bitcoin.jonasschnelli.ch/secp/stuck_debug.log
< sipa> how long have you been stuck?
< jonasschnelli> i just checked 1-2h ago and encountered the issue... so height 382748 (where i'm stuck is from today ~13:30)
< jonasschnelli> git log says: 4ee149a6db25cde31432f83369b40c92be13021c + Merge branch 'secp256k1' of https://github.com/sipa/bitcoin
< sipa> have you restarted the note yet?
< jonasschnelli> no. i guess this will cure the issue...
< sipa> (not that that should be needed, but if it's a bug related due to switchover from IBD to normal sync, that would solve it)
< jonasschnelli> but i'd like to not ruin the possibility to debug that issue
< jonasschnelli> as you can see in the debug log, IBD was already done
< jonasschnelli> but i try to restart now...
< jonasschnelli> what -debug flags would be helpful? =net?
< jonasschnelli> (in case the issue shows up again)
< jonasschnelli> michagogo : ppa?
< michagogo> Hm?
< michagogo> This is in a gitian build
< jonasschnelli> hu!
< michagogo> (That's build.log)
< * jonasschnelli> thinks michagogo should use a desktop shell to avoid the use of a magnify
< michagogo> jonasschnelli: what?
< jonasschnelli> Where do we use Qt4.6.4? Aren't the current linux builds not done with qt5?
< michagogo> (That's a screenshot from me remoting into my laptop from my phone)
< jonasschnelli> ^^
< michagogo> jonasschnelli: 0.10.4rc1
< sipa> jonasschnelli: this may be related to the IsInitialBlovkdownload logic wumpus and suhas have been talking about
< sipa> see #6971 and #6974
< jonasschnelli> right.. could be. Restarting the node now...
< jonasschnelli> (dbcache=6000, takes a while)
< sipa> yes, leveldb always does a full compaction at startup
< sipa> rather than at shutdown
< jonasschnelli> jup... sync up to the "most recent block"
< jonasschnelli> getchaintips still reports a "headers-only" chain on height (382649 , height I have ben stuck +1).
< sipa> where are you syncing from?
< jonasschnelli> sipa: before i restarted i started from genesis (fresh node sync, full IBD)
< jonasschnelli> with dbcache 6GB
< jonasschnelli> i keep an eye on that node and will report if it happens again...
< morcos> jonasschnelli: so that node is still up and running and stuck?
< sipa> but where is it pulling blocks from?
< jonasschnelli> morcos: a restart cured the issue
< morcos> whoops, was behind in my history
< jonasschnelli> sipa: you mean from which node? I think without the -debug=net tag i can't see the node,.. i just get the UpdateTip log entry
< sipa> jonasschnelli: so i assume the answer is "random nodes on the internet" :)
< jonasschnelli> sipa: i have set up the node with standard parameters in a non-firewalled zone.
< jonasschnelli> right
< sdaftuar> jonasschnelli: am i right in thinking your node that got stuck had inbound connections?
< jonasschnelli> sdaftuar: right..
< jonasschnelli> it was connected to 38 nodes
< michagogo> cfields: depends doesn't seem to be fetching qt 4.6.4 for the 0.10.4rc1 build
< michagogo> Nevermind
< michagogo> But do we REALLY need to be using dotfiles for download stamps, in a directory that's dedicated to those?
< morcos> I posted issued #6976 with some performance estimates. If we can merge all of these for 0.12, we'll have sped up ConnectBlock by over 400% and CreateNewBlock by 2000%.
< jcorgan> #6973 could use some additional review. i don't think it is the right approach but would like to see more input from others.
< cfields> gitian signers: osx detached sig for 0.10.4rc1: https://bitcoincore.org/cfields/bitcoin-0.10.4rc1/signature.tar.gz