< bitcoin-git> [bitcoin] fanquake closed pull request #8660: txoutsbyaddress index (take 2) (master...cozz8) https://github.com/bitcoin/bitcoin/pull/8660
< fanquake> cfields pushed my signed sigs up. Can try help debug gitian issues during today.
< fanquake> Also noticed that zlib doesn't currently build with osx depends builder
< luke-jr> re "About Qt": we're using "About %1" now anyway, so the translation issue is presumably gone, and Qt might actually be worth promoting anyway; thinking perhaps what would be an improvement there is to add an "About Bitcoin" :p
< gmaxwell> luke-jr: It might be better to put the QT link though on the about bitcoin core page though.
< gmaxwell> the about qt thing is really uninteresting.
< luke-jr> hmm
< luke-jr> so the menu would have About Bitcoin Core and About Bitcoin; and the former's dialog have a link for About Qt?
< gmaxwell> I think that would be dandy. I dunno what about Bitcoin would say.
< gmaxwell> I think all this is kind of pointless, user use of bitcoin is dying because of sync resource requirements.
< luke-jr> true, better things to spend time on esp since the translation issue is basically fixed
< gmaxwell> I'd be happy to ack a patch to improve the about messages but I think it's a waste of time to work on that.
< BlueMatt> <conman> do you understand the whitelist v addnode thing btw?
< BlueMatt> <conman> if I whitelist an IP address and addnode it, when I see it listed in the debug window it has the addnode'd IP there as not whitelisted
< gmaxwell> ugh, whitelist is not something conman should want for pratically anything. Addnode (in 0.14 at least) prevents banning, whitelisting adds to that near blindly and redundantly relaying transactions, which is not desirable for him, unless he's now become an armory developer. :P
< BlueMatt> yes, i figured, but also it is a strange thing that it doesnt whitelist for outbound
< BlueMatt> hum, yea, looks like addnode will only consider itself "connected" if IP:Port matches, not just IP, so if you, eg, addnode bidirectionally you're guaranteed to have two connections (unless the source uses 8333 for source port, which it wont)
< BlueMatt> afaict
< gmaxwell> yea, bidi results in two connections, I think I had believed there was no strong way to avoid it-- in any case, it's always been true.
< BlueMatt> i mean why do we only consider an addnode connected if its the same port?
< BlueMatt> might as well just filter by ip?
< gmaxwell> now I'm not sure why I was thinking it was hard to avoid. maybe just that being connected to the same IP is not the same as connecting to that IP due to the existance of NATs.
< BlueMatt> indeed, though if you addnode its probably fine
< gmaxwell> E.g. if you addnode my office, but have some random laptop at my office connected to you, you have not achieved your goal.
< gmaxwell> but perhaps it's close enough that the default should be to avoid the duplicate connection...
< BlueMatt> yes, that would be my vote
< gmaxwell> the other thing is that the addnode connection is privledged in ways that the inbound isn't.
< BlueMatt> true, but if the connection gets killed it'll re-open
< BlueMatt> possibly as -addnode
< BlueMatt> and we can do the same set-host-on-lookup thing we do for other connections and set the addnode flag if we find an applicable inbound connection
< gmaxwell> well as I said, inbound != addnode. This would be much better with authentication keys, since you would know if it was who you thought it was.
< BlueMatt> yup, that
< gmaxwell> We could have some other kind of cookie based dedupe that worked more robustly.
< BlueMatt> true, though im no fan of encouraging more places where you can tie two nodes together as the same one
< BlueMatt> i know we have a few places with that and just added more, buttttt
< gmaxwell> well kinda hopeless, we should set a bar that we'd like to meet there. Like you shouldn't be able to correlate tor/ipv4+ipv6 as much as we can but otherwise we don't care. And especially shouldn't be able to correlate across restarts as much as we can.
< gmaxwell> the current informal method is not working well.
< BlueMatt> true
< gmaxwell> BlueMatt: and yea, we just added a _smoking hot_ identifier... so kinda moot. without a good framework we're just going to keep undermining ourselves while also missing good functionality.
< BlueMatt> i mean where all do we have that? double spend-based net splitting, compact blocks, addr stuff, what else?
< BlueMatt> i mean the compact block one is easy to split-by-netgroup
< gmaxwell> well lets ask the question, would we want to 'logically' behave like we had two mempools for ipv4 vs tor, or just give up and say that the strongest property we try to achieve is unlinkablity across restarts?
< gmaxwell> Because thats the first choice there, if we're unwilling to do that then the best I think we can do is unlinkability across restarts.
< BlueMatt> heh, +/- saving mempool on disk :p
< gmaxwell> yea, well peers.dat is the big limitation on restarts right now, in fact. much more than mempool.
< BlueMatt> yea, ok
< BlueMatt> yea, its a shame keeping multiple peers.dat per-netgroup would kinda defeat the purpose of peers.dat :/
< gmaxwell> One of the open questions about mempool sync is do we want to make mempools convergent ... e.g. that you would switch to doublespends under some criteria. Without that, I think our worst case synchronization bandwidth will always be quadratic.
< BlueMatt> yea, good question...i mean we could do some kind of lowest-hash metric, but that seems to amount to a not-by-fee-rbf
< BlueMatt> which also sucks, though i suppose the bw usage could be mitigated
< gmaxwell> BlueMatt: well the sorting criteria can be {fee,hash} which is basically what I menat by convergent.
< BlueMatt> yes, i assumed that
< gmaxwell> (or in particular, fee,H(salt,hash) where salt is a random beacon to avoid being able to grind preferable txn.)
< gmaxwell> and the fact that it's non-fee based replacement is actually not harmful... so long as the sync process rate limits itself.
< BlueMatt> random how? wouldnt it have to be network-wide? which would mean grindable
< BlueMatt> yesyes, generally agreed there
< gmaxwell> I think I can show that the bandwidth of that is polylog.
< gmaxwell> BlueMatt: e.g. using block hashes, so it would be network wide but your grindablity would be a finite window.
< BlueMatt> heh, well you only need to grind a lower hash...that shouldnt take you 10 minutes
< BlueMatt> anyway, just means an attacker can always create infinite amount of mempool divergence in a single tx, though they'd have to spend fee to actually put it in a higher-sync-rate-bucket
< BlueMatt> lol, recaptcha wasnt working so i tried the "audio version" and it just said "we're sorry, your computer may be sending automated queries, so we cannot process your request"
< BlueMatt> fuckers
< gmaxwell> better than the google search captcha that just sends you really hard captchas and doesn't let you through no matter what.
< BlueMatt> thats what it was doing for the non-audio form, though
< BlueMatt> it just kept saying "try again"
< gmaxwell> yea, I think google does that intentionally on search... so perhaps they also do on recaptcha.
< BlueMatt> yea, likely, except they give it away if you try to switch to audio
< gmaxwell> Same shadowbanning BS that reddit does, so profoundly anti-social. The hope is to make spammers waste their time/money, and it ignores the people they harm when there is a false positive.
< BlueMatt> hum, im thinking its just straight up broken, doesnt work on a different browser/ip either
< achow101> cfields: I fixed the osx build. I finally fixed the fucking osx build. All I had to do was make a new base vm. it only took me the better part of the last 3-4 hours to figure out that vmbuilder has a bug and that I had a zombie vm screwing things up
< wumpus> luke-jr: well I managed to screw up my base image, wouldn't be the first time, I do some optimizations to speed up the "upgrading operating system" step, I don't think there's any reason to dig deeper
< luke-jr> ah
< luke-jr> I just worry that the "rebuild base" solution is prone to malware getting in
< luke-jr> otoh, so is the updating step
< wumpus> if you feel like digging deeper I may be able to dig up my old base image and send it to you but I relaly don't have the time
< luke-jr> nah, probably not worth it all things considered
< luke-jr> if we weren't autoupdating, maybe, but we are
< wumpus> there's just no way to do this properly as long as we're not using a deterministically built OS for building
< luke-jr> right
< wumpus> I think it's more curious that making a new base image solved achow101 osx non-determinism - we have no reliance on the OS's build tools in that
< wumpus> at least I had three versions of gcc installed :)
< luke-jr> indeed
< wumpus> please just leave the "About Qt" link where it is, that's a common fixturefor all Qt programs and it's useful to be able to see what version of Qt is used for troubleshooting problems
< wumpus> there's no reason at all to remove it, it's not like we lack space in that menu and absolutely need to change something
< Arvidt> ) is expired
< luke-jr> wumpus: the original reason I brought it up, was translators confusing it for "About Bitcoin-Qt". but that has since been resolved in other ways, so
< Arvidt> Wladimir J. van der Laan (Bitcoin Core binary release signing key) <laanwj@gmail.com> 01EA 5486 DE18 A882 D4C2 6845 90C8 019E 36C2 E964 is expired
< wumpus> no, it's been updated and bumped forward 2 years
< wumpus> try gpg --refresh-keys
< * luke-jr> wonders if this should be added to release announcements
< wumpus> that makes little sense, you could add arbitrary steps to release announcements to make the signature check out as ok
< wumpus> you can't trust any steps in it before you've verified the signature
< luke-jr> yes, but this one makes sense
< wumpus> it's not our job to explain people how to use gpg
< wumpus> at least not in the release notes
< fanquake> I've still got the same issues even after re-building the base image. So I'm guessing I've somehow done that incorrectly..
< wumpus> I'm very surprised that the base image solved it for achow101
< wumpus> osx build hardly uses anything from the base image
< wumpus> we should check what the differences are
< fanquake> Looking at his new PR, it looks like the previous build was using the old version of Qt?
< fanquake> Actually, no.
< wumpus> I don't think so
< wumpus> normally would suggest "remove the gitian cache directories" but due to that switch, 0.14 uses new ones anyway
< wumpus> so it can't be using anything stale
< wumpus> your gitian-builder is up to date?
< fanquake> I might be missunderstanding, but why do the hashes for the same file change in his new PR?
< fanquake> Yes gitian-builder up to date.
< fanquake> Talking about something like native_mac_alias-1.1.0-f064d7cfd4c.tar.gz
< wumpus> curious. yes.
< fanquake> Same for cctools andbiplist.
< wumpus> not sure where those files come from. cfields?
< fanquake> That hasn't changed in depends recently as far as I remember. And those hashes don't even match the ones we have in depends anyways.
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/7ff4a538a868...1f9e904f4556
< bitcoin-git> bitcoin/master 5c8fd50 Pieter Wuille: Avoid VLA in hash.h
< bitcoin-git> bitcoin/master 1f9e904 Wladimir J. van der Laan: Merge #9791: Avoid VLA in hash.h...
< bitcoin-git> [bitcoin] laanwj closed pull request #9791: Avoid VLA in hash.h (master...novla) https://github.com/bitcoin/bitcoin/pull/9791
< wumpus> maybe the gitian "tarballs are not deterministic" issue
< wumpus> eh GITHUB not gitian
< fanquake> except biplist comes from bitcucket
< fanquake> actually, from pypi.python.org
< wumpus> oh right, maybe bitbucket has that issue too, or has it now that github no longer has it, I don't know \
< wumpus> good catch though, that could very well be the reason
< wumpus> we'd need to find the old and new versions of those files and compare
< fanquake> They should still be in the /inputs folder
< wumpus> df26cbb976befd59ba20b73ee33676f22b6ed7aa1329e373307e0a21055f5a5a ./cache/bitcoin-osx-0.13/x86_64-apple-darwin11/native_biplist/native_biplist-0.9-d766a97a608.tar.gz
< wumpus> ed6c2d2b2839164e592859f5eff62df7fedde9657c9e7a853720d0a768b8b756 ./cache/bitcoin-osx-0.14/x86_64-apple-darwin11/native_biplist/native_biplist-0.9-d766a97a608.tar.gz
< wumpus> those are mine. Both have different hash from achow101's new and old one
< wumpus> maybe they regenerate that file on every download?!
< fanquake> Surely we would have noticed if that we happening before now ?
< wumpus> for all the other downloaded dependencies we check the hash
< wumpus> apparently not for this one
< fanquake> My cached biplist files don't even match yours or his..
< fanquake> Well, I even have a different file cached for 0.13 compared to you.
< fanquake> 490e9049e68ec8c7010be3cd19a6237e463d5115839c4cbccbb1dff1196129b4 native_biplist-0.9-1c39c8871c7.tar.gz
< wumpus> so my hypothesis of it changing every time isn't far off the mark, probably
< fanquake> Looking at my two asserts, my biplist files haven't changed between the two.
< fanquake> My other theory is something zlib related. Given it's new to our build.
< fanquake> It also doesn't compile using the depends system on OS X.
< wumpus> we should probably start by comparing the resulting files, it maybe some silly metadata thing too
< fanquake> Yes hopefully.
< fanquake> Something like #8373
< gribble> https://github.com/bitcoin/bitcoin/issues/8373 | Fix OSX non-deterministic dmg by theuni · Pull Request #8373 · bitcoin/bitcoin · GitHub
< wumpus> yes, like that one
< wumpus> fanquake: could you upload your bitcoin-0.14.0-osx64.tar.gz somewhere?
< wumpus> I can't guarantee anything, I don't have the tooling to compare or analyze osx executables, but maybe I can find something obvious
< fanquake> wumpus probably. Sketchy internet at the moment. Will find a site.
< bitcoin-git> [bitcoin] kirit93 opened pull request #9798: Fix Issue #9775 (master...issue) https://github.com/bitcoin/bitcoin/pull/9798
< gmaxwell> uh. hey, so the release notes don't mention bumpfee.
< gmaxwell> thats like, a major feature, no? :P
< wumpus> yes
< gmaxwell> Relase notes make it sound like getinfo is hard-cut disabled in this version.
< gmaxwell> (I'm answering questions about the RC on reddit)
< wumpus> well, it mentions that getinfo was deprecated, not that it was removed. But yeah I guess it could be clearer
< wumpus> I implemented getinfo client-side at some point (https://github.com/bitcoin/bitcoin/pull/8843) so that anyone using bitcoin-cli would hardly notice the command being removed from the server
< wumpus> but that only resulted in a big argument, so I gave up on it
< luke-jr> not the first time someone confused deprecated to mean removed
< wumpus> on the dev side getinfo was already deprecated for years, iti's just that this was never communicated on the RPC interface
< achow101> wumpus: fanquake: I did delete gitian-builder and redownload it before making the base vm, so maybe that could be part of why the build was fixed, although I don't see why it should
< wumpus> achow101: yes, it's no longer possible to isolate what the cause of the problem was
< wumpus> although the same is not working for fan
< wumpus> fanquake
< gmaxwell> I wonder if the next step there shouldn't be to disable getinfo but allow a config option to reenable it. Another half measure would be to make the error field there always preface with a warning.
< wumpus> I think the next major release we should just get rid of it
< wumpus> we've delayed that enough
< wumpus> announcing it in the release notes in strong language was a good start
< wumpus> and we provide *exactly* a table of how to get the information
< wumpus> no need to draw this out like the account deprectation, where it's not 100% clear what the replacement for some use cases
< da2ce7> if you decide now to remove it in the next version, state. "in version 0.15, getinfo will be removed from bitcoin-core". :)
< wumpus> yes, would make sense to add that
< wumpus> also clears up that it is not removed yet
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/1f9e904f4556...390a39bb5cf4
< bitcoin-git> bitcoin/master eb49101 Wladimir J. van der Laan: doc: Update manpages for master...
< bitcoin-git> bitcoin/master 390a39b Wladimir J. van der Laan: Merge #9795: doc: Update manpages for master (laanwj)...
< bitcoin-git> [bitcoin] laanwj closed pull request #9795: doc: Update manpages for master (laanwj) (master...Mf1602-docManMaster) https://github.com/bitcoin/bitcoin/pull/9795
< gmaxwell> wumpus: the problem with hard cuts like that is predictable people will just not upgrade their nodes when their software is incompatible, and in doing so harm the network.
< gmaxwell> especially since its super easy to just have never noticed a release note.
< gmaxwell> (not really trying to argue it out with you now, just defending why I'd even suggest such a thing)
< achow101> wumpus: it's still non-deterministic :( I just ran gitian again and got my original results
< gmaxwell> instagibbs: you're double counted in the contributor list
< bitcoin-git> [bitcoin] gubatron opened pull request #9801: Removed redundant parameter from mempool.PrioritiseTransaction (master...refactor-mempool-prioritisetx) https://github.com/bitcoin/bitcoin/pull/9801
< bitcoin-git> [bitcoin] dooglus opened pull request #9803: Fix typo in release notes. (0.14...patch-5) https://github.com/bitcoin/bitcoin/pull/9803
< bitcoin-git> [bitcoin] JeremyRubin opened pull request #9804: Fixes subscript 0 (&var[0]) where should use (var.data()) instead. (master...fix-subscript-0) https://github.com/bitcoin/bitcoin/pull/9804
< achow101> cfields: I now have what appears to be a consistently inconsistent gitian build for osx. Each time I build it, it alternates between the hashes that everyone else has, and the hashes that fanquake and I have
< sipa> "consistently inconsistent"
< achow101> cfields: I put the two different assert files in different branches: https://github.com/achow101/gitian.sigs/tree/0.14.0rc1-match and https://github.com/achow101/gitian.sigs/tree/0.14.0rc1-mismatch the only difference between the two are the output files. everything else matches according to diff
< bitcoin-git> [bitcoin] petertodd opened pull request #9805: Add seed.btc.petertodd.org to mainnet DNS seeds (master...2017-02-add-pt-mainnet-seed) https://github.com/bitcoin/bitcoin/pull/9805
< boab> Looking for readme on upgrade of core from 0.8.x to 0.10.x Help?
< gmaxwell> uh? why run such an old version as 0.10? I assume everyone has long forgotten about it, it's old and insecure.
< boab> Yep, good ppoint. My main mission is to get off of 0.8.x ... and how-to's please?
< gmaxwell> the blockchain files and wallets are all compatible. shut down, backup your wallet (for good measure), install 0.13.2. start it.
< boab> Fantastic...that's all the assurance I needed. Just did not want to trash something in the process.
< sipa> make a backup of your wallet after shutting down 0.8 if you want to be sure
< sipa> (which you should have anyway)
< boab> Thanks mate. I used the [backup wallet] option from within 0.8 and have stored off site.
< luke-jr> need to backup the wallet before shutting down..
< boab> ?? I think that's right: I was running 0.8, used the GUI option to create a backup, and then have shut 0.8 down.
< luke-jr> sounds good
< sipa> sounds good
< boab> fingers crossed...about to fire up new core now...
< boab> Rock n roll! v013.2 seems to have read the wallet and blockchain and is happily syncing right now. Thanks for your hand-holding...much appreciated.
< gmaxwell> Great! congrats on joining the modern era!
< boab> heheh...respkt ur elders?
< sipa> boab: also, 0.14.0 will be released in a few weeks likely