< GitHub44> [bitcoin] gmaxwell opened pull request #8800: Fetch w/o CB if mempool empty, don't use HB mode if blocks only. (master...no-hb-in-bo) https://github.com/bitcoin/bitcoin/pull/8800
< gmaxwell> hm. that allow edits thing defaults to on, so next time someone PRs their whole altcoin to our repo we could go and replace their code with bitcoin? :P "we upgraded your altcoin, enjoy!"
< luke-jr> lol
< sipa> gmaxwell: only the PRed branch opens up
< sipa> ah, for that example that may be sufficient, actually...
< gmaxwell> yes, that was what I was thinking.
< sipa> we should see what happens if you PR to two projects at once
< luke-jr> do we currently have a way to turn off NODE_NETWORK without enabling pruning?
< gmaxwell> no, why would we?
< luke-jr> so people don't IBD off me <.<
< gmaxwell> oh well set the upload limiter to a tiny number. :)
< gmaxwell> and you'll hang up on them when they do.
< luke-jr> yeah, I'd rather just not have them ask though
< luke-jr> otoh, I guess I have no reason to discourage light clients from connecting to me..
< gmaxwell> fwiw, I found more bandwidth was being used by bitcoinj clients than ibding bitcoin core.
< luke-jr> O.o
< luke-jr> that's unexpected
< phantomcircuit> i was thinking of using an RAII wrapper for the CWalletDB pointer in CWallet
< phantomcircuit> does anybody have an opinion on how that should be structured
< phantomcircuit> luke-jr: they do all kinds of silly things compared to 1MB/10 minutes
< luke-jr> phantomcircuit: IBD isn't 1 MB/10 min
< wallet42> is there a proposal of having NODE_2WEEKS or something like that? a node that guarantees the serve of all block headers + the bodies for the past 2016 blocks but nothing earlier?
< luke-jr> wallet42: no, I don't know that there's value to it
< luke-jr> wallet42: next step seems like it would be something where nodes all store a random assortment of history similar to bittorrent swarms, but even that's unnecessary today
< wallet42> i think sipa posted a while ago a histogram showing the #getblocks[height]
< wallet42> and its obviously heavily dispropotionate towards the recent blocks
< wallet42> if you can configure the node to only serve the last 2 weeks (especially with the NODE_BLOOM overhead) it makes it cheaper. also 2 weeks of blocks can be held completly in RAM/mmap so running the bloom filter doesnt even requires HDD/SDD access
< luke-jr> yes, but there is no shortage of dumb servers that serve the full chain anyway
< blaker> hi bitcoin core devs. I am at a conference where a talk later today called 'exploiting trust in deterministic builds' is going to be presented. Since you use reproducible builds maybe it is of interest. paper should be available via springer
< jl2012> It seems #8499 is considered as a blocker for 0.13.1. Could I do anything to make the review easier?
< wumpus> gmaxwell: it ended up where most of my projects go, unfortunately. to the long backlog, I still intend to do it some time. Or anyone else can pick it up if they want.
< GitHub132> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/2b514aa2eae6...5d0219d983b6
< GitHub132> bitcoin/master f839350 Pavel Janík: Do not shadow in src/qt
< GitHub132> bitcoin/master 5d0219d Wladimir J. van der Laan: Merge #8793: Do not shadow in src/qt...
< GitHub128> [bitcoin] laanwj closed pull request #8793: Do not shadow in src/qt (master...20160922_Wshadow_qt) https://github.com/bitcoin/bitcoin/pull/8793
< phantomcircuit> wallet42: there have been various ideas for doing some kind of sharding they all have issues though
< phantomcircuit> you really dont want to rebalance constantly
< phantomcircuit> but you also dont want the blocks a node stores to be based on when it joined the network
< phantomcircuit> achieving both goals turns out to be non trivial
< sipa> i've been keeping statistics on the depth of blocks being requested from my node
< sipa> the tip is of course by far the most requested (though i'm not counting CB HB mode sends)
< sipa> but after about 2 months deep the distribution is almost uniform down to genesis
< luke-jr> how long have you been keeping statistics? (2 months? :P)
< sipa> eh, yes
< * luke-jr> curious what it will look like 3 months from now
< sipa> but the numbers i'm aggregating are for the block depth compared to the tip at the time it was requested
< luke-jr> oh, hm
< sipa> not absolute height
< sipa> it's much more spread out than i had anticipated, actually
< sipa> this implies that a lot of people sync while being 1-2 months behind
< sipa> overall, it's 7.5 million block downloads
< sipa> so not just some random outliers
< sipa> about 100k of which are at the tip
< sipa> and there are strange peaks around 820 deep and 1970 deep
< GitHub8> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/5d0219d983b6...d2e46558ba0e
< GitHub8> bitcoin/master 6d0ced1 Gregory Maxwell: Do not set an addr time penalty when a peer advertises itself....
< GitHub8> bitcoin/master d2e4655 Wladimir J. van der Laan: Merge #8661: Do not set an addr time penalty when a peer advertises itself....
< GitHub141> [bitcoin] laanwj closed pull request #8661: Do not set an addr time penalty when a peer advertises itself. (master...addrman_nopenalty_self) https://github.com/bitcoin/bitcoin/pull/8661
< jonasschnelli> Anyone interested in reviewing the "generic"-statistics PR (currently mempool only)? https://github.com/bitcoin/bitcoin/pull/8501
< jonasschnelli> The GUI mempool stats are based on that PR
< dgenr8> sipa: it sounds like an exponential dropoff from existing nodes syncing, plus a constant level of new nodes being spun up or reloaded
< jonasschnelli> To bad github doesn't allow editing a review after posting it
< jonasschnelli> or do I miss something?
< achow101> jonasschnelli: nope, not missing anything.
< jonasschnelli> hmm.. I hope they will add this soon
< GitHub48> [bitcoin] jonasschnelli pushed 8 new commits to master: https://github.com/bitcoin/bitcoin/compare/d2e46558ba0e...24f72e9f3fd0
< GitHub48> bitcoin/master 0904c3c Jonas Schnelli: [Refactor] refactor function that forms human readable text out of a timeoffset
< GitHub48> bitcoin/master bd44a04 Jonas Schnelli: [Qt] make Out-Of-Sync warning icon clickable
< GitHub48> bitcoin/master a001f18 Jonas Schnelli: [Qt] Always pass the numBlocksChanged signal for headers tip changed
< GitHub168> [bitcoin] jonasschnelli closed pull request #8371: [Qt] Add out-of-sync modal info layer (master...2016/07/UI-out-of-sync) https://github.com/bitcoin/bitcoin/pull/8371
< morcos> cfields: gentle ping
< cfields> morcos: pong
< morcos> just wanted to follow up on the copy-move
< morcos> i thought you had already had move to prevector though in that branch?
< cfields> morcos: i messed with the move stuff yesterday. I can push my changes if you'd like, but i traced through to see where it's called (I put asserts in the moves), and they're almost zero
< morcos> ah, so we need to make moves of transactions or scripts or something?
< cfields> morcos: so i think if there's any benefit that you're seeing, it's probably in a very subtle change:
< cfields> morcos: that's what i did in the commit, so they're all movable now. but, we're pretty good about not doing that
< cfields> (there are .swap()s all over the place)
< morcos> its a clear benefit, but unfortunately in one of the vagaries of this stuff, the benefit is reduced with jeremy's replacement for the boost lockfree queue. not sure why in the world that would be the case
< morcos> where are the few places its called?
< cfields> i have few possible explanations there
< cfields> brb 2 min
< cfields> morcos: ok, back. First let me push up those changes so we can be sure we're comparing the same things. 1 min.
< cfields> morcos: sigh. as usual, i had my wires crossed. I was looking at the wrong branch. Yea, the moves are already in there so you're already testing with that
< cfields> apparently i re-did that yesterday with no memory of doing it the first time...
< morcos> cfields: np. but do you still think it shouldn't be much improvement?
< morcos> oh no
< morcos> ha
< cfields> morcos: i don't think there are many moves, no. I think jeremy's code might've gotten rid of the few that are there. sec
< cfields> hm, maybe not. looks like it just saves on allocation
< cfields> morcos: it should be pretty easy to compare copies/moves before/after the checkqueue changes. We could just add static incrementors for all copies and moves and see how they change
< cfields> morcos: do you have a specific bench you've been using to test?
< morcos> cfields: sorry i wasn't clear. it wasn't the checkqueue changes. it was the sigcache lock contention fix. i had been using a boost lockfree queue and jeremy wrote a custom "cuckoo cache" which i like the design of a lot
< morcos> without your copy/move changes they are equivalent , but with, the boost lock free queue improves more
< morcos> very odd
< morcos> should be unrelated i think
< morcos> anyway, don't worry about it
< morcos> only thing that would be useful if you could point me to where the moves happen the most.. but i'm going down 3 other rabbit holes at the moment, so don't get distracted from whatever you're doing
< cfields> oh, i see. Sounds like you had some copies/moves in your boost branch, then
< sipa> cuckoo cache, sounds fancy!
< morcos> sipa: it's cool!
< sipa> i know cuckoo hashtables
< sipa> it is related?
< morcos> yep
< cfields> morcos: i believe the only hot move is in the checkqueue. https://github.com/bitcoin/bitcoin/blob/master/src/checkqueue.h#L150
< cfields> and obviously that's not very significant
< morcos> cool thx
< cfields> morcos: and here are my changes to make the moves go away: http://pastebin.com/raw/LpGPzjFu
< cfields> the emplace in CheckInputs could be significant if don't have something like that in your branch already
< jeremyrubin> ls
< jeremyrubin> oops
< jeremyrubin> cfields: that code already exists on his branch
< morcos> sipa: if you don't know, i'll try to figure it out. But among the consequences of the txChanged PR that made syncWithWallets happen in ActivateBestChain instead of ConnectTip, is you could get a reordering of when SyncWithWallets is called on various txs
< morcos> For instance if during the processing of ABC, you both connect tips and disconnect them trying to find the longest valid chain... for all the connects you'll be building up syncWithWallets to call at the end and for all the disconnects you'll be calling SyncWithWallets as you go
< morcos> i don't have a good mental model for how SyncWithWallets should be used
< morcos> perhaps i should be asking jonasschnelli
< cfields> jeremyrubin: ah, thanks
< morcos> sipa: jonasschnelli: well, as far as i can tell it seems ok i guess.. but i don't know how to be very confident
< GitHub166> [bitcoin] tjps opened pull request #8801: [trivial] Switching from Boost for-each macros to C++11 for-each (master...tjps_foreach) https://github.com/bitcoin/bitcoin/pull/8801