< esotericnonsense>
this PR does not seem popular (sdaftuar's comments on flushes will likely lead to a better solution)
< esotericnonsense>
but from the log file it should be possible to guess at time wasted :)
< cluelessperson>
esotericnonsense: These are just tests pointed at the localhost node, on a PCISSD, so yeah, I can share
< cluelessperson>
esotericnonsense: you want me to just attach the log file to that link?
< esotericnonsense>
cluelessperson: re your chat in #bitcoin if you actually want to and have time it would be nice to see a benchmark of non-pruning sync time against the same target (e.g. without network node variance)
< esotericnonsense>
sure maybe gzip it or something though :P
< cluelessperson>
esotericnonsense: to summarize agreed point, you'd like me to resync under the same conditions, without pruning?
< cluelessperson>
esotericnonsense: allowing you to view pruned and non-pruned results?
< esotericnonsense>
yeah. it's not that important, I already know the results, i'm just interested in precisely how much fastr it ends up being
< esotericnonsense>
faster*
< cluelessperson>
esotericnonsense: how much faster pruning ends up being? or not pruning ends up being than pruned?
< cluelessperson>
and GO!
< cluelessperson>
it's running
< esotericnonsense>
cluelessperson: not pruning should be faster because the utxo cache gets flushed completely on every prune event
< cluelessperson>
esotericnonsense: I have the dbcache set to 10000
< cluelessperson>
esotericnonsense: it looks faster
< esotericnonsense>
it won't make much difference until you get a sizeable dbcache but yeah it is faster. just wondering whether it's actually say half the time. i killed it at block 200K because I only have one ssd and need the space :P
< cluelessperson>
esotericnonsense: it's already at 10%
< cluelessperson>
esotericnonsense: well, unfortunately, I ran out of space
< cluelessperson>
esotericnonsense: but it seems WAY faster
< bitcoin-git>
[bitcoin] practicalswift opened pull request #11577: Fix warnings (-Wsign-compare) when building with DEBUG_ADDRMAN (master...DEBUG_ADDRMAN-warnings) https://github.com/bitcoin/bitcoin/pull/11577
< bitcoin-git>
[bitcoin] practicalswift opened pull request #11578: net: Add missing lock in ProcessHeadersMessage(...) (master...ProcessHeadersMessage) https://github.com/bitcoin/bitcoin/pull/11578
< bitcoin-git>
[bitcoin] TheBlueMatt opened pull request #11580: Do not send (potentially) invalid headers in response to getheaders (master...2017-10-getheaders-valid-only) https://github.com/bitcoin/bitcoin/pull/11580
< wxss>
sipa mentioned undocumented -stopatheight here yesterday. Is the opposite possible, to restart downloading at/from a specified block height?
< promag>
BlueMatt: mapBlockIndex can contain "invalid" blocks (where IsValid(BLOCK_VALID_SCRIPTS) is false)?
< sipa>
promag: yes, absolutely
< sipa>
as blocks are only validated after storing them
< jimpo>
promag: They wouldn't necessarily be "invalid", just not known to be valid.
< sipa>
mapBlockIndex only guarantees that the *headers* are valid
< jimpo>
Though I think the index includes invalid blocks as well
< sipa>
the blocks themselves are downloaded afterwards, and may turn out to be invalid
< sipa>
jimpo: yes, we store invalid blocks in the index
< promag>
and once invalid it is kept in the index?
< sipa>
yes
< sipa>
the header will be marked as invalid even
< promag>
so the following can be simplified:
< promag>
if (punish_duplicate_invalid && mapBlockIndex.find(first_invalid_header.GetHash()) != mapBlockIndex.end())
< promag>
because if the header is invalid then it is not in the block index right?
< promag>
since above you said "mapBlockIndex only guarantees that the *headers* are valid"
< luke-jr>
hmm, is there a good overview of what is needed to add -usehd=0 back in?
< BlueMatt>
promag: you mean in sdaftuar's pr?
< luke-jr>
basically just make sure it works without a default key, right?
< BlueMatt>
promag: no, the point of that check is to see if the header was accepted (ie if it is known to be a valid header)
< promag>
@BlueMatt please correct me if this isn't the same: if (punish_duplicate_invalid && !first_invalid_header.IsNull())
< sdaftuar>
can someone tag #11578 for backport to 0.15.0.2?
< achow101>
luke-jr: it has to work without a default key and cannot be downgraded
< achow101>
promag: the first_invalid_header could be in mapBlockIndex
< achow101>
promag: specifically if a block was invalid but the header was valid, the header would be in mapBlockIndex. if another peer sent us the exact same header, it would still be in mapBlockIndex and that is what that if statement is checking for
< sdaftuar>
has travis been flaky lately? i'm seeing a wallet.py error in one of the functional tests runs for #11560
< sdaftuar>
yeah i don't know what would cause that
< sipa>
let's try restarting
< sdaftuar>
restarted
< BlueMatt>
ryanofsky: did you actually see the test in #11531 hang?
< gribble>
https://github.com/bitcoin/bitcoin/issues/11531 | Check that new headers are not a descendant of an invalid block (more effeciently) by TheBlueMatt · Pull Request #11531 · bitcoin/bitcoin · GitHub
< BlueMatt>
I can bump the timeout there
< ryanofsky>
no
< ryanofsky>
the code just seems unnecessarily complicated and probably flaky
< BlueMatt>
I mean all I can do is bump the timeout there, really
< BlueMatt>
I think when i go to rewrite the dos logic stuff soon it will change behavior
< BlueMatt>
and no need for the test to test for it
< ryanofsky>
can you simplify the test by skipping the ping and just waiting for the disconnect?
< BlueMatt>
no, because I believe it currently does not disconnect
< BlueMatt>
right now it uses the ping path
< BlueMatt>
but really it should be disconnecting
< ryanofsky>
then i don't see what you lose by dropping the exception handler completely and updating the test when you do add a disconnect
< BlueMatt>
because I'm probably gonna need the exception handler in like a month
< BlueMatt>
and testing for something that bitcoind really shouldnt be doing anyway is kinda nonsense
< ryanofsky>
how do you even know the exception handler works?
< BlueMatt>
i tested it :)
< BlueMatt>
iirc you can just change the dos points for whatever case its using to make the block invalid trivially and it'll keep chugging along
< ryanofsky>
ok well do what you want, but if were me i would opt for not adding a potentially flaky 1-sec timeout and confusing code that doesn't get executed
< BlueMatt>
i mean its not like we dont have other timeouts in the tests, no?
< BlueMatt>
I can set it higher if that makes it much less flaky
< maaku>
does anyone have a list of version bits currently being used, including unofficially?
< ossifrage>
I wonder what the 0.15.x based btc1 code did to break networking during initial block download (it runs for a while, then all the peers timeout, then I get nothing but version handshake timeouts and it makes minimal further progress with 0 or 1 peers)
< gmaxwell>
ossifrage: sounds like you're confirming something like wxss's result-- but sadly, this is offtopic for here, as we don't debug adversarial fork's problems here. :)
< ossifrage>
gmaxwell, I agree, I was just curious how it could break in such a odd way.
< wxss>
gmaxwell, absolutely agree. My issue was different, but this is not the place for analyzing it further
< bitcoin-git>
[bitcoin] TheBlueMatt opened pull request #11583: Do not make it trivial for inbound peers to generate log entries (master...2017-10-no-net-log) https://github.com/bitcoin/bitcoin/pull/11583