< esotericnonsense>
hm. is there a reason why one my node would suddenly dump a bunch of tx from its' mempool (with maxmempool set at default)?
< esotericnonsense>
why one of my nodes*
< esotericnonsense>
e.g. different behaviour on one node versus another
< sipa>
a replacement was accepted?
< esotericnonsense>
hm. perhaps. the vsize change is 1MB. there could be some bug in my monitoring software and they're not running the same version unfortunately.
< esotericnonsense>
actually, they might both be on the same. either rc2 and rc3 or both rc3, will check
< esotericnonsense>
ah hang on now. it's like a puzzle waiting for me to solve it for new insights. i still don't understand the difference in behaviour, but i can see a reason my mempool analysis stuff is less useful in its current form than i thought
< esotericnonsense>
RBF limits the evictions to 100 transactions, but if you decide to send an opt-in RBF low-ish fee tx with say ten 50kB children and then replace it you can invalidate 500kB
< gmaxwell>
esotericnonsense: yes, though you have to pay a pretty phenomial fee to do that.
< esotericnonsense>
ah yes. the fee would have to include the children's fees. clever.
< esotericnonsense>
this has gone quite OT for this channel now, sorry. :)
< meshcollider>
ok I can't see anything wrong in the log, guessing it was a random failure again. Can someone restart it please :)
< meshcollider>
I feel like I'm always asking for travis restarts, sorry about that but hopefully the random errors get fewer and fewer
< wumpus>
meshcollider: yes it needs to run after master re-cached
< wumpus>
restarted the build
< meshcollider>
thanks :) is there any way to enable me to restart builds myself without write access or whatever, so I can stop bugging people
< meshcollider>
not sure how travis permissions work tbh
< wumpus>
I hope so too, though it's always a fight with travis, for everyone not just you
< wumpus>
I think all organization members (no matter what their permissions) can restart builds
< wumpus>
and that's useful anyhow as it means issues can be assigned to you, so I'll invite you
< gmaxwell>
cfields: hopefully you can get the 0.15 release signatures up ASAP so we can get the gitian builds out. :(
< meshcollider>
Oh cool, sounds good
< meshcollider>
Is anyone working on segwit wallet support for 0.15.1 yet? I would kinda like to take a stab at that if no one else has had time yet
< sipa>
i haven't done it, but was planning to look into it
< sipa>
i expect it to be extremely easy
< bitcoin-git>
[bitcoin] MeshCollider closed pull request #11297: Make sure ~/.bitcoin doesn't exist before build (master...201709_travis_delete_dir) https://github.com/bitcoin/bitcoin/pull/11297
< meshcollider>
sipa: alright I'm happy to have a go at it if you want to offhand it, just let me know what the general strategy is :)
< morcos>
esotericnonsense: if you get any more insight into your issue, i'm interested. i looked at several nodes that seemed to mirror your node that didn't experience the drop
< esotericnonsense>
morcos: sure. they've almost converged now
< esotericnonsense>
morcos: in particular the block at 0046 UTC (000000000000000000071cf8891c31cb5361797a24cf78e14e2979e9c8c65e29) knocked out a bunch of 10-15 sat/b transactions (in terms of byte size, i'm working on charting tx counts now)
< esotericnonsense>
right, so there were two blocks one after the other there
< esotericnonsense>
the first one had a similar effect on both nodes, the second one took less out of the mempool than it contributed by approx 300kB
< esotericnonsense>
(on the node that experienced the drop). i interpret that as being 300kB of transactions disappeared from the mempool that were included in that block.
< esotericnonsense>
i don't have logs of the txids unfortunately :(
< esotericnonsense>
(oh, relevant detail is that the non-drop node observed 600kb gone from mempool and 600kb block size approximately, so they weren't direct-from-miner tx)
< esotericnonsense>
morcos: digging about a bit and hacking my tool to show tx counts i think i've found the culprits but can't be entirely sure, that block has a few ~30kB consolidation tx that pay 3sat/b
< esotericnonsense>
vague tinfoil idea of a thing you could do: send child tx to some of the network that increase the total required for RBF, while simultaneously sending a tx to other nodes which RBF's the initial tx - if i'm interpreting things right, if my node is in the latter set it would drop the transaction and the others would ignore the fee replacement.
< esotericnonsense>
no idea why this is a thing you'd want to do
< morcos>
esotericnonsense: i'm not sure i follow all that, but how is this explained by a block, or are you just saying that the block ended up having txs that you think could have been the relevant RBF txs?
< earlz>
Can someone give me an explanation for this exploit that is apparently fixed by moving from a database storing per-tx to per-txout? I can't find any good write ups and maintain my own fork and trying to figure out if I'm affected
< esotericnonsense>
i'm just guessing really. it's not that important, just thought I would share as you asked.
< esotericnonsense>
if my node (or some set of nodes close to me) replaced the tx, but the rest of the network didn't, it would sort of make sense. basically you don't actually pay the large 'replace by fee' because you've deliberately seeded miners with an extra child first.
< morcos>
earlz: you can watch the presentation from breaking bitcoin today, but essentially the fact that the pre-0.15 utxo set stored unspent outputs per transaction meant that a new tx could pull in an input that was 1 of many outputs in a previous transaction
< morcos>
so processing that required pulling the whole previous tx into memory. this is true for each input of a transaction you are processing.
< esotericnonsense>
i can't see what it actually achieves other than mucking about with my mempool and possibly fee estimation if it were based on mempool (which it shouldn't be for that reason)
< morcos>
depending on your threat model, it can be bad in the regular case for low memory nodes, or bad in a contrived attack case for virtually any node
< morcos>
recommend you watch the presentation if you're concerned
< earlz>
thanks, I had similar thoughts last time I worked with coindb that it might be more expensive than expected, but not "can break bitcoin" or crash a node. I'll watch the presentation
< bitcoin-git>
[bitcoin] MeshCollider reopened pull request #11297: Make sure ~/.bitcoin doesn't exist before build (master...201709_travis_delete_dir) https://github.com/bitcoin/bitcoin/pull/11297
< meshcollider>
Just in case ^
< meshcollider>
wumpus: 21630 failed again with the ~/.bitcoin issue
< meshcollider>
oh actually I probably need to rebase to make it work anyway, dw
< sipa>
yeah
< fanquake>
I've just pushed up sigs for 0.15
< midnightmagic>
Would it be useful to push sigs for 0.15.0 in advance of the detached sigs "officially" becoming avail for v0.15.0 ? Or should I wait?
< midnightmagic>
(or rather, open a PR, not push, obviously)
< meshcollider>
useful because the detached sigs "officially" become available only after at least 3 people have (correct me if I'm wrong)
< fanquake>
Yes we need people to build 0.15.0 and push sigs before we can create the detached sigs.
< meshcollider>
im building it atm, should have it up soon too
< mryandao>
is there a threshold of signatures required before a release can be made?
< morcos>
sipa: I was just discussing BIP 173 with sdaftuar and I agree with him that the upper case and lower case rules for the hrp are very unclear.
< morcos>
In particular the BIP says that the hrp is restricted to ASCII characters in the range of 33-126
< morcos>
but I assume you don't want to allow a human readable part where "Aa" is allowed? or "AA" is not the same thing as "aa"?
< morcos>
I think the intention is pretty clear from the uppercase/lowercase section, but it seems like we should just require that the hrp not be allowed tocontain the characters in 65-90 (upper case) and that the rules of converting upper to lower before processing apply to hrp and data?
< sipa>
morcos: sounds good
< morcos>
And maybe add just a little bit that to the QR section that indicates the alphanumeric mode may be possible depending on the hrp choices
< sipa>
morcos: i do think it's already implied... as the text says that mixing cases is not allowed, and that for computing the checksum the lowercase version of the hrp is used
< sipa>
it's probably clearer if we point out that the hrp is just always lowercase
< morcos>
yes, but if you pick some of the allowed characters for hrp, they dont' fall in alphanumeric such as '<'
< morcos>
so they are still allowed for a bech32 code, but don't want to confuse people that all bech32 codes will be always able to take advantage of alphanumeric QR encoding
< morcos>
oh sorry you were responding to the first part
< sipa>
alphanumeric mode is chosen by the encoder on a character-by-character basis
< sipa>
it's not tyat your whole string needs to be uppercase to use the advantages
< sipa>
but they do disappear when you switch all the time
< gmaxwell>
sipa: that was my ultimate answer to sdaftuar on that too, but I admit I struggled with extracting the justification from the spec.
< bitjazz>
hi
< bitcoin-git>
[bitcoin] practicalswift closed pull request #11261: scripted-diff: Use <cxxx> instead of deprecated <xxx.h> when including C compatibility headers (master...cinclude) https://github.com/bitcoin/bitcoin/pull/11261
< morcos>
sipa: is anything holding up #11100, I know luke-jr does not agree, but we're not goign to get him on board. Let's just merge it so we can stop fighting about it an move on.
< gribble>
https://github.com/bitcoin/bitcoin/issues/11100 | Fix confusing blockmax{size,weight} options, dont default to throwing away money by TheBlueMatt · Pull Request #11100 · bitcoin/bitcoin · GitHub
< gmaxwell>
morcos: fwiw, since that last discussion I've heard from two other miners that accidentally produced undersized blocks due to the defaults.
< morcos>
yeah it just needs to be merged, it has plenty of ACK's. I was going to see if I needed to spend more time reviewing it or something, but thats not the hold up.
< cfields>
gitian builders: v0.15.0 detached sigs are pushed
< gmaxwell>
\O/
< BlueMatt>
heh, in bench we go out of our way to cache countMaskInv (inverse of countMask, an int, as a double) to avoid a div during the inner loop...but in order to avoid that div we first do a convert from int to float, then a mul, then a convert back, which by my instruction tables is roughly the same cost (a tiny bit less, but not enough to care) as just doing the regular int div
< BlueMatt>
anyone care if I nuke that?
< gmaxwell>
I think that was benchmarked, but I hate this code.
< gmaxwell>
the adaptive interval counts stink and make doign a/b tests basically impossible.
< BlueMatt>
yep, if the interval counts change you are *guaranteed* to get a different result, in my experience, even for 10ish-ms runtimes
< gmaxwell>
thats suspicious, but I've seen it too
< BlueMatt>
lol, I know you've run the fibre bench stuff....I think its super sensitive to cache effects given what it does, so that may make it worse
< BlueMatt>
and possibly malloc time
< BlueMatt>
anyway, I'm gonna nuke this int-div conversion crap, if someone wants to actually make the framework work, they can, but this isnt the issue with it
< BlueMatt>
heh, reading wallet, "who in the fuck wrote this garbage"...."oh, I did...in 2011"
< achow101>
BlueMatt: are you volunteering to be wallet maintainer?
< gmaxwell>
sipa is already the wallet maintainer.
< BlueMatt>
no, I thought we nominated sipa
< BlueMatt>
:p
< sipa>
i thought i had already been volunteerified