< GitHub193>
bitcoin/master 9a227e9 Wladimir J. van der Laan: tx: change slug to `bitcoin.qt-translation-013x`...
< GitHub193>
bitcoin/master 3b2dadc Wladimir J. van der Laan: qt: Periodic translations update
<@wumpus>
morcos: thanks for the info; yes that makes sense, I think we should (separately) consider lowering the default mempool size
<@wumpus>
but not for 0.13, maybe, as you say it warrants more testing
< phantomcircuit>
sipa, i'd like to implement mempool sync using short ids
< phantomcircuit>
looking at the trickle logic for this
< phantomcircuit>
im thinking this is no longer trivial
< phantomcircuit>
is there a shortcut you can point me towards?
< phantomcircuit>
:)
< sipa>
can you elaborate?
< phantomcircuit>
sipa, the goal is to essentially prime the cache for compact block
< gmaxwell>
phantomcircuit: the lack of any fetch by short ID might get in your way. :P
< phantomcircuit>
peers request the top ~2MiB of a nodes mempool (ancestor index) and the node responds with short ids
< phantomcircuit>
gmaxwell, well yeah i'd be adding that
< phantomcircuit>
keyed on the last block the requestee had seen
< gmaxwell>
perhaps better is to request a "fakeblock", and then use the getblocktxn mechenism?
< phantomcircuit>
so if you weren't at the same tip you'd just get gibberish
< gmaxwell>
why require same tip? just have the sender pick a nonce.
< phantomcircuit>
because this can be cached by the node receiving the request
< gmaxwell>
or even generate a block, cache it, and serve requests on it.
< gmaxwell>
you can cache a generated block.
< phantomcircuit>
hmm
< gmaxwell>
what kind of timeframe do you think you'd do this in? for mempool sync in general, set reconcillation should be used. PinSketch is super easy to use C++ code, but we couldn't put it into production until rewriting it. (due to incompatible licensing and because NTL is huge)
< gmaxwell>
set recon may or may not make sense for block relay, because the cpu time to decode is prohibitive enough to get in the way of latency... but for mempool sync no such issue exists.
< gmaxwell>
and it sends data exactly equal in size to the symmetric difference.
< phantomcircuit>
i'd like something really simple that we can do like
< phantomcircuit>
nowish
< phantomcircuit>
i have some private patches which are kind of stupid that do this for mining nodes
< phantomcircuit>
there's a pretty significant advantage to having a mining node with a smaller mempool
< gmaxwell>
If there is we should fix that.
< gmaxwell>
because of the use of indexes I'd consider it a bug if createnew block was taking more than a tiny amount longer with a 100MB mempool than a 2MB one.
< phantomcircuit>
but yeah i guess i should try to benchmark that first
< gmaxwell>
I was wondering that as I commented, but that might just be the case where we're missing some index we should have.
< sipa>
gmaxwell: the mempool remove of conflicts code copies CTransaction objects all over the place
< sipa>
eh phantomcircuit
< phantomcircuit>
gmaxwell, i'll benchmark it
< phantomcircuit>
im prettttty sure it's slow
< gmaxwell>
great. "we should fix that"
< sipa>
we should switch that to shared_ptr based
< sipa>
so it can return the deleted transactions without copying
< gmaxwell>
phantomcircuit: I think mempool sync is useful and important, but it shouldn't be used to hack around even more important speed improvements.
< phantomcircuit>
gmaxwell, the principle reason for it is to prime the cache for compact blocks
< phantomcircuit>
but i guess that can just be a simple list of txids
< phantomcircuit>
once per connect
< gmaxwell>
yes, thats an initially useful application, though we should also be considering directing our attention toward reducing relay overhead, because with BIP152, it's now the biggest user of bandwidth on a listening node.
< phantomcircuit>
ok well i guess i'll benchmark the conflict thing
< sipa>
phantomcircuit: that would be very useful
< sipa>
i can write a patch that i think will improve its performance significantly
< phantomcircuit>
sipa, uh first off it seems we're iterating vTx twice in removeForBlock for no reason?
< GitHub19>
bitcoin/master 2129fce Pavel Janík: The bit field is shown only when status is "started"
< GitHub19>
bitcoin/master 3685e0c Wladimir J. van der Laan: Merge #8261: The bit field is shown only when status is "started"...
< GitHub152>
[bitcoin] laanwj closed pull request #8261: The bit field is shown only when status is "started" (master...20160625_sw_getblockchaininfo_bit) https://github.com/bitcoin/bitcoin/pull/8261
< GitHub17>
[bitcoin] sdaftuar opened pull request #8280: Tests: Increase sync_blocks() timeouts in pruning.py (master...fix-pruning-test) https://github.com/bitcoin/bitcoin/pull/8280
< sipa>
petertodd: re your review the witness cost check is in contextual because we need to know whether segwit is enabled when seeing a block without context
< GitHub142>
[bitcoin] laanwj opened pull request #8281: qt: Remove client name from debug window (master...2016_06_qt_remove_client_name) https://github.com/bitcoin/bitcoin/pull/8281
< GitHub20>
[bitcoin] EthanHeilman opened pull request #8282: net: Feeler connections to increase online addrs in the tried table. (master...feelers3) https://github.com/bitcoin/bitcoin/pull/8282
< paveljanik>
curl: (51) SSL: no alternative certificate subject name matches target host name 'patch-diff.githubusercontent.com'
< paveljanik>
Pavels-MacBook-Pro:tmp pavel$
< paveljanik>
their cert doesn't contain proper SAN.
< paveljanik>
reported to github
< petertodd>
sipa: but my point is, currently the way that cost check is written is such that it's applied even if segwit isn't enabled, for non-segwit blocks
< petertodd>
sipa: that said, I think you're right that the content of the footnote is wrong; I'll delete it
< sipa>
petertodd: yes, but only after checking that there is no superfluous witness record, and that check is dependent on context
< sipa>
thank you for the review though!
< sipa>
the malleability to exceed size limits needs to be addressed
< petertodd>
sipa: sure, the existing code I do believe works, I was concerned about problems if we ever add anything else to that cost check
< sipa>
ah, ok
< petertodd>
sipa: sorry if it's not clear, but most of my objections are semi-theoretical, "if we screw up x this will break", belt-and-suspenders stuff :)
< petertodd>
speaking of, I just added this sentence to the summary: "Put another way, in a number of places we either have a belt, or suspenders, when given the importance of this code we’d rather have both a belt and suspenders."
< bsm117532>
Hey petertodd, I'm doing some stuff with your python-bitcoinlib. Have you had a chance to incorporate the new address types and whatnot for segwit?
< sipa>
there are no new address tyoes
< sipa>
tyoes
< sipa>
tyPes
< bsm117532>
Oh I see, you embedded it in P2SH.
< * bsm117532>
does a little dance, and pushes the "segwit required from now on" button in his code.
< bsm117532>
I have a use case which absolutely needs the txid to be immutable. In order to achieve that, is it necessary that the *inputs* to my transaction be *outputs* of a segwit transaction?
< sipa>
yes
< bsm117532>
Ok then I'll have to test the inputs. Thanks sipa.
< bsm117532>
So the trick is that the scriptSig is only data pushes. Since the top of the stack is non-zero (unless you know a preimage that hashes to zero), pre-segwit considers it anyone can spend, and any following input with an empty scriptPubKey can spend it. No?
< sipa>
in segwit scriptSig will e empty
< sipa>
the scriptPubKey is just pushes
< sdaftuar>
well, p2sh-wrapped-segwit won't have empty scriptsig
< sipa>
right, it would just have a redeemScript
< petertodd>
bsm117532: apparently someone else has been working on adding segwit to python-bitcoinlib; I haven't looked at their code yet, which afaik isn't public yet :(