< assder>
Question: Are unspendable outputs (such as OP_RETURN outputs) stored in the UTXO set? Will pruned nodes have to store them?
< sipa>
they are not stored in the utxo set, but still occupy blockchain space
< sipa>
pruned nodes store them as long as they're in recent history
< assder>
sipa: thanks
< jtimon>
so, phantomcircuit, can we decide with respect to #8077 vs #7985 vs #7947 before deciding about #8087 vs #7310 (or whatever number my next attempt at a "moveonly + doc + trivial fixes" gets)?
< GitHub106>
bitcoin/master 581ddff Wladimir J. van der Laan: net: Add fRelayTxes flag...
< GitHub106>
bitcoin/master 1ab1dc3 Wladimir J. van der Laan: rpc: Add `relaytxes` flag to `getnetworkinfo`...
< GitHub106>
bitcoin/master c028c7b Pieter Wuille: Merge #8049: Expose information on whether transaction relay is enabled in `getnetwork`...
< GitHub121>
[bitcoin] sipa closed pull request #8049: Expose information on whether transaction relay is enabled in `getnetwork` (master...2016_05_rpc_relaytxes) https://github.com/bitcoin/bitcoin/pull/8049
< CodeShark>
the IsDevelopmentBranch() method should check whether it is a release branch or not rather than just whether it's the master branch
< CodeShark>
is there a simple way we can determine this?
< luke-jr>
CodeShark: IMO mining is the least problematic use of dev branches..
< luke-jr>
miners are used to patching and overriding stuff anyway
< luke-jr>
might make more sense to disable wallet and/or bloom
< CodeShark>
luke-jr: the goal is to make it safer to merge consensus changes before releasing
< CodeShark>
if the changes are merged into master they will get more tested
< GitHub162>
[bitcoin] CodeShark opened pull request #8101: Disable mining on nonrelease branches. (master...disable_mining_on_nonrelease_branches) https://github.com/bitcoin/bitcoin/pull/8101
< sipa>
sdaftuar: what type is 'f' in the deserialize methods in mininode?
< sipa>
or more specifically, can i test whether there are more bytes to read?
< sdaftuar>
f?
< sdaftuar>
oph
< sipa>
i'm adding a test that fRelayTxes in version is correct (because it's currently broken in master, and no test detected it)
< sipa>
but fRelayTxes is currently not deserialized
< sipa>
and it's optional per bip37
< sdaftuar>
ah ok
< sdaftuar>
i assume it's possible to tell if there are more bytes to read but i don't know how off the top of my head. f is a BytesIO i think?
< sipa>
i am going to guess it has a eof() method
< sdaftuar>
doesn't seem to? docs i'm reading suggest you just call read() and see if you don't get anything back
< sipa>
yup
< * sipa>
curses extensively at python
< btcdrak>
sipa: it needs feeding.
< CodeShark>
sipa: throw a mouse at it
< sipa>
sdaftuar: so the bug is that a connecting node currently never sets fRelayTxes... and it seems we have not a single test for that
< sipa>
sdaftuar: the p2p tests only connect a testnode to a real node, and not the other way around
< GitHub39>
[bitcoin] sipa opened pull request #8102: Bugfix: use global ::fRelayTxes instead of CNode in version send (master...oopsrelay) https://github.com/bitcoin/bitcoin/pull/8102
< sipa>
sdaftuar: is that possible, or am i missing an extra condition that makes this harder to test?
< sdaftuar>
sipa: you're right that in the testing framework, testnodes connect out to real nodes and not the other way around.
< sdaftuar>
i'm taking a look at 8102
< btcdrak>
sdaftuar: is the list of cfpf related pulls #7600, #7960 and #7292? am i missing any?
< sdaftuar>
btcdrak: just #7600 and #7598. 7598 is a refactor of CreateNewBlock; 7600 builds off it
< sdaftuar>
sipa: i'm baffled that this breakage wasn't caught in our existing RPC tests. surely anywhere we call sync_mempools() we would have seen a test failure?
< sipa>
sdaftuar: perhaps there are more requirements before this triggers
< sipa>
i saw this bug when syncing over thr internet... perhaps the rpc tests run fast enough
< sipa>
it is related to responses to version messages
< wumpus>
I'm not going to be able to attend the meeting today probably
< GitHub132>
bitcoin/master 52b02ec Pieter Wuille: Use global ::fRelayTxes instead of CNode one
< GitHub132>
bitcoin/master 425278d Pieter Wuille: Merge #8102: Bugfix: use global ::fRelayTxes instead of CNode in version send...
< GitHub160>
[bitcoin] sipa closed pull request #8102: Bugfix: use global ::fRelayTxes instead of CNode in version send (master...oopsrelay) https://github.com/bitcoin/bitcoin/pull/8102
< GitHub124>
[bitcoin] sdaftuar opened pull request #8104: Tests: add timeout to sync_blocks() and sync_mempools() (master...improve-rpc-sync) https://github.com/bitcoin/bitcoin/pull/8104
< sipa>
meetink?
< jonasschnelli>
Yes.
< CodeShark>
let's do it
< sipa>
waiting for some more people
< * btcdrak>
raises hand
< cfields_>
here
< paveljanik>
here
< cfields_>
sipa: interestingly: on the net refactor branch I'm rebasing, it magically quit working after rebasing to (this morning's) master. after nabbing your fix, all is good now
< kanzure>
btw i heard from someone that they were surprised that libconsensus refactoring was considered lower priority than segwit
< kanzure>
just some interesting feedback.
< btcdrak>
kanzure: could be arranged.
< btcdrak>
sipa: ack
< cfields_>
sipa: sure. Though i suspect the (my, anyway) answer will be "segwit comes first, hands-down"
< sipa>
cfields_: ok, settled :)
< sipa>
kanzure: good feedback... i think it's mostly that segwit has much more buy-in as a roadmap (but i'm biased here)
< btcdrak>
was that even a question. segwit first.
< cfields_>
sipa: for sure. I'm working on it in parallel, but I have no desire to slow segwit down for it in any way
< kanzure>
right, right. it's more of a long-term note--- but eventually we will have to bite the bullet and absorb the pain of the refactor impacting everyone's branches.
< CodeShark>
libconsensus is strategically at least as important as segwit - but the coordination issues required are considerable
< wumpus>
what do segwit and net refactor compete on?
< sipa>
wumpus: code :)
< wumpus>
which parts? does net refactor give you significantly more trouble rebasing?
< CodeShark>
also, libconsensus isn't as glitsy :)
< sipa>
probably not
< kanzure>
what is the status of net refactor things?
< btcdrak>
where does compact blocks fit in?
< sipa>
libconsensus refactoring worked well in the 0.10 window, because it seemed there was a clear goal (getting script exposed) and a clear way to do it... further refactors seem to be more one-person shows (not that i blame those people, but if we want them to happen, i think we'll need to agree on a plan beforehand)
< wumpus>
well I can understand how libconsensus conflicts with segwit
< kanzure>
wasn't aware of previous concerns about libconsensus plan synchronization, good to know
< sipa>
libconsensus conflicts with everything :)
< wumpus>
there's also some network changes for segwit, but they're at a message level
< wumpus>
whereas cfields' network refactor is at a lower level
< sipa>
yeah, network refactor probably hurts compact blocks more than it hurts segwit
< cfields_>
kanzure: see #8085. I addressed wumpus's notes from Zurich, but that made it rough to read. I'm working on another version of the same thing with a clean history, done by tomorrow for sure
< CodeShark>
as much as it pains me to sacrifice on architecture, I think holding up segwit right now is much more costly in terms of the public's goodwill
< jcorgan>
+1
< wumpus>
I agree progress in the protocol is more important, I think segwit even affects the proposed libconsensus API
< sipa>
worse, it affects the current libconsensus API :)
< sipa>
ok, other topics?
< sipa>
i have a few more
< sipa>
CPFP will also need to go in at some point, and also conflicts with many in-flight things
< sipa>
and a whole range of relay improvement
< wumpus>
libconsensus feels more like some checkbox people want checked than something that will actually have a lot of users but feel free to prove me wrong
< sdaftuar>
if we think segwit will be backported to 0.12, then probably CPFP should wait until afterward?
< wumpus>
it should be done but unless someone has a clear example of an application using it and contributes to it it has not much priority
< sipa>
when i talk about libconsensus i mean "abstracting out consensus logic"... not so much an actual API exposure
< sdaftuar>
to avoid dealing with the CNB refactor in 0.12
< wumpus>
sipa: that's what I mean right, I'm not sure other people talkinga bout it mean the same thing
< wumpus>
at some point it becomes just a buzzword... :)
< sipa>
maybe i should formulate the question this way: segwit, compact blocks, CPFP... all for 0.13?
< wumpus>
that's a bit much
< wumpus>
for just before the feature freeze
< sdaftuar>
sipa: yes!
< wumpus>
(2016-06-16)
< sipa>
i would very much like to have at least compact blocks in before segwit, to alleviate the extra relay latency
< wumpus>
I don't think we should make a habit of merging such big things just before a release
< sdaftuar>
segwit is clearly the heaviest lift here to review... i think the otherw two things can be knocked out very quickly
< wumpus>
segwit is obvious
< sdaftuar>
but that's the thing where it's not clear to me if it'll be sufficiently reviewed by 6/16
< wumpus>
yes that'st he thing what counts
< btcdrak>
compact blocks would be good in 0.13
< wumpus>
segwit should be merged soon so that we can do 0.12.1 before 0.13
< CodeShark>
on that topic...
< sipa>
we can merge segwit with no softfork defined for it on mainnet
< wumpus>
we've already moved the release for 0.13 with a month so I'd really like to not move it again
< sdaftuar>
sipa: interesting! i hadn't considered that
< CodeShark>
sipa: indeed!
< CodeShark>
and even once we do add the segwit softfork we can disable mining on it until release
< sipa>
#idea merge segwit without defined softfork
< cfields_>
hmm
< wumpus>
sure
< sdaftuar>
i guess the thing to worry about is if we have testing gaps, things might break without anyone noticing?
< wumpus>
just to have the code in?
< luke-jr>
need vb gbt before segwit tho..? maybe not if left undefined, unsure
< sipa>
luke-jr: yup
< sipa>
luke-jr: will look at that
< wumpus>
sdaftuar: well the tests need to cover it
< wumpus>
if there's a testing gap it should not be merged in any case
< sipa>
all the segwit tests use regtest
< wumpus>
right
< wumpus>
for that it doesn't matter whether a softfork is defined on mainnet
< sipa>
indeed
< sipa>
and for script/tx tests, the softfork is not relevant
< luke-jr>
sipa: should I look into merging vbgbt w segwit?
< achow101>
what benefit would there be to not define the softfork when mergin segwit
< sipa>
achow101: segwit conflicts with a lot of code, having it in would simplify further development on the branch
< btcdrak>
sipa: we should merge without mainnet, because it will allow people to test on testnet now (it's already been activated in testnet).
< wumpus>
achow101: to have the code in, so that development happens on top
< wumpus>
achow101: and so that people acn use it on the regtest/testnet network
< sipa>
btcdrak: another good reason, indeed
< wumpus>
btw: should we keep the segnet?
< sipa>
no, i want to drop it
< btcdrak>
wumpus: no segnet should go
< sipa>
unless there is a good reason to keep it
< wumpus>
ok
< wumpus>
(no opinion either way just wondering whether that's supposed to end up in master)
< btcdrak>
sipa: there's no need to drop it in merge to master right away imo, but certainly before we add parameters to mainnet.
< wumpus>
yes as it has been triggered on testnet
< sipa>
i will prioritize the testnet dns seed filtering, vb/gbt changes, and doing another batch
< kanzure>
merging segwit without activation might lessen the pressure on reviewers
< kanzure>
which might be a negative side effect
< sdaftuar>
kanzure: agree
< wumpus>
anyhow if the segnet network helps testing I don't have problems with temporarily having it in master, as long as it is clearly communicated that people shouldn't rely on it
< sipa>
i wasn't planning on including it in master
< wumpus>
well playing psychological meta-tricks on reviewers doesn't play much of a role imo, we should do whatever is practical
< wumpus>
if merging segwit helps make progress on other fronts so that 0.13 can be a better release
< wumpus>
we should do that
< wumpus>
also having it merged in master usually means it gets more testing and review, not less
< sipa>
i'll do one more batch, and if there are some ACKs then, i'll squash
< kanzure>
so it would be active in testnet segnet and regtest when merged, but not mainnet, and letting others maintain segwit for other 0.13 changes?
< sipa>
kanzure: don't understand the last part
< sipa>
if there are changes necessary to the code post-merge but pre-release, they can just go in master
< kanzure>
the ideal of merging soon is to let others maintain segwit for possibly conflicting 0.13 changes?
< wumpus>
there's not *that* much time left for 0.13, it's good to decide now what we still want to have in and focus on that
< sipa>
kanzure: to let others rebase their own patches on top
< wumpus>
well not 'maintain segwit' but work on top, yes
< btcdrak>
kanzure: the only difference is not having activation params on mainnet. it would really help by not holding up other work.
< sdaftuar>
sipa: would you still plan to backport to 0.12?
< sipa>
sdaftuar: yes, but after merge in master
< sipa>
(but before defining activation)
< sdaftuar>
ok
< sipa>
ok, other topics?
< gmaxwell>
The non-merged status of segwit has kinda been holding up other work, unfortunately.
< kanzure>
maybe bip151 things?
< sipa>
status bip151: waiting for implementation
< sipa>
i'd say :)
< instagibbs>
sdaftuar, is that list of testing gaps public somewhere?
< petertodd>
re: bip151, I mentioned it today at the conf I was at to some cryptographers, and their response to it not being an off the shelf standard was horror :) might be a pr issue
< kanzure>
were they horrified about the current implementation at all
< petertodd>
sipa: good, we should make that 110% clear then
< sipa>
petertodd: maybe that needs to be made more clear
< gmaxwell>
it's pretty clear in the BIP now, I thought.
< gmaxwell>
maybe it could be moved up to the top.
< petertodd>
gmaxwell: yeah, move it to the top - I just looked at it and didn't see that
< sipa>
#action jonasschnelli make it more clean that bip151 uses openssh's chacha20-poly1305 standard
< btcdrak>
ok make that an action point for the logs
< sipa>
jinx
< petertodd>
make it clear that the standard *describes* a subset of openssh's standard, and that bitcoin's use of it is identical and can reuse the existing code
< sipa>
i for one welcome the new recursively compressing overlords
< sipa>
s/compressing/encrypting/
< petertodd>
I for one welcome sipa's welcoming nature
< btcdrak>
+1
< petertodd>
C-C-C-C-C-C-C-Combo Breaker!
< gmaxwell>
sipa: why didn't warnings save us from the global aliasing bug you fixed today?
< sipa>
gmaxwell: #8105
< sipa>
and travis didn't warn because of #8103
< gmaxwell>
LOL
< sipa>
"oh, machine shutdown
< sipa>
"oh, machine shutdown... ALL TESTS PASS"
< gmaxwell>
Perhaps we need to make sure all commiters have access to super fast machines, and make running the full tests part of the merge script?
< gmaxwell>
sipa: "No failures detected."
< cfields_>
hmm, we've seen that before. thought it was fixed on their end.
< cfields_>
checking
< sipa>
fair enough, that's a subtle but relevant distinction
< cfields_>
one thing we could do is add the rpc-tests to 'make check'
< cfields_>
especially once the java blocktester is replaced in python, that'll make it so that travis would basically just be running 'make check', same as what everyone could do manually
< sipa>
cfields_: ack
< cfields_>
sdaftuar: how's that replacement coming, btw?
< sipa>
together with parallel checking, it's even not that painful anymore