< GitHub50>
[bitcoin] jtimon opened pull request #7665: Contrib: Introduce script to tag compiled binaries for convenience (py) (master...0.12.99-contrib-tag) https://github.com/bitcoin/bitcoin/pull/7665
< GitHub191>
[bitcoin] MarcoFalke opened pull request #7666: [0.12.1] Fix doc and output for rpcauth (0.12...Mf1603-012fixdocrpcauth) https://github.com/bitcoin/bitcoin/pull/7666
< go1111111>
so, wumpus commented on PR 7635 with "utACK after squash". As the PR author does this mean I should make another PR with the changes all squashed into one commit? or will whoever merges it squash them and I take no more action? (this is my first non-trivial PR)
< MarcoFalke>
git checkout docs4; git log -1
< jonasschnelli>
go1111111: squash and force push on the same branch/PR
< MarcoFalke>
You should not see you last commit 429
< jonasschnelli>
I do that with git rebase -i HEAD~X (X needs to be replace with >= amount of commits)
< MarcoFalke>
git rebase --interactive ed067f722a244a31809eab633fbe316028a6f80b~
< MarcoFalke>
and then select `f` or `s`
< go1111111>
ok, will do. thank you
< MarcoFalke>
finally git push $origin docs4 -f
< go1111111>
random github newb question: in the comments on that PR I referred to another branch in my repo (zmq3). it seemed like github then automatically included that branch in the PR also. which seems dangerous if I want to refer to a branch without including it. is my interpretation of how github handles that correct?
< go1111111>
i couldn't find that behavior documented
< MarcoFalke>
no, should not happen
< MarcoFalke>
maybe you fiddled locally with it.
< MarcoFalke>
Try git branch -v
< MarcoFalke>
to see what your branches look like locally
< go1111111>
ok, thanks.
< wumpus>
go1111111: should be no reason to make a new pull request - rather not - if you need help with git just let me know
< wumpus>
I've listed the steps in various PRs scattered over the place, probably would make sense to add it to some developer faw
< wumpus>
faq
< morcos>
btcdrak: ok. i got sucked in. i'm writing rpc/p2p tests for 68/112/113.
< morcos>
i'm basically taking one of your tests that tests the activation logic and modifying to try many variations of different txs before and after soft fork activates and make sure they are accepted/rejected appropriately
< btcdrak>
morcos: we can never have enough test :)
< morcos>
btcdrak: well i was letting you know in case someone else was working on them
< morcos>
but as far as i know we don't have any other than the unit tests so far right?
< sipa>
morcos: awesome, thanks
< sipa>
so what is the current state of the pull request against my bip9 pr?
< sipa>
i agree with you that it makes sense to have separate tests for "50% of the past N blocks have a version we don't know", and one for "an unknowm versionbits deployment is lockedin/activated"
< morcos>
sipa: hmm, i guess thats up to you
< sipa>
though the latter one should be permanent i think... if you happened to be offline during the window in which it activated, you won't ever know itherwise
< morcos>
what do you mean by permanent?
< sipa>
that was a downside we get from bit flags that expire: they are not visible forever
< morcos>
i guess its not clear to me how these alerts work
< morcos>
if you're not using QT, there is no permanent way is there?
< sipa>
if any vb deployment ever activated in your currently active blockchain, you get a warning
< sipa>
eh, no clue what qt does
< morcos>
so right now that strMiscWarning isn't displayed in the errors field
< morcos>
i don't think
< sipa>
hmm?
< morcos>
so the errors that are printed in getInfo won't include this
< morcos>
i think
< morcos>
even for ISM soft forks
< morcos>
the only way you see it is if you look at your debug log
< morcos>
or if you're using QT or the -alertnotify script
< sipa>
what?
< sipa>
really?
< morcos>
maybe?
< sipa>
well, i have no counterevidence
< sipa>
so i think that needs investigating
< morcos>
maybe i taket hat back
< morcos>
sorry, let me see
< morcos>
yeah i think thats right
< morcos>
look at GetWarnngs in main.cpp
< morcos>
strRPC isn't updated by strMiscWarning
< sipa>
heh
< sipa>
that variable name does not even look familiar to me
< morcos>
i also find it a bit disturbing that one warning can wipe out another
< sipa>
i doubt i have ever looked at that code
< morcos>
so for instance if PartitionCheck causes "abnormally high number of blocks" that wipes out your warning that a soft fork has activated
< morcos>
and even in the ISM code, that warning only happens once
< morcos>
so.... i'm going to quietly slink away and go back to my test writing...
< sipa>
we should probably concatenate all warnings...
< sipa>
and have a boolean for "serious" or so
< sipa>
that can make RPCs fail or the GUI show bigger animated themed warning boxes
< morcos>
yeah i think the quick and dirty way is to have separate variables though for each of the warnings
< morcos>
b/c some occur repeatedly or can get cleared out or whatever
< morcos>
and then the display mechanism can check each one and do the right thing?
< sipa>
yeah
< morcos>
i guess it wouldn't be that hard to just have a set of strings, and then you can clear them to
< sipa>
anyway, the point i was trying to make (and which is independent of the warning presentation problem) is that imho a versionbits deployment from anywhere in your past should remain visible (but probably should not be considered serious)
< morcos>
wait, an unknown one should not be considered serious?
< gmaxwell>
Meeting in half an hour, everyone should think back to all the great topics we ran out of time for last week and have them ready for the agenda this time.
< morcos>
sipa: i suppose what we could do is everytime you start the node we rescan for unknown deployments from the beginning (which would speak to using a real cache).. b/c otherwise there is a risk that you shutdown your old node and when you restart it you've lost the warning, is that what you meant?
< sipa>
morcos: i think that the presence of an unknown softfork should not be treated as something serious
< sipa>
morcos: or just have a versionbits cache per bit for warnings
< sipa>
without special logic
< morcos>
sipa: yes thats what i meant by speaking to using a real cache
< morcos>
perhaps its worth the 300k extra memory
< morcos>
but why isn't it serious?
< morcos>
i think it should be considered quite serious
< morcos>
we have in the past, we told you your node was obsolete and you must upgrade
< morcos>
there is no way to know whether the soft fork was relatively benign or not
< morcos>
but if we give the information on the bit/activation height, then it should be easy for people to look up that soft fork, and decide for themselves whether they care
< morcos>
but to suhas's point, thats why its important to be able to warn on the same bit more than once?
< sipa>
hmm
< sipa>
maybe i misunderstood the code
< sipa>
it seemed to me that it would not catch historical deployments
< morcos>
my code will catch historical deployments but only once
< morcos>
i mean once per each deployment, will catch multiple on the same bit
< morcos>
so sipa, i think if it were me, i'd just change my PR to run on start up on the entire blockchain, instead of actually populating a cache that is mostly usless and whose only effect is to obscure multiple deployments on the same bit
< morcos>
we have a list of remaining segwit items that were written on whiteboard at MIT... should we assign those?
< wumpus>
sure
< * sipa>
hides
< * btcdrak>
locked the door
< wumpus>
#topic remaining segwit items
< jonasschnelli>
I'm happy to help,... probably in the wallet section.
< morcos>
seems to me it would nice to buckle down and prioritize BIP 9, BIPs 68,112,113 , segwit. i mean i think we are all working on those things, but there is still more to do on all of them
< sipa>
my plan was to rebase segwit on top of bip9, add the rewind logic to continie after post-softfork uograde, and do a new testnet
< sipa>
eh, new segnet
< btcdrak>
great
< CodeShark>
when do you think to deploy the new testnet?
< sdaftuar>
we also need to discuss standardness rules
< sdaftuar>
(or rather, propose and discuss)
< warren>
What was decided for safety on the edge of the rule change in case of reorg?
< morcos>
I think my suggestion would be to push that problem to wallet users
< sdaftuar>
warren: i believe we decided to advise wallet authors to wait some time after segwit activates before using
< sipa>
one possibility is not enabling relay/mempool logic for segwit transactions until 2 period after activatation
< morcos>
yes authors i meant
< gmaxwell>
warren: I don't understand the context of your question. Generall new soft fork rules should not be used immediately.
< sipa>
another is to just be cautious and nit enable the functionality in wallets until a post segwit release
< sdaftuar>
gmaxwell: this one is more dangerous than usual though
< gmaxwell>
sdaftuar: it's just like p2sh.
< warren>
OK, that seems adequate enough.
< CodeShark>
I'm not that concerned about edge case reorg situations
< morcos>
gmaxwell: its a bit riskier than p2sh... don't need a preimage
< sipa>
it's not situation*s*
< gmaxwell>
in core I would recommend that we would switch to using segwit as default in a subsiquent release after the SF activates.
< jonasschnelli>
so have it ready and auto-use it whiteout creating another release?
< morcos>
so in any case, i think we just advise wallet authors to wait some minimum amount of time after activation to send segwit txs.. they are already going to have to put in some code to wait until it activates
< jonasschnelli>
*without
< CodeShark>
either wait until next release - or wait a few blocks after activation before enabling it in wallet
< sipa>
ok
< jonasschnelli>
auto-enable in in the wallet after 95%?
< gmaxwell>
no.
< btcdrak>
I think better to wait a release code afterwards
< sipa>
not even at 100%
< jonasschnelli>
Okay. Then do it over the release-cycle..
< gmaxwell>
I recommend using a release. Automatic behavior is not needed here. Also-- it's a pretty big behavioral change to use it, e.g. you'd be issuing other address styles in response.
< jonasschnelli>
Agree.
< morcos>
also we haven't written that code yet anyway
< sipa>
that is an open question
< jonasschnelli>
I just think there is a hight incentive to use it (fees)
< CodeShark>
this is something that can be decided once segwit has already been deployed and is awaiting activation
< gmaxwell>
having that triggered by invisible-to-the-user network behavior isn't reat.
< gmaxwell>
great*
< sipa>
i wish we had an address style for segwit part of the standard :(
< gmaxwell>
sipa: no you don't. Deploying a new address style takes forever. :P what you wish for is a magic wand.
< btcdrak>
sipa I thought we did? BIP142?
< CodeShark>
btcdrak: we haven't been pushing it because of concerns regarding scary "new addresses"
< jonasschnelli>
P2WPKH?
< morcos>
i think we should do that though, otherwise everyone is going to embed them in p2sh which is kind of silly
< sipa>
i think we have more buy-in from wallet authors about segwit now, than p2sh had at the timr
< CodeShark>
p2sh was almost entirely under the radar as far as the community at large
< gmaxwell>
also, continued use of base58 is awful. I am going to refuse to discuss address encodings with anyone who hasn't read an address to me over the phone. :)
< btcdrak>
CodeShark: I would suggest brining it back. There's no problem.
< gmaxwell>
CodeShark: thats not the case; besides there is a material difference in the sheer mass of code that must be updated. Besides why is this being discussed there.
< CodeShark>
btcdrak: there sort of is a problem still...
< gmaxwell>
btcdrak: I oppose it in the strongest possible terms.
< CodeShark>
both internal and external
< btcdrak>
CodeShark: you can change you mind :)
< CodeShark>
internally some people hate base58, externally some people are still grandstanding with the "segwit is too hard" crap
< BlueMatt>
gmaxwell: there is certain value in user expectations of things that look like base58
< CodeShark>
I think it's a battle not worth fighting right now
< jonasschnelli>
+1
< btcdrak>
+1
< morcos>
gmaxwell: whats your number
< jonasschnelli>
haha
< gmaxwell>
BlueMatt: You are on subject-matter-ignore until you call me and successfully read me a base58 address. :)
< sipa>
i would really like to have some psegwit afdressddress as part of the "package"
< gmaxwell>
morcos: PM :P
< BlueMatt>
gmaxwell: I'm not suggesting I'm necessarily in support of base58 addresses, but my point is that it is not up to us here to figure out addressing for segwit...that is something WALLETS need to be involved in...people who actually have some ux experience, which does not exist here
< morcos>
gmaxwell: BlueMatt says "..."
< gmaxwell>
yes indeed. but thats also why taking on a new address type at the same time is not a good idea, it would get in the way of that kind of collaboration.
< gmaxwell>
hahha
< BlueMatt>
agreed
< sipa>
do you think so?
< sipa>
i think it's the opposite
< BlueMatt>
I think the idea of pushing off address types for segwit for broader feedback is a good thing
< Luke-Jr>
sipa: that was a lot of backspaces.
< CodeShark>
good UI design wouldn't even burden users with having to deal with copy/pasting cryptographic keys
< CodeShark>
but that's an entirely separate discussion
< Luke-Jr>
FWIW, I've been thinking of implementing the payment protocol better in Core
< Luke-Jr>
including the new BIP 75 stuff
< jonasschnelli>
We are far away from designing the UX... this is a topic we can talk about in 2-3 years.
< sipa>
:)
< jonasschnelli>
But sipa: is the P2WPKH address type not okay?,.. addresses like "p2xtZoXeX5X8BP8JfFhQK2nD3emtjch7UeFm"?
< sipa>
ok, let's postpone that address discussion
< sipa>
jonasschnelli: you mean bip142?
< jonasschnelli>
Yes. Easy to handle by existing wallets?
< BlueMatt>
jonasschnelli: ACK
< gmaxwell>
sipa was raising the concern that if something weren't done sooner we'd be left stuck with 80 bit security forever, I reminded him in PM that we have an upcoming checksig improvement to reduce transaction sizes by 30% which would be a nice time to do a new address type too. :)
< wumpus>
would be good to have concrete proposals for address formats ,say as BIPs
< CodeShark>
we'll have plenty of opportunities to introduce new stuff in the future
< CodeShark>
right now we should focus on the path of least resistance
< sipa>
the reason against incorporating bip142 is people yelling "see! sehwit needs new address types! everyone has to upgrade! not backward compatible!"
< gmaxwell>
The amount going on right now is too great to be able to afford forced seralization.
< CodeShark>
eventually my hope is we'll actually have a competent ecosystem that can actually do tech
< CodeShark>
but for now we must deal with what we have
< morcos>
Yeah I think it would make a lot of sense to release just the P2WPKH address
< Luke-Jr>
sipa: we could add sending support, and discourage anyone from using it to receive for now
< jonasschnelli>
sipa: You could still uses it over P2SH... but we don't live for the critics.
< morcos>
sipa: don't you think everyone is going to say that even more if we don't have address types
< sipa>
i don't know
< sipa>
i'm an engineer, not a marketer
< jonasschnelli>
+1 for P2WPKH bip142
< wumpus>
shouldn't be expected from you to act as marketeer sipa
< wumpus>
you have enough on your plate
< sipa>
next topic?
< CodeShark>
yes please
< wumpus>
suggestions?
< Luke-Jr>
(I don't see value in p2wpkh, but let's move on)
< wumpus>
ok, if not, that was a quick meeting
< gmaxwell>
there were many things last week that got cut off.
< wumpus>
(I'm still too tired and jetlaggy to contribute much now)
< jonasschnelli>
If no one has something important... what do you think about my approach of IBD with a pregenerated signed UTXOset?
< btcdrak>
Can I ask people to review the backport PRs for BIP68 and 112? They were straight cherry-picks into 0.12 but they do need a couple eyes on them.
< morcos>
jonasschnelli: i think thats a bad idea
< Luke-Jr>
jonasschnelli: sounds like a waste of time at best, to be frank. :x
< wumpus>
#action review backport PRs for BIP68 and 112
< morcos>
jonasschnelli: core devs (or anybody for that matter) should not be authorizing the state of the ledger at any time
< wumpus>
#topic IBD with a pregenerated signed UTXOse
< wumpus>
it's risky, it brings trust into the system
< Luke-Jr>
much prefer to see SPV mode until IBD completes
< jonasschnelli>
Luke-Jr: Yes. Agree.
< wumpus>
who would you trust to sign something like that?
< sipa>
Bob.
< jonasschnelli>
Just thought we might want to reduce the amount of block-serving even more.
< wumpus>
yes, definitely Bob
< Luke-Jr>
XD
< jonasschnelli>
wumpus: could be signed by the same users/devs who are signing the gitian builds.
< sipa>
jonasschnelli: that's not reducing block serving; it's changing the trust model instead
< jonasschnelli>
I agree. It does change the trust model.
< wumpus>
jonasschnelli: hmm don't put too much on their shoulders
< CodeShark>
without TXO commitments it's unfixable
< CodeShark>
:p
< morcos>
if we go back to the backport question, we also need to backport all these SF's to 0.11 is that correct?
< jonasschnelli>
Okay... so. Then let me not implement it. :)
< wumpus>
yea, UTXO commitments would make this somewhat more realistic, although it'd still reduce the overall trust model to SPV
< wumpus>
jonasschnelli: yes, better not for now I think
< wumpus>
#topic backports to 0.11
< gmaxwell>
morcos: I was thinking about this this morning. The normal release cadance would have us do so, I believe.
< jonasschnelli>
SPV during IBD and a throttled(CPU) IBD is the better approach.
< wumpus>
jonasschnelli: yes, definitely those would be good to have
< Luke-Jr>
how practical is it to backport to 0.10?
< warren>
why 0.10?
< wumpus>
topic is backport to 0.11 Luke-Jr :p
< sipa>
0.10 and 0.11 are close
< morcos>
0.10? i'd hoped you guys would be willing to skip 0.11
< sipa>
but agree; just 0.11
< btcdrak>
morcos: I would skip 0.11
< sipa>
we have a published release schedule, no?
< wumpus>
I think backporting to 0.11 is fairly necessary, that's only one release back
< Luke-Jr>
sipa: of guaranteed support, not limited-to
< Luke-Jr>
0.11 is necessary; 0.10 would be ideal
< morcos>
wumpus: i suppose, i'm worried about how well tested these major changes will be
< sipa>
master, 0.12, 0.11
< Luke-Jr>
but I'll deal with 0.10 later I guess
< jonasschnelli>
0.10 is not necessary... we once agree in supporting only two major versions.
< Luke-Jr>
jonasschnelli: it was never agreed to drop support for older ones, only to not promise support for them; but that's on me for 0.10 I think
< wumpus>
morcos: it's a challenge, but I think you can't get around it, some people won't be ready yet to upgrade to 0.12
< btcdrak>
the backports to 0.12 are trivial, but 0.11 will be a little more annoying, especially for BIP68 I believe
< morcos>
btcdrak: we should just backport them kind of altogether right?
< btcdrak>
btw the backports for BIP68,112 are #7543 and #7544, I forgot to mention the numbers earlier
< morcos>
i suppose they don't overlap that much
< morcos>
i think i can make sure BIP68 for 0.11 backports properly
< btcdrak>
morcos: you'd be the best person to backport BIP68 to 0.11.
< morcos>
i will take the approach of not keeping the mempool consistent and checking sequence values in the mining code, which will probably not make much of an effect on the already absurdly slow mining code
< morcos>
thats why 7187 was separate, that won't be backported to 0.11
< btcdrak>
morcos: plenty miners have upgraded to 0.12 fwiw
< btcdrak>
well pools*
< morcos>
ok. i'll work on that
< wumpus>
the other path would be to wait for 0.13, then only backport to 0.12, but then you'll have to wait 6 months :p
< Luke-Jr>
>_<
< btcdrak>
wumpus: nice joke there :-P
< wumpus>
well not exactly 6 anymore but ok...
< wumpus>
I don't think it's realistic no
< sipa>
i think we can do 9/68/112/113 soon
< wumpus>
aim for 0.13 would be july
< morcos>
sipa: agreed!
< wumpus>
sipa: I hope so!
< btcdrak>
sipa: I also believe so. 68/112/113 are done from my side, morcos wants to add more RPC tests which is fine.
< morcos>
did you ever reply to my comment on block version = 4
< morcos>
btcdrak: there are NONE!
< CodeShark>
any plans to remove the default version crap out of primitives?
< btcdrak>
morcos: yes there are. same standard as for BIP65
< sipa>
CodeShark: yes, see bip9
< sipa>
morcos: it was needed in combination with the warning logic
< btcdrak>
morcos: see #7648
< sipa>
it may not be needed anymore with imoroved logic
< morcos>
btcdrak: i saw, and i agree, BIP65 made it out withotu adequate tests
< sdaftuar>
sipa: right now #7575 will revert back to version 4 blocks after the first soft fork activates, if no other soft forks are in the queue
< jonasschnelli>
bip65 had rpc tests from petertodd?!
< sdaftuar>
i assume that is unintentional?
< btcdrak>
morcos: I would disagree they were not adaquet, you can always have more sure.
< btcdrak>
bip65 had RPC tests.
< sipa>
sdaftuar: that seems correct to mr
< sdaftuar>
oh, i didn't realize that!
< morcos>
sipa: huh? correct as in you wanted that?
< sipa>
the old version is used to indicate "no versionbits"
< btcdrak>
bip68/112/113 have the p2p RPC tests in #7648
< sipa>
morcos: yes, otherwise the version is ambiguous based on what software miners use, ignoring deploymemts
< sipa>
which the warning logic can't deal with
< sipa>
it must be absolute "these deployments -> this nVersion"
< morcos>
sipa: all prior soft forks required minors to upgrade
< sipa>
miners :p
< btcdrak>
lol
< CodeShark>
minors can always get a fake ID
< morcos>
sipa: what i would like to do is with this first soft fork, require nVersion > 4
< sipa>
why?
< btcdrak>
morcos: why?
< morcos>
then we can warn on anything that is not 001... unless it is <=4 which we know are invalid
< morcos>
that seems consistent with how we've done other soft forks
< morcos>
there is an expected version number, unknown differences are warned on
< gmaxwell>
so old software versions warn.
< gmaxwell>
and continue warning.
< morcos>
gmaxwell: yes, old software versions are incapable of noticing transient changes
< morcos>
thats why we need to rework alerts/warning to correctly identify them now
< morcos>
in fact if you turned off a 0.12 node for a couple months
< morcos>
and then turned it back on after all these SF's activated
< morcos>
you would have no idea
< morcos>
even if you looked at your debug logs
< morcos>
b/c the warning logic doesn't run in IBD
< morcos>
i feel like that makes it a requirement that we permanently increase the version number
< sdaftuar>
agreed
< sipa>
wth are you talking about?
< morcos>
who?
< CodeShark>
I don't follow
< morcos>
me? which part?
< sipa>
versionbits is deterministic based on the chain histotu
< morcos>
yes 0.12 doesn't have version bits
< morcos>
0.12.0
< CodeShark>
after versionbits deployment, all block versions would be > 4
< sipa>
ah
< gmaxwell>
sipa: he's talking about software which doesn't have versionbits.
< sipa>
oh, now i get it
< sipa>
sorry
< morcos>
CodeShark: thats what i'm trying to say, thats not how its written
< sipa>
so increase to 5 after the first vb deployment?
< morcos>
no no no
< sdaftuar>
why not increase to 001...?
< morcos>
not 5
< sdaftuar>
consensus rule is >= 5 i guess
< morcos>
just > 4, but you expect to see 001
< sipa>
consensus >=4, but by default set 001...000....?
< jonasschnelli>
Agree that we should also warn in IDB (but low prio IMO).
< sdaftuar>
jonasschnelli: that code doesn't run during initialblockdownload
< morcos>
the assumption is if you see something that starts with 001 you think you know whats happening and you're unknown activation detector can alert you with specific information
< jonasschnelli>
Then lets improve that and BP.
< morcos>
and if you see soemthing that doesn't start with 001 such as a 5 or a 110...
< morcos>
jonasschnelli: thats what we're doing , but we can't retroactively change old code
< CodeShark>
110... would be treated as negative, no? because for some reason signed types were used
< morcos>
then you assume that someone is doing something you really don't understand
< jonasschnelli>
Yes. Sure... we might not be capable of warning for the next SF.
< morcos>
CodeShark: sure i mean 010
< btcdrak>
morcos: there is already 011 on the network too
< morcos>
btcdrak: exactly, and we should be warning about that (and we are now) b/c its changes our software doesn't understand
< btcdrak>
right
< morcos>
and if > 50/100 blocks then we turn that warning into an alert
< jonasschnelli>
Agree. But that raises also the question how to deploy an alert... debug.log? Yes. No.
< morcos>
jonasschnelli: see scroll back before meeting
< * jonasschnelli>
has only a 500 lines scrollback >_<
< morcos>
sipa: so agreed that we should increase version number? should i make a new BIP for that? and we'll just soft fork it in at same time, or add to existing BIP
< sipa>
morcos: consensus or not?
< morcos>
sipa: consensus rule that nVersion must be >= 5.
< sipa>
morcos: why consensus?
< btcdrak>
morcos: confused
< CodeShark>
before or after versionbits activation?
< CodeShark>
still don't follow
< morcos>
sipa: well i suppose it doesn't have to be a consensus rule...
< morcos>
CodeShark: with the first SF that activates
< morcos>
sipa: but i think its more clear to me that it doesn't have problems if it is a consensus rule
< morcos>
b/c thats how the other ones worked
< gmaxwell>
making it consensus would cause gratitious orphaning, which we were generally trying to avoid in the design of segwit.
< morcos>
gmaxwell: this will be before segwit
< sipa>
i prefer not introducing new consensus logic, especially when the only argument for it is better guarantees for warnings
< morcos>
sipa: yes thats the only difference i see. is that now we somehow can't warn on version = 4 blocks
< sipa>
and if it's not consnesus, we can say bip9 miners without active rollouts use 001000...
< gmaxwell>
I like 001000 in that it would encourage visualization tools to parse the bitfield instead of just displaying an integer.
< btcdrak>
yes
< morcos>
sipa: if its not a consensus rule, you can't be SURE that old nodes will be warned that the rules have changed... perhaps thats not worth worrying about
< sipa>
miners can already cause total network forks
< btcdrak>
That was my initial understanding that the new version would be 00100..0 when no sfs are being deployed
< morcos>
its just cleaner to me
< sipa>
are we worried that they can bypass a warning mechanism?
< btcdrak>
sipa: no
< morcos>
sipa: ok i guess. i'm ok with not making it a consensus rule.. but just seems weird to me... i feel like we're transitioning from an old system to a new system, and the transition should conform to the old system
< gmaxwell>
Lets wrap the metting now and talk more about warning after. Unless there are any other subject to squeek in at the end?
< wumpus>
meeting: 3 minutes to go
< morcos>
but as long as we make the default 00100 then i think its just theoretical concerns
< sipa>
yeah
< wumpus>
#endmeeting
< lightningbot>
Meeting ended Thu Mar 10 19:58:17 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< gmaxwell>
sipa: So a concern I raised on the PR is that I'd like nodes to stop mining (e.g. error on GBT results) after a SW upgrade happens. The issue I'm trying to address is say the network successfully upgrades, and then a couple huge pools reboot and end up on pre softfork code-- e.g. bootscripts start the wrong versions; things run happily for a while, but then someone mines an invalid transaction a
< gmaxwell>
nd then a huge amount of hashpower begins a fork. Nversion progression previously prevented this, but at the expense of creating gratitious orphans around the switchover time.
< morcos>
gmaxwell: i was wondering about that, but do miners change their version number outside of bitcoind anyway?
< gmaxwell>
morcos: in the past some mining hardware had hardcoded versions numbers, but I think the last couple softforks probably shook out a fair amount of that.
< sipa>
we hope...
< morcos>
gmaxwell: so this is another reason to make >= 5 a consensus rule
< Luke-Jr>
morcos: they set it outside bitcoind generally
< morcos>
if you don't you can't fix that problem for 0.12.0 miners
< sipa>
how does that help?
< gmaxwell>
sipa: the idea is that post SW miners would do the voluntarily shut off unless overridden.
< morcos>
sipa: well i don't know. i took gmaxwell's scenario at face value.. but maybe it doesn't make sense. i guess the difference is whether you find out right away or not?
< gmaxwell>
It's a difference in the size of the fork; and who takes the risks.
< morcos>
gmaxwell: are you specficially talking about SW script upgrades? or do you mean BIP 9 soft forks?
< gmaxwell>
BIP9 softforks.
< morcos>
gmaxwell: can you explain a bit why the fork size would be bigger?
< sipa>
if the vereion number is set outside of bitcoim core, then logic there won't help
< sipa>
you just need logic that stops GBT in case the vb warning logic triggered
< morcos>
that seems a bit risky to me to automatically stop GBT
< gmaxwell>
morcos: Because only an exceptional circumstance would begin such a fork, it might happen months or years after the change, with no one paying attention.
< morcos>
i mean i guess it does take 95% of miners to cause a fake trigger
< gmaxwell>
morcos: the existing warning stuff is stupid, boarderline broken-- which is why I was suggesting it only for BIP9 activation.
< gmaxwell>
morcos: the idea that the network in bulk could silently regress rules (though detectable by the nodes themselves) concerns me; but I don't know how much of a pratical concern it should be.
< morcos>
gmaxwell: the fact that nVersion is set outside bitcoind is disturbing, but imagine 1% of miners don't upgrade to versionbits and the 68/112/113 soft fork
< morcos>
they will happily mine version 4 blocks for months until they accidentally mine an invalid BIP68 tx, and then all the SPV miners will just build off them
< morcos>
as opposed to htat thing happening right away if the nVersion had to increase
< * Luke-Jr>
wonders if versionbits has GBT extensions yet
< sipa>
Luke-Jr: is it not enough to report nVersionm
< sipa>
?
< gmaxwell>
morcos: right, so two questions: avoiding that for this one change vs in the long term.
< Luke-Jr>
sipa: no, because nVersion could mean different things
< morcos>
gmaxwell: yes, i agree that perhaps your long term change makes sense
< sdaftuar>
gmaxwell: morcos: seems to me like we should do both things you guys are suggesting
< morcos>
i mean your change now to solve the longer term problem
< sipa>
Luke-Jr: ?
< Luke-Jr>
sipa: version=69632 could mean different softforks depending on context of the block
< sipa>
so?
< gmaxwell>
morcos: in the long term I think it's adequate to refuse to serve GBT requests after a BIP9 activiation triggers. (and perhaps mine only empty blocks during the quiet period for further visibility)
< gmaxwell>
sipa: matters if you're adding additional txn to the gbt output.
< Luke-Jr>
sipa: so the GBT client won't know which softforks to enable
< gmaxwell>
morcos: for the current issue, I agree that an upgrade to >=5 may be needed.
< sipa>
Luke-Jr: nobody is doing that, right?
< gmaxwell>
Luke-Jr: I think you actually want to expose the mempool validation flags in GBT for that, not just softforks.
< Luke-Jr>
sipa: adding transactions, not at the moment AFAIK, but GBT clients always must be aware of what the block rules are to some extent
< sdaftuar>
note that the version bits do not indicate what the consensus rules are
< sipa>
Luke-Jr: i guess those can be per-softfork extensions to GBT
< sipa>
as their effects can be hard to generalize
< Luke-Jr>
sipa: some way to indicate them is still needed, but yes
< Luke-Jr>
ie, we don't want old GBT clients to think they can handle bit X, but interpret it differently
< morcos>
Luke-Jr: any time you are setting bit X it by definition means that soft fork isn't even active yet
< morcos>
or at least by implementation
< Luke-Jr>
more likely the GBT client would be the side setting the bits it implements
< gmaxwell>
Luke-Jr: trying to move network consensus logic into gbt clients seems inadvisible and should only be done where strictly needed.
< sipa>
gmaxwell: +1
< BlueMatt>
instead we should be pulling out as MUCH logic as possible from gbt clients/pool servers into bitcoind
< BlueMatt>
and also kill getblocktemplate
< BlueMatt>
because no one uses it and everyone hates it
< gmaxwell>
BlueMatt: so mean. By "kill" you mean "optimize".
< BlueMatt>
gmaxwell: I mean replace entirely
< sipa>
making it observable what is being mined makes sense
< BlueMatt>
gmaxwell: no one cares about the features provided by gbt, and every time I talk to /anyone/ using it (except eloipool) the response is "omg, just replace it with a binary something-or-other that has nearly opposite features of what gbt provides"
< sipa>
but i don't think GBT's goal should be letting block construction outside of bitcoind
< BlueMatt>
^that
< BlueMatt>
is also what people want
< BlueMatt>
most folks just want the minimal data they need to be able to twiddle extranonce
< gmaxwell>
if the nonce in header fork were done we could go back to getwork. :)
< BlueMatt>
and that
< btcdrak>
BlueMatt: is there any more progress on HF contents?
< BlueMatt>
btcdrak: I havent seen anything on bitcoin-dev?
< BlueMatt>
btcdrak: also #bitcoin-dev
< Luke-Jr>
sipa: so just give up on decentralised mining entirely?
< BlueMatt>
Luke-Jr: that is a false dichotomy
< gmaxwell>
there are other ways to do that, e.g. telling bitcoind what the requirements are for a block it creates and letting it create it.
< BlueMatt>
Luke-Jr: the amount of data you need to twiddle the extranonce is almost the same as the amount of data you need to create a coinbase for gbt in the same way you want people to do decentralized mining
< gmaxwell>
E.g. coinbase must pay to X.
< BlueMatt>
gmaxwell: you can create the coinbase downstream of bitcoind
< gmaxwell>
only by implementing consensus rules outside of bitcoind, however.
< BlueMatt>
bitcoind should be able to tell you things like requirements for what scriptSig should look like
< gmaxwell>
BlueMatt: you are reinventing GBT. :)
< Luke-Jr>
gmaxwell: this is higher latency
< BlueMatt>
gmaxwell: no, gbt gives you all kinds of shit like details for every transaction in the block
< BlueMatt>
you should only be providing minimal merkle tree info
< gmaxwell>
BlueMatt: which is actually needed if you're building your own coinbase, because you may need to drop out transactions to make room.
< BlueMatt>
gmaxwell: also, phantomcircuit has a protocol designed for this
< BlueMatt>
that is pretty minimal
< BlueMatt>
gmaxwell: "meh", your coinbase shouldnt be /that/ large
< gmaxwell>
go look at a p2pool coinbase txn.
< BlueMatt>
I'm aware
< BlueMatt>
ehh, whatever, fine, give bitcoind a coinbase output list
< BlueMatt>
either way, push complexity into bitcoind :)
< * Luke-Jr>
notes that's the approach he took originally and gave up after years of it not being merged.
< morcos>
i know nothing about mining, so hopefully im not wasting peoples time
< morcos>
but it seems to me that it would be reasonable to require miners to run a bitcoind
< * gmaxwell>
plays taps for 'coinbaser'
< morcos>
and we should provide an API by which they can submit a block to to a pool and say hey, is this acceptable
< morcos>
and then they can work on that
< morcos>
i know thats the idea behind GBT (sort of)
< morcos>
but instead of building all the functionality into the API
< BlueMatt>
Luke-Jr: to be fair, people also hate json, so need something outside the existing rpc
< Luke-Jr>
BlueMatt: it wasn't via RPC ;)
< BlueMatt>
hmm, fair enough
< morcos>
just have a testBlock RPC call that the pool can say, yeah thats cool pays the right coinbase or whatever and is valid according to our consensus rules
< Luke-Jr>
preferably there'd be a way to avoid sending the entire gen-tx around.
< BlueMatt>
Luke-Jr: so revive it now :)
< Luke-Jr>
BlueMatt: ?
< Luke-Jr>
not sure the point
< jtimon>
sipa morcos, what about always producing 0010... (even when no deployments are being checked) after the the starttime of the first deployment? (my own solution was to just do it right away for new miners, even before any start time, but sipa complained that old nodes would get warnings before start time [they're going to receive warnings before activation anyway, so I wasn't too concerned about that])
< morcos>
jtimon: yeah i think its strictly better for the old nodes to get the warnings earlier
< jtimon>
that was my point, the dicsussion is there in some outdated comment in #7575
< jtimon>
and that solution doesn't require any additional consensus checks, which seems to be what people dislike more from your proposal
< e0_>
The variables pathCached and pathCachedNetSpecific in util.cpp appears to be unused (find doesn't return any results of them being set) but can cause interesting segfaults since depending on the order in which objects are created they can be uninitialized. Are they actually unused