< Luke-Jr>
sipa: indeed, I think the decision whether to translate or not really lies with the target languag
< Luke-Jr>
when it is inappropriate, the translators can just leave it as-is
< Luke-Jr>
anyone happen to remember what the whitelist-peer hack is needed to get RPC tests to pass on 0.10/0.11? :/
< jayd3e>
I'm receiving this error "scheduler.h:14:35: fatal error: boost/chrono/chrono.hpp: No such file or directory", even though I have installed libboost-dev-all
< jayd3e>
followed the directions for 12.04 exactly
< phantomcircuit>
jayd3e, iirc chrono is missing in older versions of ubuntu
< jayd3e>
so use 14.04?
< jayd3e>
phantomcircuit:
< phantomcircuit>
jayd3e, sorry i dont know which ubuntu versions will work
< phantomcircuit>
i dont think 12.04 will though
< jayd3e>
kk
< morcos>
CodeShark: sorry, dirty way of getting your attention
< CodeShark>
No worries :)
< morcos>
Are we still trying to get this rolled out as the next soft fork
< morcos>
if so we should be concentrating core development effort on all of these things so we can show progress. versionbits, BIP68/CSV, segwit
< morcos>
but there doesn't seem to have been any movement on the versionbits PR for quite some time
< btcdrak>
who is doing the final implementation now, CodeShark or rusty?
< morcos>
oh
< morcos>
see i didn't know about that
< CodeShark>
I think the segwit testnet has taken priority right now
< morcos>
is there a different implementation?
< morcos>
is that something that is being worked on?
< morcos>
i asked sipa about that earlier and he didn't reply
< morcos>
where is that happening?
< btcdrak>
morcos: even if versionbits was not ready we can still deploy using ISM though...
< petertodd>
morcos: and there's still my pseudo-versionbits proposal that we can use too
< morcos>
sure i agree we don't have to have versionbits before segwit
< morcos>
i just thought that was what wumpus laid out in his plan
< petertodd>
also, for the record, I would NACK segwit myself without a validationless mining fix
< CodeShark>
wanna help implement that, petertodd?
< petertodd>
CodeShark: yes, but note that I have no funding right now for core dev work, so I can't promise anything
< petertodd>
CodeShark: that said, I'm fairly certainly I'll be able to get some soon
< CodeShark>
the short-term objective is to deploy a functional testnet without all the intended features and to set up a framework for adding more features
< petertodd>
CodeShark: makes sense to me
< CodeShark>
deployment to mainnet is still at least another couple months away, I'd say...once we near that we can figure out which activation mechanism we'll use
< CodeShark>
So versionbits is now on the backburner
< CodeShark>
If you have some extra time and want to dig in, petertodd, I'd love to hear your suggestions
< petertodd>
CodeShark: well, I'll see that I can do
< morcos>
/window 24
< morcos>
oops
< sipa>
petertodd: your suggestion is easy to implement for consensus rules; i just worry about the logic needed to get mining software to support it
< sipa>
anything that changes the coinbase outputs needs extra logic
< btcdrak>
CodeShark: what about rusty, he has code done also.
< btcdrak>
oh wait, wrong conversion :)
< petertodd>
sipa: I'm not suggesting changing the coinbase outputs, just committing to something calculated form them
< petertodd>
sipa: equaly, the idea probably would work even without the coinbase output commitment
< petertodd>
sipa: important thing is that miners never soft-fork in a commitment to the prior-block possession proof in the prior block
< sipa>
petertodd: you're suggesting making block X commit to for example H(full block X-1 incl witness, X's coinbase outouts) ?
< sipa>
s/,/||/
< sipa>
right?
< petertodd>
sipa: yes
< sipa>
which means that any software which computes the coinbase outouts will need updated code to compute the commitment
< petertodd>
sipa: right, that's fair
< sipa>
which for example makes the 21 computer incompatible with it
< petertodd>
sipa: so maybe make it commit to H(full block X-1 + nonce)
< sipa>
ah!
< sipa>
what is nonce?
< petertodd>
sipa: then soft-forking in a later committment to the coinbase outputs is possible, as is anything else
< petertodd>
sipa: it's probably ok if the nonce can be picked arbitrarily - so long as miners don't actually soft-fork in a commitment to that commitment in the previous block, we're ok
< petertodd>
sipa: and in fact, if it's the *full* block that's impossible :)
< sipa>
i'm very interested in including solutions for this, but i think it will need input from people with more experience with mining software/hardware
< petertodd>
sipa: agreed
< petertodd>
sipa: part of why I want to do this now, is because I think if we *don't* fix this, we will have significant pushback on fixing it later
< sipa>
petertodd: also, i think you want the full block data at the end of the hash, so you can't compute a midstate for it
< petertodd>
sipa: IE, we probably will have to allow a validationless mining alternative where you leave off the commitment, in exchange for an empty block
< petertodd>
sipa: well, if the nonce is picked arbitrarily, that's technically not an issue, but yeah, best to do H(nonce + <data>)
< sipa>
petertodd: i mean, you probably want to prevent a setup where someone can send a midstate of the full prevblock
< sipa>
petertodd: i think there may be significant pushback against this already; any device that decides the coinbase but leaves tx selection up to something else will now need to be burdened with full blocks, where it only needed very low bandwidth beforehand
< petertodd>
sipa: well, the fact that you can do that is probably a flaw in the bitcoin protocol
< morcos>
sipa: "pushback against this" you mean segwit itself?
< petertodd>
sipa: I'm just trying to at least get us back to thecurrent situation, where if you do validationless mining you're trusting others
< petertodd>
sipa: IE, don't make the situation worse
< btcdrak>
morcos: against validationless mining fix
< morcos>
but the issue with the coinbase needing more data exists with just segwit
< petertodd>
morcos: not quite - with just segwit the rest of the coinbase outputs can be picked freely
< sipa>
morcos: segwit makes the coinbase input depend on tx selection
< morcos>
petertodd: so what was the reasoning of having it commit to block X's coinbase outputs
< sipa>
morcos: the fact that segwit allows easy mining without validating signatures
< petertodd>
morcos: so, the reasoning of committing to H(nonce + prev-block-data) is to force you to have that data to be able to mine; making it H(coinbase-outputs + prev-block-data) would additionally constrain that calculation to be miner-specific
< morcos>
yes i understand why to commit to previous blocks witness, but the point of it being your coinbase in there is just to prove that the computation had to be done individually for each miner
< morcos>
right, ok
< petertodd>
morcos: like I saidabove, it's probably sufficient to just make it H(nonce+prevblockdata), and if coinbase is worth committing too we can do that later with a H(H(nonce+coinbase)+prevblockdata) softfork
< morcos>
so personally i'm not convinced of why this is so critically important, i think if you don't have enough full nodes validating everything to keep the network honest then we've failed anyway.
< morcos>
the only thing from your email that i found particularly compelling though was maybe there is some gaming behind witholding your witness data
< petertodd>
morcos: sure, but this makes it much easier for miners to skip validating - in practice that'll be a major engineering risk at the very least
< petertodd>
morcos: minres don't like to do anything they don't have too- the existing validationless mining code they run is scary enough
< morcos>
so anyway, to sipas point about pushback.. there would no additional pushback to the H(nonce + prevblockdata) approach then
< petertodd>
morcos: I don't think there would be, same kind of data needed to calculate segwit
< petertodd>
(calcualte it honestly anyway)
< sipa>
petertodd: not true
< morcos>
ok, that was my original confusion
< petertodd>
sipa: why not?
< sipa>
petertodd: with segwit, the code doing the tx selection needs to be able to make coinbase commitments
< sipa>
petertodd: with your proposal (the eventual one), the code doing coinbase output selection needs to be able to make coinbase commitments
< sipa>
this is already not the same hardware in some cases now
< petertodd>
sipa: sure, but I mean the arbitrary nonce version - committing to the coinbae output is *think* optional
< morcos>
sipa: yes but we're talking about just the H(nonce + prev) version
< sipa>
who chooses the nonce?
< petertodd>
sipa: minerdoes
< petertodd>
sipa: they'll probably all choose 0x00 :)
< sipa>
you can construct a validationless mining scheme where everyone just agrees on a constant noncr
< sipa>
*nonce
< morcos>
there are other ways you could make it unique'ish to the miner if that was important, that doesn't depend on coinbase outputs anyway
< petertodd>
sipa: no! because you don't know for sure if the nonce the previous miner gave you with their block is actually correct, and they can't commit to it in the block itself
< sipa>
petertodd: so?
< sipa>
they can make a deal that they have to
< petertodd>
sipa: like I say, that gets us back to the status quo, where such arrangements involve trust
< petertodd>
sipa: I'm certainly not claiming this isa perfect solution, but keeping at least the status quo is important
< sipa>
does segwit-without-witness-validation not give you just a weaker version of that?
< sipa>
that also requires trust, but only trust in valid signatures
< sipa>
rather than trust in fully valid block
< petertodd>
sipa: which is bloody dangerous given how many SPV nodes there are
< petertodd>
sipa: note that my *other* proposed solution was that we allow invalid txs to be committed to in blocks, and have miners commit to the subset that were valid n-1 blocks ago, but I figured that was too far out to get any acceptance
< sipa>
my point is: if they're going to bother not doing full validation, why wouldn't they go all the way
< sipa>
nowitvalidation gives you a small constant factor bandwidth improvememt
< petertodd>
sipa: whatdo you mean by "all the way", empty blocks?
< sipa>
Ah.
< sipa>
now i see the distinction: it requires giving up fees
< petertodd>
indeed
< petertodd>
equally, giving an option to leave out the prev-block posession proof,in exchange for an empty block, is a nice way to make gmaxwell's validationless mining flag actually have some consensus enforced teeth
< morcos>
yeah i guess i agree it could be a step backwards that if you withold your witness data without one of these fixes, other miners won't have the fee incentive to ignore you, and will be more likely to build on top
< petertodd>
yup
< morcos>
thats more compelling to me than the miners won't bother with the witness data
< petertodd>
well, I'm sure some won't bother... it's a lot of engineering work to do it right
< morcos>
so couldn't we just do H(cur block wit data + prev block data) or something to make it unique to miner
< petertodd>
morcos: hmm, actually I think that works
< morcos>
if someone doesn't give you the witness data, you aren't free to pick your own txs to include in the block
< petertodd>
morcos: though you'd have to recalculate the commitment every time you change the tx contents of the block, but that's not too expensive - just hashing a few MB of data
< petertodd>
morcos: makes sense
< morcos>
you have to do that anyway for your own witness commitment
< petertodd>
morcos: yup
< sipa>
... of course, a fraud proof for that requires access to the whole block
< petertodd>
sipa: that's kinda the point :)
< petertodd>
sipa: fraud proofs are themselves a dangerous double-edged sword...
< sipa>
petertodd: i'm aware that you feel that way
< sipa>
i'd prefer a mechanism that didn't preclude compact fraud proofs though
< petertodd>
to use the terminology of banking/fintech, fraud proofs lull you into an ecosystem without strong transaction finality: you never know if there's a fraud proof out there that you're being prevented from learning about via a sybil
< petertodd>
sipa: note how, if you make the prev-block-data commitment into a tree, allowing for a compact fraud proof, you'd simultaneously make it easy and compact to prove that it was correct!
< sipa>
petertodd: but what if the alternative to those who would run partial validation node is not more full node, but more centralized api providers?
< petertodd>
sipa: that said, in that type of scheme, it may be sufficient that the sender of that "proof" can lie about a subset of the tree and get away with it
< petertodd>
sipa: the alternative isn't that, it's that we scale up bitcoin differently
< sipa>
petertodd: i think that question may be independent of scale :)
< sipa>
but i understand your concern, and i'm certainly not pushing to get the ecosystem into a fraud proof model now
< petertodd>
sipa: a 4MB fraud proof isn't all that bad
< petertodd>
sipa: remember, right now we have ~50GB fraud proofs :)
< sipa>
aware!
< sipa>
petertodd: "lull you into an ecosystem without strong transaction finality"... isn't that already the case for bitcoin? you never know that the block you saw may be reorganized away
< sipa>
the difference is of course that PoW, under certain assumptions about mining strategies and distribution, allows you to compute an upper bound on the chance for that
< sipa>
and that you can observe long term sybils
< sipa>
but bitcoin doesn't have the strong transaction finality either as such
< petertodd>
sipa: bitcoin has very strongly *defined* transaction finality: at a given # of confirmations from a point after which you're sure is in the longest chain, you know exactly what it'dcost to reorganize
< sipa>
fair enoigh
< petertodd>
sipa: fraud proofs aren't like that at all
< sipa>
nope, for fraud proofs it is just assuming no censorship
< petertodd>
sipa: especially in an envifronment where we're relying heavily on them
< petertodd>
not quite, remember a fraud proof can't be made from data that you don't have access too, so the actual mechanism of a fraud proof will always be challenge->response - it's not fraud proofs that matter, but rather *non-fraud* proofs
< sipa>
?
< sipa>
well SNARKs that can encompass the entirety of a block validation would of course solve many things :)
< kanzure>
once the SNARK scheme is figured out, you can often figure out a way to do the same thing without SNARKs
< petertodd>
sipa: with SNARKs even treechains is trivial, so... :)
< petertodd>
sipa: consider the case of a prev-block-possession commitment, proved via merkle tree of H(nonce + <part of prev-block>)
< sipa>
well, if computing the proof takes a google-size datacenter to produce within 10 minutes, we get a different type of centralization concern
< petertodd>
sipa: now, if I commit fraud in that tree, meaning one of the leaves isn't valid, I'm obviously not going to give you that leaf - you'd have a handy fraud proof! instead, I'll distribute all but the data required to prove that fraud
< petertodd>
sipa: now, what'll happen is nodes will notice "hey, how do I know leaf #123 is valid?", and they'll ask they're peers to prove it to them - that's a challenge
< sipa>
petertodd: a partial node wouldn't accept a block that lacks revealing all its commitments
< petertodd>
sipa: the peers will response with a *non-fraud/validity proof*, showing that that part of the block is correct
< kanzure>
wasn't there a proposal about making an explicit commitment(?) to the sum of the values of the outputs
< petertodd>
sipa: however, at no point is a fraud proof ever going to be generated
< petertodd>
sipa: right, but in that case, the partial node doesn't even need fraud proofs
< sipa>
how so?
< petertodd>
sipa:you just said thhe partial node is validating everything relevant...
< sipa>
no
< petertodd>
sipa: IOW, you just said 'full node' and made a typo :)
< sipa>
i said it is receiving everything relevant
< sipa>
not validating it
< petertodd>
sipa: right, in which case it's just a dumb transmitter of information - fraud proofs don't come into it
< petertodd>
sipa: again, in what scenario would the fraudster ever reveal the data required to prove fraud? I'd argue never
< sipa>
petertodd: in that scenario, no node will accept its block
< sipa>
because it is failing to reveal commitments
< petertodd>
sipa: ok, but this goes back transitively - a block may be in itself 100% locally valid, but it's invalid because it depends on an invalid block - why would the partial node eversee thatblock?
< petertodd>
but who forces it to revealthe commitments? fraud challenges!
< sipa>
petertodd: a partial node sees everything; it just doesn't verify everything
< petertodd>
sipa: that's a contradiction...
< petertodd>
sipa: what do you think a partial node sees exactly?
< sipa>
petertodd: all blocks and witnesses
< sipa>
(in the current model, more things can be added of course)
< petertodd>
sipa: all blocks from the genesis block?
< sipa>
down to a block it trusts; that may be the genesis block or not
< sipa>
same as a full node now verifies down to genesis, or gets a utxo set copied from a trusted place
< petertodd>
sipa: right, so again, the only time fraud ever comes into it is fraud in a block prior to the one it trusts
< sipa>
that makes no sense
< petertodd>
sipa: and again, under what circumstance would the fraudster ever give that partial node the data necessary to prove fraud?
< petertodd>
sipa: like, come up with a concrete example!
< sipa>
i said no node accepts without seeing the commitment
< sipa>
there is no need to prove fraud
< sipa>
nobody accepts it
< petertodd>
sipa: ok, so... I think we're in violent agreement :) I'm just saying, there's actually no scenario where fraud proofs themselves make sense - what's actually important is fraud challenges, responded to via local validity proofs
< sipa>
i disagree
< petertodd>
ok, so where's the concrete example of a fraud proof?
< sipa>
for example, you could build a node that maintains only a utxo set for txids whose hash starts with a 0
< petertodd>
right
< sipa>
the assumption is that other nodes are either full, or do together validation for all txids whose hash starts with (not 0)
< petertodd>
right
< sipa>
and that you're not censored from them
< sipa>
this partial node would still download full blocks and witnesses
< petertodd>
sure, so when exactly does the fraud proof get generated?
< sipa>
and will not accept a block that hash merkle malleability, pow failure, mismatching txids, mismatching witness hashes, ...
< petertodd>
ok, so lets come up with a concrete example: tx in block n spends an output that doesn't exist
< sipa>
the witness will be required to contain a reference (for example, height) to the block that created the output, and an index into it the block (transaction number 5 in that block)
< petertodd>
now, as a miner, why would I give a partial node that data? instead, knowing that'll instantly get turned into a fraud proof, I just don't distribute that part of the block, and instead create block n+1 that spends output in block n from tx A, where tx A simply isn't valid
< sipa>
nobody will accept n+1 without seeing n
< petertodd>
ah! but then, why do we need a fraud proof system? you just said no-one will accept n+1 without n, so fraud doesn't come into it
< petertodd>
again, I'm just not seeing a concrete example where a fraud proof would ever be generated
< sipa>
the fraud is needed so the partial node can be told its block n spends from an invalid output
< petertodd>
lol, yubikey
< sipa>
yeah, i agree with that
< petertodd>
good thing that's one-time :)
< sipa>
*fraud proof
< petertodd>
right, and like I say, in an ecosystem with partial nodes, no miner making an invalid block would ever publish the parts of theirblocks that allow fraud to be proven, so the fraud proof itself is irrelevant
< sipa>
that's like saying full nodes are irrelevant because no miner will produce a block that is invalid
< petertodd>
it's pointless to do that - better to just setup a bunch of sybil's that prevent to SPV clients that everything is ok
< sipa>
that's the point obviously
< petertodd>
if everyone ran full nodes, then yes, that'd be irrelevant, but only on a tautological level :)