< moneyball>
dongcarl your topic mentioned earlier today has been added to the CoreDev list
< gmaxwell>
Decreasing the number of people who can review the software by five orders of magnitude doesn't sound like a particularly strong way to achieve a security improvement.
< gmaxwell>
in particular because modern-ish C++ is a lot safer 'by default' than C code... and a lot of the constructs that you need to use in rust to get acceptable performance, like iterators, are also safe in C++. Not that I dislike rust, but overhype is bad for everyone.
< kallewoof>
I'm honestly confused what this guy is concerned about: https://twitter.com/DrRoyMurphy/status/1039512043567497221 He calls glaring problems about a sign/verifymessage proposal, then talks about wallet security, unescaped loop injection (?), and then goes on an opcode rant.
< kallewoof>
Is he talking about Johnson Lau's OP_MESSAGE comment, maybe..?
< luke-jr>
kallewoof: IIRC, that's a CSW troll pretending to be an "early dev"; likely has no idea what he's talking about, just trying to sound smart
< kallewoof>
luke-jr: Okay...
< luke-jr>
if you try to get a sensible explanation out of him, he'll probably end up blocking you :P
< kallewoof>
Unescaped loop injection. What on earth is that anyway?
< kallewoof>
luke-jr: I don't care about being blocked, tbh. I guess I'll just ask him about it.
< gmaxwell>
kallewoof: guy appears to just be a drug addict nutcase from prior discussions I saw from him.
< kallewoof>
gmaxwell: at least he knows how to make someone feel insecure about themselves. i still have no clue what an unescaped loop injection is.
< gmaxwell>
kallewoof: nor does he.
< gmaxwell>
If no one does, then he can't be wrong. Checkmate Engineers.
< sipa_>
kallewoof: the proper response is muting him
< kallewoof>
I guess I'll see what his response is and if that makes no sense either I'll just mute and move on
< gmaxwell>
just don't let him keep you going by changing the subject.
< sipa_>
kallewoof: no, mute him, nos
< sipa_>
there is nothing useful he says
< gmaxwell>
lol
< sipa_>
all his claims are total garbage bullshit
< gmaxwell>
in any case, he came up before, he's some bitconnect promoter that claims the be one of the first bitcoin developers after satoshi and a bunch of other stuff that made no sense.
< gmaxwell>
(where pieter made a mistake in talking to him)
< kallewoof>
Ahh... Yeah looks like he is just a slimy clueless person. How quaint..
< kallewoof>
Johnson Lau is suggesting reserving OP_MESSAGEONLY = 0xf0 as opcode for message signing, or alternatively "OP_RETURN msgXXX". It feels wasteful to take an opcode, but feedback would be nice: https://github.com/bitcoin/bips/pull/725#issuecomment-420421058
< kallewoof>
The OP_RETURN thing seems like it could be a bit of a pain to implement for verifiers.
< kallewoof>
Also still looks like 50/50 'want actual transaction format' vs 'do not want actual transaction format'.
< echeveria>
I'm not sure why you'd want that.
< echeveria>
using RETURN seems kind of nasty too, as there's probably actually scripts in a lot of forms people would be signing.
< kallewoof>
echeveria: It lets you use existing HSMs to sign messages, proofs of funds, etc. Apparently some HSMs are specifically built to never be updated, but it would be handy to be able to create proofs with them.
< echeveria>
kallewoof: the exchange I'm aware of with that issue has others that mean they need to replace their hardware anyway.
< echeveria>
kallewoof: only uncompressed point keys makes it sort of sucky to use for anything non-joke.
< kallewoof>
echeveria: To quote argument for a tx-like protocol: "It also works well with proof of reserve: the proof of reserve is a bitcoin transaction spending all the funds, but with an additional input (covered by SIGHASH_ALL) that points to a fake/invalid tx. This has the additional benefit of working in a forward compatible way with any future bitcoin extension, like confidential transactions or mimblewimble: your proof of
< kallewoof>
reserve could have blinded inputs and outputs as well, or whatever else the bitcoin protocol is made to allow. As long as the spends are tangled up with the fake input (via SIGHASH_ALL or a mimblewimble kernel, or whatever), it doesn't matter."
< echeveria>
that sounds like it's attracting foot gunning, honestly.
< kallewoof>
Sounds a bit dangerous, yeah.
< echeveria>
that sort of cute thing really doesn't sit well with me. it's where I've seen people make really expensive mistakes in the past.
< aj>
kallewoof: the backwards compatibility thing seems like the difference between "i trust the code handing stuff off to my hsm, but can't upgrade my hsm" vs "my hsm is an essential line of defense, i don't trust anything else"
< echeveria>
aj: agreed entirely.
< kallewoof>
aj: That's a good way of putting it
< aj>
kallewoof: i wonder if adding an input txid=00..00,n=0,value=21e14 would be an interesting invalidator?
< kallewoof>
aj: i don't know if it's really necessary. btw, i assumed from your comment that you were against the idea of a transaction based protocol, but now i'm unsure.
< aj>
kallewoof: i'm still trying to form an opinion :)
< kallewoof>
Ahh, okay
< aj>
kallewoof: the thing i don't quite get... suppose you come up with a non-tx based thing, it's perfect, great, everyone puts it on their roadmap. but maaku or someone says no, here's a tx based plan that's compatible with existing signing hw, and posts a bip and sample implementation that works with trezors or whatever
< kallewoof>
I've been going back and forth on the subject several times already. It seriously seems like everyone's opinion is split down the middle
< aj>
kallewoof: all the people who say "my hsm is an essential line of defense" still have to be able to not accidently sign stuff via maaku's hypothetical method in that case, your avoidance of a tx based format doesn't save them any effort?
< echeveria>
aj: the assumption seems to be kind of strange. they're using a HSM that supposedly has some more rules than "sign what comes to me", right?
< echeveria>
if it's a dumb interface that signs anything hat comes to it, sounds like a shitty HSM.
< aj>
echeveria: it could be one with an embedded ruleset, or it could be one with a display that lets you check the amt/address and say yes/no out of band
< echeveria>
for the former, I'm not sure this hack would work a lot of the time anyway.
< luke-jr>
it would be nice if it was such that scripts could have a different execution path for messages
< aj>
luke-jr: why??
< echeveria>
?
< kallewoof>
I think that's what Johnson Lau wants with the OP_MESSAGEONLY opcode?
< aj>
luke-jr: jl2012's OP_MESSAGEONLY or OP_RETURN special handlying does that, but i don't get why
< luke-jr>
aj: so you can sign with a key that can't spend
< echeveria>
lol no
< luke-jr>
aj: and thereby avoid multiple signatures with the same key
< aj>
luke-jr: wouldn't you be better of signing with K+y*G in that case? otherwise that you know key_m doesn't really prove you know key_s
< kallewoof>
key_m would be signing that you know key_s, though
< aj>
kallewoof: claiming, but not proving
< luke-jr>
aj: you might not want to disclose the spending pubkey
< luke-jr>
if the messageonly path just had a hash of another script, it would save on-chain space; but maybe that can just wait for MAST
< luke-jr>
aj: I can't think of a reason to require messages prove ability to spend directly; that's already not the case for current signed messages
< luke-jr>
(and couldn't be guaranteed by using the same key here either)
< kallewoof>
luke-jr: For SignMessage case, yes, but for ProveFunds this becomes a bit of a problem
< luke-jr>
kallewoof: does it?
< kallewoof>
Unless we all just assume that key_m proves ownership of key_s
< luke-jr>
by convention
< kallewoof>
Is that sufficient for a proof of funds?
< luke-jr>
ProveFunds is confusing anyway; how would a shared wallet with hot/cold separation give fund proofs for all their users?
< luke-jr>
I'd guess they'd reuse the same UTXOs, and the convention would say something like "this only proves an amount, not specific UTXOs"
< luke-jr>
in which case, the key difference is pretty minor IMO?
< kallewoof>
Not sure what you mean by "shared wallet" -- are you talking about an exchange wallet or similar now?
< luke-jr>
kallewoof: yes, something like that
< luke-jr>
multiple people using the same hot/cold wallet combination
< kallewoof>
That is probably tricky. They could still prove solvency, but I don't think the exchange could prove anything to individual users
< luke-jr>
well, as long as the convention is to only verify the amount, and the hot wallet has sufficient UTXOs, it could work
< luke-jr>
of course, if there's separate spending vs message keys, they could possibly just keep all the message keys hot and avoid the overlaps too
< kallewoof>
how do i know the exchange didn't prove my X BTC and your X BTC using the same UTXO's?
< luke-jr>
kallewoof: maybe they did. is that a problem?
< kallewoof>
I would assume a user wants a proof because they wanna make sure the exchange actually has their coins set aside in some fashion
< luke-jr>
even with a single user wallet, perhaps I proved I have funds to buy two houses using the same UTXOs ;)
< kallewoof>
Hm, right..
< luke-jr>
hmm, I was thinking the user wants to prove to someone else
< luke-jr>
proof of solvency might be a specialised case that needs a third type? :/
< luke-jr>
I guess the only way to prove funds uniquely, is to escrow them
< kallewoof>
I admit I don't know enough about how a proof of solvency would work to comment.
< luke-jr>
IIRC petertodd has dabbled in that area
< kallewoof>
He has? Didn't know. Will try to find
< kallewoof>
luke-jr: To backtrack a bit, you're for using a transaction pair to do the verification thing, right? Is that to allow dusty DSM:s or are there other benefits that I haven't thought/heard of yet? (achow101 noted it would be simpler to do tx based, so that's another one I guess)
< kallewoof>
luke-jr: I guess the OP_MESSAGEONLY approach would let you prove without revealing the pubkey which is nice
< kallewoof>
luke-jr: Actually, you can still do that even w/o transactions, since the verification call executes the scripts as is. I.e. even w/o transactions, you can do the key_m/_s stuff.
< gmaxwell>
I don't see _any_ value in message signing that uses special opcodes.
< gmaxwell>
There is value in being able to sign with an existing script pubkey, because then you can at least sign with the same criteria used to spend coins.
< gmaxwell>
Making something that isn't that is just going down the route of reinventing gpg.
< luke-jr>
kallewoof: I don't see any strong reason to care if it's tx pair format or not. But I haven't read the whole thread, so maybe there's a reason for it
< gmaxwell>
Tx format is nice, because it allows near perfect reuse of code. Which, assuming the goal is "be able to sign with existant coins" (for example) thats a pretty good match.
< luke-jr>
gmaxwell: re special opcodes, think of it as pay-to-contract, but with the contract separate and updatable? (and people can always opt not to use the special opcodes if they want to share a key)
< sipa_>
what's the point?
< luke-jr>
the whole purpose of (the current) signed messages is to verify the [future] recipient of a payment agrees to terms before making the payment
< luke-jr>
I don't see how using a separate key hurts that at all
< sipa_>
i must be missing something, but what is being proven, if it's not the same key?
< luke-jr>
that the recipient agrees to some terms
< sipa_>
but how do you know it's the recipient?
< luke-jr>
because it's the address you're paying
< sipa_>
i'm very confused
< sipa_>
i'll read the ML posts later :)
< kallewoof>
sipa_: No one is responding on the ML unfortunately, but there are comments on the PR
< kallewoof>
To be honest, I get more confused as time passes. I didn't realize there were so many different opinions on so many aspects of this.
< luke-jr>
:/
< sipa_>
kallewoof: yeah, don't bring works in progress to the ML, you'll just get bikeshedding and design by committee :)
< kallewoof>
lol
< kallewoof>
I think I'll make a section describing how to convert to/from transaction pair format and then let implementers choose whether they go the optional step. That would give them the ability to pass onto set-in-stone HSM:s.
< sipa_>
also don't have optional features that you aren't sure when people would use them :_
< sipa_>
they don't know better than you
< kallewoof>
Optional features, like the transaction pair conversion? It seems like it could be useful for some people, considering like... 4-5 people have already said they would prefer it to be IN that format.
< kallewoof>
(and an equal amount of people have said they prefer it wasn't)
< sipa_>
meh
< sipa_>
make a choice
< kallewoof>
Fair enough.
< sipa_>
if you can't write explicit advice "in case A, you should use this, otherwise you should use that", it shouldn't be an optional feature
< * sipa_>
zZzZ
< aj>
kallewoof: solvency proofs are "all my debts are in this merkle tree, whose total value is X; here's proof of funds of value Y; Y >= X" possibly with some zkp magic to avoid revealing X or Y. https://crypto.stanford.edu/~dabo/pubs/abstracts/provisions.html
< kallewoof>
aj: Nice. Thanks, will read
< wumpus>
c++ shouldn't need ? 1 : 0 for booleans, right?
< wumpus>
e.g. a bool will always coerce to either 0 or 1
< wumpus>
I don't think the added compexity really makes it more readable
< wumpus>
let's type more to make the compiler happy
< wumpus>
aanyhow I'll tell him to squash it and we can merge it, no need to get held up on this, though I do think practicalswift is kind of going too far here in seeding confusion
< aj>
seems like ec_privkey_export_der should be accepting a bool rather than an int anyway?
< wumpus>
unfortunately secp256k1 lives in 1989, booleans weren't invented back then
< aj>
... it's in src/key.cpp as well as secp256k1/ ?
< aj>
i'm confused :(
< wumpus>
it's calling a secp256k1 function right? or not?
< wumpus>
no, I'm sure I'm confused
< aj>
i think it's cut and pasted from secp256k1 into src/key.cpp and marked as static?
< wumpus>
AhhhAHHhahHa
< wumpus>
you're completely right: there is a ec_privkey_export_der function in secp256k1, but it is not used by us, we have our own version
< wumpus>
which could, if we're willing to diverge from upstream take an ActuallyBool
< wumpus>
(we have already diverged from upstream in some regards, in 63179d028347bf3e32c7ea61386df4c44307b4a7)
< aj>
we've already diverged in a few trivial ways, more so for the _import_der version
< wumpus>
yep !
< wumpus>
changing the argument to bool is the most sensible solution, suggested it in the PR
< Jmabsd>
Where in Bitcoin's source code is the TxFee estimation logics for when planning to make a transaction?
< wumpus>
src/policy/fees.cpp mainly
< Jmabsd>
Thanks!
< echeveria>
< Jmabsd>
wumpus: do you know how the fee estimations are estimated? the code goes something like ""a 60% threshold required at target / 2, an 85% threshold required at target and a 95% threshold required at 2 * target ... Conservative estimates, however, required the 95% threshold at 2 * target", so, there are three different algorithms running, i don't understand the sense
< wumpus>
fee estimation is indeed very complicated
< wumpus>
I don't know all the details either
< wumpus>
this is the right place to ask specific questions about the code though, though I'll likely not be able to answer them, someone else might...
< instagibbs>
quick N(ACK)s please: hidden(?) RPC calls exposing compact blocks work flow. We have use in Elements/Liquid for passing blocks around the federation outside of p2p.
< instagibbs>
(N)ACKs* :)
< gmaxwell>
so you want basically a GETCOMPACTBLOCK and a SUBMITCOMPACTBLOCK?
< instagibbs>
~much yeah
< andytoshi>
also a way to get tx indices
< andytoshi>
missing tx indices*
< instagibbs>
andytoshi, that message already exists
< andytoshi>
oh ok ignore me
< instagibbs>
and "consume" compact block, which returns the indices, then responding to getblocktxn, yada
< gmaxwell>
I don't see any harm in it. I'm not sure if it would have much use, the json serialization/deserialization is going to be slow, perhaps slow enough that this wouldn't be interesting for anyone to build alternative transports. Might it be more interesting to have a hidden generic RPC that lets you send any P2P message?
< instagibbs>
I'll take that as 0+ :P
< gmaxwell>
yea, it's a 0+
< andytoshi>
hmm actually a hidden generic RPC would be better imho
< instagibbs>
andytoshi, seems useful overall, I'd have to think about our application a bit more
< instagibbs>
i'll take a swing at it
< gmaxwell>
I'm not opposed it it but skeptical that its all that useful for bitcoin in general. I suggested the generic because I could imagine more uses for that. (even if just test shims)
< andytoshi>
this isn't the first time i've considered connecting to p2p alongside rpc to get extra stuff
< instagibbs>
one * is that we're passing unsigned blocks(kalle's signet may also want something like this)
< instagibbs>
which means it'll fail PoW checks
< gmaxwell>
Oh I see, you actually want to use compact blocks for pre-pow blocks.
< gmaxwell>
BlueMatt: ^ this might be relevant to your interests too.
< gmaxwell>
(as matt's mining protocol work also exchanges not-pow-valid-blocks)
< sipa>
andytoshi: there may be good reasons to do so; p2p is far more optimized than rpc
< gmaxwell>
in general passing through the json is going to hurt performance alone.
< BlueMatt>
instagibbs: I dont think the compact block protocol makes sense for low-bandwidth relay outside of p2p
< BlueMatt>
oh, wait, scratch that, I was thinking the wrong direction
< BlueMatt>
that said, I'm with gmaxwell, I'm skeptical anyone would use it, because its round-trip-required its not really gonna be all that much better for most use-cases
< BlueMatt>
eg if you're in a protocol where one side is pushing out to multiple clients it doesnt make any sense, and doesn't make any sense for unidirectional things ala blocksat
< gmaxwell>
BlueMatt: well in the one side pushing, much more efficient things are possible but compact blocks already exists and is at least a large constant factor better than sending out the whole block.
< BlueMatt>
you misunderstood my point there, by "one side pushing" i meant any unidirectional comms
< gmaxwell>
yea, okay, sure, but unidirectional is kind of a special case. The betterhash thing could support round trips, no?
< BlueMatt>
well round trips means bad performance if you're mega-latency-sensitive
< BlueMatt>
so betterhash *could* (with more complication of an already-overcomplicated-protocol imo), but then you'd break anything like running betterhash over quic to get low-latency
< gmaxwell>
fair point. Right your current weak block thing still gets guarenteed no RT.
< gmaxwell>
BlueMatt: s/latency sensitive/latency sensitive OR on very high latency links/
< BlueMatt>
yea
< jarthur>
Would Lightning Network watchtowers have such a use-case?
< gmaxwell>
I'm not aware of lightning watchtowers relaying bitcoin blocks.
< BlueMatt>
yea, I dont see why that would be needed?
< jarthur>
Yea, n/m, my mind is in the pre-conf realm.
< BlueMatt>
I'm not aware of any lightning watchtowers.
< BlueMatt>
but, yea, instagibbs, I cant say I'm a fan of putting it in the rpc, just seems like vaguely-useless-features that no one will use
< BlueMatt>
but otoh I'd doubt its *much* code, so whatever
< gmaxwell>
^ yea my view too.
< BlueMatt>
I do plan on putting some able-to-support-compact-encodings-ala-betterhash into core prs soonish, though, cause betterhash needs it
< gmaxwell>
alternatively, if there are refactors that would make it easier for you to carry a patch, those would probably be more interesting.
< BlueMatt>
but I dont think thats really that related
< instagibbs>
alright im -0 it myself, we'll just carry it
< instagibbs>
no biggie, thanks for the input
< jarthur>
There has been a trend of "personal stratum daemon"s recently, that don't use txindex, might they benefit from such an RPC call?
< gmaxwell>
again same question as watchtowers, why would they be relaying blocks to other nodes?
< jarthur>
No, but if you took out the stratum daemon, and wanted to just use Core for this purpose, and your light-wallet spoke the language, might there be some benefit from you getting your tx history knowledge in this fashion?
< gmaxwell>
No.
< jarthur>
I guess making the light wallet speak p2p is just as good?
< gmaxwell>
jarthur: I'm not sure about what parts of the above we read, but we were talking about compact blocks which are _utterly_ useless to litewallets.
< gmaxwell>
They're only potentially useful to full nodes.
< jarthur>
Cool, sorry, didn't think it through.
< phantomcircuit>
i cant decide between setting up Register/Unregister calls for poll/select or replicating the existing logic and setting up the structures every call
< phantomcircuit>
the event(ish) based thing is easier to make a mistake with but setting up all the structures is potentially expensive with many peers
< gmaxwell>
phantomcircuit: what is easier for people to review and get merged?
< gmaxwell>
even if we go with something slower, it can be optimized later.
< gmaxwell>
and I doubt anything 'slow' is going to matter with just 125 peers...
< phantomcircuit>
rebuilding the structure... maybe?
< phantomcircuit>
the logic about whether to set the send/recv flag is surprisingly not trivial so that kind of makes it harder