< phantomcircuit>
gmaxwell, what's the newhope stuff for?
< sipa>
phantomcircuit: post quantum link layer encryption
< gmaxwell>
Post-quantum might over/undersell it a bit. Its recently become a fashionable design for ephemerial key agreement to make use a hybrid ciphersuite where your key is H(ecc || some thing else), with the rational being that some attacker could be logging all your traffic in the hopes that 20 years from now your ECC group will pratically be broken and the decrypted data will be useful to them. Majo
< gmaxwell>
r fad is to pick the other thing out of a set of things conjectured to hold up to quantum computers, but it could be helpful against more conventional crypto breaks.
< gmaxwell>
kallewoof: I've reread the whole PR there and I am not seeing the rationale for that.
< phantomcircuit>
gmaxwell, ah
< phantomcircuit>
sipa, ah
< gmaxwell>
In any case, might well turn out that any particular candidate scheme isn't secure against classical computers, much less quantum ones... thus combining it with ECC, so at worst you just waste a bit of cpu and bandwidth.
< Jmabsd>
Wait, in what situations are sighash bytes not added vs. added, to ecdsa signatures in Bitcoin? With a sighhash byte the DER signature max size is 73B (if low-s, 72B) whereas without sighash byte the max size is 72B (if low-s, 71B).
< Jmabsd>
..so "contract endorsement" is DER signature + endorsement byte (= max 73B), and the DER signature is max 72B.
< sipa>
what do you mean by 'situations' ?
< sipa>
every signature in bitcoin script has a sighash byte
< Jmabsd>
sipa: ah right, so, the zoomed-out situation in Bitcoin is that DER situations are in sigscript, redeem script (part of sigscript right) and P2WSH witness script (input's last witness element), only.
< sipa>
"only" ?
< sipa>
what else is there?
< Jmabsd>
sipa: so those three are the only 'situations', and in each of them there's a sighash byte, and, the OP_CHECKSIG operator processes the sighash byte too.
< sipa>
yes
< sipa>
OP_CHECKSIG and friends all require a sighash byte
< Jmabsd>
and so Bitcoin is really only dealing with the "contract endorsement" structures, and hence in the real world those are 72B+1B sighhash=73B max.
< Jmabsd>
great. thx for confirming.
< Jmabsd>
where is CHashWriter's sourcecode? hash.h defines the interface but its logics are not in hash.cpp
< Jmabsd>
oh there we go, CHashWriter wraps CHash256, which is a double-SHA256 hashing facility.
< Jmabsd>
why is double-pass SHA256 used when signing?
< sipa>
Jmabsd: ask satoshi :)
< Jmabsd>
aha.
< sipa>
bitcoin basically everywhere either uses ripemd160(sha256(x)) or sha256(sha256(x))
< Jmabsd>
sipa, right - and normally when you pass around (communicate between parties) a hash, using double-hash is a nice convention as it protects from extension attacks right
< Jmabsd>
however in the particular case of using the hash as input for signature, it doesn't make sense...
< Jmabsd>
or can some actual utility of double-hashing the signtext when signing be guessed / thought of?
< sipa>
nope. i believe it is pointless
< sipa>
protecting against length extension attacks is generally a good reason, but it doesn't apply here
< Jmabsd>
aha.
< Jmabsd>
sipa, two quick off-topic but conceptually related questions, as general guidance for design of any new signing protocols, would you suggest using a single hash instead of double, or could it make sense to recycle Bitcoin's double-hash for symmetry with Bitcoin? also for guidance for other protocols, if you're going to do a salted hash of some message e.g. SHA256HMAC(protocolmessage,"SIPAPROTOCOL"), then for extension
< Jmabsd>
protection would you do the SHA256HMAC doubly i.e. SHA256HMAC(SHA256HMAC(protocolmessage,"SIPAPROTOCOL"),"SIPAPROTOCOL") or would you do an inner SHA256 i.e. SHA256HMAC(SHA256(protocolmessage),"SIPAPROTOCOL") ?
< sipa>
Jmabsd: a good guideline is to make sure that the data you're hashing is always deserializable
< sipa>
whenever you're writing things into a hash with self-descriptive lengths, length extension attacks generally don't ezist
< sipa>
and i would use single hashes
< Jmabsd>
ahh so you mean, if your message protocol has the format <length><message data>, then noone could meaningfully attack a single-hash anyhow.
< Jmabsd>
because they can't forge the <length> part in a hash anyhow - however, if the message format was just <message data> then they could (given a single hash). noted.
< Jmabsd>
sipa, couldn't double hashing provide some other benefit.. or you mean, hashes may be data-destructive so risk that some rainy day there'd be hash collision attacks could be higher with double-hash or?
< Jmabsd>
like what's motivating arguments in the choice for and against
< Jmabsd>
..ok so i take your suggestion that single-hash, and have the message format be well-defined and mark out length, should suffice. thanks for suggesting!
< Jmabsd>
luke-jr: ah, so an optimized double-SHA256 for a 64-bytes input. so a double-sha256 optimized for merkle node calculation. neat. thanks.
< Jmabsd>
luke-jr: is any particular byte ordering or "endianness" to the SHA256 used here, i've seen some Bitcoin merkle node calculation use byte-reverse logics
< sipa>
Jmabsd: as i've said before, all hash functions are treated as black box that convert from byte arrays to byte arrays
< sipa>
no endianness is relevant
< gmaxwell>
Jmabsd: Sipa already answered this question previously. The bitcoin protocol uses sha256 exactly as it is.
< gmaxwell>
Asking again won't make the answer change.
< gmaxwell>
oops. didn't mean to pile on.
< luke-jr>
XD
< sipa>
:)
< gmaxwell>
the stuff you're seeing is probably working from hex in UIs or something, and got reversed there.
< gmaxwell>
Bitcoin "UI" hashes are backwards for path dependant but otherwise explicable reasons...
< luke-jr>
it's probably because the block hash is interpreted as a 256-bit little endian number for target comparison, or something like that
< Jmabsd>
gmaxwell, yes interesting, i see clearly in Bitcoin Core and other implementations that indeed there is no reversing whatsoever, no "endianness" or the like here. you are right that the code in the wild must have been mitigating some kind of user-facing hex representation quirk.
< gmaxwell>
luke-jr: right, and then you want to print it so that it starts with leading 0s so that the 'less than' comparison makes sense.
< Jmabsd>
i think here it's only about transaction hashes
< gmaxwell>
Jmabsd: the point is that bitcoin's UI uses the same stuff to print all these 256 bit hashes.
< luke-jr>
^
< gmaxwell>
block hash is interpreted as a 256-bit little endian number for target comparison as luke says, so then bitcoin needed to print it that way too, otherwise the 0s would have been on the right which would have been confusing... and the same code was then used to print other hashes.
< gmaxwell>
But thats just a UI thing.
< Jmabsd>
gmaxwell: ahaaa so for symmetry reasons there's reverse output order of transaction hashes too, i understand - "big endian".
< Jmabsd>
noo, little-endian.
< Jmabsd>
gmaxwell, this applies only to the string representation of block hashes and transaction hashes - any more?
< luke-jr>
Jmabsd: the hash is interpreted little-endian, and printed big-endian. don't try too hard to make endian sense of it or you'll hurt your brain :P
< Jmabsd>
luke-jr: this transformation when providing a string serialization, has it been applied (assumed as convention to be applied) to any more data than block hashes and transaction hashes?
< luke-jr>
off-hand, I don't think we print any other hashes in the UI, but I could be forgetting something
< sipa>
Jmabsd: it's used everywhere hashes are printed for human consumption
< sipa>
so txids and block hashes
< sipa>
also the chainstate hash in gettxoutsetinfo
< Jmabsd>
hash prints.. hm.. ok so it won't apply to other data like merkle tree root printouts? or signatures and such. ok. (y)
< luke-jr>
I'm glad getwork is gone :D
< luke-jr>
Jmabsd: probably there too
< luke-jr>
but that's not really exposed to the user
< luke-jr>
(other than RPC stuff)
< Jmabsd>
aha. ok so the quirk of this reverse ordering UI representation thing, is that transaction hashes have the funny representation order too, and this needs a bit of extra consideration when making a simple example script to calculate merkle nodes, as they need to reverse the byte order of the transaction hash. gotcha.
< luke-jr>
Jmabsd: no, not there
< sipa>
Jmabsd: no, it's only used when printing for human consumption
< sipa>
internally there is no reversing ever
< luke-jr>
I'm not 100% sure, but I suspect the merkle root is printed reversed in getblock RPC for example; but never internally
< sipa>
oh, yes it is
< sipa>
but again, just when converting to hex
< Jmabsd>
luke-jr, what's the code location for this hex serialization?
< Jmabsd>
(yes I totally see that Bitcoin Core or the Bitcoin protocol never represent data in reverse order internally - it's only a human representation thing)
< Jmabsd>
sipa, right, so it's when you're making a test script on the theme "user enters two+ transaction hashes for you to calculate merkle nodes / merkle root out of it", you take the human-output form as input there.
< Jmabsd>
also when making a block explorer and taking transaction hashes, you take the reverse form as input there too no?
< sipa>
Jmabsd: uint256's ToString() method
< airplanemod>
Hey guys, I would like to know if it is possible to set up a development environment for bitcoin-qt and/or bitcoind that will allow me to set breakpoints and step through live code as it is running? I cannot seem to find a tutorial or any good starting point for how to do this. I have been testing my own edits in a simple text editor and using gitian builder to compile and test using
< airplanemod>
the debug.log file but that is tedious and time-consuming work. Can anyone give any tips or pointers for how I might get a proper developement environment set up for testing?
< sipa>
airplanemod: sure, you can run bitcoind in gdb or so
< sipa>
setting up a development environment isn't very bitcoin core specific, so i wouldn't say is on topic here
< sipa>
but it's no different than other c/c++ projects
< airplanemod>
ok thanks for replying sipa. Ive done tons of development but never touched c++ so I wasnt sure if there was a recommended IDE or anything specific that would cater to blockchain dev specifically or if its easier on linux vs windows for example
< sipa>
airplanemod: bitcoin core is primarily developed on unix systems (and the windows binaries are produced through cross compiling on linux)
< sipa>
you can use an IDE if you like, but i believe most developers use just text editors
< luke-jr>
I use a text editor with syntax highlighting
< airplanemod>
sipa: thats kind of what I was wondering about. so most developers are editing the source and then building binary executable files to test their changes? thats what Ive been doing up to now but I consider that to be working blind
< sipa>
yes, i just use a text editor
< airplanemod>
im guessing because of the complexity of how the bicoin wallet works that it isnt possible to run it in a debugger. i couldnt find any info online to suggest this would be possible which is why i came here to ask
< luke-jr>
I'm not aware of any way to modify code at runtime
< luke-jr>
airplanemod: running it in a debugger just works
< luke-jr>
(in valgrind, is a bit more painful..)
< airplanemod>
luke-jr: not modify code at runtime but see the contents of variables
< sipa>
airplanemod: i occasionally run bitcoind in a debugger
< sipa>
which lets you put breakpoints and see the contents of variables etc
< sipa>
inside functional tests it's harder though, as they spin up multiple communicating bitcoinds
< airplanemod>
sipa: is that with gbd? (i havent used gdb before but i did come across it already tonight in my search)
< luke-jr>
while ! gdb -p $(ps ax | grep node0 | cut -b 1-5); do true; done
< luke-jr>
<.<
< sipa>
airplanemod: yes, gdb
< airplanemod>
great, thanks guys. I will look into gdb and keep using my text editor for qt debugging
< kallewoof>
gmaxwell: since sdaftuar objected, I figured I could make a separate PR with that enabled. I'll do that soon.
< wumpus>
arhhhhh noooo why does restarting your node cut up the debug.log again
< wumpus>
lost more than a day of perf data
< wumpus>
luke-jr: whooops, okay, will build windows then
< wumpus>
ok, everything pending for 0.17 has been merged again, time to tag rc2?
< wumpus>
it should at least be deterministic this time
< MarcoFalke>
go ahead
< instagibbs>
SGTM
< instagibbs>
appreciate the psbt fix merges last-minute
< fanquake>
wumpus 👍
< fanquake>
I’d like to be able to re-create the NSIS issue at some point. Just for interest
< wumpus>
* [new tag] v0.17.0rc2 -> v0.17.0rc2
< jonasschnelli>
\o/
< jonasschnelli>
Is there a known issue with ./test/functional/wallet_backup.py? It failes often on my branch (though did some unrelated changes)?
< wumpus>
not known to me, at least
< wumpus>
wallet_backup.py passed, Duration: 106 s
< jonasschnelli>
okay... probably a local issue
< fanquake>
\o/
< wumpus>
/o\
< achow101>
\o\
< wumpus>
/o/
< wumpus>
the new version of lxc did make a long-standing issue go away for me where the first build try after starting the VM would always fail
< achow101>
apparently kvm is still broken even with the updated vmbuilder :(
< wumpus>
so if you can build a bionic image succesfully, what is the problem?
< achow101>
err, I mean that vmbuilder still doesn't work
< achow101>
so you can't build the bionic image
< wumpus>
maybe one of the ready-made ubuntu cloud images would be useful?
< wumpus>
that's how I spin up new VMs locally, download the appropriate ubuntu cloud image, verify gpg signatures, convert to qcow2 format, resize, then launch it. I don't use this process for gitian, but I don't see why it wouldn't be similar.
< jonasschnelli>
wumpus: wallet_backup.py is also failing on master on my debian system
< jonasschnelli>
'importwallet' RPC took longer than 60.000000 seconds
< jonasschnelli>
(4 instances of Core mainnet are runnning in the background though)
< wumpus>
jonasschnelli: maybe you can make it pass by increaing the timeout
< wumpus>
if it's really crashing, it would be interesting to attach a debugger and see where
< fanquake>
wumpus I think I’ve seen the same issue on my old Debian setup. Haven’t seen on the newer setup.
< jonasschnelli>
wumpus: I think importwallet is just slower then the timeout,... but I'll investigate
< wumpus>
right-importwallet is a call that can be slow so it doesn't surprise me
< MarcoFalke>
jonasschnelli: You could maybe fix that with "self.rpc_timewait = 90"
< MarcoFalke>
like in feature_dbcrash
< jonasschnelli>
MarcoFalke: Yes. I think that is necessary
< MarcoFalke>
or wallet_dump.py
< jonasschnelli>
MarcoFalke: IMO wallet_dumps.py:L84 self.rpc_timeout = 120 is invalid
< jonasschnelli>
It should be self.rpc_timewait = 120
< MarcoFalke>
oh
< jonasschnelli>
I saw both (wallet_dump and wallet_backup) failing on my system (and on travis)
< jonasschnelli>
Maybe they don't fall on bitcoin/bitcoin travis due to the exta CPU cycles we bought
< MarcoFalke>
right
< MarcoFalke>
Looks it is wrong in wallet_dump
< MarcoFalke>
Not sure about travis
< MarcoFalke>
We only bought additional machines, but the machines are itself identical
< jonasschnelli>
Okay. But when I enabled travis for my local branch, master failes
< jonasschnelli>
And master fails always on my debian machine
< MarcoFalke>
rerun all jobs and cancel the linter
< MarcoFalke>
They are known to be broken and only meant to run on bitcoin/bitcoin pull requests
< jonasschnelli>
Okay. Done.. but meh
< MarcoFalke>
Agree on meh, but I am not going to touch the linters again ;)
< wumpus>
fyi: gitian 0.17.0rc2 sigs up
< provoostenator>
Building... By the way, the web chat logs now say "BotBot disconnected, possible missing messages"
< wumpus>
I was kicked off too (and couldn't easily come back, SASL timeout issue)
< wumpus>
(so with sigs up, I mean my own, not codesigned sigs)
< kanzure>
can someone kill the spam plzkthx
< kanzure>
on github
< sipa>
blocked him
< wumpus>
thanks
< gmaxwell>
Is there a proper way to get information out of connOptions from some random place in the code? I want to make the tip may be stale stuff not run in all the cases where its pointless, (reindex/network inactive/-noconnect...)
< wumpus>
the variables from connOptions end up in CConnman, so I guess only where the code has access to that