< GitHub115> [bitcoin] sipa opened pull request #9039: Various serialization simplifcations and optimizations (master...simpleserial) https://github.com/bitcoin/bitcoin/pull/9039
< Bi> Guys, if I have a block with only the coinbase tx and one segwit tx. It takes sha256->sha256 on 64 bytes of data to calculate the witness' merkle - right?
< Bi> And the first 32 bytes are zero?
< Bi> While the other 32 bytes is sha256->sha256 of the raw segwit transaction?
< Bi> It seems very simple, but I must be doing something wrong, beacuse my merkle roots dont match the expected values
< Bi> Can someone please help me?
< Bi> Or maybe you could at leat tell me what is wtxid of this transaction: 01000000000101f15a2db447e8316e9f20073ab8bdeac53df1a53d415fbd41e73929699b889e7a0000000017160014fb459699ce9ca78de6d0790121dfb215883acb64ffffffff0100111024010000001976a914fb459699ce9ca78de6d0790121dfb215883acb6488ac02483045022100c5feadba5fbcc28afb940c7ee664bba81830f19a5407ae49cf75f565534b149702200a7c2b6b1b5846a37c434ffe0cae168d6b797de9087a21176f2c9ef6fa9f47aa014104b848ab6ac853cd
< Bi> 69baaa750c70eb352ebeadb07da0ff5bbd642cb285895ee43f2889dd12642ca0ff9b2489bd82d4d5ca287c4ebdf509f1bcd02d6fa720542e2900000000
< Victorsueca> Bi: your transaction is supposed to have 2 IDs, the txid as it has always been and the new wtxid
< Victorsueca> the txid should be sha256(sha256(original serialization format))
< Victorsueca> the wtxid should be sha256(sha256(new serialization format with witness data))
< Victorsueca> if your transaction does not have any witness data the txid and the wtxid should be the same
< Bi> Victorsueca, yes I know all of this
< Bi> Bu my merkle roots dont match :(
< Victorsueca> hmmmm
< Bi> If someone could give me the expected wtxid for the given raw transaction?
< Victorsueca> weird.... can't decode your hex string using decoderawtransaction
< Victorsueca> also tried blockchain.info transaction parses but I wouldn't be surprised if that failed anyway
< arubi> the transaction is fine
< Bi> This is segwit transaction
< Bi> it decodes properly here: http://n.bitcoin.ninja/checktx
< Bi> It is a real tx from testent block #895231
< Victorsueca> why does bitcoin-cli decoderawtransaction not work then?
< Bi> I dont know
< arubi> it works for me..
< Bi> Make sure to remove all the spaces
< Victorsueca> ahhh yeah
< Bi> So what is its wtxid?
< Victorsueca> IRC added a line jump, forgot about that
< arubi> Bi, as reported by bitcoin-cli, it's 508e4409c2cdff3d08b54370147cf6bf4d328f8248e4c60941a528b4d0375159 . not sure how much that'll help you
< Bi> ok, thank you - I have the same one
< Bi> But then it doesnt create a correct merkle hash
< arubi> I am
< Bi> ... it expects segwit merkle root of 554949ea7454928f42158d3554817f70c1fa8ba75e769faa58128ac05d333d28
< arubi> looks like it, right. hopefully the hash in the op_return output is actually reversed like you say.. I'm not sure
< Bi> and when you hash null hash (wtxid of coinbase) followed by this 508e4409c2cdff3d08b54370147cf6bf4d328f8248e4c60941a528b4d0375159 - I get something else
< Bi> I was trying swapping and not swapping - no success :(
< Victorsueca> Bi: here's your transaction in JSON format https://softnet.homenet.org/zerobin/?11d833ece4506b4e#IYNmeQ7+/uvT/uZUW6TbikcrQus5TJa8AJruObsa3bY=
< arubi> but what's in that return output is not just the wtxid merkle root
< Victorsueca> jus queried those from my node
< arubi> the merkle root is appended the witness reserved value and then hashed hash256 again
< Bi> oh?
< Bi> is it?
< arubi> at least that's what I gathered from bip141
< Bi> No, it says "A witness root hash is calculated with all those wtxid as leaves, in a way similar to the hashMerkleRoot in the block header."
< arubi> right, and then...
< arubi> Double-SHA256(witness root hash|witness reserved value
< arubi> )
< Bi> right - thank you!
< Bi> this must be it
< arubi> also, pretty sure that at least with the wtxid I gave you, you /should/ reverse the bytes. and also treat the bytes like they are in the commitment hash
< arubi> (so not reversed like you pasted)
< Bi> yes, reversing I knew about
< Bi> Double-SHA256(witness root hash|witness reserved value) - was the part I missed
< Bi> thanks again
< arubi> cheers Bi
< Bi> Yes, it works no!
< Bi> Cheers
< arubi> yep, tried it too :)
< Bi> now*
< tonikt> Is the presence of the witness root field (in the coinbase output) mandatory, after SEGWIT activation?
< tonikt> (assuming there are any segwit txs inside the block)
< tonikt> Sorry, I found it. It's the "unexpected-witness" code :)
< sipa> tonikt: not, not mandatoy
< tonikt> sipa: I think it becomes mandatory, after the activation.
< tonikt> yes?
< tonikt> .. it would go to "if (!fHaveWitness) ..." in ContextualCheckBlock()
< tonikt> .. and then it would find a tx with segwit, and then it would return error "unexpected-witness"
< Victorsueca> tonikt: non-segwit blocks are supposed to still be valid even after activation
< tonikt> Sure. I was just wondering, after the activation, whether I have to reject blocks without the segwit root
< tonikt> It seemed logical to reject them, but could not find that part of code
< sipa> tonikt: no, if no transaction contains a witness, there is no requirement for a witness commitment
< tonikt> yes - only if there is at least one witness transaction
< sipa> of course
< sipa> then it is mandatory
< tonikt> but I think it also counts coinbase transaction as a possible witness-type?
< tonikt> I have another, more high-level question...
< tonikt> I am on testnet network now and trying to sync up the chain and all my current peers are pre 0.13.0
< tonikt> And I am already on block #893669
< sipa> tonikt: the coinbase witness is not committed to anywhere excet by the commitment in the output
< tonikt> I should not be taking any block data from these peers, should I?
< sipa> tonikt: which means that if you don't have a commitment, you can just remove the coinbase witness
< sipa> without invalidating anything
< sipa> we only download blocks from witness peers, indeed
< tonikt> sipa: OK - that's interesting. I forgot that removing the witness from CB would not change txid, so it would still validate the block :)
< sipa> indeed
< sipa> if there is no witness commitment, it means that no tx has a witness
< sipa> including the coinbase
< tonikt> OK - all clear now
< tonikt> thanks
< tonikt> do you know any running testnet node where I could connect to fetch some segwith blocks?
< tonikt> all my peers are < 0.13.0 now :)
< sipa> use x9.seed.bitcoin.sipa.be
< sipa> eh, that's mainnet
< sipa> for testnet i think jonasschnelli's and petertodd's seed support filtering
< tonikt> if you have a running node, would you just give me an IP, please?
< sipa>
< tonikt> thank you!
< BlueMatt> uhhhh, preferential peering is quite impressive... https://imgur.com/a/jwH7R
< sipa> BlueMatt: 174 connections here
< Victorsueca> used to have 70-80 peers, I have 121 right now
< sipa> i temporarily increased my max connections
< BlueMatt> sipa: i had something like 250 on that node
< BlueMatt> (with some blacklisting of bad nodes)
< sipa> seems my dns seed has a bug... it's using 400% cpu and is not responding to dns requests
< sipa> yay for multiple implementations
< * btcdrak> forks sipa
< sipa> fixed, i think
< sipa> jonasschnelli, petertodd, luke-jr: pushed a bugfix to bitcoin-seeder which could cause infinite loops in the dns handling thread
< luke-jr> sipa: I'm intentionally holding back my version for a little while just to avoid us all using the same code immediately
< luke-jr> is it an exploit, or am I probably okay?
< sipa> luke-jr: you're probably already stuck in an infinite loop
< sipa> is the 'DNS requests' number going up?
< luke-jr> CPU is 100% idle, so I doubt it?
< luke-jr> yes
< sipa> indeed, then i doubt it
< luke-jr> [16-10-29 19:23:24] 591/5717464 available (5713103 tried in 132614s, 3081 new, 1280 active), 4288200 banned; 1851875 DNS requests, 316442 db queries
< sipa> did you upgrade to have filtering support?
< luke-jr> -rw-r--r-- 1 bitcoinpool bitcoinpool 849M Oct 29 18:49 dnsseed.dat <-- is this right? O.o
< luke-jr> no
< luke-jr> (the size)
< sipa> with 11M addresses in your database, i guess yes
< sipa> the question is why do you have so many addresses in there
< sipa> i have 2.5M
< luke-jr> plaintext dump is only 117 MB
< sipa> the dump doesn't contain banned/terrible IPs
< luke-jr> would it matter that I dropped crawler threads to 20 for a long time?
< sipa> 160M -rw-rw-r-- 1 pw pw 160M Oct 29 12:27 dnsseed.dat 54M -rw-r--r-- 1 pw pw 54M Oct 29 12:27 dnsseed.dump
< sipa> it seems you're spending a huge amount of time retrying IPs that are gone
< sipa> your node needs 2 days to cycle through everything
< sipa> it's 15 minutes here
< luke-jr> :o
< luke-jr> I guess variety is worth the overhead?
< sipa> i would say so
< sipa> but i'd still like to understand why yours behaves so differently from mine
< luke-jr> only difference in mine from 11e935b72020607e5c3ce85a88209bc34e427a06 is REQUIRE_VERSION 70001
< luke-jr> ./dnsseed -h dnsseed.bitcoin.dashjr.org -n jun.dashjr.org -p 1053 -t 80
< luke-jr> sipa: some years ago, my server was blackholed semi-regularly for a while due to an upstream/backbone router hating the crawler's access patterns; could blackholing do it?
< luke-jr> ie, maybe it thinks all IPs are sometimes terrible
< sipa> you only have 591 good IPs
< sipa> i have 4000
< sipa> wow, only 3000 suddenly... perhaps due to being offline for a while as a result of that bug
< sipa> usually it's around 4000