< bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/aa8d76806c74...0aa2ff0f660e
< bitcoin-git> bitcoin/master e8a8cff fanquake: build: enforce minimum required Windows version (7)
< bitcoin-git> bitcoin/master 0aa2ff0 fanquake: Merge #18956: build: enforce minimum required Windows version (7)
< bitcoin-git> [bitcoin] fanquake merged pull request #18956: build: enforce minimum required Windows version (7) (master...window_enforce_runtime_version) https://github.com/bitcoin/bitcoin/pull/18956
< jb55> I'm still new to the codebase, sorry if dumb question: but looking at undo blocks/rev*.dat files: does a node really need to store (37GB on my node currently) worth of undo blocks? is this just an optimization? couldn't you do this on the fly?
< sipa> define "on the fly" ?
< sipa> if you mean "at disconnect time", no, that would be extremely slow (it would need a linear scan through all previous block to find the outputs being spent)
< sipa> if you mean created at connect time, that's what it's doing
< sipa> in non-pruning mode you could see it as an optimization, but only in so far that the concept of a UTXO set is an optimization...
< sipa> in pruning node it's data that's not otherwise available
< jb55> ah ok I think I was missing the linear scan to find outputs bit
< sipa> you can see blocks as patches to the UTXO set "delete this UTXO, add this UTXO", plus witnesses that authenticate the patch
< sipa> the undo data is the reverse patch; it contains the actual UTXO data deleted by a block
< jb55> sipa: how many patches are kept, is there a way to tweak that?
< sipa> jb55: for every block you keep, the undo data is kept
< sipa> if you prune blocks, the undo data is also pruned along with it
< sipa> there could be a knob to change that, in case you're ok with not being able to reorg back to block height H, but still want to be able to that block to others on the network
< sipa> or want it for rescanning
< sipa> undo data is around 9x smaller than the corresponding block data iirc
< jb55> sipa: so if you pruned the undos you couldn't rescan before a certain point? it has to apply those patches back to the rescan point to determine the utxo state at that height?
< sipa> jb55: if you prune you don't have the blocks either, so you lose the ability to rescan anyway
< sipa> and rescanning is about finding transactions, not UTXOs
< sipa> undo data is solely used for disconnecting block (though there is a PR open to also use it for some block statistics, i think)
< jb55> was just thinking if there was an option to prune the undo blocks, was wondering what the consequences were
< sipa> you'd be unable to reorg back
< sipa> which is extremely unlikely to be a problem beyond a few 100 or 1000 blocks
< jb55> yeah I would like to prune undo blocks past 100 or so, and would try to PR it if it makes sense
< sipa> ah, i forgot: https://github.com/bitcoin/bitcoin/pull/16083 added statistics using undo data
< sipa> pruning is restricted to keeping at least 288 blocks, i think
< sipa> it may make sense to have the same limit
< jb55> sure
< gwillen> I think it's already the case that if we reorg more than 100 blocks deep, things have gone very wrong, yeah?
< jb55> proof of aliens trolling us
< sipa> jb55: on the other hand... if you care getting rid of 10% of disk usage, maybe you should consider just pruning
< sipa> it's not a perfect argument; there are things you can do with blocks that don't need undo data... but undo data is also an order of magnitude smaller
< jb55> sipa: I like the full db for analysis and rescan. any optimization on top of that is nice.
< bitcoin-git> [bitcoin] fanquake opened pull request #19025: [0.19] Backports (0.19...0_19_2_backports) https://github.com/bitcoin/bitcoin/pull/19025
< fanquake> Thanks elichai2. Looks like GitHub does send two email notifications
< elichai2> actually it's not that ridiculous because its probably sending 2 unrelated emails. 1 for notifying about the PR. 2. for telling you you were chosen to review a PR. right?
< fanquake> Correct
< elichai2> ha, too bad they didn't special-case the CODEOWNERS to merge this emails into 1
< elichai2> at least it's under the same subject though
< willcl_ark> Does BIP30 / BIP34 completely prevent there from existing two transactions with same txid, or are there still edge-cases where this can happen?
< willcl_ark> BIP30 appears to leave some room for re-orgs, but I can't tell if "spent" transactions (in different blocks) could still be allowed to have had identical txids
< sipa> willcl_ark: BIP30 doesn't prevent duplicate txids; it only prevents one transaction that would overwrite an earlier one
< sipa> BIP34 prevents duplicate txids (assuming collision resistance of double-SHA256...)
< PaulTroon> BIP34 was an interesting historic read - is Gavin's speculation about a duplicate coinbase in 2048 technically impossible for a malicous miner? https://github.com/bitcoin/bitcoin/pull/1526#issuecomment-6796145
< PaulTroon> s/impossible/possible
< sipa> 2048?
< sipa> ah
< PaulTroon> A pre-BIP34 coinbase with matching signature for block height 1,983,702 (approx year 2048)
< sipa> BIP141 solved that
< PaulTroon> 'signature' I mean push of block height in coinbase
< yevaud> PaulTroon: that's super implausible at best to begin with.
< yevaud> and any impact is pretty minimal even if it wasn't.
< PaulTroon> yevaud: sure, just a thought experiment
< sipa> all txouts of the block 164384 coinbase transaction are redeemed
< sipa> so it won't matter even if it could happen
< PaulTroon> sipa: because BIP30 allows duplicate tx's if all outputs are spent?
< sipa> yes
< sipa> well, yes, but that's not what I mean
< yevaud> PaulTroon: I still have no clue why the format specified is a partly valid CScript.
< sipa> the problem that BIP30 was trying to solve is txout _overwriting_
< sipa> since all the outputs are already spent, there is nothing to be overwritten, so the problem doesn't exist in the first place
< sipa> BIP30 thankfully has an exception that in that case doesn't prevent creation, because otherwise we'd need to keep track of every spent utxo forever
< PaulTroon> I was also trying to understand that BIP30 attack scenario, snot sure if I understand it.. a coinbase_1 with an unspent txout to A, and then create a duplicate coinbase_2 that has a txout to B ?
< PaulTroon> So then a tx that spends the txout to A is invalid because coinbase_2 does not have that txout A ?
< sipa> PaulTroon: coinbase_1 is created, the chain forks into branch A and branch B (just a 1 or 2 block reorg, otherwise harmless); in branch A, coinbase_2 overwrites coinbase_1; in branch B this does not happen
< sipa> branch B eventually wins
< sipa> nodes that have seen branch A will have overwritten coinbase_1, and then _deleted_ it when switching to branch B (as they're "undoing" the creation of coinbase_2, having forgotten something was overwritten)
< sipa> nodes that have not seen branch A will think coinbase_1 still exists
< sipa> if now someone spends the coinbase_1 output, it permanently forks the network between those that have seen branch A and those who have not
< PaulTroon> thanks sipa, the commentary from bip30 makes more sense now
< PaulTroon> hmm, one more question, if branch A wins and coinbase_2 overwrites coinbase_1, are the unspent tx_outs that spend coinbase_1 now invalid in branch A?
< PaulTroon> to try and answer my own question, in a winning branch A scenario the UTXOs are valid from both coinbase_1 and coinbase_2 as long as no rollback occurs to pull the rug out.
< bitcoin-git> [bitcoin] jonasschnelli pushed 5 commits to master: https://github.com/bitcoin/bitcoin/compare/0aa2ff0f660e...a587f85853ec
< bitcoin-git> bitcoin/master 83f69fa Russell Yanofsky: Switch transaction table to use wallet height not node height
< bitcoin-git> bitcoin/master 2bc9b92 Russell Yanofsky: Cancel wallet balance timer when shutdown requested
< bitcoin-git> bitcoin/master bf0a510 Russell Yanofsky: gui: Avoid wallet tryGetBalances calls before TransactionChanged or BlockT...
< bitcoin-git> [bitcoin] jonasschnelli merged pull request #18587: gui: Avoid wallet tryGetBalances calls in WalletModel::pollBalanceChanged (master...pr/ipc-bal) https://github.com/bitcoin/bitcoin/pull/18587
< bitcoin-git> [bitcoin] MarcoFalke pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/a587f85853ec...bd5ec7c5284d
< bitcoin-git> bitcoin/master fa83b39 MarcoFalke: init: Remove confusing and redundant InitError
< bitcoin-git> bitcoin/master fa12a37 MarcoFalke: test: Replace inline-comments with logs, pep8 formatting
< bitcoin-git> bitcoin/master faf45d1 MarcoFalke: http: Avoid crash when g_thread_http was never started
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #19006: rpc: Avoid crash when g_thread_http was never started (master...2005-httpNoCrash) https://github.com/bitcoin/bitcoin/pull/19006
< bitcoin-git> [bitcoin] MarcoFalke pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/bd5ec7c5284d...e20e964cb1ab
< bitcoin-git> bitcoin/master facdeea MarcoFalke: net: Remove un-actionable TODO
< bitcoin-git> bitcoin/master fabea6d MarcoFalke: net: Run clang-format on protocol.h
< bitcoin-git> bitcoin/master e20e964 MarcoFalke: Merge #18996: net: Remove un-actionable TODO
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #18996: net: Remove un-actionable TODO (master...2005-netNoTodo) https://github.com/bitcoin/bitcoin/pull/18996
< bitcoin-git> [bitcoin] MarcoFalke pushed 8 commits to master: https://github.com/bitcoin/bitcoin/compare/e20e964cb1ab...448bdff26307
< bitcoin-git> bitcoin/master 769ee5f Pieter Wuille: Merge BigEndian functionality into CustomUintFormatter
< bitcoin-git> bitcoin/master 6f9a1e5 Russell Yanofsky: Extend CustomUintFormatter to support enums
< bitcoin-git> bitcoin/master d06fedd Russell Yanofsky: Add SER_READ and SER_WRITE for read/write-dependent statements
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #18317: Serialization improvements step 6 (all except wallet/gui) (master...202003_noncastserial_6) https://github.com/bitcoin/bitcoin/pull/18317
< willcl_ark> sipa: thanks for your earlier reply re BIP30/34
< shesek> I released a new open-source project, an HD wallet indexer similar to EPS, implemented in Rust. Apart from supporting the Electrum RPC protocol, it also provides a more modern HTTP REST API and convenient real-time updates. https://github.com/shesek/bwt
< shesek> (apologies for the off-topicness, I figured there might be people who find it interesting here)
< jb55> shesek: nice, looks neat
< kanzure> shesek: it would be helpful if this could do something about offline consumer applications to provide updates (such as reorgs or anything modifying the application's last processed tip) to an application that was offline for a while.
< shesek> kanzure, the closest thing currently available is the /txs/since/:height endpoint, but I have an issue for something similar to what you suggested: https://github.com/shesek/bwt/issues/6
< shesek> reorgs are being reported when happen in the live server-sent-events stream, but the fact that they happened isn't kept around after that (also, bwt is currently entirely in-memory, so it would get lost if bwt itself shuts down)
< shesek> I setup a live bwt demo server on regtest if anyone wants to experiment. its on https://demo.bwt.dev/. you can try the https://demo.bwt.dev/stream endpoint (a transaction should happen every 10 seconds) or https://demo.bwt.dev/hd to see the tracked wallets
< shesek> (ugh, its not actually working, fixing it..)
< shesek> its good now
< bitcoin-git> [bitcoin] MarcoFalke opened pull request #19028: test: Set -logthreadnames in unit tests (master...2005-testThreadNames) https://github.com/bitcoin/bitcoin/pull/19028
< bitcoin-git> [bitcoin] hebasto opened pull request #19029: net: Fix CThreadInterrupt::sleep_for TSan issues (master...200520-interrupts) https://github.com/bitcoin/bitcoin/pull/19029
< hebasto> jonasschnelli: mind looking into the small #18424 ?
< gribble> https://github.com/bitcoin/bitcoin/issues/18424 | qt: Use parent-child relation to manage lifetime of OptionsModel object by hebasto · Pull Request #18424 · bitcoin/bitcoin · GitHub
< bitcoin-git> [bitcoin] vasild opened pull request #19031: Implement ADDRv2 support (part of BIP155) (master...addrv2) https://github.com/bitcoin/bitcoin/pull/19031
< jamesob> sorry for the absence last few months; IRC client back on now
< bitcoin-git> [bitcoin] dongcarl closed pull request #16748: [WIP] Add support for addrv2 (BIP155) (master...2019-07-addrv2v4) https://github.com/bitcoin/bitcoin/pull/16748
< bitcoin-git> [bitcoin] sipa opened pull request #19032: Serialization improvements: final step (master...202005_noncastserial_final) https://github.com/bitcoin/bitcoin/pull/19032
< achow101> how do I run the travis lsan build locally?
< bitcoin-git> [bitcoin] MarcoFalke pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/448bdff26307...3eda7ea9ba72
< bitcoin-git> bitcoin/master eeaaa58 Sebastian Falbesoner: test: replace inv type magic numbers by constants
< bitcoin-git> bitcoin/master b35e1d2 Sebastian Falbesoner: test: add inventory type constant MSG_CMPCT_BLOCK
< bitcoin-git> bitcoin/master 4a614ff Sebastian Falbesoner: test: explicit imports from test_framework.messages in p2p_invalid_message...
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #18764: refactor: test: replace inv type magic numbers by constants (master...20200425-test-replace_inv_types_by_constants) https://github.com/bitcoin/bitcoin/pull/18764
< bitcoin-git> [bitcoin] promag opened pull request #19033: http: Fix workQueue race on InterruptHTTPServer (master...2020-05-fix-race-interrupt-http-server) https://github.com/bitcoin/bitcoin/pull/19033
< promag> "verificationprogress": 1.882374845250915e-09 <- is it worth such precision?
< sipa> "you're REALLY far off still"
< sipa> promag: not sure what the alternative is?
< promag> heh, i'm instrumenting -qt and see where it "hangs"
< promag> sipa: round to say 5 decimal places? so it's more human friendly
< promag> "verificationprogress": 0.005865895479773644 VS "verificationprogress": 0.005867