< bitcoin-git>
bitcoin/master e8a8cff fanquake: build: enforce minimum required Windows version (7)
< bitcoin-git>
bitcoin/master 0aa2ff0 fanquake: Merge #18956: build: enforce minimum required Windows version (7)
< bitcoin-git>
[bitcoin] fanquake merged pull request #18956: build: enforce minimum required Windows version (7) (master...window_enforce_runtime_version) https://github.com/bitcoin/bitcoin/pull/18956
< jb55>
I'm still new to the codebase, sorry if dumb question: but looking at undo blocks/rev*.dat files: does a node really need to store (37GB on my node currently) worth of undo blocks? is this just an optimization? couldn't you do this on the fly?
< sipa>
define "on the fly" ?
< sipa>
if you mean "at disconnect time", no, that would be extremely slow (it would need a linear scan through all previous block to find the outputs being spent)
< sipa>
if you mean created at connect time, that's what it's doing
< sipa>
in non-pruning mode you could see it as an optimization, but only in so far that the concept of a UTXO set is an optimization...
< sipa>
in pruning node it's data that's not otherwise available
< jb55>
ah ok I think I was missing the linear scan to find outputs bit
< sipa>
you can see blocks as patches to the UTXO set "delete this UTXO, add this UTXO", plus witnesses that authenticate the patch
< sipa>
the undo data is the reverse patch; it contains the actual UTXO data deleted by a block
< jb55>
sipa: how many patches are kept, is there a way to tweak that?
< sipa>
jb55: for every block you keep, the undo data is kept
< sipa>
if you prune blocks, the undo data is also pruned along with it
< sipa>
there could be a knob to change that, in case you're ok with not being able to reorg back to block height H, but still want to be able to that block to others on the network
< sipa>
or want it for rescanning
< sipa>
undo data is around 9x smaller than the corresponding block data iirc
< jb55>
sipa: so if you pruned the undos you couldn't rescan before a certain point? it has to apply those patches back to the rescan point to determine the utxo state at that height?
< sipa>
jb55: if you prune you don't have the blocks either, so you lose the ability to rescan anyway
< sipa>
and rescanning is about finding transactions, not UTXOs
< sipa>
undo data is solely used for disconnecting block (though there is a PR open to also use it for some block statistics, i think)
< jb55>
was just thinking if there was an option to prune the undo blocks, was wondering what the consequences were
< sipa>
you'd be unable to reorg back
< sipa>
which is extremely unlikely to be a problem beyond a few 100 or 1000 blocks
< jb55>
yeah I would like to prune undo blocks past 100 or so, and would try to PR it if it makes sense
< sipa>
pruning is restricted to keeping at least 288 blocks, i think
< sipa>
it may make sense to have the same limit
< jb55>
sure
< gwillen>
I think it's already the case that if we reorg more than 100 blocks deep, things have gone very wrong, yeah?
< jb55>
proof of aliens trolling us
< sipa>
jb55: on the other hand... if you care getting rid of 10% of disk usage, maybe you should consider just pruning
< sipa>
it's not a perfect argument; there are things you can do with blocks that don't need undo data... but undo data is also an order of magnitude smaller
< jb55>
sipa: I like the full db for analysis and rescan. any optimization on top of that is nice.
< fanquake>
Thanks elichai2. Looks like GitHub does send two email notifications
< elichai2>
actually it's not that ridiculous because its probably sending 2 unrelated emails. 1 for notifying about the PR. 2. for telling you you were chosen to review a PR. right?
< elichai2>
ha, too bad they didn't special-case the CODEOWNERS to merge this emails into 1
< elichai2>
at least it's under the same subject though
< willcl_ark>
Does BIP30 / BIP34 completely prevent there from existing two transactions with same txid, or are there still edge-cases where this can happen?
< willcl_ark>
BIP30 appears to leave some room for re-orgs, but I can't tell if "spent" transactions (in different blocks) could still be allowed to have had identical txids
< sipa>
willcl_ark: BIP30 doesn't prevent duplicate txids; it only prevents one transaction that would overwrite an earlier one
< PaulTroon>
A pre-BIP34 coinbase with matching signature for block height 1,983,702 (approx year 2048)
< sipa>
BIP141 solved that
< PaulTroon>
'signature' I mean push of block height in coinbase
< yevaud>
PaulTroon: that's super implausible at best to begin with.
< yevaud>
and any impact is pretty minimal even if it wasn't.
< PaulTroon>
yevaud: sure, just a thought experiment
< sipa>
all txouts of the block 164384 coinbase transaction are redeemed
< sipa>
so it won't matter even if it could happen
< PaulTroon>
sipa: because BIP30 allows duplicate tx's if all outputs are spent?
< sipa>
yes
< sipa>
well, yes, but that's not what I mean
< yevaud>
PaulTroon: I still have no clue why the format specified is a partly valid CScript.
< sipa>
the problem that BIP30 was trying to solve is txout _overwriting_
< sipa>
since all the outputs are already spent, there is nothing to be overwritten, so the problem doesn't exist in the first place
< sipa>
BIP30 thankfully has an exception that in that case doesn't prevent creation, because otherwise we'd need to keep track of every spent utxo forever
< PaulTroon>
I was also trying to understand that BIP30 attack scenario, snot sure if I understand it.. a coinbase_1 with an unspent txout to A, and then create a duplicate coinbase_2 that has a txout to B ?
< PaulTroon>
So then a tx that spends the txout to A is invalid because coinbase_2 does not have that txout A ?
< sipa>
PaulTroon: coinbase_1 is created, the chain forks into branch A and branch B (just a 1 or 2 block reorg, otherwise harmless); in branch A, coinbase_2 overwrites coinbase_1; in branch B this does not happen
< sipa>
branch B eventually wins
< sipa>
nodes that have seen branch A will have overwritten coinbase_1, and then _deleted_ it when switching to branch B (as they're "undoing" the creation of coinbase_2, having forgotten something was overwritten)
< sipa>
nodes that have not seen branch A will think coinbase_1 still exists
< sipa>
if now someone spends the coinbase_1 output, it permanently forks the network between those that have seen branch A and those who have not
< PaulTroon>
thanks sipa, the commentary from bip30 makes more sense now
< PaulTroon>
hmm, one more question, if branch A wins and coinbase_2 overwrites coinbase_1, are the unspent tx_outs that spend coinbase_1 now invalid in branch A?
< PaulTroon>
to try and answer my own question, in a winning branch A scenario the UTXOs are valid from both coinbase_1 and coinbase_2 as long as no rollback occurs to pull the rug out.
< bitcoin-git>
bitcoin/master faf45d1 MarcoFalke: http: Avoid crash when g_thread_http was never started
< bitcoin-git>
[bitcoin] MarcoFalke merged pull request #19006: rpc: Avoid crash when g_thread_http was never started (master...2005-httpNoCrash) https://github.com/bitcoin/bitcoin/pull/19006
< willcl_ark>
sipa: thanks for your earlier reply re BIP30/34
< shesek>
I released a new open-source project, an HD wallet indexer similar to EPS, implemented in Rust. Apart from supporting the Electrum RPC protocol, it also provides a more modern HTTP REST API and convenient real-time updates. https://github.com/shesek/bwt
< shesek>
(apologies for the off-topicness, I figured there might be people who find it interesting here)
< jb55>
shesek: nice, looks neat
< kanzure>
shesek: it would be helpful if this could do something about offline consumer applications to provide updates (such as reorgs or anything modifying the application's last processed tip) to an application that was offline for a while.
< shesek>
kanzure, the closest thing currently available is the /txs/since/:height endpoint, but I have an issue for something similar to what you suggested: https://github.com/shesek/bwt/issues/6
< shesek>
reorgs are being reported when happen in the live server-sent-events stream, but the fact that they happened isn't kept around after that (also, bwt is currently entirely in-memory, so it would get lost if bwt itself shuts down)
< shesek>
(ugh, its not actually working, fixing it..)
< shesek>
its good now
< bitcoin-git>
[bitcoin] MarcoFalke opened pull request #19028: test: Set -logthreadnames in unit tests (master...2005-testThreadNames) https://github.com/bitcoin/bitcoin/pull/19028