00:11 <bitcoin-git> [bitcoin] achow101 opened pull request #26418: Fix signing of multi_a and rawtr scripts with wallets that only have corresponding keys (master...fix-psbt-multia) https://github.com/bitcoin/bitcoin/pull/26418

00:12 <bitcoin-git> [bitcoin] jamesob opened pull request #26419: log: mempool: log removal reason in validation interface (master...2022-10-log-mempool-removal-reason) https://github.com/bitcoin/bitcoin/pull/26419

00:18 <BlueMatt[m]> oookayyy, so my question yesterday was hitting on a not-quite-bug-but-definitely-usability-issue

00:20 <BlueMatt[m]> the issue was basically that some folks were running bitcoin core on regtest and calling `generatetoaddress`, some software was seeing the block, persisting its "latest chain tip", then the bitcoin node was crashing (even after an hour, but basically docker container shutdown was just kill'ing) and on restart the block was gone. the aforementioned software would then later load its "latest chain tip", not have any idea where that

00:20 <BlueMatt[m]> is on the chain and get very sad cause it couldnt ask for the block.

00:22 <BlueMatt[m]> so, kinda annoying that no flush happens if bitcoin core is idle for an hour, but ultimately the issue could also appear in prod if there's a reorg while a node that just crashed is offline

00:22 <BlueMatt[m]> at least on regtest

00:22 <BlueMatt[m]> doubly annoying cause on regtest you're not generating large blocks, so flushing is rare, afaiu?

01:31 <luke-jr> BlueMatt[m]: "don't do that"

01:33 <luke-jr> killing a program forcefully bypassing its shutdown sequence is not reasonable behaviour. of course the program should recover, but not necessarily in the most efficient manner

01:36 <BlueMatt[m]> apparently that's "Just How Docker Works", cant help that.

01:36 <BlueMatt[m]> also, machines lose power 🤷

01:56 <luke-jr> yes, but the current behaviour is acceptable for power loss type scenarios

01:57 <luke-jr> can perhaps improve it, but I'd consider it a feature, not a bugfix

01:57 <luke-jr> IIRC the reason we don't flush more, is that it also clears the cache

01:57 <luke-jr> I think there's a PR to change that though

02:00 <luke-jr> here it is, #17487

02:00 <gribble> https://github.com/bitcoin/bitcoin/issues/17487 | coins: allow write to disk without cache drop by jamesob · Pull Request #17487 · bitcoin/bitcoin · GitHub

02:01 <luke-jr> looks fairly ready to merge, but considering a prior version had a consensus bug, it could benefit from additional review BlueMatt[m]

02:03 <BlueMatt[m]> sure, but if you're talking about regtest and its been an hour, it seems like flush should be done :)

02:03 <sipa> I had no idea anyone was even interested in having a regtest node run for more than a few minutes

02:03 <luke-jr> special-casing for regtest seems not useful

02:04 <sipa> Regtest or not could change some default timeout for flush configuration option, perhaps.

02:05 <luke-jr> could, but I don't think it's worth that

02:05 <luke-jr> especially with 17487 around the corner enabling a more aggressive flush without drop

02:05 <sipa> I don't think 17487 changes much.

02:06 <sipa> Flushing without dropping the cache is fairly terrible for performance.

02:06 <sipa> I can imagine there are special cases where that's useful, but it's not a good idea to use it for normal regular flushes

02:07 <sipa> Hmm, actually, that's not true.

02:07 <sipa> You can't use it for cache-size-limit-exceeded flushes, but you could use it for just regular time-based one if the cache isn't full.

02:15 <luke-jr> I would think it would be ideal to flush after each block, outside of IBD

02:16 <luke-jr> and keep the cache for processing subsequent blocks

02:16 <sipa> That may make sense, or every few blocks.

02:18 <sipa> The main benefit of not flushing (whether you drop the cache or not) is avoiding that UTXOs, created before the flush point but spent shortly after, hit disk at all.

02:18 <sipa> But outside of IBD that's not nearly as important.

02:20 <luke-jr> right, it could perhaps be configurable for situations where I/O is more precious

02:20 <luke-jr> but I think the norm is that outside IBD, it's no big deal

02:21 <sipa> well one downside is that flushing does need a somewhat extended lock on the cache data structures, and if a block arrives in that time, processing it will be delayed (perhaps just a few ms, but possibly more)

02:22 <sipa> i can imagine that some network users prefer not to risk that latency hit

02:22 <luke-jr> hmm

02:22 <luke-jr> can abort the transaction in that case?

02:23 <luke-jr> maybe not so simple, since that would require 2 passes over the dbcache

02:23 <luke-jr> (one to do the actual flush, and another to update the flags)

08:01 <bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/8b050762b1b4...984a01589b7b

08:01 <bitcoin-git> bitcoin/master 984a015 MacroFake: Merge bitcoin/bitcoin#26408: test: Remove spam from debug log

08:01 <bitcoin-git> bitcoin/master ef97b89 Jeff Ruane: Exclude rand from debug log

08:01 <bitcoin-git> [bitcoin] MarcoFalke merged pull request #26408: test: Remove spam from debug log (master...2210-fix-noisy-tests) https://github.com/bitcoin/bitcoin/pull/26408

08:11 <bitcoin-git> [bitcoin] TheQuantumPhysicist closed pull request #24922: Isolate the storage abstraction layer from the application/serialization layer (master...feat/storage-isolation) https://github.com/bitcoin/bitcoin/pull/24922

09:15 <bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/984a01589b7b...4f270d2b6391

09:15 <bitcoin-git> bitcoin/master 8a9f1e4 Martin Zumsande: test: fix intermittent failure in rpc_getblockfrompeer.py

09:15 <bitcoin-git> bitcoin/master 4f270d2 MacroFake: Merge bitcoin/bitcoin#26404: test: fix intermittent failure in rpc_getbloc...

09:15 <bitcoin-git> [bitcoin] MarcoFalke merged pull request #26404: test: fix intermittent failure in rpc_getblockfrompeer.py (master...202210_testfix_blockfrompeer) https://github.com/bitcoin/bitcoin/pull/26404

14:55 <bitcoin-git> [bitcoin] fanquake opened pull request #26421: build: copy config.{guess,sub} post autogen in zmq package (master...fix_macos_zeromq_older_ubuntu) https://github.com/bitcoin/bitcoin/pull/26421

16:31 <sipa> @[luke-jr] Yeah, and without significant refactor, flushing does take time proportional to the cache size, even when it has few dirty entries (because it requires iterating the whole cache)

17:33 <bytes1440000> jonatack: Sorry, for some reasons i could not test this today: #26328 looks easy and will do tomorrow and a few others

17:33 <gribble> https://github.com/bitcoin/bitcoin/issues/26328 | netinfo: fix relaytxes doc, display 3 relaytxes states by jonatack · Pull Request #26328 · bitcoin/bitcoin · GitHub