<bitcoin-git>
[gui-qml] johnny9 opened pull request #487: Use handlers with explicit parameters in main.qml and DesktopWallets.qml (qt6...modern-handlers) https://github.com/bitcoin-core/gui-qml/pull/487
joetor5 has joined #bitcoin-core-dev
joetor5 has quit [Ping timeout: 252 seconds]
l0rinc has joined #bitcoin-core-dev
TheRec has quit []
l0rinc has quit [Quit: l0rinc]
l0rinc has joined #bitcoin-core-dev
cmirror has quit [Remote host closed the connection]
cmirror has joined #bitcoin-core-dev
TheRec has joined #bitcoin-core-dev
TheRec has quit [Changing host]
TheRec has joined #bitcoin-core-dev
TheRec has quit []
aleggg has quit [Ping timeout: 240 seconds]
aleggg has joined #bitcoin-core-dev
TheRec has joined #bitcoin-core-dev
TheRec has quit [Changing host]
TheRec has joined #bitcoin-core-dev
l0rinc has quit [Quit: l0rinc]
robobub has joined #bitcoin-core-dev
brunoerg has quit [Read error: Connection reset by peer]
<bitcoin-git>
bitcoin/master 6653caf fanquake: ci: allow libc++ instrumentation other than msan
<bitcoin-git>
bitcoin/master b09af2c fanquake: ci: instrument libc++ in TSAN job
<bitcoin-git>
bitcoin/master 7aa5b67 fanquake: ci: remove DEBUG_LOCKORDER from TSAN job
<bitcoin-git>
[bitcoin] fanquake merged pull request #33099: ci: allow for any libc++ intrumentation & use it for TSAN (master...ci_generic_libcpp_instr) https://github.com/bitcoin/bitcoin/pull/33099
Guyver2 has joined #bitcoin-core-dev
jespada has quit [Ping timeout: 272 seconds]
jespada has joined #bitcoin-core-dev
robszarka has joined #bitcoin-core-dev
joetor5 has quit [Quit: joetor5]
szarka has quit [Ping timeout: 276 seconds]
bitcoinlover has quit [Remote host closed the connection]
<corebot>
https://github.com/bitcoin/bitcoin/issues/29675 | wallet: Be able to receive and spend inputs involving MuSig2 aggregate keys by achow101 · Pull Request #29675 · bitcoin/bitcoin · GitHub
<achow101>
#topic orphan resolution WG Update (glozow)
<achow101>
#topic QML GUI WG Update (jarolrod, johnny9dev)
<kanzure>
hi
Christoph_ has quit [Quit: Christoph_]
<johnny9dev>
We're making good progress on patching remaining issues with moving the project to submodule, cmake, and qt6. Deer gee has ported over his AssumeUTXO PR over to the new structure already (bitcoin-core/gui-qml#485) . I should have my previous PRs ported over this weekend.
<phantomcircuit>
i've been looking at initial block download times and noticed that we're recalculating block header hashes many many times (like 800+ times in cases), this can be fixed forever with an immutable block/header class(es)
<phantomcircuit>
does anybody have thoughts on such an endeavor
<pinheadmz>
i believe that PR will also require an upstream PR to expose some utxo snapshot thing in the interface
<fjahr>
hi
<johnny9dev>
It will. I suggested we maintain a patch until then
<pinheadmz>
so thatll be the new, somewhat frustrating, process but i think the separation of interests there is good
<achow101>
phantomcircuit: we're in a meeting, can add that as a topic if you want
<l0rinc>
phantomcircuit: I have investigated that before, but don't have exact numbers
<darosior>
phantomcircuit: interesting, but let's wait till other prepared meeting topics are done?
<Murch[m]>
achow101: How long as #32144 been RFM when you say "it’s still probably rfm"?
<l0rinc>
dbwrapper writes currently pretend to return bool but always return true, while real errors surface via exceptions – #33042 makes that explicit, but we need to decide the canonical error path
<willcl-ark>
The benefits and various tradeoffs of such a move are detailed in the top comment in the PR, but in summary it should make the CI maintainable by more people, not be reliant on the current infrastructure, and also provide "easier scaling"; if you want to scale more machines, just throw more $$ at the problem. These are fleshed out further in the PR, and we'd welcome feedback there.
<achow101>
ack
<willcl-ark>
Moving to our own runners also has a knock-on effect that all the (current) "cirrus" jobs will run on your own forks slower, on GH free runners, so self-hosting a cirrus runner may no longer be necessary if speed it not an issue for you. Forks (like inquisition) would be able to buy their own runners if they wanted faster runners for their repos too, and enable with a trivial patch.
<willcl-ark>
I wanted to highlight it in a meeting just to try and alert as many people to it as possible, in case there was any opposition to it which hadn't been raised yet. Naturally, we are also looking for more reviewers too! So if anyone has any interest, then feel free to consider this a review-beg as well :)
<willcl-ark>
That's all from me on that topic, unless anyone has any questions they'd like to ask about it here.
<achow101>
we don't store the hash in CBlockHeader?
<phantomcircuit>
so i instrumented cblockheader::gethash to count how many times we reclculate the headers hash and i got some extreme outliers like 800+ times, i think this has to do with receiving blocks out of order during IBD and the behavior of activatebestchain
<achow101>
If there's a measuable performance improvement in IBD, then I think that seems like a good idea
<phantomcircuit>
achow101, so the block headers are about the same length as the hash and we reference the block headers with the hash, so in most places we should already have the hash and know that we haven't changed it. though the logic for that doesn't flow through everywhere, so we end up calling gethash more than we should
<l0rinc>
phantomcircuit: I can take another look at it, though making blocks mostly immutable isn't trivial
<darosior>
sha256 over 80 bytes of data should be quite fast. But yeah if we do indeed do it 800+ times (!!) for every single header in the chain that does sound like it could speed things up.
<phantomcircuit>
it trades memory usage for performance but the dbcache is already huge so whatever, an additional 45mb of memory usage to cache the block header hashes along with the the headers
<achow101>
hmm, that is 10% of default dbcache though
<darosior>
That sounds non-trivial
<phantomcircuit>
we usually do it several times per header but in the perverse case it is 800+ times
<l0rinc>
I've pushed a few changes deduplicating hash calculations already - should be slightly less than that now. Are you saying that the exact same block has 800+ recalculations?
<fjahr>
how often have you seen the perverse case?
<darosior>
ceteris paribus i expect 45mb of dbcache to speed IBD more so than caching a sha256 over 80 bytes
<phantomcircuit>
it depends on the behavior of activatebestchain when the blocks are received out of order, I think
<achow101>
if we already know the hash elsewhere, can we use it in more places rather than caching the hash again?
<phantomcircuit>
l0rinc, yes, one block header has 800 recalculations
<phantomcircuit>
fjahr, my graphs suck so I don't know, but it was bad
<darosior>
I'd be very curious to learn more about the situation in which we re-calculate the same header hash 800 times
<l0rinc>
We can likely cache some of those locally, I tried making the blocks immutable and it was quite messy
<TheCharlatan>
achow101 that seems like my intuition too. After all we do save it in the index with every header already.
twistedline has quit [Ping timeout: 252 seconds]
<phantomcircuit>
darosior, it's dependent on the out of order activatebestchain, so it won't show in normal unit tests or anything
twistedline has joined #bitcoin-core-dev
<l0rinc>
I have doubts about the 800 recalculations, seems like a miscalculation to me (I have also investigated this and haven't found anything like that) - but we can do that after the meeting as well
<lightlike>
why 45MB? isn't the CBlockHeader objects freed once the block is no longer in memory (other than the CBlockIndex)?
<phantomcircuit>
l0rinc, it's annoying and you won't see the behavior with blocks received in order
<achow101>
it definitely seems there's something to look at here. phantomcircuit I suggest you work with l0rinc and the others who have also been looking at ibd improvements and benchmarking
<darosior>
+1
<achow101>
#topic Benchmarking WG Update (josie, l0rinc)
<furszy>
memory shouldn't be a concern, could directly use the phashBlock field from the index.
<phantomcircuit>
the way to have the immutable header and block classes is to have it silently convert between the two with constructors
<achow101>
(missed this during the wg update section)
<l0rinc>
I have doubts about the 800 recalculations, seems like a miscalculation to me (I have also investigated this and haven't found anything like that) - but we can do that after the meeting as well
<phantomcircuit>
then slowly change from mutable to immutable everywhere
<l0rinc>
sorry, wrong message
<l0rinc>
#31144 and #32279 were just merged, we're making good progress. The next important ones where reviewers are needed are:
<l0rinc>
#31645 The batch size for UTXO set writes is now calculated based on the maximum dbcache size to ensure that with the default values, memory usage doesn't increase, while reducing flushing time when there is enough memory available.
<l0rinc>
The change reduces the IBD time by a fixed amount, it's speeding up a critical part of saving the state for long-term storage.
<achow101>
phantomcircuit: if you have things you'd like to discuss in a meeting, you can use prepent #proposedmeetingtopic to your message
w0xlt has quit [Ping timeout: 252 seconds]
twistedline has quit [Ping timeout: 276 seconds]
twistedline has joined #bitcoin-core-dev
Robotico has joined #bitcoin-core-dev
Robotico has quit [Remote host closed the connection]
twistedline has quit [Ping timeout: 240 seconds]
twistedline has joined #bitcoin-core-dev
<Murch[m]>
Thanks glozow
eugenesiegel has quit [Quit: Client closed]
kevkevin has quit [Quit: Leaving...]
l0rinc has quit [Quit: l0rinc]
l0rinc has joined #bitcoin-core-dev
rkrux has quit [Quit: Client closed]
l0rinc has quit [Quit: l0rinc]
Christoph_ has joined #bitcoin-core-dev
Christoph_ has quit [Client Quit]
l0rinc has joined #bitcoin-core-dev
<bitcoin-git>
[bitcoin] glozow opened pull request #33106: [WIP] policy: lower the default blockmintxfee, incrementalrelayfee, minrelaytxfee (master...2025-07-minrelay) https://github.com/bitcoin/bitcoin/pull/33106
<darosior>
Oo
freesprung512697 has quit [Quit: zzz]
<l0rinc>
phantomcircuit: looks like I was wrong, thanks for the hint, I have instrumented `CBlockHeader::GetHash` to display the hash that it just calculated and even the first 200k blocks contain hundreds (sometimes even thousands) of hash calculations, e.g. `0000000000000475da936e766d0e3ebf8eb41f0f21f6bcdc7f8a03d80bd495a` is calculated 1018 times. I'll investigate further.
jonatack has joined #bitcoin-core-dev
jespada has joined #bitcoin-core-dev
<lightlike>
we also call CBlockHeader::GetHash for different objects, for example we process a header from a peer (even if we already know the header) we calculate the hash. immutability or cashing won't help with that.
<l0rinc>
lightlike: Yes, it also doesn't help when re-reading from disk. But since I have only seen 4-8 recalculations per block so far (and it looks indeed like some scenarios does a lot more, which we can likely deduplicate locally) an investigation makes sense.
jespada has joined #bitcoin-core-dev
<phantomcircuit>
l0rinc, so i instrumented it to keep a counter because we end up with headers before blocks, so instead of looking at how many times we're calculating the same header hash, i was looking at how many times we're calculating the hash of a distinct object
<phantomcircuit>
since i expect us to be calculating the same header a few times legitimately
jon_atack has joined #bitcoin-core-dev
jespada_ has quit [Ping timeout: 252 seconds]
<lightlike>
l0rinc: yes - as furszy said above, in many situations we should also have the CBlockIndex available, so we could take the hash from there instead of recalculating it from the CBlockHeader.
jonatack has quit [Ping timeout: 260 seconds]
enochazariah has quit [Quit: Connection closed for inactivity]
Guest11 has joined #bitcoin-core-dev
l0rinc has quit [Quit: l0rinc]
Guest11 has quit [Quit: Client closed]
Naiyoma has quit [Quit: WeeChat 4.1.1]
w0xlt has joined #bitcoin-core-dev
w0xlt has quit [Ping timeout: 260 seconds]
kevkevin has joined #bitcoin-core-dev
jon_atack has quit [Ping timeout: 248 seconds]
jonatack has joined #bitcoin-core-dev
jon_atack has joined #bitcoin-core-dev
jonatack has quit [Ping timeout: 276 seconds]
Talkless has joined #bitcoin-core-dev
l0rinc has joined #bitcoin-core-dev
jonatack has joined #bitcoin-core-dev
jespada_ has joined #bitcoin-core-dev
jon_atack has quit [Ping timeout: 248 seconds]
jespada_ has quit [Client Quit]
jespada has quit [Ping timeout: 265 seconds]
jonatack has quit [Ping timeout: 252 seconds]
jonatack has joined #bitcoin-core-dev
kevkevin has quit [Remote host closed the connection]
<darosior>
"DoS resistance is clearly insufficient even at 1s/vB". Rationale: "trust me bro". Reasons to give weight to my opinion: "i also believe Bitcoin's value has gone *down* since the 1sat/vb limit was introduced".