<bitcoin-git>
[bitcoin] maflcko opened pull request #32881: Turn rpcauth.py test into functional test (master...2507-test-rpcauth-fun) https://github.com/bitcoin/bitcoin/pull/32881
joetor5 has quit [Ping timeout: 252 seconds]
VonNaturAustreVe has joined #bitcoin-core-dev
VonNaturAustreVe has joined #bitcoin-core-dev
l0rinc has joined #bitcoin-core-dev
VonNaturAustreVe has quit [Client Quit]
VonNaturAustreVe has joined #bitcoin-core-dev
joetor5 has joined #bitcoin-core-dev
joetor5 has quit [Quit: joetor5]
joetor5 has joined #bitcoin-core-dev
<bitcoin-git>
[bitcoin] HowHsu opened pull request #32882: index: remove unnecessary locater cleaning in BaseIndex::Init() (master...read-best-block) https://github.com/bitcoin/bitcoin/pull/32882
l0rinc has quit [Quit: l0rinc]
hacker4web3bitco has quit [Ping timeout: 248 seconds]
joetor5 has quit [Quit: joetor5]
fearbeag has quit [Ping timeout: 252 seconds]
hernanmarino has quit [Ping timeout: 244 seconds]
l0rinc has joined #bitcoin-core-dev
hernanmarino has joined #bitcoin-core-dev
hacker4web3bitco has joined #bitcoin-core-dev
l0rinc has quit [Ping timeout: 248 seconds]
bitdex has quit [Remote host closed the connection]
bitdex has joined #bitcoin-core-dev
nickygreen85 has joined #bitcoin-core-dev
bitdex has quit [Ping timeout: 244 seconds]
jonatack has quit [Ping timeout: 248 seconds]
kevkevin has joined #bitcoin-core-dev
nickygreen85 has quit [Remote host closed the connection]
kevkevin has quit [Ping timeout: 276 seconds]
nickygreen85 has joined #bitcoin-core-dev
hacker4web3bitco has quit [Ping timeout: 248 seconds]
SpellChecker has quit [Remote host closed the connection]
SpellChecker has joined #bitcoin-core-dev
kevkevin has quit [Ping timeout: 260 seconds]
l0rinc_ has joined #bitcoin-core-dev
l0rinc_ has quit [Quit: l0rinc_]
kevkevin has joined #bitcoin-core-dev
kevkevin has quit [Remote host closed the connection]
kevkevin has joined #bitcoin-core-dev
Talkless has quit [Quit: Konversation terminated!]
___nick___ has joined #bitcoin-core-dev
___nick___ has quit [Client Quit]
___nick___ has joined #bitcoin-core-dev
l0rinc has joined #bitcoin-core-dev
probot has joined #bitcoin-core-dev
bitdex has joined #bitcoin-core-dev
___nick___ has quit [Ping timeout: 244 seconds]
<bitcoin-git>
[bitcoin] vtjl10 opened pull request #32883: Rename `shuffled_indeces` to `shuffled_indices` in `rpc_packages.py` (master...master) https://github.com/bitcoin/bitcoin/pull/32883
sbddesign has joined #bitcoin-core-dev
sbddesign has quit [Ping timeout: 244 seconds]
sbddesign has joined #bitcoin-core-dev
jadi has quit [Ping timeout: 252 seconds]
jadi has joined #bitcoin-core-dev
l0rinc has quit [Quit: l0rinc]
kevkevin has quit [Remote host closed the connection]
<phantomcircuit>
sipa: the only thing i can think of like that is the og ibd algorithm, is that what you're thinking of?
Cory48 has quit [Quit: Client closed]
Cory48 has joined #bitcoin-core-dev
kevkevin has joined #bitcoin-core-dev
bitdex has quit [Ping timeout: 244 seconds]
probot has quit [Quit: Connection closed for inactivity]
jadi has joined #bitcoin-core-dev
Cory48 has quit [Quit: Client closed]
Cory48 has joined #bitcoin-core-dev
sbddesign has joined #bitcoin-core-dev
<sipa>
phantomcircuit: i'm more thinking of all our tests which would break, by sending a ping and waitong for a pokg to know a node is done processing
<sipa>
(not that that is a problem, but it's an indication of the kind of things can clearly be relied upon right now by peers)
kevkevin has quit [Remote host closed the connection]
<sipa>
lacking a formal specification of what the expected behavior of the P2P protocol is, i don't think this is something that can just be changed (at least not without a way to negotiate it as an optional thing)
sbddesign has quit [Read error: Connection reset by peer]
<phantomcircuit>
sipa: what response do we even have for the block message?
<sipa>
phantomcircuit: none, but responses to later messages can depend on it
<phantomcircuit>
i can definitely see changing the orderr breaking a bunch of the functional networking tests
<phantomcircuit>
like basically all of them right
<sipa>
yes
<phantomcircuit>
so in theory i could also just have multiptl message handling threads and try to reduce the lock contention on cs main by finding individual things locking it that don't need to
<phantomcircuit>
like im pretty sure i can remove the lock from IsInitialBlockDownload with a handful of atomics
<phantomcircuit>
tthe alternative is to dilligently identify all of the actual data in memory and reduce the scope of locks while ensuring no lock orderring issues... which um sounds harder
<sipa>
lock annotations are pretty good these days, you'll have more effort convincing clang's analyser that a change is safe, than accidentally introducing a lock ordering issue
<sipa>
but really, i think the way to go is a validation queue for tx/blocks like i suggested earlier
<sipa>
that won't on itself give you more parallellism, but it means all effort to improve that will be localized in validation code, not spread everywhere
sbddesign has joined #bitcoin-core-dev
sbddesign has quit [Read error: Connection reset by peer]
<phantomcircuit>
sipa: without a specific path to parallelism by doing that im afaird it'll make things more complicated, but not actually any faster
<sipa>
well, yes, obviously
<sipa>
but right now i don't even want to try reasoning about it
<sipa>
at least things like validating multiple transactions in parallel seems pretty doable
<sipa>
(individual transactions, received from different peers)
<sipa>
and it would mean without any further changes that for example we can process and respond to ADDR or PING messages even while a block is being processed
<sipa>
i'm considering working on something like this post-clustermempool, fwiw
<phantomcircuit>
sipa: so that's what i thought to but there's enough random other things that acquire cs_main that the msghand thread ends up waiting until the blocks processed
<phantomcircuit>
IsInitialBlockDownload being the one that im 99% sure can be lock free
Guest21 has joined #bitcoin-core-dev
<sipa>
well cs_main would move to validation, and net_processing would become independent of ot
<phantomcircuit>
to test this i put ActivateBestChain in a background thread and ran it on a 10 minute timer, while it's running absolutely nothing will happen because it's holding the chainstate lock and cs_main
<phantomcircuit>
yeah but doing that for the things that wouldn't be put in a queue seems like the first step
<sipa>
anything that needs validation-related data structures (chainstate, headers, mempool, block tree) would need to go through the queue mechanism, effectively making it async, and letting validation decide when to process them
<phantomcircuit>
we're accessing chainstate in places where the result can be pretty fuzzy, like IsIBD doesn't need to be absolutely correct, it's already an approximation
<sipa>
right
Guest21 has left #bitcoin-core-dev [#bitcoin-core-dev]
<sipa>
my goal is reducing latency much more than increasing throughput, but i think it's a good step towards that too
<phantomcircuit>
sipa: so do you think looking for things accessing chainstate which don't really need it is a good first step here?
<phantomcircuit>
seems like ehem *atomic* improvements
<sipa>
yeah, i don't think that on itself will gain us much
<sipa>
but it'll be necessary anyway
<phantomcircuit>
sipa: yeah that's what im thinking