jerryf has quit [Remote host closed the connection]
jerryf_ has joined #bitcoin-core-dev
timbo_xyz has quit [Ping timeout: 265 seconds]
jerryf_ has quit [Remote host closed the connection]
jerryf has joined #bitcoin-core-dev
jerryf has quit [Remote host closed the connection]
jerryf has joined #bitcoin-core-dev
szarka has joined #bitcoin-core-dev
timbo_xyz has joined #bitcoin-core-dev
adys has quit [Read error: Connection reset by peer]
adys has joined #bitcoin-core-dev
timbo_xyz has quit [Ping timeout: 265 seconds]
memset has quit [Remote host closed the connection]
<bitcoin-git>
[bitcoin] fanquake opened pull request #34923: depends: remove workaround for Make older than 4.2.90 (master...drop_make_cross_workaround) https://github.com/bitcoin/bitcoin/pull/34923
memset has joined #bitcoin-core-dev
jerryf has quit [Remote host closed the connection]
jerryf has joined #bitcoin-core-dev
timbo_xyz has joined #bitcoin-core-dev
Guest505 has joined #bitcoin-core-dev
Guest505 has quit [Client Quit]
timbo_xyz has quit [Ping timeout: 265 seconds]
timbo_xyz has joined #bitcoin-core-dev
timbo_xyz has quit [Ping timeout: 265 seconds]
timbo_xyz has joined #bitcoin-core-dev
timbo_xyz has quit [Ping timeout: 265 seconds]
kevkevin has quit [Remote host closed the connection]
timbo_xyz has joined #bitcoin-core-dev
conman has quit [Quit: Konversation terminated!]
timbo_xyz has quit [Ping timeout: 265 seconds]
kevkevin has joined #bitcoin-core-dev
conman has joined #bitcoin-core-dev
timbo_xyz has joined #bitcoin-core-dev
kevkevin has quit [Remote host closed the connection]
kevkevin has joined #bitcoin-core-dev
robszarka has joined #bitcoin-core-dev
kevkevin has quit [Ping timeout: 246 seconds]
timbo_xyz has quit [Ping timeout: 265 seconds]
szarka has quit [Ping timeout: 245 seconds]
timbo_xyz has joined #bitcoin-core-dev
timbo_xyz has quit [Remote host closed the connection]
cmirror has quit [Remote host closed the connection]
cmirror has joined #bitcoin-core-dev
kevkevin has joined #bitcoin-core-dev
timbo_xyz has joined #bitcoin-core-dev
SpellChecker has quit [Remote host closed the connection]
SpellChecker has joined #bitcoin-core-dev
roconnor has joined #bitcoin-core-dev
memset has quit [Remote host closed the connection]
memset has joined #bitcoin-core-dev
memset has quit [Remote host closed the connection]
memset has joined #bitcoin-core-dev
kevkevin has quit [Ping timeout: 256 seconds]
Guest52 has joined #bitcoin-core-dev
Guest52 has quit [Client Quit]
kevkevin has joined #bitcoin-core-dev
ghost43_ has quit [Ping timeout: 265 seconds]
ghost43 has joined #bitcoin-core-dev
ghost43 has quit [Remote host closed the connection]
ghost43 has joined #bitcoin-core-dev
Holz has joined #bitcoin-core-dev
vasild_ has quit [Ping timeout: 265 seconds]
imakeconfou has joined #bitcoin-core-dev
vasild has joined #bitcoin-core-dev
timbo_xyz has quit [Remote host closed the connection]
timbo_xyz has joined #bitcoin-core-dev
Guyver2 has joined #bitcoin-core-dev
Guyver2 has left #bitcoin-core-dev [#bitcoin-core-dev]
S3RK has joined #bitcoin-core-dev
S3RK_ has quit [Ping timeout: 276 seconds]
vasild has quit [Ping timeout: 265 seconds]
<bitcoin-git>
[bitcoin] MkDev11 opened pull request #34924: test: add feebumper coverage for combined bump fee failure (master...fix/issue-34902-feebumper-coverage) https://github.com/bitcoin/bitcoin/pull/34924
timbo_xyz has quit [Remote host closed the connection]
timbo_xyz has joined #bitcoin-core-dev
ourson has joined #bitcoin-core-dev
ourson has left #bitcoin-core-dev [#bitcoin-core-dev]
johnzweng has quit [Quit: Leaving...]
johnzweng has joined #bitcoin-core-dev
johnzweng has quit [Quit: Leaving...]
johnzweng has joined #bitcoin-core-dev
timbo_xyz has quit [Ping timeout: 265 seconds]
timbo_xyz has joined #bitcoin-core-dev
serbtc has joined #bitcoin-core-dev
ozdeadman has joined #bitcoin-core-dev
deadmanoz has quit [Ping timeout: 244 seconds]
serbtc has left #bitcoin-core-dev [ERC 5.6.0.30.1 (IRC client for GNU Emacs 30.1)]
timbo_xyz has quit [Ping timeout: 265 seconds]
apollodorus has joined #bitcoin-core-dev
l0rinc has joined #bitcoin-core-dev
<bitcoin-git>
[qa-assets] ekzyis opened pull request #268: Add shebang to delete_nonreduced_fuzz_inputs.sh (main...delete-nonreduced-fuzz-inputs-shebang) https://github.com/bitcoin-core/qa-assets/pull/268
nymius has joined #bitcoin-core-dev
imakeconfou has quit [Ping timeout: 248 seconds]
<sipa>
#proposedmeetingtopic asmap file format & tooling
jerryf has quit [Remote host closed the connection]
jerryf has joined #bitcoin-core-dev
apollodorus has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
eugenesiegel has joined #bitcoin-core-dev
jerryf has quit [Remote host closed the connection]
<fjahr>
There is one pre-proposed meeting topic this week. Any last minute ones to add?
<abubakarsadiq>
hi
<epicleafies>
hi
<fjahr>
Let's start with the Working Groups
<fjahr>
#topic Kernel WG Update (sedited)
<sipa>
hi
Guest3 has joined #bitcoin-core-dev
<furszy>
hi
<sedited>
no updates from me this week, and am going afk for the next three weeks.
<fjahr>
#topic Benchmarking WG Update (l0rinc, andrewtoth)
purpleKarrot has joined #bitcoin-core-dev
<andrewtoth_>
nothing from me this week
<kanzure>
hi
<fjahr>
#topic Net Split WG Update (cfields)
isaquef has joined #bitcoin-core-dev
<cfields>
Finally some progress! I have a branch which cleans up all of the local address handling which is currently just a bunch of global functions used all over the place (GetLocal(), AddLocal(), etc). Ultimately they are all used for GetLocalAddrForPeer().
<cfields>
My first step has been just to remove the dependencies on CNode and move all of the functions into a new LocalAddressManager. For now, it's instantiated as a static global. Functionally there's no change but it's now 10x easier to understand how it all works and test. The next step will be to actually instantiate it and store it in node.
nervana21 has joined #bitcoin-core-dev
<cfields>
While working on it, I discovered a few nasty leaks that should potentially be fixed. I'm working on testing and documenting so that I can propose some fixes and behavioral changes.
<cfields>
Both streams (refactor and fixes) could potentially happen in parallel, but imo it's _much_ easier to understand what's going on and test after refactoring to a sane manager class. So I'll probably PR that work first.
<fjahr>
Cool, anything else on net split?
<cfields>
Nope, that's it for now.
<fjahr>
#topic QML GUI WG Update (johnny9dev)
<johnny9dev>
The decoupling of qml from the qt widgets gui is done. A part of that also included the last piece of automated test tools for the project. Specifically gmock for mocking the wallet and node interfaces
<eugenesiegel>
hi
<johnny9dev>
so the project now has comprehensive testing. unittests, qml tests, and end to end gui tests
musaHaruna has joined #bitcoin-core-dev
<johnny9dev>
Feature wise wallet import/restore was merged and a pr for wallet migration is up and this week i did all of the features for fee setting. Those will be PR'd the next couple of days
<johnny9dev>
epicleafies has a bunch of PRs lingering for features he completed so my top priority is to finish reviews of all of those now that the test frameworks are settled and they all no longer have conflicts
<johnny9dev>
epicleafies: can you share your status?
<fanquake>
gmock as in Google Test / Mock?
<epicleafies>
Yeah, this past week I've created PRs for adding desktop tray icon/functionality and the rpc console page
<johnny9dev>
yeah thats what I know for mocking. open to swapping it out later but I've always liked it.
<johnny9dev>
only using gmock, nothing from gtest
<darosior>
hi
<fjahr>
seems like that's it for gui?
<johnny9dev>
yeah thats it. we'll be PRing a few more features and then i will likely do another assestment to see what the remaining gap is and I will share that
<pinheadmz>
#32061 has been rebased following great reviews on code style and deeper http protocol. I am running fuzzers on the branch this week and will re-run my integration tests with lnd, electrs, etc as well. The first 7 commits add tests and utilities, and are split off in to two small PRs which are in review, and I'm pushing updates today: #34772 and #34905
<corebot>
https://github.com/bitcoin/bitcoin/issues/32061 | Replace libevent with our own HTTP and socket-handling implementation by pinheadmz · Pull Request #32061 · bitcoin/bitcoin · GitHub
<fjahr>
From my side, I am still getting some good on the torcontrol PR (#34158) and respond to that as quickly as possible. I think it looks to be in pretty good shape by now.
<fjahr>
That concludes the WGs unless I missed someone
<fjahr>
#topic asmap file format & tooling (sipa)
<sipa>
Hi!
<sipa>
With the asmap data now built-in to the bitcoin core binary, i had a look at the tooling around it
<sipa>
and it's not entirely satisfying i think that the only actual encoder/decoder for the format is a largely untested (and probably barely reviewed) python script
<sipa>
we do have a well-tested interpreter inside bitcoin core that can lookup ASNs, but there is no end-to-end testing that the logic we use for creating the files (in python) actually matches what the interpreter reads
<sipa>
so what do people think about adding a bitcoin-asmap binary, with a c++ implementation of what is currently in the python script?
<sipa>
the code would be able to be covered by testing, but still only the interpreter would end up in bitcoind
<sedited>
sgtm
Emc99 has joined #bitcoin-core-dev
<fanquake>
Probably depends where the rest of the tooling ends up?
<fjahr>
sounds good to me
<sipa>
so this is just about the "ASN mapping to binary" part, and possibly the diffing/etc, it's not about the actual sourcing of the mapping, which is the kartograf project
<fjahr>
FWIW, there seemed to be diverse opinions about what should go where so I planned to have that part of the discussion in person
<Murch[m]>
sipa: Would that be something that you hack up in a few days or a bigger project?
<fjahr>
Replacing the encoder/decoder doesn't change the status quo so should be uncontroversial in that regard?
<fanquake>
There doesn't seem to be a good split on what should live where, or if asmap = core or not, and who's responsible for maintainig various things
isaquef has quit [Ping timeout: 245 seconds]
<fanquake>
(especially since this is all becomming (potentially) release blocking)
<sipa>
Murch[m]: Opus 4.6 converted the python code in one go to C++, produces bit-identical files, and is 10x faster... i'd obviously clean it up myself, but having something that already works is nice
<sipa>
with that, another thing i want to bring up... i've been experimenting with replacing the binary file format with some new insights and better compression
<sipa>
and have something that can realistically be 20% smaller, while still being interpretable on-the-fly, though the format does get more complex
<sipa>
does that sound like a trade-off that's worth looking into?
<cfields>
+1
<sipa>
i expect it to be equally fast or faster
<fjahr>
how much more complex? But 20% sounds great
isaquef has joined #bitcoin-core-dev
<sipa>
fjahr: the interpreter is probably not that much worse, the encoder is pretty nontrivial
<Murch[m]>
Isn’t it pretty small already?
<cfields>
sipa: I suppose the format changes could be done in python too? So they don't depend on figuring out the c++ tool first?
<fjahr>
yeah, I was going to say the order of these is worth thinking about
<sipa>
cfields: i already have it working in c++... this was the actual motivation for me to start looking into it, because the new encoding scheme is significantly more computationally costly, so didn't want to keep doing that in python
<Murch[m]>
Or to put it differently, how big is the file?
<cfields>
ok, makes sense
<sipa>
the current code i have is not nearly production ready... but it's good enough to give me an idea of how much smaller it can get
<sipa>
Murch[m]: around 1.6 MiB, and shipped inside our binaries
<fjahr>
1.4 with filling
<sipa>
so we might cut down on some 250 KiB in binary size
<fanquake>
So about 10% of our releasebinary, maybe more
<Murch[m]>
Then I don’t think it makes sense to spend a lot of engineering on making it smaller unless there are other benefits
<sipa>
there is the possibility of shipped the data in a separate file that needs to be installed in some discoverable location, rather than inside the binary
<sipa>
i think there are convenience arguments against that, but if installer size is a concern, that's a much more significant improvement
<Murch[m]>
Right, but anyone running it will immediately download hundreds of gigabytes of data and store at least several. In that context, does it matter much?
<Murch[m]>
Maybe I’m missing something here
<furszy>
Murch[m]: math is fun
<Murch[m]>
Yeah, sure. Just trying to understand the cost-benefit heer
<sipa>
Murch[m]: that's a good point yes - i feel our binaries are big, but looking at the blockchain size does put it all in perspective
<Murch[m]>
s/heer/here/
<sipa>
well i think i'll start with c++ifying with the current format
<sipa>
based on the discussion here
<fjahr>
Murch: valid point, but i am still very interested in seeing what we can get
<Murch[m]>
Not trying to tell sipa what to do, but I’m not sure spending several weeks to optim
<sipa>
Murch[m]: too late
<fjahr>
sipa: better testing is a very good argument for that
<Murch[m]>
s/optim/shave a MB off of that file is the biggest bang for the buck ^^/
<darosior>
I'm not sure how relevant is the comparison between binary size and block chain size. But the point probably holds for our RAM usage defaults anyways.
<sipa>
it's 3 weeks worth of block headers worth of RAM that this may save :)
<sipa>
which is probably not relevant
<darosior>
Unless we start having one dbcache defaults per piece of hardware ever manufactured :p
* darosior
hids
<cfields>
which in 2026 is $10k...
<darosior>
sipa: right
<sipa>
cfields: :D
<Murch[m]>
Also, I think we’re getting a bit lost in details here
<sipa>
agreed
<sipa>
that's it for me, unless people want to discuss other asmap related things
<cfields>
sipa: presumably it's irrelevant for runtime ram once loaded though? Or is this representation persistent in memory?
<sipa>
cfields: the cool thing about this format is that it's interpretable without decompression
<sipa>
so it's literally just a blob built-in to the binary, and directly interpreted there
<cfields>
oh, neat.
nymius has quit [Ping timeout: 245 seconds]
brunoerg has quit [Remote host closed the connection]
<fjahr>
Anything else to discuss?
brunoerg has joined #bitcoin-core-dev
<hodlinator>
hi
<fjahr>
Quick shill for #34636 since optimizing RAM was mentioned. A lot of room for improving the index allocation there.
<corebot>
dzxzgthree: Error: That URL raised <HTTP Error 404: Not Found>
isaquef has quit [Ping timeout: 245 seconds]
<dzxzgthree>
"As a -blocksonly node, should we immediately drop the compact block (since we did not send sendcmpct to our peer) or should we process the header if we receive the compact block?"
<bitcoin-git>
[bitcoin] maflcko opened pull request #34927: test: Check that RPCs do not time out, even under load (master...2603-test-rpc-sla) https://github.com/bitcoin/bitcoin/pull/34927
<sipa>
dzxzgthree: also impressed with your 8 consecutive consonants
<darosior>
I would have to read about the tradeoff here. Obviously it seems to me that we should always (tm) process headers, but that also doesn't seem crazy to ignore broken peers that send cmpctblocks to blocksonly nodes.. 🤷
<sipa>
i don't know if it matters much
<sipa>
it feels wrong to try to deal with broken peer implementations, as it might encourage relying on that behavior
<sipa>
but also processing headers is so cheap...
<Murch[m]>
Only orthogonally related, but we’ve seen more BIP submissions in the past couple years (probably because they get processed?), but especially lately, there have been a few that I think I’ve been the only reader of. If anyone is looking for some fun reading material, please feel free to review some BIPs.
<dzxzgthree>
sipa: darosior: right this is the exact thing that I'm stuck on, in general it seems advisable to ignore out-of-protocol behavior from peers, but I also feel like processing a header that you have is so cheap and so potentially valuable that it might be an exception
epicleafies has quit [Quit: Client closed]
<Murch[m]>
sipa: Processing headers early has some downsides. We have been looking at the recent 2-block reorg and there is a relay anomaly there
<sipa>
Murch[m]: i think if that's related, then that sounds like an unrelated issue
<sipa>
wait, that sentence makes no sense
<dzxzgthree>
that's unrelated I think, although notably I think that the 10-minute stalling delay has hit a lot of nodes: https://github.com/bitcoin/bitcoin/issues/33687 in the recent re-org
<dzxzgthree>
one of achow101's nodes and one of b10c's nodes were both stalled by what I'm pretty sure is the same strange peer
<sipa>
i mean: if header processing somehow contributed to the 2-block reorg issue, then we have an issue in our code that should be addressed independently of the question of whether to process headers in unrequested compact blocks
<Murch[m]>
sipa: started writing that when fjahr asked whether there were other topics, but posted it a bit delayed.
<Murch[m]>
It looks like there is a node blasting out header INV of every new block to loads of other nodes without sending the bodies.
<fanquake>
Murch[m]: are there still 4/5 BIP editors?
<fjahr>
Murch[m]: stalling peers on IRC with delayed messages?
<Murch[m]>
As far as I can tell, there are currently 6, but the vast majority of activity has been from a single dev in the last three months
<sipa>
Murch[m]: these are bip130 headers announcements, not compact blocks, right?
eugenesiegel80 has joined #bitcoin-core-dev
<Murch[m]>
fjahr: Yeah, it looks like several nodes we looked at got the header for the Foundry block pretty early, but then waited ten minutes because they didn’t get a high-bandwidth compact block announcement of the block
<Murch[m]>
Yes, not compact blocks
<dzxzgthree>
sipa: the peer aggressively inv's new blocks, responds to our getheaders with a headers, and then stalls the getdata for the block
<Murch[m]>
achow101 and dzxzgthree looked into it more yesterday
<dzxzgthree>
if there's ever a race and all of your HB peers happen to be on the same side of it as you you get stalled for 10 min
<sipa>
dzxzgthree: so it's not even using sendheaders, interesting
eugenesiegel has quit [Ping timeout: 245 seconds]
<lightlike>
why would that broken/attacking node reach many nodes with that strategy, when other nodes using direct headers and need a roundtrip less? is it connecting to everyone?
<eugenesiegel80>
If b10c still has the logs, it might be worth checking if the stalling node then and now have similar characteristics (same entity)?
<dzxzgthree>
lightlike: sipa: That's still a question, but it seems still to be relatively succesful, b10c I think will share their timeline table soon on bnoc but from glancing at it, this node also seems to have a lot of success being the first to announce HEADERS to one of b10c's nodes, although it is sometimes beat by direct headers annoucements
purpleKarrot has quit [Quit: purpleKarrot]
TallTim has joined #bitcoin-core-dev
purpleKarrot has joined #bitcoin-core-dev
<dzxzgthree>
I don't want to share it on irc, but I posted the full log message on bnoc for the stall disconnect, which includes the ip address of this entity if anyone else wants to check their logs for similar strange behavior
<achow101>
they also have a unique user agent
<darosior>
I'll check a node i have handy here
<darosior>
What's the UA?
<achow101>
"semikek"
<darosior>
lol
<sipa>
(an outburst of audible laughter was heard in the chaincode office)
<achow101>
I grepped my historical debug logs for them. They were first seen by this node in Nov 2024
<dzxzgthree>
still I wonder why it is so often faster to announce headers than other peers in the logs I've looked at so far
<phantomcircuit>
sipa, the better question is do we know of any nodes that actually send compactblocks to blocks only peers?
<dzxzgthree>
blocksonly is not the only instance under questioning in that PR, another situation is a node that fails to send a SENDCMPCT, and I think the question can just be put more broadly: Should headers of out-of-protocol CMPCTBLOCK messages be processed?
<phantomcircuit>
my first guess is that it really doesn't matter
<dzxzgthree>
with the question put broadly it's also easier to express the general risk: if nodes don't process the headers from out-of-protocol CMPCTBLOCK's, then any bug or vulnerability that causes a node to behave out-of-protocol with CMPCTBLOCK's becomes a much more dangerous DoS vector
dzxzgthree has quit [Quit: Konversation terminated!]
<Murch[m]>
b10c: wb
<Murch[m]>
You got any new hot goss on the reorg?
b10c[m]1 has quit [Quit: Reconnecting]
b10c[m]1 has joined #bitcoin-core-dev
memset has quit [Remote host closed the connection]
memset has joined #bitcoin-core-dev
kevkevin has joined #bitcoin-core-dev
timbo_xyz has joined #bitcoin-core-dev
eugenesiegel80 has quit [Quit: Ping timeout (120 seconds)]