thelounge496669 has quit [Read error: Connection reset by peer]
thelounge496669 has joined #bitcoin-core-dev
Christoph_ has joined #bitcoin-core-dev
thelounge496669 has quit [Read error: Connection reset by peer]
thelounge496669 has joined #bitcoin-core-dev
Cory30 has joined #bitcoin-core-dev
Cory44 has quit [Ping timeout: 250 seconds]
nanotube has quit [Ping timeout: 252 seconds]
robszarka has joined #bitcoin-core-dev
Earnestly has quit [Ping timeout: 260 seconds]
helo_ has quit [Ping timeout: 260 seconds]
rszarka has quit [Ping timeout: 260 seconds]
helo has joined #bitcoin-core-dev
Earnestly has joined #bitcoin-core-dev
Cory44 has joined #bitcoin-core-dev
Cory30 has quit [Ping timeout: 250 seconds]
nanotube has joined #bitcoin-core-dev
l0rinc has joined #bitcoin-core-dev
PaperSword1 has joined #bitcoin-core-dev
Christoph_ has quit [Quit: Christoph_]
PaperSword has quit [Ping timeout: 244 seconds]
l0rinc_ has quit [Ping timeout: 244 seconds]
_durandal has quit [Ping timeout: 244 seconds]
dzxzg2 has quit [Ping timeout: 244 seconds]
nanotube has quit [Ping timeout: 244 seconds]
PaperSword1 is now known as PaperSword
nanotube_ has joined #bitcoin-core-dev
purpleKarrot has joined #bitcoin-core-dev
purpleKarrot has quit [Remote host closed the connection]
purpleKarrot has joined #bitcoin-core-dev
twistedline_ has quit []
twistedline has joined #bitcoin-core-dev
l0rinc has quit [Quit: l0rinc]
Cory47 has joined #bitcoin-core-dev
_durandal has joined #bitcoin-core-dev
l0rinc has joined #bitcoin-core-dev
Cory44 has quit [Ping timeout: 250 seconds]
l0rinc has quit [Quit: l0rinc]
l0rinc has joined #bitcoin-core-dev
jonatack has joined #bitcoin-core-dev
l0rinc has quit [Quit: l0rinc]
jon_atack has quit [Ping timeout: 250 seconds]
Cory51 has joined #bitcoin-core-dev
Cory47 has quit [Ping timeout: 250 seconds]
l0rinc has joined #bitcoin-core-dev
thelounge496669 has quit [Read error: Connection reset by peer]
thelounge496669 has joined #bitcoin-core-dev
_durandal has quit [Ping timeout: 256 seconds]
_durandal has joined #bitcoin-core-dev
jon_atack has joined #bitcoin-core-dev
jonatack has quit [Ping timeout: 244 seconds]
TheRec has quit [Ping timeout: 265 seconds]
TheRec has joined #bitcoin-core-dev
jonatack has joined #bitcoin-core-dev
jon_atack has quit [Ping timeout: 258 seconds]
l0rinc has quit [Quit: l0rinc]
SpellChecker has quit [Remote host closed the connection]
jerryf_ has quit [Remote host closed the connection]
SpellChecker has joined #bitcoin-core-dev
jerryf has joined #bitcoin-core-dev
jon_atack has joined #bitcoin-core-dev
jonatack has quit [Ping timeout: 258 seconds]
cmirror has quit [Remote host closed the connection]
cmirror has joined #bitcoin-core-dev
l0rinc has joined #bitcoin-core-dev
l0rinc has quit [Ping timeout: 256 seconds]
l0rinc has joined #bitcoin-core-dev
<_aj_>
sipa: mostly i'd expect it to reduce the instances where we get a backlog in the first place, which the scaling can't really help with since it only hits when there's already a backlog. in the last couple of days, i see a bit under 60 ten minute blocks with more than 4200 txs received; versus about 8 with more than 8400 reecived (highest was ~11,540 txs between 10:50 and 10:59 on the 16th)
jon_atack has quit [Ping timeout: 256 seconds]
phantomcircuit_ is now known as phantomcircuit
<phantomcircuit>
it would be nice if the leveldb background compaction thread got renamed so it doesn't look like there's two msghand threads
<phantomcircuit>
(or whatever it gets duplicated from)
<phantomcircuit>
i've not a clue how to make that happen tho
jackielove4u has joined #bitcoin-core-dev
Christoph_ has joined #bitcoin-core-dev
Christoph_ has quit [Quit: Christoph_]
l0rinc has quit [Quit: l0rinc]
nanotube_ has quit [Ping timeout: 256 seconds]
juleeho has joined #bitcoin-core-dev
juleeho has joined #bitcoin-core-dev
juleeho has quit [*.net *.split]
thelounge496669 has quit [*.net *.split]
PaperSword1 has joined #bitcoin-core-dev
thelounge496669 has joined #bitcoin-core-dev
nanotube_ has joined #bitcoin-core-dev
helo_ has joined #bitcoin-core-dev
Earnest has joined #bitcoin-core-dev
jackielove4u has quit [*.net *.split]
cmirror has quit [*.net *.split]
TheRec has quit [*.net *.split]
purpleKarrot has quit [*.net *.split]
PaperSword has quit [*.net *.split]
Earnestly has quit [*.net *.split]
helo has quit [*.net *.split]
robszarka has quit [*.net *.split]
javi404 has quit [*.net *.split]
hardtotell has quit [*.net *.split]
pablomartin4btc has quit [*.net *.split]
stratospher[m] has quit [*.net *.split]
cotsuka has quit [*.net *.split]
corebot has quit [*.net *.split]
PaperSword1 is now known as PaperSword
TheRec_ has joined #bitcoin-core-dev
javi404 has joined #bitcoin-core-dev
hardtotell has joined #bitcoin-core-dev
robszarka has joined #bitcoin-core-dev
pablomartin4btc has joined #bitcoin-core-dev
stratospher[m] has joined #bitcoin-core-dev
cotsuka has joined #bitcoin-core-dev
corebot has joined #bitcoin-core-dev
hardtotell has quit [Max SendQ exceeded]
robszarka has quit [Max SendQ exceeded]
hardtotell has joined #bitcoin-core-dev
robszarka has joined #bitcoin-core-dev
vincenzopalazzo has quit [Ping timeout: 256 seconds]
vincenzopalazzo has quit [Read error: Connection reset by peer]
Saturday7 has joined #bitcoin-core-dev
sdfgsdfg3d has joined #bitcoin-core-dev
f321x has quit [Quit: f321x]
f321x has joined #bitcoin-core-dev
sdfgsdfg3d has quit [Quit: Client closed]
<sipa>
_aj_: hmm, looking at the formula, it's just BASE + (to_send.size() / 1000) * 5, so the adaptive behavior does not kick in until the backlog is 1000 txn
<sipa>
if we think that's too much, maybe it's just the 1000 constant that should be reduced?
Christoph_ has joined #bitcoin-core-dev
Christoph_ has quit [Client Quit]
<sipa>
any reason not to do (BASE + to_send.size() / 200) ?
f321x has quit [Remote host closed the connection]
f321x has joined #bitcoin-core-dev
robszarka has joined #bitcoin-core-dev
rszarka has quit [Ping timeout: 256 seconds]
robszarka has quit [Quit: Leaving]
szarka has joined #bitcoin-core-dev
Cory51 has quit [Quit: Client closed]
Cory51 has joined #bitcoin-core-dev
abubakarsadiq has joined #bitcoin-core-dev
<hebasto>
#proposedmeetingtopic Support for Ubuntu 22.04 LTS as a build platform
<_aj_>
sipa: the behaviour i was seeing is more the inv queue spiking to ~5000 then sticking there for a little while for a handful of nodes (presumably spy nodes who neer inv us, or poorly connected 0.1sat/vb nodes who don't have another source for many txs), for which i don't think that alternative would help much. i thought the argument that "7 tx/sec was chosen pre-segwit, and we've doubled capacity
<_aj_>
since then so should have already switched to 14 tx/sec" made sense though? although i guess that argument was in #27630 but not re-stated in the new pr.
<_aj_>
sipa: iirc i felt like immediately bumping the rate by +1 when the queue was >200 entries was a potential privacy leak, so thought some quantizing was a good idea
<sipa>
_aj_: yeah, i don't object to raising it 2x, but want to reason through it a bit more
<darosior>
The last setting was chosen in April 2016, when there was between 200-220k transactions a day. Nowadays there is between 400k and 600k transactions a day.
<darosior>
200k transactions a day is 2.3 tx/s. Why was the value chosen to be thrice as much?
<sipa>
darosior: well you don't want the rate limit to be set to the expected value, because then you'll always have a backlog
<sipa>
so you want some overshoot to account for temporary variations around the average
<instagibbs>
aside: should we hard cap the inv queue size at some count? theoretically hard to hit but unbounded growth (even with fee payers) seems not great
<darosior>
Ok, i was thinking of it as the default value should roughly match usage, and the temporary increase would kick in for periods of above-average usage.
<darosior>
instagibbs: can it really be hit? Because the number of transaction you send will also grow arbitrarily large with the size of inv_to_send
<sipa>
darosior: in the steady state, the size of the queue will be ~200x the number of transactions you receive between any two sends
<sipa>
for sufficiently large inbound tx rate
<instagibbs>
Probably impossible in practice due to rate limiting of peers incoming messages or something else I missed? Have a set of 100 peers send tiny high paying fee txns directly, see what happens to inv queues?
<darosior>
instagibbs: yeah would be interesting to have a functional test for this. I suspect it would just lead you to blast INV messages to your peers.
<sipa>
per peer you're limited by the roundtrip time of an INV-GETDATA-TX cycle, times the number of transactions permitted in txrequest
<instagibbs>
I suppose it would be hard for attacker to get their hands on the txs fast enough, be first to send them to you unilaterally
<darosior>
sipa: sorry could you expand on the ~200x figure? Not sure to follow.
<sipa>
darosior: the formula for how much to send, approximated, is (invs_to_send.size() / 200)
<sipa>
in the steady state, the amount of tx you send equals how many transactions get added to the queue in between two sends
<sipa>
so if you receive say 300 transactions between every two sends, the queue size will tend to grow to a point where invs_to_send.size() / 200 equals 300
<sipa>
which is when invs_to_send.size() == 200 * 300 = 60000
<darosior>
Ok it's the "steady state" definition that i was missing. I wasn't seeing how if you receive on average 2.3 tx per second and send 35 every 5 seconds you end up with a queue of size 200 * 5 * 2.3. Thanks.
<sipa>
darosior: well the steady state is what you end up
<sipa>
the queue will grow until it hits that point
Christoph_ has joined #bitcoin-core-dev
Christoph_ has quit [Client Quit]
* darosior
clarified with sipa IRL, he was talking about instagibbs' question i was talking about _aj_'s proposal to increase the default rate
jespada has joined #bitcoin-core-dev
<sipa>
_aj_: do we actually care about treating the size of the queue as secret for fingerprinting reasons? because an attacker observing it gets to see the actual transactions too, which seems like a much stronger signal anyway
<sipa>
if not, i'd suggest something simpler like always send the top 20-40% (randomly selected) of the queue?
<sipa>
darosior points out that that may mean never actually sending the bottom part
<instagibbs>
darosior ok them I'm confused now
<instagibbs>
just throwing an idea out there, I've not thought hard about it
<sipa>
instagibbs: my answer to your question is: in the steady state, queue sizes (with the current formula) will grow proportionally with the incoming transaction rate
<instagibbs>
sipa 👍
<sipa>
so they can be unbounded, but only if the incoming transaction rate is unbounded, and txgraph prevents that
<sipa>
eh, txrequest
<sipa>
but the rate can be pretty high
<darosior>
txgraph txrequest txdownload
<instagibbs>
I think it only seems problematic if nodes arent sending invs to you and direct sending
<darosior>
tx* is the new *man
<sipa>
needs more "mini"
<sipa>
minitxman
<darosior>
if we call it mini it cannot grow unbounded
<sipa>
right
Christoph_ has joined #bitcoin-core-dev
Cory51 has quit [Quit: Client closed]
jespada has quit [Ping timeout: 244 seconds]
Cory51 has joined #bitcoin-core-dev
<darosior>
_aj_: so assuming we keep the same mechanism it makes sense to me to increase the default relay rate. This is because we've observed average transaction rate in the past years that were higher than 7 tx/s. But i'm not sure about doubling it though. We've never witnessed a sustained rate anywhere close to 14 tx/s. According to
<darosior>
https://mainnet.observer/charts/transactions-per-day/ the highest we've been using a 30-day moving average is ~7.75 tx/s, and using a 7-day moving average 8.6 tx/s. So it seems that using 14 tx/s would not be different from not having any limit 99.9% of the time?
<sipa>
another idea: add insertion time into the to_send queue, and always send everything whose insertion time is more than X seconds ago (which becomes some sort of timeout)
<sipa>
plus X% of the highest-feerate stuff in the queue that isn't timed out yet
<instagibbs>
dont have to sort timed out stuff either
<instagibbs>
iiuc
<sipa>
we may still want to, to guarantee topology
<sipa>
and probably better for privacy anyway
<instagibbs>
ah right
<instagibbs>
doing stuff before sorting can help if you're *dropping* it for various reasons
<sipa>
yet another idea is to see where transactions fit in your mempool (in terms of how many MvB from top), and send everything that's up to say 2 MvB from the top, and X% of the rest
Cory51 has quit [Quit: Client closed]
Cory51 has joined #bitcoin-core-dev
<_aj_>
instagibbs: capping the inv queue isn't great, because we sort high-fee child transactions after low-fee txs that only spend confirmed txs. maybe we could sort better with cluster mempool? alternatively, if we had rebroadcast behaviour, dropping some txs would be less of a problem
<instagibbs>
clearly not optimal, only better than crashing
<_aj_>
sipa: i suspect we don't care about keeping the queue size secret -- we effectively consider the contents of the mempool at inv-send-time public anyway
<sipa>
right
<sipa>
the primary privacy benefit the queueing/batching has is hiding the relative order of transactions
<_aj_>
darosior: "mini-inf, another name for aleph0 or the cardinality of the integers, ..."
<_aj_>
darosior: 2024-04-23 on that graph claims 927010 txs per day, which is 10.7/second
<sipa>
hmmm, i think my conclusion is that we should really set the amount of transaction sent as a function of the time since the last send
<sipa>
because it's a bandwidth consideration... we have some tx send rate that we think would be harmful for (some?) peers, and when we approach that, and can't send everything anymore, we should prefer higher-feerate things
<sipa>
but in addition, from a personal DoS protection perspective, we need to limit the size of the queue
dzxzg has joined #bitcoin-core-dev
<sipa>
what about: send_now = std::min(to_send.size(), time_since_last_send * R + (to_send.size() ** 2) / M)
<sipa>
where R is the "everyone can handle this tx rate" number, and M is the maximum queue size
<sipa>
R could be different for outbound and inbound; if not, the number of txn per inv will be lower for outbounds than for inbounds, as they have more frequent sends - but maybe that's ok
<_aj_>
i think the main benefit of the queue is that it allows the network to prioritise relaying the high-pri/high-fee txs at a rate similar to what can be confirmed, relegating low fee backlog txs to only being relayed when there's a lull. my take is that in that context, just dropping the low fee txs and not relaying them at all would be better, provided there's some way for them to get picked back up
<_aj_>
when the mempool empties out (which could just be "the sending wallet resubmits")
<sipa>
_aj_: agreed, but i think dropping would be bad absent mempool retransmission - i'm not sure how reliably the network deals with resubmission on its own
<_aj_>
sure, bip153/sendtemplate's my answer to that :)
<_aj_>
anyone remember how many txs were in the mempool back when we had huge backlogs?
<_aj_>
hmm, i have significantly fewer txs in my mempool than mempool.space reports (87k vs 130k?)
<sipa>
is there an RPC to get node uptime?
<sipa>
ah, `uptime` ~!
<fanquake>
uptime
<sipa>
that was way too obvious
<sipa>
why isn't it getnodeuptimeinfo ?
<sipa>
_aj_: i have 117230 txn right now on my long-running node's mempool (which was recently updated to 0.1 sat/kvB policy ~33 days ago)
jadi has joined #bitcoin-core-dev
<sipa>
eh
<sipa>
100 sat/kvB or 0.1 sat/vB
<fanquake>
sipa: how long running?
<sipa>
33 days
<_aj_>
#10400 ; could have gone into `getinfo`, but that was already being deprecated
<corebot>
https://github.com/bitcoin/bitcoin/issues/10400 | [RPC] Add an uptime command that displays the amount of time (in seconds) bitcoind has been running by rvelhote · Pull Request #10400 · bitcoin/bitcoin · GitHub
dzxzg has quit [Quit: Konversation terminated!]
dzxzg has joined #bitcoin-core-dev
purpleKarrot has quit [Quit: purpleKarrot]
jadi has quit [Ping timeout: 256 seconds]
<_aj_>
ah, a thought i had was that you could: (a) add a relayed_at timestamp to txs in the mempool, with index; (b) add a relayed_upto timestamp for each peer; (c) add txs to a peer's inv_to_send and update relayed_upto correspondingly in order, but stop once the inv_to_send hits 1000 txs; (d) sort and drain inv_to_send as currently. that way lagging peers will just get worse tx info during bursts,
<_aj_>
without using up extra memory or compute
<sipa>
the mempool relayed_at index would use memory?
<_aj_>
whether or not peers lag wouldn't change how much memory was used, i mean
<_aj_>
i'm not sure why i wanted a timestamp rather than just the mempool seq number? maybe so you could time things out? or maybe i was just confused
purpleKarrot has joined #bitcoin-core-dev
<_aj_>
ohhh, i should read my notes. the idea was to have "rebroadcasting" happen by updating the relayed_at timestamp, but that could mean a parent's relayed_at timestamp was later than one of its children
jonatack has joined #bitcoin-core-dev
<_aj_>
(ie, rebroadcasting via existing methods that currently work such as the wallet or sendrawtransaction)
Cory44 has joined #bitcoin-core-dev
dzxzg has quit [Quit: Konversation terminated!]
alvroble has joined #bitcoin-core-dev
Cory51 has quit [Quit: Client closed]
f321x has quit [Quit: f321x]
QUBIT_ABHAY has joined #bitcoin-core-dev
BGL has quit [Ping timeout: 260 seconds]
enochazariah has joined #bitcoin-core-dev
jeremyrubin has joined #bitcoin-core-dev
<QUBIT_ABHAY>
hi
dzxzg has joined #bitcoin-core-dev
enochazariah70 has joined #bitcoin-core-dev
enochazariah70 has quit [Client Quit]
enochazariah89 has joined #bitcoin-core-dev
enochazariah24 has joined #bitcoin-core-dev
enochazariah24 has quit [Client Quit]
enochazariah has quit [Ping timeout: 250 seconds]
enochazariah has joined #bitcoin-core-dev
QUBIT_ABHAY has quit [Quit: Client closed]
eugenesiegel has joined #bitcoin-core-dev
QUBIT_ABHAY has joined #bitcoin-core-dev
enochazariah89 has quit [Ping timeout: 250 seconds]
Cory90 has joined #bitcoin-core-dev
Christoph_ has joined #bitcoin-core-dev
Christoph_ has quit [Client Quit]
Cory44 has quit [Ping timeout: 250 seconds]
<achow101>
#startmeeting
<corebot>
achow101: Meeting started at 2025-09-18T16:00+0000
<fanquake>
sjors: how are you planning to handle version in the multiprocess subtree going forward?
<fanquake>
*versioning
<Sjors[m]1>
fanquake: do you mean versioning of the interface or of the multiprocess dependency?
<fanquake>
I'm guessing release branches to match Core? Otherwise I'm wondering what the plan is if you need to land bugfixes from multiprocess that wont apply to release branches, due to api changes or similar
<Sjors[m]1>
For the latter we should probably tag whatever is in v30
<fanquake>
We also don't generally backport subtree updates, so this would also be new for multiprocess
<sipa>
if a subtree has a bugfix that matters, it does seem reasonable to backport it
<fanquake>
Sure, but that might have to be selective, rather than a full pull
<sipa>
yeah, makes sense
<fanquake>
so by default isn't a clean backport
<fanquake>
so just wondering what the plan is there, and if anything is going to be done to track things upstream in the multiproess repo
<Sjors[m]1>
Maybe ryanofsky has a specific plan. I would imagine indeed having a seperate branch to track minimal changes if anything needs to be backported to v30.1+
<fanquake>
Ok, can leave it up to you and him to figure out going forward
<achow101>
#topic MuSig2 WG Update (achow101)
<achow101>
No major updates. PR to review is #29675 and I've been addressing comments.
<corebot>
https://github.com/bitcoin/bitcoin/issues/29675 | wallet: Be able to receive and spend inputs involving MuSig2 aggregate keys by achow101 · Pull Request #29675 · bitcoin/bitcoin · GitHub
Cory4 has joined #bitcoin-core-dev
<achow101>
#topic v30.0
<achow101>
Any new issues found in testing rc1?
<darosior>
I think we backported a few issues we found already, any plans on an rc2?
<l0rinc_>
for the record, I've compared V30 to previous releases (23-29) and it seems V30 is exactly 2x faster than v25 thanks to all the optimization efforts in the past releases, see https://x.com/L0RINC/status/1968392472717033927
Cory4 has quit [Ping timeout: 250 seconds]
<achow101>
#topic Support for Ubuntu 22.04 LTS as a build platform (hebasto)
<hebasto>
By the time Bitcoin Core v31.0 is released, Ubuntu 26.04 LTS will be available.
jon_atack has joined #bitcoin-core-dev
<hebasto>
Ubuntu 22.04’s default repositories provide dependency packages that are outdated and problematic to maintain, such as: CMake 3.22.1, Qt 6.2.4, Cap'n Proto 0.8.0
<hebasto>
Furthermore, Ubuntu 22.04’s default repositories do not provide the minimum required Clang version.
<hebasto>
My initial proposed action was to stop considering Ubuntu 22.04 as a build platform starting from the v31 release cycle.
<hebasto>
However, during conversation on IRC on 2025-09-09, sipa argued that we may end "in a situation where some important user chooses not to upgrade because they want to build from source, and they're stuck on an older platform"
<hebasto>
And fanquake pointed on such real world case where "a lot of Armory users are stuck on v27 branch".
<sipa>
do we actually have a "list of build platforms we support"?
<hebasto>
nope, I guess
<fanquake>
we've also since heard that a bunch of nodl (box?) users are also stuck on 27.x
<achow101>
why are they stuck?
jonatack has quit [Ping timeout: 258 seconds]
<hebasto>
Considering, that every problematic dependency package can be build by our own depends build subsystem, the remaining issue boils down to the minimum supported CMake version, which is 3.22 now.
<sipa>
my impression is more that for individual upgrades/dependency requirements, we have an ad-hoc discussion for benefits vs. which use/build platforms those would drop
<hebasto>
The question for my fellow collegues is can we ask users who will build Bitcoin Core v31+ from source on Ubuntu 22.04 to install newer CMake from https://cmake.org/download/ or from some other package manager?
<hebasto>
If so, we can bump the minimum supported CMake version in the v31 release cycle up to 3.25, same as in Debian Bookworm (oldstable).
<darosior>
Tangentially, i've had report of people stuck on 27 because of our glibc bump. fanquake also had some. I would suspect there is more. It probably means a lot of people that won't upgrade in case of vulnerability disclosures. I don't know the upsides to breaking support for some platform in this instance, but i'd like to underline there are
<darosior>
substantial downsides to doing so.
<sipa>
fanquake, darosior: but that's due to runtime requirements, not build time?
QUBIT_ABHAY has joined #bitcoin-core-dev
<fanquake>
Why do we need to bump the minimum required CMake version?
<hebasto>
This gives us a few useful features such as (1) file sets; (2) `COMPILE_WARNING_AS_ERROR`; (3) better support for IPO/LTO; (4) scope management
<sipa>
hebasto: can we use those feature optionally?
<darosior>
achow101: the nodl boxes are stuck because of our glibc bump in 28
<hebasto>
Very useful for ongoing libkernel development
<sipa>
(i'm guessing no from 1 and 4, yes for 2 and 3)
jadi has joined #bitcoin-core-dev
<fanquake>
These seem like things that might be mostly nice for cmake devs/
<fanquake>
*?
<achow101>
I would prefer that we support platforms that are still in use and supported by upstream
<hebasto>
sipa: some of them, yes
<achow101>
if ubuntu 22.04 is supported until 2027, i'd rather not drop support for it until it is eol
<darosior>
sipa: yes, my examples are runtime, that's why i said it's only tangentially related
<sipa>
achow101: agreed
<darosior>
achow101: +1
<hebasto>
achow101: users still able build on ubuntu 22.04; they will need to install newer cmake
<fanquake>
It'd atleast be good to see an example of changes that could be made after dropping. Hard to infer the benefits from that list
<fanquake>
Also, what version are you trying to bump the minimum to?
<achow101>
hebasto: I don't think asking users to update a dependency from external sources is a reasonable ask
<hebasto>
fair
<sipa>
specifically for (2), i think it's not unreasonable to enable it whenever cmake is recent enough, and not otherwise - which will give the benefit for most people who compile, but not break compatibility?
<fanquake>
Oh, I see 3.25
<fanquake>
sipa: not that we already also have that feature
<fanquake>
*note
QUBIT_ABHAY has quit [Ping timeout: 250 seconds]
<fanquake>
We just aren't using the CMake *thing*
<sipa>
fanquake: sure, but we can simplify the cmake code with that feature, i assume - which is still possible by using it optionally
<fanquake>
I'd think that'd be more code, if we retain the old way, and add the new way
<hebasto>
^ true
<fanquake>
unless you mean drop support of the feature for older users
<sipa>
fanquake: no, i mean use the new way, drop the old way
<hebasto>
that needs bump the minimum supported version
<sipa>
because people who care about the feature will almost certainly be on a new platform anyway (they're developers, not builders-on-old-platform-who-just-want-it-to-work)
<sipa>
hebasto: can you not enable it optionally?
<hebasto>
features that depends on cmake policy can be enabled optionally
<dzxzg>
Won't the flag just be ignored if used in a cmake version below 3.24?
<hebasto>
but also there are language features
<l0rinc_>
do we need to add a separate CI build with an old cmake version in that case?
<hebasto>
yes; with more coding
<sipa>
(if we're done with this topic, i'd like to also discuss the possibility of bringing back runtime compatibility with glibc 2.28 (or whatever it is that causes the reports of people unable to upgrade))
<achow101>
how difficult would it be to do that?
<fanquake>
Have to look at undoing a number of changes in Guix
<fanquake>
If people are trying to run on glibc < 2.28, I'm guessing this is on Ubuntu Bionic. Which shipped with 2.27
<hebasto>
if I recall correctly, there were issues with qt as well
<darosior>
One thing they could do to have new-bitcoind compatibility in their old machines would be to dockerize it. It's not impossible, but a bunch more work for them and therefore they're not doing it.
<fanquake>
Bionic went EOL sometime in 2023 I think
<fanquake>
Athough not sure we'd take that approach
<sipa>
fanquake: what are the downsides, besides it being less well tested?
Cory21 has quit [Quit: Client closed]
Cory21 has joined #bitcoin-core-dev
<fanquake>
Less tested, would need to check if all the targets we support would work
<darosior>
sipa: might not be consensus compatible
<fanquake>
Although I guess if we ship x86_64 and aarch64, that'd probably cover most users
* darosior
hides
<sipa>
darosior: only if there are bugs in it :p
<sipa>
another possibility is having separate static-linux and modern-linux release builds
<darosior>
Differential fuzzing across libc's is something that came up recently when chatting with external people looking into fuzzing Core
<fanquake>
would also mean splitting the guix build in two, either to do a lts & the current build, or some sort of static bitcoind + utils & gui separate
<sipa>
i'm less a fan of that due to splitting testing, though
<sipa>
fanquake: multiprocess gui lalala
<fanquake>
we are already shipping releases built against different libc++ flavours
<darosior>
fanquake: for MacOS, right?
<fanquake>
I will do a POC in any case
<sipa>
fanquake: cool, curious about what you find
<fanquake>
darosior: yea, clang & libc++ for macOS, gcc & libstdc++ for linux
<darosior>
_aj_: re "2024-04-22 on that graph claims 927010 txs per day" -> sure if you use a 1-day moving average but in this case you should rely on the adjustment mechanism, shouldn't you? It seems suboptimal to use the highest, least sustained, transaction rate as the default. Because it means every day of the year except one you will end up not limiting
<darosior>
anything?
Talkless has joined #bitcoin-core-dev
<sipa>
darosior: i think the number should be more picked based on what we see averaged over windows of maybe minutes or so?
<sipa>
which i'm guessing can be many times larger
jon_atack has quit [Ping timeout: 258 seconds]
jonatack has joined #bitcoin-core-dev
<darosior>
Isn't it more appropriate for short burst to rely on the dynamic relay rate mechanism, and choose the default for what's most commonly observed?
<sipa>
i think the dynamic rate is to protect against "inv queue growing unbounded", not for normal the this-is-fine rate
<sipa>
like, i assume the network can deal just fine with a fairly nontrivial multiple of the average rate
<sipa>
there's no reason to limit anything when the actual rate is below that, if we don't care about the hiding-size-of-queue privacy
l0rinc has quit [Quit: l0rinc]
l0rinc has joined #bitcoin-core-dev
dzxzg has joined #bitcoin-core-dev
<darosior>
Ok that makes sense to me, thanks.
<sipa>
11:01:15 < sipa> what about: send_now = std::min(to_send.size(), time_since_last_send * R + (to_send.size() ** 2) / M)
<darosior>
11:01 <sipa> where R is the "everyone can handle this tx rate" number, and M is the maximum queue size
jadi has joined #bitcoin-core-dev
<darosior>
So assuming R would be our current relay rate, time_since_last_send * R is what we do under normal conditions today. So this only differs in how we clear the queue faster when it gets too large, which today is `to_send.size() / M * 5`.
<fanquake>
Do we want to do this PR for 30? (#28592)
<darosior>
It kinda makes sense to go along with reducing the minrelaytxfee
bugs_ has quit [Quit: Leaving]
<fanquake>
Will put it on the milestone, can decide
<darosior>
(re sipa's formula) The real difference is taking the square of the size of the queue rather than multiplying it by 5. That makes sense, but does it make that much of a difference in practice? I guess it gives clearer guarantees we won't OOM instagibbs.
<sipa>
darosior: at low rates i think it's more consistent too (by scaling with actual time between sends, not just the average)
<sipa>
but that's a minor point
<dzxzg>
dumb question: why not just be a function of the queue size? is it to set a floor on the relay rate?
<darosior>
Ah right time_since_last_send isn't exactly what we do today
<sipa>
dzxzg: it is a function of the queue size right now
<sipa>
and yes, i think it should be
jadi has quit [Ping timeout: 244 seconds]
<sipa>
currently it is std::min(to_send.size(), B + (to_send.size() / 1000) * 5)
<darosior>
sipa: ACK you formula. I think it makes it easier to reason about the growth of the queue.
jonatack has quit [Read error: Connection reset by peer]
<sipa>
dzxzg: i think i don't understand your question
<dzxzg>
sipa: My question is why use `R` instead of only being a function of `M`
<dzxzg>
sorry not M I mean queue size
<dzxzg>
but I guess then the queue would not clear when it gets smal
jonatack has joined #bitcoin-core-dev
<sipa>
dzxzg: yeah, it'd mean the queue really never completely drains, which seems silly (for propagation speed, etc) when it's low enough that the network can just deal fine with it
jackielove4u has joined #bitcoin-core-dev
jadi has joined #bitcoin-core-dev
jon_atack has joined #bitcoin-core-dev
jonatack has quit [Ping timeout: 258 seconds]
l0rinc has quit [Quit: l0rinc]
jonatack has joined #bitcoin-core-dev
jon_atack has quit [Ping timeout: 256 seconds]
l0rinc has joined #bitcoin-core-dev
jonatack has quit [Ping timeout: 244 seconds]
jerryf has quit [Ping timeout: 272 seconds]
jerryf has joined #bitcoin-core-dev
Christoph_ has joined #bitcoin-core-dev
Cory21 has quit [Quit: Client closed]
l0rinc has quit [Quit: l0rinc]
Cory21 has joined #bitcoin-core-dev
enochazariah has quit [Quit: Connection closed for inactivity]
l0rinc has joined #bitcoin-core-dev
Christoph_ has quit [Quit: Christoph_]
Cory33 has joined #bitcoin-core-dev
Cory21 has quit [Ping timeout: 250 seconds]
Talkless has quit [Quit: Konversation terminated!]
conman has quit [Ping timeout: 258 seconds]
hernanmarino has joined #bitcoin-core-dev
enochazariah has joined #bitcoin-core-dev
Cory56 has joined #bitcoin-core-dev
Cory33 has quit [Ping timeout: 250 seconds]
thoragh has joined #bitcoin-core-dev
<hodlinator>
I've shaped up #33422 (the one added to the v30 milestone today) a bit and added some illustrative/motivating screenshots. Just waiting for fellow Windows fans to review. :)
<dzxzg>
ah, ok so the reason the current formula is so bad is that the inbound transaction rate (n) required to sustain a queue size of 1000
<dzxzg>
is n - R = 5
<dzxzg>
with sipa's formula it's n - R = M
<dzxzg>
(n = R + 5M/M and n = R + M^2/M)
<dzxzg>
and the time to clear (when n = 0)for a queue k if no new tx'es arrive is sqrt(M / R) * arctan ( k / sqrt(M * R) ) in sipa's suggested formula and (M / 5) * ln|1 + 5k/MR| in the current formula
<dzxzg>
with M=1000 and R=7: 11.95 * arctan(k/83.6) in the quadratic formula and 200 * ln|1 + k/1400| in the linear formula, with arctan(inf) = pi/2, the quadratic formula is bounded to never take longer than ~18.77 seconds to clear the queue if there are no inbound transactions
<sipa>
... arctan :o
<sipa>
(i didn't expect that function to show up here, not saying it's wrong!)
jadi has quit [Ping timeout: 256 seconds]
<dzxzg>
My math could definitely be wrong: I got that from integrating dk / (R + k^2/M) and using ∫dx/(a^2 + x^2) = 1/a * arctan(x/a) + C
conman has joined #bitcoin-core-dev
cman has joined #bitcoin-core-dev
conman has quit [Ping timeout: 256 seconds]
Cory56 has quit [Quit: Client closed]
jadi has joined #bitcoin-core-dev
Cory56 has joined #bitcoin-core-dev
kevkevin has joined #bitcoin-core-dev
kevkevin has quit [Remote host closed the connection]
jadi has quit [Ping timeout: 256 seconds]
thoragh has quit [Read error: Connection reset by peer]
<dzxzg>
ah ok, there was a little bit of a mistake in my equilibrium formula above for the current fomula: to sustain a queue size of 1000 you need n - R = 1
<dzxzg>
a new transaction rate of 8 tx/s would cause the queue to grow until it reached 1000 and then stay there
Christoph_ has joined #bitcoin-core-dev
enochazariah has quit [Quit: Connection closed for inactivity]
kevkevin has joined #bitcoin-core-dev
kevkevin has quit [Remote host closed the connection]
kevkevin has joined #bitcoin-core-dev
kevkevin_ has joined #bitcoin-core-dev
l0rinc has quit [Quit: l0rinc]
l0rinc has joined #bitcoin-core-dev
kevkevin has quit [Ping timeout: 258 seconds]
kevkevin_ has quit [Remote host closed the connection]
dzxzg has quit [Remote host closed the connection]
aleggg has quit [Remote host closed the connection]