ChanServ changed the topic of #bitcoin-core-dev to: Bitcoin Core development discussion and commit log | Feel free to watch, but please take commentary and usage questions to #bitcoin | Channel logs:, | Weekly Meeting Thursday @ 14:00 UTC | Meeting topics
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 260 seconds]
kevkevin has joined #bitcoin-core-dev
kevkevin_ has joined #bitcoin-core-dev
kevkevin has quit [Ping timeout: 268 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 260 seconds]
kevkevin_ has quit [Remote host closed the connection]
kevkevin has joined #bitcoin-core-dev
vincenzopalazzo has quit [Remote host closed the connection]
vincenzopalazzo has joined #bitcoin-core-dev
vincenzopalazzo has quit [Remote host closed the connection]
vincenzopalazzo has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 260 seconds]
kevkevin has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 252 seconds]
jonatack has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 268 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 264 seconds]
kevkevin has joined #bitcoin-core-dev
kevkevin has quit [Remote host closed the connection]
kevkevin has joined #bitcoin-core-dev
kevkevin has quit [Remote host closed the connection]
pablomartin has quit [Ping timeout: 260 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 255 seconds]
cmirror has quit [Remote host closed the connection]
cmirror has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 260 seconds]
BUSY has quit [Quit: Leaving]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 252 seconds]
Chris_Stewart_5 has quit [Ping timeout: 255 seconds]
Chris_Stewart_5 has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
benwestgate has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 264 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 256 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 260 seconds]
brunoerg has joined #bitcoin-core-dev
javi404 has quit [Ping timeout: 260 seconds]
javi404 has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 264 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 268 seconds]
PaperSword has joined #bitcoin-core-dev
zato has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 268 seconds]
salvatoshi has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 264 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 252 seconds]
Guyver2 has joined #bitcoin-core-dev
Guyver2 has left #bitcoin-core-dev [#bitcoin-core-dev]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 264 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 252 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 268 seconds]
dviola has joined #bitcoin-core-dev
Guest47 has joined #bitcoin-core-dev
Guyver2_ has joined #bitcoin-core-dev
Guest47 has quit [Client Quit]
mudsip has joined #bitcoin-core-dev
mudsip has quit [Client Quit]
ghost43 has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
ghost43 has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 256 seconds]
boris- has quit [Quit: ZNC 1.8.2 -]
AaronvanW has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 252 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 255 seconds]
vasild has quit [Ping timeout: 240 seconds]
vasild has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 264 seconds]
brunoerg has joined #bitcoin-core-dev
darosior6 has joined #bitcoin-core-dev
darosior has quit [Read error: Connection reset by peer]
darosior6 is now known as darosior
<glozow> instagibbs: right, I think EA anchor without its child should be removed from mempool, yes? (it’s always 0fee iirc so 27018 should suffice?)
kashifs has joined #bitcoin-core-dev
<bitcoin-git> [bitcoin] maflcko opened pull request #28972: test: Add and use option for tx-version in MiniWallet methods (master...2311-test-mw-version-)
<Sjors[m]> #proposedmeetingtopic: Stratum v2
<Sjors[m]> #proposedmeetingtopic Stratum v2
guest-127 has joined #bitcoin-core-dev
instagibbs_ has joined #bitcoin-core-dev
<achow101> #startmeeting
<glozow> hi
<RubenSomsen> hi
<hebasto> hi
<Murch[m]> Hi
<fjahr> hi
<achow101> #bitcoin-core-dev Meeting: achow101 _aj_ amiti ariard aureleoules b10c BlueMatt brunoerg cfields darosior dergoegge dongcarl fanquake fjahr furszy gleb glozow hebasto instagibbs jamesob jarolrod jonatack josibake kallewoof kanzure kouloumos kvaciral laanwj LarryRuane lightlike luke-jr MacroFake Murch phantomcircuit pinheadmz promag provoostenator ryanofsky sdaftuar S3RK stickies-v sipa theStack TheCharlatan vasild
<Sjors[m]> hi
<brunoerg> hi
<darosior> hi
<furszy> hi
<vasild> hi
<gleb> hi
<achow101> There are 2 pre-proposed meeting topics, any last minute ones to add to the list?
<kanzure> hi
<RubenSomsen> I'll do another silent payments update
<achow101> #topic package relay updates (glozow)
<sipa> hi
<glozow> 2 PRs are open for review: #28848 and #28948.
<glozow> The game plan is in 2 tracks mempool and p2p: v3 (#28948), then package RBF for 1p1c (#25038), then EA (#26403). And 1p1c package relay (#28970), then more robust orphan resolution (#28031), then orphanage protection.
<gribble> | bugfix, Change up submitpackage results to return results for all transactions by instagibbs · Pull Request #28848 · bitcoin/bitcoin · GitHub
<gribble> | v3 transaction policy for anti-pinning by glozow · Pull Request #28948 · bitcoin/bitcoin · GitHub
<gribble> | v3 transaction policy for anti-pinning by glozow · Pull Request #28948 · bitcoin/bitcoin · GitHub
<gribble> | policy: nVersion=3 and Package RBF by glozow · Pull Request #25038 · bitcoin/bitcoin · GitHub
<gribble> | policy: Ephemeral anchors by instagibbs · Pull Request #26403 · bitcoin/bitcoin · GitHub
<gribble> | [WIP] p2p: opportunistically accept 1-parent-1-child packages by glozow · Pull Request #28970 · bitcoin/bitcoin · GitHub
<gribble> | Package Relay 1/3: Introduce TxDownloadManager and improve orphan-handling by glozow · Pull Request #28031 · bitcoin/bitcoin · GitHub
<theStack> hi
<_aj_> hi
<glozow> The very exciting thing is we’ll have 1p1c package relay at the end of this. And then cluster mempool!
<b10c> hi
<achow101> cool
<achow101> #topic silent payments updates (RubenSomsen)
<RubenSomsen> Still actively taking review on BIP352, responding to feedback, and wanting review on #25273.
<gribble> | wallet: Pass through transaction locktime and preset input sequences and scripts to CreateTransaction by achow101 · Pull Request #25273 · bitcoin/bitcoin · GitHub
<RubenSomsen> A change we're considering in outpoint hashing (to make things simpler for hardware wallets) has an issue with forced collisions (worst case is address reuse), a possible fix is being discussed at
<RubenSomsen> Could use more eyes on that
<sipa> fwiw, we've been having some of our cluster mempool discussions on delving:
<maxedw> hi
<Murch[m]> Nice to see the discussion accessible in public!
<achow101> #topic multiprocess updates (ryanofsky)
Murch has joined #bitcoin-core-dev
<achow101> looks like the tracking issue was updated and several new PRs have been opened
<achow101> #28921 seems to be next
<gribble> | multiprocess: Add basic type conversion hooks by ryanofsky · Pull Request #28921 · bitcoin/bitcoin · GitHub
<ryanofsky> Hi, yes there are a few new prs opened
<ryanofsky> Currently working on a design doc which I want to post in a new PR this week
<ryanofsky> Tracking issue is if anyone is looking what to review
<achow101> #topic Ad-hoc high priority for review
<achow101> Anything to add or remove from
<gleb> can you add #28765?
<gribble> | p2p: Fill reconciliation sets (Erlay) by naumenkogs · Pull Request #28765 · bitcoin/bitcoin · GitHub
<achow101> gleb: added
<gleb> thanks
<_aj_> #27432 has concept acks, could be dropped from there, presumably?
<gribble> | contrib: add tool to convert compact-serialized UTXO set to SQLite database by theStack · Pull Request #27432 · bitcoin/bitcoin · GitHub
<maflcko> Can I have #28924 ? (It is a blocker for a bugfix)
<gribble> | refactor: Remove unused and fragile string interface from arith_uint256 by maflcko · Pull Request #28924 · bitcoin/bitcoin · GitHub
<hebasto> #26762 seems almost rtm?
<gribble> | bugfix: Make `CCheckQueue` RAII-styled (attempt 2) by hebasto · Pull Request #26762 · bitcoin/bitcoin · GitHub
<achow101> _aj_: maflcko: done
_flood has quit [Ping timeout: 260 seconds]
<maflcko> thanks
<theStack> can i have #28336 added?
<gribble> | rpc: parse legacy pubkeys consistently with specific error messages by theStack · Pull Request #28336 · bitcoin/bitcoin · GitHub
<achow101> hebasto: yes, going to try to get to that soon
<Murch[m]> Chasing concept acks:
<hebasto> achow101: thanks
<achow101> theStack: done
<Murch[m]> Also, related to my topic later
<_aj_> achow101: #28592 should be easy to review, and basically RTM
<gribble> | p2p: Increase tx relay rate by ajtowns · Pull Request #28592 · bitcoin/bitcoin · GitHub
<achow101> murch[m]: done. _aj_: will look
<Murch[m]> Thanks
<theStack> achow101: thx
Murch has quit [Quit: Connection closed]
<achow101> #topic Deficient coin selection behavior at high feerates (murch[m])
<Murch[m]> The mempool hasn’t cleared since end of April, and generally the blockspace market has seen a lot of demand in that time.
<Murch[m]> We recently had a period of time when the feerates peaked above 300 ṩ/vB, but they generally were significantly higher than usual for long stretches this year.
<instagibbs> _aj_ might be helpful to know what the potential downsides are to the PR (I have not thought about this stuff deeply)
<Murch[m]> Bitcoin Core wallet currently uses three different coin selection algorithms and generates up to ~12 candidate input sets from those in the first attempt, then picks the least wasteful among those per the waste heuristic.
<_aj_> instagibbs: ie, why there's a limit at all?
<instagibbs> _aj_ 👍
<Murch[m]> However, when wallets have a very large UTXO pool, it may be that even the "least wasteful" candidate set uses a ton of inputs.
<Murch[m]> We recently had another user submit disbelief when their Bitcoin Core wallet used over 500 inputs at a feerate of 75 sat/vB, when they had multiple UTXOs that could have funded the transaction by itself.
<Murch[m]> This wallet behavior is unexpected to users and can cause significant unnecessary cost
<Murch[m]> While adding a different coin selection algorithm would be a new feature and therefore not something that we’d put out in a point release, could we please at least aim to improve this behavior by the time of the next release?
<sipa> concept ack, but the devil is in the details
<Murch[m]> I have had a pull request open with #27877 since July that would minimize the weight of the input set on transactions over 30 sat/vB
<gribble> | wallet: Add CoinGrinder coin selection algorithm by murchandamus · Pull Request #27877 · bitcoin/bitcoin · GitHub
<Sjors[m]> Why don't any of the three algo's come up with a more sane result?
<Murch[m]> BnB doesn’t always have a solution, Knapsack optimizes for the least overshoot, i.e. minimizes the difference in satoshis between target+min_change and selection amount, and SRD is random.
<Murch[m]> If you have a huge number of UTXOs, with a few large ones, BnB might strike out, and the other two just give back crap.
<Murch[m]> It is by the way my opinion that Knapsack is utterly useless and should be destroyed.
<theStack> :D
<Murch[m]> And especially at high feerates e.g. above 30 sat/vB or 50 sat/vB, the wallet should be more thrifty for the sake of our users’ financial outcome.
<achow101> ack CoinGrinder, ack delete knapsack :)
<sipa> Sjors[m]: there is a difficulty here, i think, because the waste metric (which we nomiminally optimize for) doesn't account for "wallet composition health" so there is a concern that having something too minimizy might actually "win" (by waste metric) too much and result in a terrible wallet utxo set over the long term
<Murch[m]> Anyway, if people agree that it’s insane our wallet might use 500+ inputs above 75 sat/vB, when a single one suffices, I would appreciate if such people consider taking a look at #27877 which has been chasing concept review since July
<achow101> I'm slightly concerned that users are going to always want to use CoinGrinder since it should always produce small input sets, at the expense of grinding coins to dust
<gribble> | wallet: Add CoinGrinder coin selection algorithm by murchandamus · Pull Request #27877 · bitcoin/bitcoin · GitHub
<Murch[m]> Thanks :)
<sipa> coingrinder is much better at finding small inputs, but i'm a bit concerned that if we'd always enable coingrinder, long term wallets would degrade for exame
<Murch[m]> I have another idea, which would limit the input set to use 3 more inputs than the minimal necessary by iterating over a random shuffle of the UTXO pool and dropping the UTXOs with the least effective value until it has sufficient
<Murch[m]> I’ll probably be opening a PR with that in the next couple weeks
<fjahr> i guess there needs to be a plan for the extreme case where we never go below 30 sat/vB again (or whatever the limit will be)
<achow101> murch[m]: have you run simulations with CoinGrinder?
<Murch[m]> A very fragmented wallet would then probably end up using 3 more inputs than necessary, which would at least be a net-negative of 2 UTXOs per transaction (after accounting for the change) and it should be much simpler to review
<vasild> murch[m]: picking the one input instead of the 500 ones would create smaller/cheaper tx now, but does this not delay the usage of the 500 smaller inputs? it is inevitable, we would have to use them at some point. Plus at 80sat/vbyte maybe the best time to do that if the fee rates have been e.g. 150 sat/vbyte one week before and will be 150 one week later, e.g. 80 may be a temporary dip where it
<vasild> is good to spend the 500 small inputs
<achow101> vasild: unfortunately we haven't figured out how to accurately predictthefuture
<Murch[m]> vasild: Yes, if this point were the lowest feerate it would be good to use many, but that’s not even true across the last few weeks
<sipa> vasild: always using the largest utxo(s) necessary would result in smallest input set right now, but in the long term, grind down your wallet utxos to dust
<Murch[m]> Feerates have been going below 30 sat/vB nightly, and about a month ago were below 15 sat/vB every day
<vasild> sipa: yes
<Murch[m]> When the feerates are high, we should use few UTXOs, but many utxos at low feerates
<sipa> but on the other hand, at times of extremely high fee, nobody is going to want to pay for a hihe excess in input set just for some abstract long term wallet health concern - the price for it is just too high
<Murch[m]> Always using largest first is a terrible strategy, as I have show with my master thesis in 2016
<vasild> my point is "high" and "low" is relative, don't bind that to actual numbers, e.g. 75 is "high", because over time 75 may become the new "low"
<Murch[m]> CoinGrinder does not use largest first, it uses the input set with the least weight, and on ties prefers the one with a lower total amount
<_aj_> 500 inputs seems like a lot even when feerates are low
<instagibbs> 500 inputs is like.. we need a consolidate rpc
<sipa> vasild: we have the long-term fee estimate whose algorithm is currently "return 10 sat/vb;", but in theory, this value influences the waste metric
<_aj_> instagibbs: "sendall"
<Murch[m]> vasild: Yes, you’re right in principle, but 75 sat/vB is a high feerate across the last year, including the last few months.
<sipa> vasild: coingrinder in the current PR triggers based on a multiple of the long-term feerate estimate
<instagibbs> _aj_ that is triggered by utxo health metric or something :P
<_aj_> instagibbs: "bitcoin-cli wallet-wizard" - evaluates the health of your wallet utxos, and autoconsolidates
<sipa> i think i like this "look for randomish solution not exceeding N extra inputs over optimal" idea more than coingrinder, as i don't think it needs guards like "only at high feerate"
<sipa> instagibbs: if we'd have a wallet health metric we could optimize for it directly
<Murch[m]> Yeah, we can bikeshed over the exact strategy later, I mainly just would like to see if people agree that people unintentionally consolidating their wallet at high feerates is a bug
<sipa> i guess that is a bug, though not a regression
<achow101> I think we've known about this bug for about a decade
<_aj_> murch[m]: bug/misfeature, sure
<sipa> well i think it has become more concrete - it's been known since forever that coin selection is suboptimal
<fjahr> what sipa said, yes, but the devil is in the details
<sipa> but "it's spending two orders more UTXOs than necessary at very high feerate" is still something else than "suboptimal"
<_aj_> sipa: "do your utxo sizes follow a log-normal distribution" would be my guess at a metric
<Murch[m]> Yeah, personally I’d be pretty pissed if my wallet spent 25 mBTC in fees more than necessary.
<achow101> #topic Stratum v2 (Sjors[m])
<Sjors[m]> Basically I plan to take over #27854
<gribble> | [WIP] add a stratum v2 template provider by ccdle12 · Pull Request #27854 · bitcoin/bitcoin · GitHub
<Sjors[m]> I already managed to rebase it, will make a PR shortly.
<sipa> Sjors[m]: oh cool
<Sjors[m]> And then actually try understand what it's doing and read the spec :-)
<Sjors[m]> I have a little vintage s9 to test with too.
<achow101> have you talked to the original author?
<Sjors[m]> Nobody has in the past several months unfortunately
<Sjors[m]> I did talk to the Stratum v2 folks on their discord.
<Sjors[m]> There's a newer branch by Fi3 that I'm starting from.
<sipa> Sjors[m]: i'd at least comment on the PR to announce your intention
<Sjors[m]> Yes, and we can even leave that PR open for a bit.
<fanquake> sounds like someone has spoken to them in the past several months then?
<fanquake> and the response was "a six-month window for refactoring, adding a lot more tests and structuring the proposed changes in a way that is easier for contributors to review sounds doable to me."
<fanquake> what's the impetus to take the PR over?
<fanquake> Or is the above no-longer true, and the author no-longer working on it?
<Sjors[m]> fanquake: Pavlenex asking me to
<achow101> fanquake: he hasn't really responded to any of the previously left review
<maflcko> I think it would be better if you offered to the author to help or take it over. Just opening an alternative and closing seems a bit rushed, without any prior communication.
<sipa> yeah, i think it'd be good to discuss the plan, whatever it is, on the currently open PR
<sipa> maybe there is a miscommunication, or unclear expectations, because from that comment linked to, it doesn't sound like the author has given up on it at all
<Sjors[m]> Miscommnication is certainly possible.
<sipa> but the lack of activity is worrying
marcofleon has joined #bitcoin-core-dev
<Sjors[m]> I'll leave a link to my branch on that PR and wait to see what happens.
<achow101> any other topics to discuss?
<Sjors[m]> This is the branch people are currently testing with:
<Sjors[m]> Last commit from original author on that one is Oct 26, which isn't ages agao.
<sipa> right
<achow101> #endmeeting
kashifs has quit [Quit: Client closed]
guest-127 has quit [Quit: Client closed]
<bitcoin-git> [bitcoin] fanquake pushed 2 commits to master:
<bitcoin-git> bitcoin/master 66c4b58 fanquake: guix: switch from guix environment to guix shell
<bitcoin-git> bitcoin/master c4d47d2 fanquake: Merge bitcoin/bitcoin#26077: guix: switch from `guix environment` to `guix...
<bitcoin-git> [bitcoin] fanquake merged pull request #26077: guix: switch from `guix environment` to `guix shell` (master...guix_shell_over_environment)
<pinheadmz> #27375 has an ACK from vasild, willcl-ark_ and maflcko any time to rereview this week?
<gribble> | net: support unix domain sockets for -proxy and -onion by pinheadmz · Pull Request #27375 · bitcoin/bitcoin · GitHub
<fjahr> murch[m]: would you agree generally that the coin selection algorithm will only make all users happy in all feasible scenarios if the users provide additional information: i.e. “I think this is a high fee environment now” or “consolidate a bit because I think fees will get higher soon”? I vaguely remember discussions on passing additional information to coin selection but it was years ago. Not saying that this will
<fjahr> be easy but are there plans to go into this direction in the future?
<bitcoin-git> [bitcoin] fanquake pushed 3 commits to master:
<bitcoin-git> bitcoin/master fafcee4 MarcoFalke: ci: Rename test script to
<bitcoin-git> bitcoin/master fad82fe MarcoFalke: ci: Reduce use of bash -c
<bitcoin-git> bitcoin/master d80318d fanquake: Merge bitcoin/bitcoin#28954: ci: Reduce use of bash -c
<bitcoin-git> [bitcoin] fanquake merged pull request #28954: ci: Reduce use of bash -c (master...2311-ci-)
<bitcoin-git> [bitcoin] fanquake pushed 2 commits to master:
<bitcoin-git> bitcoin/master e67634e Sebastian Falbesoner: fuzz: BIP324: damage ciphertext/aad in full byte range
<bitcoin-git> bitcoin/master 05d3f8e fanquake: Merge bitcoin/bitcoin#28951: fuzz: BIP324: damage ciphertext/aad in full b...
<bitcoin-git> [bitcoin] fanquake merged pull request #28951: fuzz: BIP324: damage ciphertext/aad in full byte range (master...202311-fuzz-bip324-damage_in_full_byte_range)
<bitcoin-git> [bitcoin] fanquake pushed 4 commits to master:
<bitcoin-git> bitcoin/master 2d2ef2f Hennadii Stepanov: msvc: No need to specify the default feature for `libevent` package
<bitcoin-git> bitcoin/master 1f97e51 Hennadii Stepanov: msvc: Update vcpkg manifest baseline up to "2023.08.09 Release"
<bitcoin-git> bitcoin/master 6d05c4f Hennadii Stepanov: msvc: Specify `boost-date-time` package explicitly
<bitcoin-git> [bitcoin] fanquake merged pull request #28938: msvc: Update vcpkg manifest (master...231125-vcpkg)
<vasild> fjahr: indeed! I totally agree
darosior has quit [Ping timeout: 252 seconds]
<sipa> _aj_: distribution of UTXO values is one aspect of wallet UTXO health, but just the number of UTXOs matters too (and unfortunately, the optimal number probably depends on your activity - more frequent outbound payments may mean you need more UTXOs to not tie up your entire balance in in-flight transactions e.g.)
theStack has quit [Quit: theStack]
theStack has joined #bitcoin-core-dev
<sipa> fjahr: i'm not sure that making users responsible for guessing long-term feerate is a very relevant change (no objection to allowing that as an expert option, but i can't imagine many users to bother or have a good idea about it)
<bitcoin-git> [bitcoin] fanquake pushed 2 commits to master:
<bitcoin-git> bitcoin/master a4980da fanquake: guix: remove input labels
<bitcoin-git> bitcoin/master cdb7723 fanquake: Merge bitcoin/bitcoin#28965: guix: remove input labels
<bitcoin-git> [bitcoin] fanquake merged pull request #28965: guix: remove input labels (master...guix_drop_input_label)
darosior has joined #bitcoin-core-dev
<vasild> sipa: I think this is all about "guessing long-term feerate" and somebody has to do it, either the software or the user, or both
<_aj_> sipa: i'm not sure that's very meaningful: if your outgoing payments are all valued at 0.42 BTC (salary payments?), but your incoming coins are all much smaller (0.003 BTC textbook sales?), then every tx is going to spend 140 inputs, but that's not a bug, it's just how things are.
<fjahr> sipa: yes, but forcing the user to make a decision would lead to less surprises. Just asking the user to decide if it's a good time to consolidate or not would probably have solved the issue with the 500 inputs.
<sipa> fjahr: fair enough; i think something similar can be achieved by just offering an explicit consolidation option too
<vasild> a slider with "cheap" on one end and "consolidate" at the other end
<sipa> fjahr: i don't think the 500 inputs issue is due to not knowing long-term feerate
<fjahr> sipa: yeah, that would be the most simple option, but maybe there are more user friendly ideas
<sipa> we *have* a long-term feerate, it's not particularly sophisticated, but it does something
<bitcoin-git> [bitcoin] fanquake closed pull request #28846: depends: fix libmultiprocess build on aarch64 (master...fixup_multiprocess_arm64)
<sipa> the problem is that we have no coin selection algorithm that's really aimed at minimizing the input set; if we had one, it *would* win
<fjahr> sipa: The way i imagine it, the wallet would have restricted the number of inputs if the user had said, now is not a great time to consolidate
<sipa> fjahr: we run multiple coin selection algorithms, and then pick the best one we found, according to the waste metric
<sipa> an input with 500 inputs, today in a high-fee environment, *would be* considered terrible waste; it gets selected anyway because it's the only solution we found at all
<sipa> adding a coin selection algorithm that minimizes the number of inputs would be considered much better
<sipa> and there is a PR that does that, but at least my concern is that it - at much lower feerates - would still be picked
<fjahr> sipa: yes, that was discussed earlier, i am aware of the different coin selection algos on a high level. I guess they would still need to be composed in a better way and letting the user indicate if they think it's a good time to consolidate could help with that.
<fjahr> *when that new algo is added
<bitcoin-git> [bitcoin] fanquake opened pull request #28973: ci: remove `libz-dev` from macOS build deps (master...macos_drop_libz_dev)
baakeydow has joined #bitcoin-core-dev
<sipa> fjahr: my point is that the current problem is not due to not knowing long-term feerate
<sipa> that's an independent improvement
<fjahr> sipa: ack
flooded has joined #bitcoin-core-dev
marcofleon has quit [Quit: Connection closed]
zato has quit [Read error: Connection reset by peer]
realies has joined #bitcoin-core-dev
zato has joined #bitcoin-core-dev
<instagibbs> sdaftuar does it make sense to use bypass_limits to allow ephemeral anchor-having tx in on its own? Right now I think all ephemeral anchor txns are simply dropped because of one-by-one submission, which will fail in PreCheck. Then we can(or not) check if the anchor has been spent for filtering(or not).
<sdaftuar> i was just thinking about that issue myself this morning
<sdaftuar> i think that seems safe, but it's sort of a shame to add an extra condition to the filter that applies to all transactions, when we know the scope of what could be affected in this way is really limited to what was just re-added from a block.
<sdaftuar> probably the performance concern is negligible compared to what we're already doing though.
<instagibbs> I think we want the ephemeral txns to re-enter the mempool (hence bypass_limits), question is more should we filter, I think. Are there situations where a spend of the output wouldn't make it back in? I presume there are cases
<instagibbs> I'm less familiar with this section of the code
<sdaftuar> yes i think so
<sdaftuar> in a reorg, it's entirely possible that a spend of the EA output could be non-standard to your node
<instagibbs> ah yeah, miner mines something non-standard
<instagibbs> f.e.
<sdaftuar> yep
<instagibbs> let's say we're in a priority txn world but your node isn't, could definitely happen
<instagibbs> hmm
<sdaftuar> although i think glozow made the point somewhere that we'd evict 0-fee transactions, which would also fix that for you\
<instagibbs> Oh.... yeah
<instagibbs> well, unless a standard spend of *other* outputs are entered
<sdaftuar> good point!
<instagibbs> so you'd still have the dust
<sdaftuar> right, probably we should filter
<instagibbs> if we get this wrong, probably still mostly ok in that these anchors could be cleaned up by a nice miner 🙏
<instagibbs> but I agree, err on cleanliness
<_aj_> there's dust in the utxo set due to spending to 0 as a p2pkh; random dust created by reorgs doesn't seem like that big a problem?
<instagibbs> I could go either way. I don't want the work to get derailed arguing that some sweepable dust in some rare cases is ok, but if there's not a lot of pushback, I'm fine.
<_aj_> yeah, just agreeing that it's not a disaster if things go wrong
instagibbs_ has quit [Quit: Connection closed for inactivity]
realies has quit [Quit: ~]
<bitcoin-git> [bitcoin] ryanofsky pushed 6 commits to master:
<bitcoin-git> bitcoin/master fa7eb4f MarcoFalke: fuzz: Drop unused version from fuzz input format
<bitcoin-git> bitcoin/master fae00fe MarcoFalke: Remove unused CDataStream
<bitcoin-git> bitcoin/master fa0ae22 MarcoFalke: Remove unused SER_NETWORK, SER_DISK
<bitcoin-git> [bitcoin] ryanofsky merged pull request #28451: refactor: Remove unused SER_DISK, SER_NETWORK, CDataStream (master...2309-no-ser-hash-)
kevkevin has joined #bitcoin-core-dev
preimage has joined #bitcoin-core-dev
dviola has quit [Quit: WeeChat 4.1.1]
Guest56 has joined #bitcoin-core-dev
Guest56 has quit [Client Quit]
Guest18 has joined #bitcoin-core-dev
<bitcoin-git> [bitcoin] BrandonOdiwuor opened pull request #28974: doc: explain what the wallet password does (master...wallet_passphrase)
bugs_ has joined #bitcoin-core-dev
Guest18 has quit [Ping timeout: 250 seconds]
<fanquake> We've merged a few Guix related changes into master
<fanquake> As far as we can tell, this shouldn't break anything for anyone (tested with Guix 1.2.0 from ~3 years ago, and builds worked ok)
<fanquake> But if anyone who regular builds, wants to build master / happens to find any issues, please open an issue
Talkless has joined #bitcoin-core-dev
kevkevin has quit [Remote host closed the connection]
kevkevin has joined #bitcoin-core-dev
zato has quit [Quit: Om mani padme hum]
kevkevin has quit [Ping timeout: 256 seconds]
test_ has joined #bitcoin-core-dev
<sdaftuar> instagibbs: this is an implementation detail, but the way the mempool filter after a reorg works is that we evaluate each transaction in the mempool in isolation, and potentially mark it for removal. it's possible that an EA transaction would pass the filter (because it has a child spending its EA output) but the child gets marked for removal.
<sdaftuar> this would result in the EA transaction sticking around without a spend. to correctly filter those transactions, you have to look at them after the filtering has happened
<instagibbs> mmm right
flooded has quit [Ping timeout: 256 seconds]
<sdaftuar> hmm, i guess this is a more general issue than just reorgs -- whenever a tx is removed from the mempool, you have to consider whether an EA output is now unspent?
<sdaftuar> that seems somewhat tedious
<instagibbs> I don't think in normal operation it can happen. A spend being added is checked that it's spending all parent anchors(the one)
<sdaftuar> i think it can. suppose P is a parent transaction with an EA output, and a regular output. It gets added to the mempool with an EA spend. later, a higher feerate spend of the non-EA output is added, and eventually the EA spend transaction (which has lower descendant feerate now, or is in a lower chunk in the cluster mempool world) gets evicted
<instagibbs> > later, a higher feerate spend of the non-EA output is added
<instagibbs> that's two descendants of parent?
<sdaftuar> so that's parent P, child C1 (EA spend) and child C2 (non-EA spend)
<sdaftuar> P+C1 is added to the mempool initially; later C2 is added also spending another output of P.
<sdaftuar> C1 could be removed from the mempool due to eviction or RBF, for that matter
<instagibbs> you can't have two descendants of parent
<instagibbs> it's v3
<sdaftuar> ahh, ok
<instagibbs> and any child MUST\ spend the anchor
<instagibbs> """sibling eviction"""
<instagibbs> in a non-v3 world the exact requirement would be "only one direct child of ephemeral anchor tx", otherwise you'd end up as you say
<Murch[m]> fjahr: You can configure the long term feerate estimate, which would change the behavior in that manner
<sdaftuar> so this works because (a) it must be v3 (so only one child), (b) and EA transactions must have 0 fee (so that if its child ever disappears, it gets evicted from the mempool).
<sdaftuar> and then it's just in the reorg case where that last property might not hold
<instagibbs> 👍
kevkevin has joined #bitcoin-core-dev
<instagibbs> right now it gets evicted in reorg, but even in the case where it has a spend
<instagibbs> (I need a test for this, heh)
<sdaftuar> right, and with the bypass_limits extension we can fix that i guess
<Murch[m]> and as sipa said, yes the issue is that we can’t predict whether the feerate is lower than the future or not
<Murch[m]> Yet, even though we currently treat 75 s/vB as a high feerate, the input set selection doesn’t reflect that information
<instagibbs> sdaftuar hmmm think I can actually delete code. Thought I was missing test coverage from your line of questioning, realized I literally can't hit the check, assuming V3 is working properly
<instagibbs> nice
<instagibbs> well, except reorgs 😬
salvatoshi has quit [Ping timeout: 260 seconds]
salvatoshi has joined #bitcoin-core-dev
boris- has joined #bitcoin-core-dev
pablomartin has joined #bitcoin-core-dev
benwestgate has quit [Quit: Leaving.]
<bitcoin-git> [bitcoin] achow101 pushed 7 commits to master:
<bitcoin-git> bitcoin/master be4ff30 Hennadii Stepanov: Move global `scriptcheckqueue` into `ChainstateManager` class
<bitcoin-git> bitcoin/master d03eaac Hennadii Stepanov: Make `CCheckQueue` destructor stop worker threads
<bitcoin-git> bitcoin/master 9cf89f7 Hennadii Stepanov: refactor: Make `CCheckQueue` constructor start worker threads
martinus has joined #bitcoin-core-dev
<bitcoin-git> [bitcoin] achow101 merged pull request #26762: bugfix: Make `CCheckQueue` RAII-styled (attempt 2) (master...221228-queue)
kashifs has joined #bitcoin-core-dev
kashifs has quit [Client Quit]
benwestgate has joined #bitcoin-core-dev
kevkevin has quit [Remote host closed the connection]
Chris_Stewart_5 has quit [Ping timeout: 260 seconds]
Chris_Stewart_5 has joined #bitcoin-core-dev
<jonatack> Sjors[m]: cool, would be good to see progress on stratum v2. I sent a message linking to today's discussion.
Talkless has quit [Quit: Konversation terminated!]
<jonatack> murch[m]: thank you for the reminder about the coingrinder pull
kevkevin has joined #bitcoin-core-dev
<jonatack> With respect to, is there an advantage to having discussions there, rather than in a github issue? afaik the github metadata is backed up regurlarly -- it is good that the discussions take place publicly though :+1:
<_aj_> i find discussions in github issues/prs/gists get lost pretty easily; opening up topics/etc on a forum vs filing random issues seems less annoying; having admin control over adding plugins and moderation may be better than relying on github/etc.
<_aj_> jonatack: also, #28318...
<gribble> | logging: Simplify API for level based logging by ajtowns · Pull Request #28318 · bitcoin/bitcoin · GitHub
pablomartin has quit [Ping timeout: 255 seconds]
<jonatack> _aj_: yes, that pull is on my list, i've been prioritising the p2p bugfixes but will get there, ty for the reminder
<jonatack> _aj_: i like having the discussions in one place and haven't signed up yet for delving (it is better than the ML though)
<jonatack> i don't have the habit of looking in delving yet but have opened it a few times
pablomartin has joined #bitcoin-core-dev
<sipa> fwiw, delving is also archived
___nick___ has joined #bitcoin-core-dev
___nick___ has quit [Quit: - Chat comfortably. Anywhere.]
BlueMatt[m] has joined #bitcoin-core-dev
<BlueMatt[m]> apparently bitcoin core doesn't properly decode 0-input transactions which are encoded with the segwit marker byte when passed to decoderawtransaction?
<_aj_> they can be ambiguous, yeah
<BlueMatt[m]> rust-bitcoin deliberately encodes 0-input transactions with the marker bytes because it resolves the ambiguity
<BlueMatt[m]> if you have the marker bytes its not ambiguous
<jonatack> Sjors[m]: discussed with moneyball____ and he said they haven't been getting updated, "So Sjors bless his heart is helping"
<BlueMatt[m]> or, at least, if you know they're there its not ambiguous, but maybe still is if you're not sure? i long ago swapped this all out of my head :(
<_aj_> there's an "iswitness" flag for decoderawtransaction that you can use if you know they've been encoded with witness flag?
___nick___ has joined #bitcoin-core-dev
<sipa> BlueMatt[m]: BIP144 says you cannot use the extended encoding if there are no witnesses
___nick___ has quit [Client Quit]
<BlueMatt[m]> yes, but BIP144 is about consensus, and these are not consensus :p
<BlueMatt[m]> _aj_: ah, that's helpful, thanks
Guest77 has joined #bitcoin-core-dev
<jonatack> Sjors[m]: so if you don't see a reply soonish to your comment today at it sounds good to move forward
___nick___ has joined #bitcoin-core-dev
<sipa> BlueMatt[m]: no, BIP144 is P2P protocol; though fair enough, RPC isn't P2P either
<Sjors[m]> jonatack: thanks, I'll take a look tomorrow or Monday
Guest77 has quit [Client Quit]
<BlueMatt[m]> sipa: in general, somehow I had recalled bitcoin core's "heuristic" here to be "try both, and if only one manages to read the full buffer successfully use that"
<BlueMatt[m]> which would imply that setting the marker bytes is more likely to resolve correctly
<BlueMatt[m]> but i guess thats not what it does?
<sipa> BlueMatt[m]: yes, in case of ambiguity; but it will never permit marker bytes in case there is no witness, because that's just not valid
<_aj_> there's other heuristics like is the output amount sensible iirc
<BlueMatt[m]> sipa: I mean in general it seems like modern software really should be setting the marker bytes, exactly to avoid the ambiguity.
<sipa> BlueMatt[m]: nacl
<sipa> nack
<BlueMatt[m]> _aj_: ah, fair, but in any case that seems to imply it should work fine with zero-input-marker-bytes
<BlueMatt[m]> sipa: can you elaborate :)
<_aj_> nah, sipa's right; CTransaction deserialize just fails in that case
<BlueMatt[m]> right, okay, so ignoring for a moment what it does, what should it do :)
<sipa> i think it does the right thing
<BlueMatt[m]> yes, so I've provided my justification, please provide yours, then lets talk about the relative tradeoffs :)
<bitcoin-git> [bitcoin] achow101 opened pull request #28976: Migrate blank (master...migrate-blank)
<sipa> it accepts all valid encodings; for valid transactions that's unambiguous; if you have invalid things (like ones without inputs...) there format is ambiguous, so it guesses
<_aj_> could add a special case for !tx.HasWitness() && == 0
<sipa> i strongly disagree with having multiple encodings for the same data
___nick___ has quit [Ping timeout: 240 seconds]
<_aj_> better than than different data having the same encoding?
<BlueMatt[m]> so if you're writing a bitcoin parsing library, and you have to parse transactions with no context, and sometimes without a known length, istm you should almost certainly rely on the segwit marker for 0-input transactions
<sipa> i mean... why do you have transactions with 0 inputs anyway?
<BlueMatt[m]> cause people havent yet adopted psbt universally, sadly
<BlueMatt[m]> I mean if we try to decode all "0 inputs" as using the segwit marker byte from day one, I'd argue at this point we should be removing the legacy encoding logic for 0 inputs entirely today :)
<BlueMatt[m]> so long-term we'd end up with one encoding
<BlueMatt[m]> ah! its instagibbs' fault
<sipa> i guess we could say the correct serialization for transactions with 0 inputs but more than 0 outputs is using the extended format too there?
<BlueMatt[m]> that's what rust-bitcoin does
<_aj_> or we could say the correct format is psbt :)
<sipa> BlueMatt[m]: i don't care that much about 0-input transactions, but i'm vehemently against permitting extended encoding for *valid* transactions without witness
<BlueMatt[m]> cause we didnt want to deal with the ambiguity (which usually requires retrying deserialization to resolve, which is annoying)
<BlueMatt[m]> sipa: fair enough, yes, that makes sense
<BlueMatt[m]> _aj_: ha, well if you read the rust-bitcoin issue that was also the response lol
<sipa> let me see how much breaks if i change that
phantomcircuit_ has quit [Quit: ZNC -]
<BlueMatt[m]> yea, indeed, rust-bitcoin will reject transactions serialized with > 0 inputs with no witnesses
<achow101> BlueMatt[m]: have you heard about our lord and savior psbt?
<sipa> achow101: he has, but apparently some of his users haven't seen The Light yet.
<BlueMatt[m]> achow101: you can join the "just use psbt" bashing party at if you want :)
phantomcircuit has joined #bitcoin-core-dev
<bitcoin-git> [bitcoin] murchandamus opened pull request #28977: Add Gutter Guard Selector (master...2023-11-gutter-guard-selector)
<bitcoin-git> [bitcoin] ryanofsky opened pull request #28978: doc: Add multiprocess design doc (
Guyver2_ has left #bitcoin-core-dev [#bitcoin-core-dev]
<Murch[m]> Re today’s meeting. Here’s a draft of a stupid simple coin selection algorithm that curbs unnecessary fee spending at high feerates while still usually being slightly consolidatory. Basically, a bound on the worst case:
<instagibbs> BlueMatt[m] I remember literally none of this
<Murch[m]> cc: fjahr, Sjors, vasild, instagibbs
<instagibbs> I need a gutter guard, how did you know
martinus has quit [Quit: Konversation terminated!]
<Murch[m]> I’ll add some simulations by next week
<BlueMatt[m]> <instagibbs> "Matt Corallo I remember literall..." <- convenient excuse
flooded has joined #bitcoin-core-dev
test_ has quit [Ping timeout: 276 seconds]
kevkevin has quit [Remote host closed the connection]
preimage has quit [Quit: WeeChat 4.1.1]
kevkevin has joined #bitcoin-core-dev
kevkevin has quit [Ping timeout: 268 seconds]
pablomartin has quit [Ping timeout: 245 seconds]
pablomartin has joined #bitcoin-core-dev
benwestgate has quit [Quit: Leaving.]
bugs_ has quit [Quit: Leaving]
<bitcoin-git> [bitcoin] ishaanam opened pull request #28979: wallet, rpc: document and update `sendall` behavior around unconfirmed inputs (master...sendall_ancestor_aware_funding)
AaronvanW has quit [Ping timeout: 252 seconds]
raini_ok has joined #bitcoin-core-dev
pablomartin has quit [Ping timeout: 256 seconds]
Guest16 has joined #bitcoin-core-dev
Guest16 has quit [Client Quit]