< fanquake> sipa: I often see similar timeouts with descriptor related tests
< fanquake> Ideally the CreateWallet tests shouldn't take > 3 minutes, regardless of disk speed
< achow101> phantomcircuit: do you happen to know what the actual difference between synchronous=FULL and synchronous=NORMAL is?
< sipa> according to the docs, normal doesn't always sync, and thus has a small risk of corruoting the database if power fails at just the wrong timr
< sipa> (unless WAL, which is safe in combination with normal)
< bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/cb79cabdd9d9...2e9031f95d2b
< bitcoin-git> bitcoin/master 94c7dd9 Yerzhan Mazhkenov: doc: Fix typos from codespell lint
< bitcoin-git> bitcoin/master 2e9031f fanquake: Merge #21626: doc: Fix typos from codespell
< bitcoin-git> [bitcoin] fanquake merged pull request #21626: doc: Fix typos from codespell (master...master) https://github.com/bitcoin/bitcoin/pull/21626
< achow101> it looks like normal is 1 sync while full is 2 syncs. but changing to normal still results in a timeout
< achow101> i guess for tests we could just do synchronous=off
< midnight> sipa: ping, can you re-center your hashrate graphs?
< bitcoin-git> [bitcoin] achow101 opened pull request #21634: tests: Skip SQLite fsyncs while testing (master...optimize-sqlite) https://github.com/bitcoin/bitcoin/pull/21634
< bitcoin-git> [bitcoin] fanquake closed pull request #18194: Bugfix: GUI: Remove broken ability to edit the address field in the sending address book (master...bugfix_gui_edit_sendaddr) https://github.com/bitcoin/bitcoin/pull/18194
< phantomcircuit> achow101, no i don't, it's pretty vague
< bitcoin-git> [bitcoin] fanquake closed pull request #18909: [0.20] Fix release tarball (0.20...fix_release_tarball-0.20) https://github.com/bitcoin/bitcoin/pull/18909
< bitcoin-git> [bitcoin] fanquake closed pull request #19120: QA: Support making RPC calls with different authentication (master...qa_rpc_with_auth) https://github.com/bitcoin/bitcoin/pull/19120
< bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/2e9031f95d2b...6664211be2b6
< bitcoin-git> bitcoin/master 9044522 Russell Yanofsky: Drop JSONRPCRequest constructors after #21366
< bitcoin-git> bitcoin/master 6664211 MarcoFalke: Merge #21574: Drop JSONRPCRequest constructors after #21366
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #21574: Drop JSONRPCRequest constructors after #21366 (master...pr/simpreq) https://github.com/bitcoin/bitcoin/pull/21574
< bitcoin-git> [bitcoin] MarcoFalke reopened pull request #21445: cirrus: Use SSD cluster for speedup (master...2103-ciSSD) https://github.com/bitcoin/bitcoin/pull/21445
< bitcoin-git> [bitcoin] MarcoFalke closed pull request #19048: test: changing signature of wait_until(). (master...fixing-waituntil) https://github.com/bitcoin/bitcoin/pull/19048
< Kiminuo> sdaftuar, Hi, is there a reason why lines L288 and L289 were not deleted in PR #12127 (https://github.com/bitcoin/bitcoin/pull/12127/files#diff-f4f454edbfa1a878a9d4ad4371611c6c2812b7c77d2bd01baa66606524b61f9eL290)?
< gribble> https://github.com/bitcoin/bitcoin/issues/12127 | Remove unused mempool index by sdaftuar · Pull Request #12127 · bitcoin/bitcoin · GitHub
< NiamhOnMars> hi I was wondering why the original bitcoin code had some irc related code to join the #bitcoin IRC channel
< elichai2> NiamhOnMars: IIRC it was used to find other peers (before the dns seeding)
< _flow_> that
< luke-jr> #proposedmeetingtopic attempts to use "dev muscle" to force MTP against community consensus of BIP8
< wumpus> #startmeeting
< jonasschnelli> hi
< sipsorcery> hi
< hebasto> hi
< wumpus> #bitcoin-core-dev Meeting: achow101 aj amiti ariard bluematt cfields Chris_Stewart_5 digi_james dongcarl elichai2 emilengler fanquake fjahr gleb glozow gmaxwell gwillen hebasto instagibbs jamesob jb55 jeremyrubin jl2012 jnewbery jonasschnelli jonatack jtimon kallewoof kanzure kvaciral lightlike luke-jr maaku marcofalke meshcollider michagogo moneyball morcos nehan NicolasDorier paveljanik
< wumpus> petertodd phantomcircuit promag provoostenator ryanofsky sdaftuar sipa vasild wumpus
< jonatack> hi
< amiti> hi
< fjahr> hi
< jeremyrubin> hallo
< phantomcircuit> uhm hello
< achow101> hi
< jnewbery> hi
< luke-jr> hi
< wumpus> one proposed meeting topic: attempts to use "dev muscle" to force MTP against community consensus of BIP8 (luke-jr)
< wumpus> any last minute suggestions?
< wumpus> (as a reminder: you can propose a meeting topic with #proposedmeetingtopic <topic> at any time during the week)
< wumpus> #topic High priority for review
< core-meetingbot> topic: High priority for review
< wumpus> https://github.com/bitcoin/bitcoin/projects/8 has 11 blockers, 2 chasing concept ACK
< wumpus> anything to add, remove, or that is ready to merge?
< jonatack> #21392 seems closed, remove?
< gribble> https://github.com/bitcoin/bitcoin/issues/21392 | Implement BIP 8 based Speedy Trial activation by achow101 · Pull Request #21392 · bitcoin/bitcoin · GitHub
< wumpus> jonatack: looks like it
< jonatack> (not opining, just info)
< achow101> remove for now
< wumpus> should we add #21377 instead?
< gribble> https://github.com/bitcoin/bitcoin/issues/21377 | Speedy trial support for versionbits by ajtowns · Pull Request #21377 · bitcoin/bitcoin · GitHub
< luke-jr> replace with #19573
< gribble> https://github.com/bitcoin/bitcoin/issues/19573 | Replace unused BIP 9 logic with draft BIP 8 by luke-jr · Pull Request #19573 · bitcoin/bitcoin · GitHub
< luke-jr> #21377 is NACK
< gribble> https://github.com/bitcoin/bitcoin/issues/21377 | Speedy trial support for versionbits by ajtowns · Pull Request #21377 · bitcoin/bitcoin · GitHub
< achow101> We can add 21377 to hi prio
< luke-jr> it would be an abuse of position to merge 21377
< wumpus> ok never mind...
< wumpus> anything else?
< achow101> #17331 for me
< gribble> https://github.com/bitcoin/bitcoin/issues/17331 | Use effective values throughout coin selection by achow101 · Pull Request #17331 · bitcoin/bitcoin · GitHub
< jeremyrubin> +1 #21377 for high priority for review
< gribble> https://github.com/bitcoin/bitcoin/issues/21377 | Speedy trial support for versionbits by ajtowns · Pull Request #21377 · bitcoin/bitcoin · GitHub
< wumpus> achow101: added
< jeremyrubin> I think irrespective of if it is merged, reviewing it is high priority
< wumpus> in the end it's up to the author (ajtowns) if he wants it has high prio, but you're right that is separate from whether it can be merged or not
< jeremyrubin> he can ack adding it after the meeting, unclear if he's present
< wumpus> yes
< wumpus> that concludes high priority for review unless anyone else has suggstions
< wumpus> #topic Attempts to use "dev muscle" to force MTP against community consensus of BIP8 (luke-jr)
< core-meetingbot> topic: Attempts to use "dev muscle" to force MTP against community consensus of BIP8 (luke-jr)
< jonatack> #19521 freshly needs rebase but should be close to ready, it's been through a number of review rounds by provoostenator and i
< gribble> https://github.com/bitcoin/bitcoin/issues/19521 | Coinstats Index by fjahr · Pull Request #19521 · bitcoin/bitcoin · GitHub
< luke-jr> I find it quite disturbing a few devs are attempting a NYA-like push to make consensus changes outright disregarding the community consensus around BIP 8.
< wumpus> jonatack: good to know!
< jeremyrubin> As the author behind the quoted "dev muscle", I would like to note that the context in which it was used was referring to sufficient resources to review and audit the corresponding PR
< luke-jr> and for no real apparent reason but to spite UASFers by reverting all the improvements made over 2017 - widely acknowledged as improvements until LOT=True began to move forward
< jeremyrubin> It is not in the context of devs muscling the community, which i think it has been represented to be in the setting of this topic
< jnewbery> As usual, luke-jr is misrepresenting
< luke-jr> no, I am not
< luke-jr> nor is that usual
< achow101> there is a person who has raised an unaddressed objection to BIP 8, therefore it does not have consensus
< luke-jr> jnewbery: I don't know what you have against me, but trolling liek that is unproductive and malicious.
< jeremyrubin> anyone is free to review the logs for relevent context http://gnusha.org/taproot-activation/2021-04-06.log
< jnewbery> at the very least you're misrepresenting the intent of jeremy's words
< luke-jr> achow101: even if that were true, all but one person is still far more than this MTP nonsense
< luke-jr> jnewbery: read the log
< jnewbery> I've read it, thanks
< achow101> at this time, 21377 is in a state where everyone except luke-jr is okay with it, and onlyu luke-jr has stated "no" and "nack" without providing any reasoning behind those statements
< achow101> simply asserting "no" and "nack" is not an objection
< jnewbery> achow101: +1
< luke-jr> achow101: that isn't true
< luke-jr> there is practically no community support for 21377, just a handful of devs
< jeremyrubin> A NACK needs to include a rationale why the change is not worthwhile. NACKs without accompanying reasoning may be disregarded.
< luke-jr> and most of you acknowledge height is better
< achow101> luke-jr: so all of the people who participated in the meeting yesterday or all devs?
< achow101> *are
< luke-jr> achow101: I didn't say that, but it's far fewer than the consensus around BIP8
< luke-jr> not even comparable
< jeremyrubin> numerous ACKs on https://gist.github.com/michaelfolkson/92899f27f1ab30aa2ebee82314f8fe7f#gistcomment-3695024 which is nonspecific to height or MTP
< luke-jr> the whole point of ST in the first place, was to be a neutral start between the simple disagreement on LOT
< luke-jr> throwing away the consensus on everything else defeats the point
< luke-jr> jeremyrubin: that gist was about height
< jnewbery> luke-jr: most people have stopped responding to your outlandish claims because there's almost no point. There's no evidence that your proposal has "community support"
< luke-jr> jnewbery: far more evidence than this MTP nonsense
< jeremyrubin> which says: " The idea can be implemented on top of either Bitcoin Core's existing BIP9 code or its proposed BIP8 patchset.[6]"
< jnewbery> you've had plenty of opportunity to make your case already. What exactly do you hope to achieve in this meeting by restating things you've claimed many times before in many previous meetings?
< jeremyrubin> I'd also invite those unfamiliar to review a different community's operating procedure https://tools.ietf.org/html/rfc7282
< achow101> there have been a few people who pop in and nack 21377, however those nacks are always unsubstantiated and no response is provided upon asking for the motivation behind those nacks
< luke-jr> the problems with MTP are well-known and shouldn't need ot be repeated constantly
< jeremyrubin> I think aj and achow101 adressed them in a very elegent manner
< jeremyrubin> so i do not beleive any of the past issues are unaddressed
< jeremyrubin> if you would like to contest the solutions as they are, you should review the current proposed changes and comment on that
< luke-jr> hack after hack is not addressing issues. nor does it justify disregarding community consensus.
< achow101> luke-jr: you should repeat them in the context of this proposal specifically. Many contributors/reviewers may not be aware of them because either they occurred in a venue that they were not aware of, or they occurred at a time where they were not active participants in development.
< achow101> if they have already been stated in the context of the proposal being reviewed, then please link them
< jeremyrubin> further, there does seem to be a substantial body of agreeing devs on the approach put forward in 21377. so there can hardly be said to be consensus for any other approach either
< jnewbery> luke-jr: there is no evidence that your proposal has "community consensus"
< luke-jr> jnewbery: that is not true
< achow101> asserting it does not make it true or false
< jnewbery> luke-jr: I'll ask again: what is your aim for this meeting item? You've had a chance to raise your objections.
< achow101> "What can be asserted without evidence can also be dismissed without evidence."
< luke-jr> anyone involved in the community knows there is consensus around BIP8.
< jeremyrubin> I don't have anything else to add on this matter; I think it should be relatively clear what sort of communications would be required to raise a tangible objection to 21377.
< aj> wumpus: (thanks for adding 21377 to hi pri)
< jeremyrubin> #proposedmeetingtopic meeting notes & timelines implied
< luke-jr> wumpus: reminder to add #19573 too
< wumpus> #topic Meeting notes & timelines implied (jeremyrubin)
< core-meetingbot> topic: Meeting notes & timelines implied (jeremyrubin)
< gribble> https://github.com/bitcoin/bitcoin/issues/19573 | Replace unused BIP 9 logic with draft BIP 8 by luke-jr · Pull Request #19573 · bitcoin/bitcoin · GitHub
< wumpus> luke-jr: ok added
< wumpus> jeremyrubin: or was that a topic fo rnext week?
< wumpus> #endmeeting
< core-meetingbot> topic: Bitcoin Core development discussion and commit log | Feel free to watch, but please take commentary and usage questions to #bitcoin | Channel logs: http://www.erisian.com.au/bitcoin-core-dev/, http://gnusha.org/bitcoin-core-dev/ | Meeting topics http://gnusha.org/bitcoin-core-dev/proposedmeetingtopics.txt / http://gnusha.org/bitcoin-core-dev/proposedwalletmeetingtopics.txt
< core-meetingbot> Meeting ended Thu Apr 8 19:34:35 2021 UTC.
< jnewbery> thanks wumpus!
< jeremyrubin> Hi
< jeremyrubin> not sure if my previous messages made it through
< jeremyrubin> i got booted right after proposing a meeting topic and was unable to rejoin/reauth
< aj> nothing after your meeting topic made it through, so wumpus ended the meeting
< jeremyrubin> wumpus: it was intended to be for today
< jeremyrubin> aj: do you think it needs to be discussed today?
< jeremyrubin> or is next week sufficient?
< aj> jeremyrubin: could just ask outside of the meeting? waiting 'til next week probably answers the question on its own though...
< jeremyrubin> I think more or less -- for those who are still reading -- the point is that the rough timeline from the 1st meeting still seems viable, so short of something unexpected that's probably what people are working towards
< jnewbery> jeremyrubin: what is that timeline?
< jeremyrubin> starttime is relatively flexible, as with endtime (nothing wrong with capturing signals earlier because min active height)
< jnewbery> thanks!
< jeremyrubin> the activation height should just be something around 707616, but the exact value doesn't matter that much
< jeremyrubin> (well, it does matter, we can pick whatever n*2016 looks like it will be near mid novemeber)
< bitcoin-git> [bitcoin] laanwj pushed 5 commits to master: https://github.com/bitcoin/bitcoin/compare/6664211be2b6...0c9597ce7db2
< bitcoin-git> bitcoin/master 84912d4 Carl Dong: build: Remove spaces from variable-printing rules
< bitcoin-git> bitcoin/master 44f6d4f Carl Dong: guix: Record precious directories and add guix-clean
< bitcoin-git> bitcoin/master 8f8b96f Carl Dong: guix: Update hint messages to mention guix-clean
< bitcoin-git> [bitcoin] laanwj merged pull request #21304: guix: Add guix-clean script + establish gc-root for container profiles (master...2021-02-guix-clean-script) https://github.com/bitcoin/bitcoin/pull/21304
< jeremyrubin> does testmempoolaccept handle the case of [A -> B, B->C, C->D] etc?
< jeremyrubin> or is each considered independently
< glozow> jeremyrubin: what do you mean?
< jeremyrubin> (Not packages)
< glozow> are A, B, C, and D transactions?
< jeremyrubin> outputs
< jeremyrubin> like assuming A->B is testmempool accept = true
< glozow> what does A->B mean?
< jeremyrubin> can B->C be true?
< jeremyrubin> Some output A creates some output B
< jeremyrubin> can there be dependencies between the txs in the array
< glozow> yes
< glozow> with #20833 i mean
< gribble> https://github.com/bitcoin/bitcoin/issues/20833 | rpc/validation: enable packages through testmempoolaccept by glozow · Pull Request #20833 · bitcoin/bitcoin · GitHub
< jeremyrubin> packages is a different thing
< glozow> on master you can only submit 1 at a time
< jeremyrubin> that's considering CPFP
< jeremyrubin> 1. rawtxs (json array, required) An array of hex strings of raw transactions.
< glozow> rawtxs can only have 1 tx on master
< jeremyrubin> ahh ok "Length must be one for now."
< luke-jr> jeremyrubin: how is this distinct from packages?
< jeremyrubin> packages is a superset of this
< luke-jr> ?
< jeremyrubin> I'm not interested in applying CPFP logic
< glozow> do you define packages as just an array of transactions?
< luke-jr> does 20833 do CPFP?
< sipa> cpfp is a transaction selection policy
< luke-jr> "the expectation is that the results returned from testmempoolaccept are identical to what you'd get from test-then-submitting each transaction individually, in that order"
< sipa> that's independent from menpool acxwptance policies
< luke-jr> 20833's description suggests it doesn't do CPFP logic
< glozow> CPFP happens after mempool submission
< jeremyrubin> testmempoolaccept can fail due to insufficient fee tho correct?
< sipa> cpfp doesn't apply to mempool accwptance
< luke-jr> jeremyrubin: yes
< sipa> jeremyrubin: tgat's where packages come in
< jeremyrubin> that's my point
< glozow> yes, but it's a small addition to make `testmempoolaccept` use the descendant feerate instead of base feerate
< luke-jr> …
< sipa> jeremyrubin: so i have no idea what you're asking about
< jeremyrubin> testmempoolaccept does check fees
< sipa> yes
< glozow> yes, obviously
< luke-jr> jeremyrubin: for each tx, individually
< sipa> for now
< jeremyrubin> well only one tx is allowed for now anyways
< glozow> testmempoolaccept only takes 1 transaction right now.
< luke-jr> sipa: if it does anything different later, it should be optional IMO
< glozow> are we talking about behavior or master or?
< jeremyrubin> so if I have A -> B paying 10000 feerate, i'm not worried about if it can go in or not
< jeremyrubin> based on fees
< luke-jr> I imagine the use case is, you want to submit A now, B later
< jeremyrubin> but I still want to test if B -> C is valid
< luke-jr> and not wait for B, for A to get confirmed
< glozow> yes
< jeremyrubin> so I'm trying to figure out how I can test the validity of a child transaction from a parent I don't want in the mempool
< jeremyrubin> i just want to test it
< luke-jr> jeremyrubin: using that PR
< glozow> then use 20833
< jeremyrubin> Ok, but there's no current way to do this?
< luke-jr> jeremyrubin: if you need to bypass fee checks, use Knots
< glozow> no, there isn't
< jeremyrubin> no the fees aren't important
< glozow> except to disconnect your node
< jeremyrubin> I'm asking about the dependency graph
< luke-jr> actually, I didn't get 20833 in Knots yet :/
< jeremyrubin> glozow: that's probably the best bet for now
< luke-jr> jeremyrubin: then 20833 is enough…?
< glozow> 20833 works with dependencies
< jeremyrubin> 20833 would do it, I was just asking if there's something that works today that doesn't cover CPFP
< luke-jr> aha
< luke-jr> but 20833 does work today ;)
< glozow> :D
< glozow> needs review
< jeremyrubin> I'll take a look
< glozow> thanks!
< jeremyrubin> I understood 20833 to be focused on the CPFP issue
< jeremyrubin> but it's clear it's even more important now :)
< glozow> 🙏
< luke-jr> jeremyrubin: 20833 is explicitly not CPFP related
< jeremyrubin> does it apply CPFP rules?
< glozow> CPFP rules are applied during block template building
< glozow> i guess the answer to your question is no
< glozow> *not yet
< jeremyrubin> Ok, so the testmempoolaccept 20833 will fail if tx 0 of a chain pays low fee?
< glozow> yes, if the base feerate is too low, it will fail
< glozow> the fix for that is to just use descendant feerate instead, which I'm saving for a followup to 20833
< jeremyrubin> great, that answers my q
< glozow> cool!
< jeremyrubin> FWIW i think the way I define packages is cuts through the transaction graph which can be mined within a block
< jeremyrubin> so an array of txns is not a package nesc because there are arrays of txns which can't mine within a block (relative time locks, too many MB, etc)
< glozow> so if there are conflicts -> not a package?
< jeremyrubin> example?
< jeremyrubin> like [A -> B, A->C]?
< glozow> [A->B, A->C]
< glozow> yeah
< jeremyrubin> yeah I don't consider that a package?
< glozow> mm
< glozow> what about [B->C, A->B] ?
< jeremyrubin> well I think that depends on what your brackets mean
< glozow> i.e., does order matter?
< jeremyrubin> I think it would be wise to make sure that core only ever outputs packages in increasing ancestor order
< jeremyrubin> but that we should accept them in any order
< jeremyrubin> in particular, when mining you generate them in reverse order
< luke-jr> huh?
< Murch> hum?
< glozow> ?
< jeremyrubin> When you're mining you pick the thing with the best feerate and then take all the ancestors
< Murch> Nope
< jeremyrubin> and then later you sort them into ancestor order
< Murch> Transactions must be sorted in topological order
< glozow> you pick by ancestor feerate, and you mine the ancestors first
< jeremyrubin> yeah you sort them after
< jeremyrubin> but the packages are visited reverse order
< Murch> You pick whole ancestor packages
< glozow> is that necessarily true? you could have parent with higher ancestor feerate than child, and you'd pick the parent first
< jeremyrubin> it is true -- you won't pick the child till later then
< Murch> Yeah, but then the parent would have a higher ancestor feerate and take precendence by itself anyway
< jeremyrubin> anyways it's not super important, once they're in the block they're in consensus order
< Murch> It groups each transaction with all of the transactions it depends on
< Murch> Then it picks the best of these ancestor packages per ancestorfeerate (i.e. the effective feerate of the whole package)
< Murch> then updates the mempool
< jeremyrubin> I don't see that in the code?
< jeremyrubin> Can you point me to a line
< jeremyrubin> What i see is:
< jeremyrubin> pick by ancestor score
< jeremyrubin> make sure the package fits
< Murch> Yeah, but transactions without ancestors are an ancestorpackage by themselves
< jeremyrubin> compute all the ancestors
< jeremyrubin> add them to the block in block sort order
< Murch> yes?
< Murch> But the "ancestorscore" is the effective feerate of a transaction with its ancestors
< glozow> before block assembly starts, the txns already know their ancestor fees and size. you just call `CalculateMempoolAncestors` to grab the mempool entries
< jeremyrubin> Yes, and the order those are visited in shows up in a reversed order
< jeremyrubin> this is relevent because the epoch memepool patches will literall put them in order of [B -> C, A->B]
< jeremyrubin> So gloria's question is, is [B->C, A->B] still a package, or must it be [A->B, B->C]
< jeremyrubin> my answer is diff algorithms will generate a package set in different orders
< jeremyrubin> so we should recognize any order they show up in
< glozow> i mean a package communicated between peers
< jeremyrubin> but that we should also be careful to pick a single order if we ever e.g. print them out
< glozow> that's what we'd be interested in formalizing the definition for
< jeremyrubin> it's a good question, there's a DoS argument for picking a defined order because it's O(n log n) to sort self-side v.s. O(n) to verify
< glozow> what's n?
< Murch> Number of txs?
< jeremyrubin> I think n is the number of inputs
< jeremyrubin> but maybe it's just # of txs, would need to draw out the actual dependency calculating algorithm
< glozow> no, it'd depend on number of inputs
< jeremyrubin> there's also a good question of if we should have a canonical ordering that a package must be in, more so than block order
< jeremyrubin> since blocks are not canonically ordered
< Murch> Do you mean topologically ordered but with canonical order when there is multiple possibilities?
< jeremyrubin> correct
< jeremyrubin> e.g., input/vout order
< glozow> it would matter if you had package IDs
< jeremyrubin> glozow: good point, should prolly make a canonical ordering then :)
< jeremyrubin> glozow: I'd say figure out if there's any algorithmic advantage for having the txns in a specific order on the receiving side
< glozow> i'd say require them to be ordered
< glozow> it's pretty easy to figure out if a package is ordered correctly
< Murch> Well, topological order would be nice, not sure whether it's worthwhile to further restrain it
< glozow> should be the sender's burden to sort them
< jeremyrubin> Murch: package IDs
< jeremyrubin> if you want them to be hashable you need a canonical order
< glozow> e.g. topological order (by # of ancestors within package), lexicographic by txid to break ties
< jeremyrubin> but glozow in theory it's not that bad, a dirty client can always send you crap
< glozow> if they send you crap, just reject it
< glozow> the package - maybe not the txs within it
< Murch> Why would we need package ids?
< jeremyrubin> Anyways that seems like an OK plan, maybe lexicographic by wtxid tho
< glozow> to announce packages/getdata
< Murch> mh
< Murch> Doesn't that mean that you could send a peer the same txs a bunch of times by generating each valid subset and announcing it as a separate package?
< jeremyrubin> Murch: correct, but you can ban peers which send you packages containing crap you already have I suppose?
< glozow> yeah
< jeremyrubin> You can also generate the subsets yourself and add their IDs to your bloom filter
< jeremyrubin> altho idk if that's worth doing at all
< Murch> not if they send them from smal lto large
< Murch> I'd say, only allow connected graphs to be sent as packages
< jeremyrubin> I think that's already a contstraint
< jeremyrubin> glozow: did you ever get the notes suhas and I put together last year about this stuff?
< Murch> AFAIU, that restraint has been losened?
< glozow> peers can already send you crap, so it's more about limiting how much time you spend looking at crap i think
< Murch> mh, good point
< jeremyrubin> glozow: I think it's important that packages as a concept are connected components
< glozow> jeremyrubin: which notes? I've seen a couple of his gists
< jeremyrubin> we can have metapackages that can be split into packages
< Murch> But at least they'd only send you announcements once so far and then wouldn't make you download the same transactions again and again?
< jeremyrubin> but the notion of a package as a component is useful, and peers sneaking in unrelated crap to the component should be bannable IMO
< jeremyrubin> glozow: I'll try to find the notes
< glozow> i think the protocol for correctly-behaving senders would be to build packages as necessary
< glozow> and the receiver can try their best to distinguish between, e.g. an extra tx you already knew about and punishable garbage
< glozow> e.g. sender creates a package based on `filterinventoryknown` of their peers
< glozow> er, what's it called? the bloom filter for invs you've heard from the peer
< jeremyrubin> glozow: FWIW, make a hashtable with all txids ( O(n) ), scan all txns inputs against it (O(n)), count txns with all inputs not found
< jeremyrubin> I think that algorithm is O(n) and guarantees a single component
< jeremyrubin> no matter what order things are in
< Murch> I think that would still allow conflicting txs i.e. A and A' that both spend the same input?
< jeremyrubin> yep, that's still a component. conflict check can also be done either as a later part of validation or with a similar algorithm
< Murch> Right, would still be connected, but not a valid package
< glozow> you're doing this with transactions in your mempool you're relaying to peers right?
< jeremyrubin> a while ago i did some O(n) algorithm for this during block validation... but it was unpopular
< jeremyrubin> it turns out we do all the connection logic in blocks while hitting the utxo caches
< jeremyrubin> but you can context-free validate block tx connectivity
< glozow> sure. why do you want to do that? check tx connectivity before you start looking at scripts?
< jeremyrubin> it would let you parallelize better
< glozow> you wouldn't even get to this part until after u verify the pow
< glozow> why would someone put duplicate inputs + do the pow just for you to verify a tiny bit slower?
< bitcoin-git> [bitcoin] achow101 opened pull request #21640: [0.21] Introduce DeferredSignatureChecker and have SignatureExtractorClass subclass it (0.21...0.21-sig-ext) https://github.com/bitcoin/bitcoin/pull/21640
< jeremyrubin> Ah it's not just that, there are other things you can do when you check all these properties at once
< jeremyrubin> i can try to find the experimental branch if you're curious, but don't spend too much time since it seems like a dead end
< jeremyrubin> you have to process txns in block order, so it's serial O(n)
< jeremyrubin> but if you do a small amount of pre-work (which is also O(n), but small c)
< jeremyrubin> then you can do the rest of the tx validation in parallel because you're not relying on the UpdateCoins calls being serial
< jeremyrubin> not a big deal, but the old PR might be interesting to you for 'algorithms to check properties of groups of txns'
< glozow> yeah it's interesting, thanks for sharing