< achow101>
luke-jr: It might have been removed by mistake when -upgradewallet was removed
< achow101>
The test doesn't really need upgradewallet at all, but the fact that it did likely confused me when removing it
< achow101>
feature_backwards_compatibility kind of does the test, but only really on releases and actual incompatibilities. I guess it would be good to re-add it.
< achow101>
it can even be done without including a wallet.dat file by using createfromdump (after that's merged)
< luke-jr>
that one was forwards compat
< luke-jr>
with some hypothetical future wallet version
< jnewbery>
For me, I'd like to make some substantial progress in clarifying the net/net_processing and net_processing/validation interfaces. Those two projects are tracked in #19398 and #20158.
< sdaftuar>
i plan to work on block-relay-only peering too, though i'm stuck on some annoying addr-relay things that might hold me up
< troygiorshev>
With the feature freeze now thawed, I wanted to remind everyone about Per-Peer Message logging #19509. It got a lot of attention a while ago and I think it's a feature a lot of people would appreciate. Just needs a bit more review!
< amiti>
in terms of my work: hoping to make progress on 19315 (adding full-relay and block-relay-only to tests), and I'm working on reviving rebroadcast
< nehan>
hi
< jonatack>
amiti: will review
< jnewbery>
ok, anyone want to add any topics before we get onto "Reducing CVE-2020-26895 class of bugs and Tx-standardness" (ariard)
< gleb>
Hi
< aj>
touch base on wtxid backport?
< jnewbery>
hi aj. Yes, let's do that one first
< jnewbery>
#topic touch base on wtic backport (aj)
< core-meetingbot>
topic: touch base on wtic backport (aj)
< jnewbery>
*wtxid
< sdaftuar>
i'm not sure we should have backported wtxid-relay to the 0.20 branch
< aj>
so 20399 is there to revert wtxid relay from 0.20; or 20317 is there to fix up the orphan handling regression if there's a reason to keep it
< sdaftuar>
(i missed that github discussion, catching up now)
< jnewbery>
I don't have a strong opinion, but I
< jnewbery>
'm curious what has changed since this was discussed in a previous meeting, when people agreed that it should be backported
< aj>
19620 imo
< anekdotin>
great job guys
< sdaftuar>
(ok caught up)
< sdaftuar>
i think that knowing that there was a crashing bug in that line of work sort of heightens the sense that backporting a feature is not really a great idea?
< sdaftuar>
i don't think backporting it was a requirement in the first place, but i saw the reasoning that it was a nice-to-have. but given that it indeed turned out to be risky, i'd say let's drop it, as there's no compelling reason for backport after #19620 either
< sdaftuar>
so falling back on our principle of only backporting bugfixes seems like the prudent thing
< jonatack>
A propos, sharing a message from moneyball who mentioned tripling down on testing (bug discovery+fixes) between now and the 0.21 release with all of the new features.
< jnewbery>
sdaftuar: do you think it should be reverted?
< sdaftuar>
yeah
< jnewbery>
because we're not confident in the code?
< sdaftuar>
if 0.21 has some other crashign bug related to wtxid-relay, it'd be great to be able to go back to 0.20.2 or whatever and have it not crash!
< aj>
jnewbery: so i think your review beg on oct-20th should've resulted in some sort of a "do we really still need this?" response, rather than silence both here and on the PR -- i think that was the only time it was mentioned in these meetings since 19620
< jnewbery>
aj: yeah, people agreed in the original meeting that it should have been backported, and then I've raised reviewing the PR in several meetings since then
< jnewbery>
I don't think the presence of a bug that was caught in review and fixed in the follow up changes the facts, but I can understand that it makes people nervous
< jnewbery>
so if people want it out, then it should be reverted
< jnewbery>
#topic Reducing CVE-2020-26895 class of bugs and Tx-standardness (ariard)
< core-meetingbot>
topic: Reducing CVE-2020-26895 class of bugs and Tx-standardness (ariard)
< ariard>
hello
< ariard>
first and foremost, I apologize for the previous rounds of discussion on this issue, it was a bit noisy to talk grasp problem right this without this vuln being public
< ariard>
for the context, lnd didn't check that counterparty commitment signatures were in the curve order /2
< ariard>
which means a malicous counterparty was able to feed them with non-relayable witness and thus complete break its security
< ariard>
because those time-sensitive transactions won't propagate before timelock expiration
< ariard>
of the HTLCs
< ariard>
I think the lesson it's not the first time that a LN implementation had issues with tx-standard, CL had some 2y ago IIRC about minimal relay fee
< ariard>
so it sounds we have this class of vulns concerning any time-sensitive protocols, and how to mitigate it correctly is a bit unclear
< luke-jr>
be strict on transaction form..
< sdaftuar>
testmempoolaccept?
< aj>
testmempoolaccept RPC isn't useful because you don't want to finish signing the tx, i guess?
< luke-jr>
you can never rely on node/relay policies
< luke-jr>
just have to do what you can to reasonably pass them
< luke-jr>
testmempoolaccept only tells you about your own policy
< ariard>
aj: my concern with testmempoolaccept its checks are blurred with the fees ones
< ariard>
but your transaction is okay because you plan to broadcast only in the future
< sdaftuar>
luke-jr: i agree, but perhaps a belt and suspenders approach where you enforce a strict transaction form and also double-check against testmempoolaccept might be helpful
< luke-jr>
it might detect bugs I suppose
< ariard>
also you have a really code architecture thing, LN mobiles won't have a mempool but they need to do the check
< sdaftuar>
or unexpected changes
< sdaftuar>
ariard: i think you'll need to specify the problem a bit better if you are assuming not having access to a full node
< luke-jr>
ariard: it can't be secure without your own full node anyway
< sdaftuar>
for instance, do you have access to all the inputs?
< ariard>
luke-jr: you might receive the headers from a trusted connection to your full node on your LN node
< luke-jr>
so you mean you don't have RPC access?
< ariard>
sdaftuar: I think we should assume no, it's concering any kind of LN clients
< luke-jr>
too bad we removed the reject message..
< sdaftuar>
ariard: if no, then isn't this problem hopeless?
< ariard>
sdaftuar: you have access to the funding utxo of your channel
< ariard>
always, that's a protocol assumption
< sdaftuar>
ariard: but if i include an input you don't know about, then you must reject the transaction, because you have no way of knowing if it's valid, let alone standard?
< ariard>
what you want to verify is the chain of transaction built from this utxo (commitment+HTLC txn) are tx-relay "valid"
< ariard>
sdaftuar: for commitment transaction the utxo is 2-of-2
< luke-jr>
ariard: not possible to do reliably..
< ariard>
luke-jr: at least you should be able to be sure your transaction is accepted by your full-node
< luke-jr>
nodes will relay what they want; there are no consensus rules forcing relay
< luke-jr>
ariard: sure
< ariard>
luke-jr: I finally agree with you on this, it's more a LN-client to its full-node relation we should care about?
< luke-jr>
ariard: so this part of problem is because you don't necessarily have RPC access, and we no longer support the reject msg?
< ariard>
luke-jr: I think it's more subtle, you have those LN transactions, that both you and your counterparty must agree on, but ultimately their validity is decided by your full-node
< aj>
ariard: what good is a commitment tx with a low fee rate that won't be acceptable possibly for months? don't you want to know that your node will (currently) reject that?
< luke-jr>
hmm, I suppose the TXOs spent in it migth also not be something you want broadcast yet
< ariard>
aj: in the future we might have fixed-fee commitment transactions and their feerate adjusted by a CPFP+package relay
< ariard>
but I may jump one step in the reasoning here, hard to see
< aj>
ariard: right, but at that point you'd expect testmempoolaccept to support packages
< sdaftuar>
aj: bold :)
< aj>
sdaftuar: (easier to implement package testmempoolaccept than package relay anyway?)
< ariard>
luke-jr: what's concerning is right now we are leaking those policy check directly in the LN spec, but we may should do the reverse and let those tx-relay be ultimaltey defined by LN clients
< sdaftuar>
aj: fewer DoS vectors to worry about for sure!
< glozow>
aj: hey guess what
< aj>
glozow: you already did it??
< ariard>
aj: yes if this the case, it's more a cod architecture reason, to have a strict transaction form lib
< glozow>
a little bit, i have a math test this week though, and then i'll send you my draft of package testmempoolaccept
< luke-jr>
ariard: the spec just needs to be narrow enough that there's a good hope any tx will be accpeted
< ariard>
aj: we shouldn't require LN clients to have access to all the utxo set, you just care about your channel utxo only, testmempoolaccept assume you have access to the full set rn?
< luke-jr>
double-checking against your own node makes sense, but ultimately shouldn't be your security check
< luke-jr>
ariard: to open a channel, you need access to the entire UTXO set to be sure the inputs are valid..
< sdaftuar>
ariard: oh, yes you're right that testmempoolaccept requires all inputs to be available in mempool or utxo set, so if you're testing validity further down a chain that won't work right now
< aj>
ariard: i think if you pull some checks out of "here's my full node" into "here's my light library" you'll run the same risk of missing some checks that are actually needed; but ymmv of course
< ariard>
luke-jr: I see your point, but you might not have access to the UTXO set for the rest of the discussion
< ariard>
*protocol execution, sorry
< ariard>
aj: yes that's exactly my worry, mempool checks are a bit blurred, and some we are interested with are also inside the script interpreter
< aj>
ariard: maybe a psbt validator could make more sense, alternatively? partial so you don't need all the inputs or even all the signatures, but still get useful results like "fees look low!" "this is a dust output!" "this signature is wrong!"
< aj>
ariard: psbts are (probably) common enough that it would justify thorough useful (re)implementation of standardness and consensus and sanity checks
< ariard>
luke-jr: without entering in a light-client debtate, for LN if you assume people are using light-clients, it's not reason for them not being at least secure on tx-relay checks
< ariard>
lke layering security reasoning
< jnewbery>
ariard: testmempoolaccept does require the full UTXO set, but you could imagine a version where you provide the inputs, like signrawtransaction
< sdaftuar>
philosophically though there is no such thing as "bitcoin's tx relay rules". i think that's luke's fundamental point (which I agree with)
< ariard>
aj: only validating the inputs and not the other transaction check like version, number of max outputs
< ariard>
?
< aj>
ariard: validating all the other stuff seems relevant for psbts in general, i think?
< luke-jr>
ariard: your non-debatable premise is false
< ariard>
sdaftuar: I finally agree with it, what I'm trying to understand is the margin we might have to make thing better
< sdaftuar>
i think the best we can do is what luke suggested already: narrow what you accept to a small enough subset that you think will likely to always be relayable now and in the future, and hope for the best.
< sdaftuar>
testing validity against a bunch of full-node implementations is a nice backup to make sure your code isn't broken or things haven't changed out from under you
< aj>
sdaftuar: not very robust against hostile peers who have access to your code base to find checks you forgot to implement, though; hard to fuzz test for valid-but-non-standard sigs and the like even, afaik
< ariard>
I see the point, which means anytime someone is implementing or desinging a L2 protocol you have to open your full-node code and understand those blurred checks
< jnewbery>
ariard: do you have what you need from this discussion? Anything else you need or could use help with?
< ariard>
aj: we should assume your counterparty shouldn't be able to observe your full-node implementation or policy, but can we really do this?
< ariard>
jnewbery: if I'm reworking on a #18797 as a wrapper on top of testmempoolaccept like libconsensus is anyone have strong NACK?
< ariard>
jnewbery: checking if the feerate of the whole package is okay while one of the transaction might be under the mempool min fee
< jnewbery>
ariard: I believe it would reject the package if any of the txs are below the min feerate (which makes sense while we don't have package relay). That could be changed later
< sipa>
hi
< sipa>
i guess i'm a bit late
< sdaftuar>
sipa: off by one hour?
< luke-jr>
XD
< sipa>
actually i just forgot
< pinheadmz>
what time is the p2p meeting in UTC? the event just floated around my calendar due to daylight savings
< ariard>
jnewbery: right, let do step by step, it's not bad to un-blurred those checks
< ariard>
aj: I need to think about your psbt validator suggestion, because the caller being able to select the verification scope, it let you validate counterparty signature without actually to produce yours
< ariard>
and that's a good thing, for LN, in the happy case you don't have to generate your LN commitment signature
< aj>
ariard: might be something that's useful elsewhere (hardware wallets, multisig setups in general), which might help it stay maintained
< bitcoin-git>
[bitcoin] MarcoFalke opened pull request #20410: wallet: Do not treat default constructed types as None-type (master...2011-rpcWalletNoneType) https://github.com/bitcoin/bitcoin/pull/20410
< miketwenty1>
Q: is there an equivalent gitian.sigs for guix build process?
< sipa>
i assumed we'd just run the guix build inside gitian, but haven't followed in detail
< luke-jr>
same here
< luke-jr>
especially since Guix still requires trusted third-party blobs
< miketwenty1>
sipa, right now in bitcoin-core repo we have the gitian sigs for non guix process .. but would we have a sigs folder for guix build process or am i thinking about this incorrectly?
< achow101>
I would expect that we would keep the same assert files and have them in the same repo
< luke-jr>
miketwenty1: at some point, I expect the gitian descriptors will be rewritten to use Guix instead of Ubuntu, and the depends/ build system will go away
< achow101>
it would just be a slightly misnamed repo
< luke-jr>
misnamed?
< miketwenty1>
i see.. seems like the idea here is gitian build process improves with guix instead of replaces it?
< achow101>
gitian.sigs wouldn't be strictly tied to gitian because in theory you could produce them without gitian
< luke-jr>
achow101: in theory you already could :P
< achow101>
I would expect the gitian descriptors to do the guix build because that makes the transition easier
< sipa>
once we have guix-based deterministic builds, much of what gitian provides would be technically overkill
< luke-jr>
miketwenty1: gitian is providing less of the deterministic stuff with guix
< sipa>
but given that it already exists, and has infrastructure around it, it's probably easier to keep it (and just use guix inside gitian)
< achow101>
luke-jr: iirc guix can be fully bootstrapped with only trusting a single small blob that is kind of verifiable. but it also takes forever to build everything
< wumpus>
achow101: MarcoFalke: will do the split-off tomorrow morning (europe time)
< MarcoFalke>
wumpus: Thanks
< achow101>
ack
< luke-jr>
achow101: "a single small blob" is still a blob
< luke-jr>
and it's not very verifiable
< achow101>
luke-jr: it can be disassembled and verified
< luke-jr>
achow101: it's still too big for that
< MarcoFalke>
I am not sure if it is easier to set up gitian than guix these days
< luke-jr>
MarcoFalke: I have failed to setup Guix to date
< luke-jr>
after spending hours on it
< MarcoFalke>
It will still be easier to set up a debian vm and run the one-line install command for you than to setup gitian on a debian vm
< achow101>
I have it setup but I don't remember how I did it
< luke-jr>
"one-line install command" does not get you a trustless setup
< luke-jr>
achow101: mind you, I refuse to run third-party binaries, and am using ppc64le
< MarcoFalke>
Ok, then wait for debian to ship guix in the package manager, this will be as trustless as the current gitian build
< luke-jr>
I suppose it's not entirely fair since I don't have gitian working on ppc64le, and Ubuntu is a huge blob :P
< luke-jr>
MarcoFalke: I don't use Debian.
< MarcoFalke>
Our gitian descriptors use debian/ubuntu
< luke-jr>
inside the VM, yes
< MarcoFalke>
So if you don't trust them, I doubt you can trust the overall build setup
< luke-jr>
MarcoFalke: again, Guix is an improvement over Ubuntu, I fully agree with that
< luke-jr>
but it isn't an alternative for gitian entirely at this point
< luke-jr>
gitian can be setup without trusted binaries (outside the VM)