< achow101>
sipa: how come expanding a descriptor returns a vector of scriptPubKeys? Isn't there only one possible scriptPubKey that a descriptor can return?
< sipa>
achow101: combo does multiple
< achow101>
ah, so that's the only descriptor that can have multiple?
< sipa>
at a single position, yes
< sipa>
for now (though i don't think there is any good reason to add more)
< bitcoin-git>
[bitcoin] MarcoFalke opened pull request #15696: [qa] test_runner: Move feature_pruning to base tests (master...1904-qaPruning) https://github.com/bitcoin/bitcoin/pull/15696
< bitcoin-git>
[bitcoin] MarcoFalke opened pull request #15697: qa: Make swap_magic_bytes in p2p_invalid_messages atomic (master...1904-qaMagicAtomic) https://github.com/bitcoin/bitcoin/pull/15697
< bitcoin-git>
[bitcoin] practicalswift opened pull request #15699: Remove no-op CClientUIInterface::[signal_name]_disconnect. Disconnect BlockNotifyGenesisWait and RPCNotifyBlockChange properly. (master...remove-CClientUIInterface-signal_name_disconnect) https://github.com/bitcoin/bitcoin/pull/15699
< meshcollider>
Apologies, I'm travelling to Portugal for ICPC at the moment so my timezone is completely messed up, I missed the meeting yesterday and will have to miss the wallet meeting today too if its on
< meshcollider>
Do people have topics they want to discuss? It should start in half an hour if my calculation is right, anyone who wants to can chair it
< MarcoFalke>
Doesn't seem like it :)
< achow101>
I have a topic, but not at computer now. Give me a few minutes
< jnewbery>
hi! I had a small topic.
< jnewbery>
shall I wait until achow101 is back?
< sipa>
i'm here, but don't have anything
< sipa>
let's wait
< achow101>
here now
< jnewbery>
sipa: care to chair?
< sipa>
#startmeeting
< lightningbot>
Meeting started Fri Mar 29 19:10:38 2019 UTC. The chair is sipa. Information about MeetBot at http://wiki.debian.org/MeetBot.
< instagibbs>
I see we finally have rebroadcast tests :)
< jnewbery>
I've been looking at wallet rebroadcasts. The current behaviour is: set a timer for random (0, 30). When it pops rebroadcast all unconfirmed wallet txs. Set timer again.
< sipa>
but only if there has been a new block since the previous rebroadcast?
< jnewbery>
(it's not quite as bad as that because each peer has a bloom filter so we won't actually rebroadcast until it's been purged from there)
< jnewbery>
ah yes, as long as there's been a block
< jnewbery>
I have a couple of suggested changes
< jnewbery>
1. separate the scheduling, so each tx is on its own timer, instead of sending them all at once
< jnewbery>
2. have the wallet remember some number of random txs it sees from the mempool, and add those to its rebroadcast list as decoys
< jnewbery>
wanted to get some concept feedback on those before I started coding up
< sipa>
so (1) is problematic if there are interdependent unconfirmed transactions in your wallet
< sipa>
as you may end up sending a child to a different peer than the parent
< sipa>
gmaxwell: opinions? ^
< jnewbery>
in (1), I'd think we'd still send each tx to all peers
< sipa>
oh!
< jnewbery>
just not all at the same time
< instagibbs>
trickle rather than flood
< sipa>
right, but you may send the child before the parent
< sipa>
(they get sorted in the broadcast buffer before actual sending, but that's just a few-seconds timer)
< sipa>
i don't know whether the rebroadcast mechanism in the wallet currently actually cares about that
< jnewbery>
I think it doesn't, but I'd have to recheck
< sipa>
yeah, but if it sends all unconfirmed txn, they'll get sorted before broadcast
< jnewbery>
oh, where does that happen?
< sipa>
sendmessages
< sipa>
there's a poisson distributed per-peer timer (but simultaneous for all inbound connections) that flushes a buffer of to-be-broadcast txn
< jnewbery>
yeah, I see it now
< sipa>
and it sorted first by dependency order and then lexicographically
< sipa>
*sorts
< sipa>
so if you announce an txid within a few seconds of each other, there's a high chance they'll end up being sorted correctly before actual broadcast
< jnewbery>
// Topologically and fee-rate sort the inventory we send for privacy and priority reasons.
< sipa>
yeah, that
< sipa>
oh yes, feerate; not lexicographically
< gmaxwell>
(1) could consider only the deepest ancestors and announce all the parents at once.
< gmaxwell>
ignoring the details (1) sounds like a very important improvement.
< jnewbery>
is the wallet aware of dependencies in its transactions?
< sipa>
yes
< gmaxwell>
er only the deepest children.
< gmaxwell>
(2) I don't have a problem with but I think the benefit is a bit dubious, like random txn by itself very likely provides no privacy improvement, because only the real sender or recipent will send txn for all of a single address/linked address consistently. It's sometimes hard to tell where the line is between snake oil that just adds complexity to the codebase for no protection against an
< gmaxwell>
even slightly incompetent attacker, vs fuzzing things up in a way that provides incremental privacy even though far from perfect
< sipa>
jnewbery: mapTxSpends
< gmaxwell>
I would think though that matching random addresses would be a lot better than random txn.
< gmaxwell>
I think a (3) avoiding pointles retransmissions would be more helpful. But I would happily review an implementation of (2), reservations aside.
< jnewbery>
Do we know what percentage of txs are re-used addresses? And what number of those re-used addresses have multiple unconfirmed txs at the same time?
< gmaxwell>
jnewbery: 'most' (I expect sipa will provide some numbers) But it's not just a same time question:
< sipa>
jnewbery: also, poisson distributed rebroadcast events are probably more private than uniform distributed ones
< gmaxwell>
what I think it should do is per node pick a random value (e.g. the addr man randomizer) and use that to consistently select some small portion of addresses to rebroadcast.
< gmaxwell>
so that over months of time we're consisently rebroadcasting the same other addresses.
< gmaxwell>
Yeah uniform leaks information that possion doesn't, possion is the distribution you get from memoryless processes.
< jnewbery>
if you're talking about months of time, then presumably you'd need to save this to disk
< sipa>
the addrman randomizer is stored on disk
< gmaxwell>
Thats why I specified that one.
< jnewbery>
So you're saying this behaviour should live in the node rather than the wallet?
< sipa>
wallet is easier i think
< gmaxwell>
also if you wipe out your peers.day because you're trying to avoid addrman based fingerprinting then you'll also wipe rebroadcast fingerprinting.
< jnewbery>
the wallet isn't aware of peers
< gmaxwell>
I think nodes without wallets should d this too.
< sipa>
but it would possibly result in identifiable behavior that can be detected if a wallet file moves to another node
< gmaxwell>
because there are many walletless nodes that would provide cover...
< gmaxwell>
maybe thats not realistic for engineering reasons. ::shrugs:: I'm just talking spherical cows here.
< gmaxwell>
In any case, if it isn't consistent like this, it's really obvious to me how attackers will filter it out. Monitor, and look for dispositive failures to rebroadcast. The real source won't have them, the fake sources will.
< jnewbery>
I think the implementation can be quite minimal if the wallet just keeps its own list of txs and doesn't try to distinguish behaviour between peers
< gmaxwell>
and given someone that already has network wide monitoring that filtering is probably only a dozen lines of code/query.
< jnewbery>
are you saying genuine wallet rebroadcasts should also be to only one peer?
< jnewbery>
because if its to all peers that's different behaviour than decoy rebroadcasts and would be easy to spot
< gmaxwell>
no, both should be to all peers.
< jnewbery>
then I've misunderstood your 'per node' comment above
< gmaxwell>
jnewbery: my node being different from yours.
< jnewbery>
ok
< gmaxwell>
I prefer to rebroadcast 1apple you prefer to rebroadcast 1spatula.
< jnewbery>
yep
< gmaxwell>
and if I restart I still prefer 1apple.
< gmaxwell>
so an attacker can't tell if I like 1apple's addresses because of my random number or because I am 1apple.
< sipa>
so what if the address you pick for rebroadcasts suddenly gets a ton of transactions
< gmaxwell>
sipa: well you're going to relay them on the network regardless.
< gmaxwell>
So I don't think thats actually a major cost?
< sipa>
ah right, if it's per node they'd just be broadcast from the mempool directly; no need to copy/store them elsewhere
< sipa>
though you do need to keep track of txn that spend outputs assigned to the addresses you've chosen
< sipa>
but that could just be a boolean per-tx i guess
< gmaxwell>
right. all these considerations is why I'm a little dubious. I think the simplest version does little to nothing. And I'm not sure of the bound of the complexity on an effort to really be indistinguishable. I mean mean one way would be to effectively importaddress on random addresses you see with a flag to hide them in the wallet but implemented that way would have too many DOS potentials.
< gmaxwell>
it would, however, have a pretty strong guarentee that you'd treat them the same, making them very hard to distinguish.
< gmaxwell>
though it would also only work if you have a wallet.
< jnewbery>
I think just selecting random txs is surely better than little to nothing
< sipa>
another approach is having the rebroadcast mechanism be purely in the node, and have it mark some subset of transactions... but then have the wallet forcibly set that mark on its own transactions too
< sipa>
but the wallet doesn't do the rebroadcasting
< gmaxwell>
jnewbery: I think any existing deanon attackers can filter out random tx rebroadcasts fairly reliably with a line of code, maybe its still worth doing. We have other paper thin privacy mechenisms (e.g. most of the node anti-fingerprinting).
< jnewbery>
but if they're random, some of them will be for addresses that aren't re-used. How would the attacked filter those out?
< gmaxwell>
jnewbery: A couple ways: for rebroadcast purposes every coin that isn't lost is reused, since senders and recipents both rebroadcast. Also even when addresses aren't reused, they're co-used in inputs with other transactions. So broadcasting of the complete address cluster is also a fairly strong hurestic.
< gmaxwell>
I think you've convinced me that in at least some cases it would be indistinguishable.
< jnewbery>
Many txs don't get rebroadcast at all and are confirmed in the next block
< gmaxwell>
e.g. one side of send/recieve doesn't need any rebroadcast, no reuse, no useful information from clustering.
< jnewbery>
Right. An attacker can't tell if you would have rebroadcast
< gmaxwell>
(thats also, aside, a reason rebroadcasts should be possion timed)
< sipa>
maybe we should move to achow101's topic?
< jnewbery>
I don't think I need anything else on this for now. Thanks for the input!
< achow101>
I've started working native descriptor wallets and I just wanted some opinions on some parts of the design.
< sipa>
cool
< sipa>
#topic non-hardened derivation paths
< achow101>
right now I have the wallet make 6 descriptors, in pairs of 3. each pair has an internal and external descriptor, and each pair for each of the address types
< sipa>
makes sense
< achow101>
it uses derivation paths that we currently use in the wallet, but I think it would be better if we used bip 44/49/89
< achow101>
thoughts? it would mean that we use non-hardened derivation paths
< sipa>
i think using non-hardened derivation paths is fine if there is no way to export the individual private keys
< gmaxwell>
I think it's necessary to disable key export on anything non-hardened.
< achow101>
as it is now, we need to have a new seed for each address type or we'll end up using keys accidentally
< achow101>
I think that the only export that will be possible is exporting the entire private descriptor itself
< sipa>
achow101: you would certainly use distinct paths for each of the address types
< sipa>
achow101: agree
< gmaxwell>
Also, I don't think we should be defaulting to using non-hardened unless we're actually making real use of its abilityies (like to cogenerate addresses with an external signer).
< achow101>
i'll be disabling dumpprivkey and all of the imports for descriptor wallets
< sipa>
gmaxwell: it also has the advantage of not needing to decrypt the wallet to generate more addresses
< gmaxwell>
so generate 100,000 addresses the first go. :)
< gmaxwell>
it's a lot of fragility to save a megabyte of file size. :P
< achow101>
gmaxwell: it has the advantage of making the wallet file extremely small
< gmaxwell>
the wallet file grows from transactions regardless. if you're actually using addresses it'll get big.
< sipa>
anyway, i think it certainly makes sense to support nonhardened derivation
< gmaxwell>
Sure.
< achow101>
right now the thing I don't like is that it needs 3 different seeds, one for each address type
< sipa>
achow101: that shouldn't be needed; you can use distinct derivation paths even when using hardened?
< gmaxwell>
as far as making wallets small, that I think is mostly interesting for backups, and for that it might be best to have tools/rpcs that strips wallets for backup purposes.
< achow101>
sipa: yes, but I don't really like having to introduce yet another set of derivation paths that people have to consider
< achow101>
vs. using existing standards
< sipa>
achow101: that's why you encode it as a descriptor; it's universal :)
< sipa>
but sure- i agree at a high level it's unnecessary to introduce new standards when existing ones exist
< sipa>
the question is whether hardened or unhardened is more desirable - i don't know, and it may depend on the situation
< gmaxwell>
This is just broken.
< achow101>
?
< gmaxwell>
'existing standards' were made by people considering entirely different use cases and deployment enviroments (hardware wallets)
< gmaxwell>
and with a different focus on security (a lot closer to 'who cares')
< sipa>
well when we'd use a nonhardened derivation (for external reasons) it makes sense to follow those standards
< sipa>
the question remains whether or not to use hardened or not
< gmaxwell>
sipa: it might, if it is otherwise fine and would actually result in some kind of useful compatibility.
< gmaxwell>
like we'll never be compatible with electrum (say) by using the same path, simply because we generate parllel native and compatibility addresses.
< achow101>
gmaxwell: the plan for descriptor wallets is not generate them in parallel
< achow101>
each address type will have it's own derivation path base
< sipa>
yeah, so your bitcoin core wallet will correspond to a union of several things that are (possibly) compatible with an electrum wallet individually
< gmaxwell>
a wallet where you can recover part of the funds and the rest are just invisibly lost isn't compatible though, if anything its a liability.
< sipa>
right
< achow101>
anywys, it seems like the preference is to use hardened
< gmaxwell>
Generally thats my strong preference, except in cases where there is a 'application layer' gain from otherwise.
< gmaxwell>
Not just some 'its easier to write this way' or 'wallet file is somewhat smaller for unspecified benenfits'
< sipa>
seems we're out of time
< gmaxwell>
I really regret ever coming up with public derrivation and wish I could take it back, The original application I had for it is still unavailable to people (so your web server could securely generate fresh addresses for you) in practice... and it gets applied *everywhere* simply because reusing a key is simpler to implement.
< achow101>
well that's all I had for today. there was something else I wanted to discuss, but I don't remember what that was
< achow101>
so clearly it wasn't very important
< gmaxwell>
.. and its resulted in funds loss, and it also results in expectations we won't be able to support for post ECC keys.
< sipa>
#endmeeting
< lightningbot>
Meeting ended Fri Mar 29 20:04:46 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< achow101>
gmaxwell: well it's been useful for hardwre wallets. asking a hardware wallet for a few thousand keys is not exactly a fun time
< gmaxwell>
achow101: yes, though really that isn't at all fundimental.
< gmaxwell>
like if it didn't exist, hardware wallets would just cost $5 have a 10x faster cpu, and a faster interface, and no one would notice otherwise.
< gmaxwell>
(or alternatively they'd just support a single address... :P)
< gmaxwell>
er cost $5 more
< gmaxwell>
I also agree that they're not terribly bad for the hardware wallet case. That isn't really all that different from the orignal 'webserver' usecase, except your wallet is the webserver.
< gwillen>
this is more generally the "not having to expose your private key in more places than necessary" usecase, which seems laudable
< gmaxwell>
though also the security of generating addresse on an untrusted device is kinda dubious. eventually the bad guys will have taken all the low hanging fruit and the next set of attacks will just be to make createnewaddress like interfaces return attacker addresses. :P
< gwillen>
although due to not trusting software, these days I always check new addresses against each other on both machines anyway
< gwillen>
yeah
< gwillen>
you could very well generate new addresses independently in parallel on multiple devices and check them that way
< gwillen>
with none of them being the private device
< gwillen>
but ain't nobody got time for that
< gmaxwell>
gwillen: there are goals in conflict, "reuse private keys" = bad. "use keys too much" =bad. Essentially all non-hardened keys share the same private key, so it's reuse of a private key.
< gmaxwell>
gwillen: indeed.
< gwillen>
reuse of private keys being bad seems very implementation- and UX-specific somehow
< gwillen>
like, if you treat it as "a series of addresses generated by a single master key, there are no separate individual keys and the software will never give you such a thing" then you mitigate the user-confusion issue