< bitcoin-git>
bitcoin/master cb31ee0 gzhao408: [test] feefilter during and after IBD
< bitcoin-git>
bitcoin/master 19aaf79 MarcoFalke: Merge #19423: test: add functional test for txrelay during and after IBD
< bitcoin-git>
[bitcoin] MarcoFalke merged pull request #19423: test: add functional test for txrelay during and after IBD (master...ibd-txrelay-test) https://github.com/bitcoin/bitcoin/pull/19423
< wumpus>
(it makes some assumptions that cannot be stated in general, like "every connection that comes from 127.0.0.1 comes from tor")
< wumpus>
also tor should be 'onion' in the new command line option wording
< jonatack>
wumpus: thanks! yes, i've been wondering what is the best criteria for tor, was thinking "addrlocal contains .onion"
< wumpus>
that's an interesting criterion; on one hand i think that works? on the other, it relies fully on what the peer sends for your local address, not the actual network circumstances
< jonatack>
I'm looking at how to make CConnman::AttemptToEvictConnection() Tor-aware
< jonatack>
so how to know which inbounds are onion conns
< wumpus>
the only real solution to this would be #8973
< wumpus>
incoming tor connections to use an alternative, local-only port or even better, a UNIX socket to connect
< wumpus>
jonatack: but until that i guess some kind of 'connects to 127.0.0.1 AND has .onion in addrlocal' heuristic works fine
< wumpus>
i do think that for eviction, only using 'what addrlocal it sends' is too easy to manipulate
< jonatack>
super helpful -- i'll see what i can do with that
< jonatack>
would a mix of 'connects to 127.0.0.1 AND has .onion in addrlocal' be manipulable?
< jonatack>
idea being to specifically protect some onions so they aren't penalised for higher minping / lower uptime vis-a-vis clearnet peers
< jnewbery>
wumpus: can you wait for cfields to review before merging 19542 please?
< cfields>
I was just commenting there... will review in detail today. I suspect I was wrong with the concept nack anyway...
< cfields>
it's not a big deal, though.
< jnewbery>
cfields: thanks!
< cfields>
np, these things happen :)
< wumpus>
yes, though, "this causes a few PRs to need to be rebased" was another reason to be cautious about merging it in the first place
< wumpus>
I think we should avoid doing these kind of mass data type changes unless there is a really good rationale and people agree about doing it
< wumpus>
"increases code quality" is kind of subjective
< wumpus>
if it's uncontroversial and doesn't impact other people's work, okay, but not sure about this
< jnewbery>
wumpus: yes, agree. When I wrote my ACK I originally had a comment about not merging immediately because of conflicts, but I deleted that because I didn't want to tell the maintainers how to do their job
< MarcoFalke>
when merging I do check all conflicts for reviews, and in this case, all of the conflicts had either, no review, didn't compile, or didn't pass the test suite. So in all cases a rebase or force push wouldn't have been harmful or was needed anyway.
< MarcoFalke>
yeah, looks like the caches are different
< MarcoFalke>
oh that makes sense
< MarcoFalke>
the qt cache is obviously invalid
< MarcoFalke>
oh wait, no. depends should detect when a package description changes
< midnight>
dangit. we're using he new Mac SDK tarball now- but it won't let an end-user download the necessary files without setting up 2FA; except part of 2FA is a trusted phone number which I haven't been convinced yet is safe from simjacking. I'm not entirely sure that requiring a user to enable simjacking on their account is tenable for gitian builders concerned about their account security.
< MarcoFalke>
no idea what's going on. Needs more diffoscope I guess
< provoostenator>
midnight: that's annoying. I don't understand why Apple insists on putting these files behind a registration wall.
< achow101>
everything else in the tarball matches
< midnight>
:-(
< hebasto>
achow101: what does mean that diff?
< achow101>
it's the diff between the two genisoimage binaries
< hebasto>
I mean what is the root of that diff?
< achow101>
it looks like the difference between the readelf outputs, so probably symbol differences?
< achow101>
i'm not sure
< achow101>
ok.. I got the same result after cleaning the cache
< achow101>
*same result as hebasto
< hebasto>
hmm, why cache re-build was not triggered?
< achow101>
sometimes it happens
< achow101>
it may also be that one of the build tools changed and not the dependency itself
< achow101>
and so the output changed as well
< midnight>
Looks like the only real answer is to buy (as anonymously as possible) an SMS-aware new number and then.. **never tell anybody what it is**.. so appleid's 2fa potentially vuln to simjacking (still) via recovery sms. hrm.
< achow101>
I think emzy will need to rebuild then as he had the same result I did
< achow101>
midnight: I think there's a prebuilt sdk somewhere if you don't want to construct it yourself
< midnight>
I saw that. I'm waffling between setting up a pin-protected additional anti-porting number and using the one you guys are using. I'd prefer to maintain independent binary builds as much as possible, jut one of those habits I have like bothering people for additional publishing evidence re: their gitian key. :)
< midnight>
(also exercises the SDK construction instructions while I'm at it)
< emzy>
achow101: how did you fixed it?
< achow101>
emzy: deleted gitian-builder/cache and rebuilt
< emzy>
achow101: ok. I will do that.
< provoostenator>
wallet meeting?
< meshcollider>
#startmeeting
< lightningbot>
Meeting started Fri Jul 17 19:00:14 2020 UTC. The chair is meshcollider. Information about MeetBot at http://wiki.debian.org/MeetBot.
< meshcollider>
I don't remember anything being suggested during the weeks
< achow101>
2020-06-19.log:16:10 < bsm117532> #proposedwalletmeetingtopic descriptor specification for watch-only wallets, and repeated payments without address use via BIP32 paths
< meshcollider>
Is bsm117532 around?
< meshcollider>
I guess not
< achow101>
hmm, that was from a while ago
< achow101>
did we skip a meeting?
< meshcollider>
No, but we may have missed the topic?
< achow101>
probably
< meshcollider>
achow101: do you want to talk a bit about current status of SQLite replacement
< achow101>
ok
< meshcollider>
And provoostenator maybe want to talk about your current goals? Maybe current state of hardware wallet stuff?
< provoostenator>
Sure, but not much changed.
< achow101>
#19334 is nominally the last step before the sqlite PR (#19077) is ready
< gribble>
https://github.com/bitcoin/bitcoin/issues/19077 | wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets by achow101 · Pull Request #19077 · bitcoin/bitcoin · GitHub
< provoostenator>
I'd like some feedback on #16378 now that most of its prerequisites are merged.
< provoostenator>
I see it needs _another_ rebase...
< meshcollider>
Do either of the new projects need updating with new PRs btw?
< provoostenator>
Oh and I need to work something out with AppVeyor :-)
< phantomcircuit>
i dont really see the purpose of sqlite as a backend unless the database interface is changed to actively query the database when you need things
< phantomcircuit>
for sure sqlite is better than bdb, but are we really going to drop bdb support?
< achow101>
meshcollider: 19334 and 19335 need to be added
< jonatack>
hi
< achow101>
phantomcircuit: the goal is to actively query the database at some point
< provoostenator>
phantomcircuit: there's some earlier (IRC) discussion about this, which should probably be linked from that PR
< achow101>
and maybe use the relational stuff too
< achow101>
I would like to drop bdb eventually
< phantomcircuit>
achow101, sure, but that's going to require significant changes to the way the wallet works, also im not sure how useful that's really going to be, even for huge wallets
< sipa>
i don't think the actual db stuff is useful
< sipa>
for us
< achow101>
phantomcircuit: my next major project is going to be significant changes to how transactions and stored and loaded (i.e. not loading every single tx into memory)
< sipa>
but i think sqlite is just the most well-tested storage layer thete is
< sipa>
exactly designed for the sort of app-level compatibility requirements we have
< phantomcircuit>
achow101, you need to have all of the script pubkeys to quickly scan a block, is it really going to reduce memory usage that much to avoid loading the entire transaction?
< achow101>
phantomcircuit: transactions are big, keys are small
< phantomcircuit>
sipa, sure and i agree that sqlite is *better* than bdb, but we're gonna end up supporting both forever and that seems kind of sad to me
< achow101>
rescans are usually a one time thing, not something people do routinely
< sipa>
phantomcircuit: i'd day that in maybe 2-3 years the bdb support can move to some comversion tool
< phantomcircuit>
achow101, you need the script pubkeys to scan a block as they come in
< meshcollider>
Yeah we talked about that before, eventually it should be okay
< phantomcircuit>
if you're reloading those from the database every time you see a new block, you're gonna have a bad time (tm)
< achow101>
phantomcircuit: sure. for now, everything is still being loaded into memory. I would like to move the less used stuff like old txs, address book data, etc. to be loaded as needed
< meshcollider>
This also reminds me of #16910
< achow101>
you don't need to load that tx where every output has already been spent. we don't need those unless someone is digging through their history, in which case we can fetch it. and that doesn't really need to be performant
< phantomcircuit>
sure, but it seems like that work, which is certainly more annoying to do, should be done before adding another database format
< sipa>
phantomcircuit: seems orthogonal to me
< achow101>
I don't see how they're related
< achow101>
now seems to be a good-ish time to introduce sqlite wallets for descriptor wallets only because that's a new thing for storage
< meshcollider>
Plus the database work has already been done so it's kinda too late to say that ;)
< phantomcircuit>
what's the point of another database unless you can leverage that it's a relational database?
< achow101>
well the point is to get away from bdb
< achow101>
at least initially
< sipa>
phantomcircuit: no crazy flushing all the time to harness bdb in not needing active maintenance
< sipa>
phantomcircuit: not needing db environments
< sipa>
not needing a whole directory per wallet
< sipa>
not relying on 10 year old software
< achow101>
meshcollider: I think 18971 can be moved to "Design" in the sqlite project
< meshcollider>
achow101: done
< meshcollider>
sipa: is there any wallet relevant discussion re taproot at this stage?
< meshcollider>
I've vaguely seen a lot more activity around it on twitter
< achow101>
can we support taproot for descriptor wallets only?
< sipa>
achow101: yes please
< sipa>
meshcollider: not at this stage, i think
< provoostenator>
Whaha, did anyone seriously think of adding taproot to legacy wallets?
< achow101>
it would just mean the only waallet changes are descriptor changes
< achow101>
I think
< provoostenator>
We're still cleaning up the complexity from adding SegWit to that...
< sipa>
things like musig signing integration may be a bit more involved, as it requires stateful signers
< sipa>
but even that doesn't need to be supported in a very initial versiin
< provoostenator>
Musig would be real cool
< sipa>
it'll be a lot easier with musig2 ;)
< meshcollider>
What's musig2?
< sipa>
an improved version that is as of yet unpublished, but only needs 2 rounds, and supports transparent nesting (so if you have musig-in-musig you don't need to reveal to your cosigners that you in fact consist of multiple signers yourself)
< meshcollider>
Ooh that sounds very nice
< meshcollider>
With the same security assumptions as musig?
< sipa>
mostly
< meshcollider>
Cool, I like forward to seeing it
< sipa>
we should have more to show soon (real_or_random and nickler really)
< meshcollider>
Any other topics?
< meshcollider>
#endmeeting
< lightningbot>
Meeting ended Fri Jul 17 19:34:18 2020 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< sipa>
it probably helps as a basis for threshold schemes too, but i don't think there is much actual work on that
< phantomcircuit>
sipa, yes i can see why sqlite is better than bdb, but without completely dropping support i dont see how you get any of the advantages
< sipa>
phantomcircuit: i think all of those advantages matter to everyone who uses it
< sipa>
the only thing that we don't get is dropping the complexity and dependency on bdb, which unfortunately needs to wait
< achow101>
phantomcircuit: at the very least, all of the bdb nonsense is shoved into its own self contained corner
< bsm117532>
Sorry, missed that ping re: wallet descriptors.
< bsm117532>
Is anyone else interested in this idea? (Basically, making usable xpubs for watch-only wallets)
< sipa>
that's the point of descriptors
< bsm117532>
I know. I'm basically talking about slightly formalizing a descriptor specification and recommending its usage in this way.
< bsm117532>
Has anyone besides bitcoind implemented descriptors?
< sipa>
i'm hesitant about formalizing descriptors, as i expect there is still a lot of unexplored terrain there
< sipa>
and aiming for compatibility at this point may set unreasonable expectations and/or complicate improvements
< sipa>
especially with miniscript extending the language significantly
< bsm117532>
Well then let's explore it. ;-)
< bsm117532>
What would you want to see @sipa?
< sipa>
?
< sipa>
i just mean we'll likely encounter more use cases and think "oh i wish this could have made it into the descriptors spec"
< sipa>
so i'd rather have it develop organically for a while
< achow101>
we might need some kind of versioning then
< achow101>
but even just adding new expressions just means that older software throws an error saying it doesn't recognize the descriptor
< sipa>
my thinking is that a first step of standardization is having some repository of its supported keywords, but have all of them optional
< bsm117532>
Well I guess then I'm dumping "watch only wallet descriptors" as a use case, at a minimum, and I'd like to see it developed toward a spec and BIP for wallet interoperability.
< sipa>
so that different software can choose to implement a subset
< sipa>
and a descriptor will either do the right thing, or fail to parse
< sipa>
bsm117532: yes, but which descriptors?
< sipa>
do you mean wsh-multisig only?
< bsm117532>
The basics encompassing x/y/z pubs, and multisig would be a great start.
< sipa>
or also 2fa timelocked htlc-based fancy wallets?
< sipa>
bsm117532: if you use the term ypub/zpub you've missed the point
< bsm117532>
how so?
< sipa>
xpubs encode public keys, not what scripts to build with them
< sipa>
ypub/zpub are imho a silly attempt at doing so, in a completely non-existible way
< sipa>
descriptors replace that
< bsm117532>
Of course, so restrict to the most basic of wallets: P2(W)PKH and multisig.
< sipa>
pretty sure that all works fine in anything that supports descriptors
< bsm117532>
Another major problem is keypaths which are highly non-standard (but descriptors take care of).
< bsm117532>
Point is, almost nothing supports descriptors at present. ;-)
< sipa>
yes
< sipa>
that's ok
< bsm117532>
Would you be opposed to a restricted descriptor specification encompassing P2(W)PKH, multisig P2(W)SH, and corresponding keypaths, solely for watch-only wallets? Scripts can be added later.
< sipa>
that's literally exactly what is supported now
< bsm117532>
I know, but the point is to make a BIP and get other wallets to support it too. ;-)
< sipa>
nack
< bsm117532>
why?
< sipa>
i really fear that will make it impossible to make it do actually cool things later
< sipa>
without everyone going "oh, but can't use that because wallet service X only supports multi/sg/wsh descriptors"
< bsm117532>
We're going to be stuck with those kinds of wallets for a long time to come, probably forever though.
< bsm117532>
Arbitrary scripts are an interesting goal, but I think not practical for widespread support.
< bsm117532>
sipa: what more do you want to add or envision arising for descriptors?
< sipa>
i'm hopeful that if descriptors are a bit more mature, and there are libraries in multiple languages with reasonable implementations, then pushing for standardization will result in far more actually usable features
< sipa>
bsm117532: at the very least, miniscript
< sipa>
or a good formalism for dealing with extensions beyond that
< bsm117532>
I would like the ability to easily export watch-only wallets, even independent of descriptors and miniscript. Forward-translating any spec for that into a more advanced descriptor or miniscript will always be straightforward.
< sipa>
ok?
< sipa>
so you're essentially saying you don't need a standard :)
< bsm117532>
Standards are about interoperability...so yes, we need a standard.
< bsm117532>
Even if it's overly simple compared to what's possible.