< jl2012>
As I replace amount+scriptPubKey with CTxOut, the default value of amount becomes -1. It should be ok?
< sipa>
sounds fine
< sipa>
perhaps add an assert to check that it's not -1 for any actual validation
< sipa>
or at least not for a witness validation
< jl2012>
sipa: it sounds already like a serious bug if the validation would use the default value, 0 or -1. Maybe -1 is actually better because it must be invalid
< jl2012>
is there any easy way to list all segwit txs in the mempool?
< wumpus>
I won't be at the dev meeting today - just getting settled in here at SF and have a few days holiday
< instagibbs>
Synced rc3 in 5 hours on my laptop with default dbcache, very tame IO use. Very nice work!
< bitcoin-git>
[bitcoin] sdaftuar opened pull request #11203: RPC: add wtxid to mempool entry output (master...2017-08-add-wtxid-to-mempool-entry) https://github.com/bitcoin/bitcoin/pull/11203
< sdaftuar>
jl2012: i don't see an easy way to list all segwit tx's in the mempool currently, but i just opened #11203 which would make it pretty straightforward (just compare txid to wtxid)
< gmaxwell>
sdaftuar: this should work: ./bitcoin-cli getblocktemplate | jq '.transactions | .[] | select(.hash != .txid) | .txid'
< gmaxwell>
but that only does it with the blocktemplate.
< sdaftuar>
gmaxwell: yeah so that only gets segwit tx's in the top of the mempool, rather than the whole mempool
< sdaftuar>
btw you just improved my life substantially by teaching me about jq
< sstone>
hi, I have a question about SPV nodes: does bitcoin core include tx witness data in filtered blocks sent to SPV client ?
< sdaftuar>
sstone: not currently, though there's an open PR on that topic. see #10350
< gmaxwell>
sstone: if you have a usecase for that I'd like to learn more. I understand codeshark's and I assume we'll do it for his but his is kind of weird and obscure.
< sstone>
our use case is light-weight (mobile for example) LN compatible wallets. There are cases when you will want to monitor the blockchain and extract preimages from witness data
< bitcoin-git>
[bitcoin] danra opened pull request #11205: Make fixed CAmounts and related sanity function constexpr (master...refactor/constexpr-amount) https://github.com/bitcoin/bitcoin/pull/11205
< instagibbs>
gmaxwell, running fine here on windows fwiw
< gmaxwell>
ohcrap this again:
< luke-jr>
IMO a tooltip doesn't need to block final, but since we're waiting a week anyway, might as well do a RC with it? (maybe merge in the -acceptnonstdtxn fix too)
< gmaxwell>
Problem solved! Running bitcoin-qt with the '-resetguisettings' switch fixed it. Thanks to MeshCollider on github for the fix! Smiley
< gmaxwell>
^ that is now the third person I've seen screwed by this.
< gmaxwell>
did we change something that created this problem
< luke-jr>
gmaxwell: that's the crash?
< gmaxwell>
luke-jr: apparently it wasn't actually a crash
< gmaxwell>
the symptom is you start bitcoin and it vanishes after the splash
< sipa>
just the window appearing in an offscreen location?
< gmaxwell>
11:19 < jonasschnelli> gmaxwell: most possible problem of a not-appearing GUI is probably persisted windows coordinates outside of the screen boundaries.
< gmaxwell>
11:20 < jonasschnelli> could be fixed by checking the screen bounds against the QSettings coords
< gmaxwell>
11:21 < jonasschnelli> -resetguisettings will just evict all user based Qt overrides (and things like window coordinates)
< meshcollider>
#11171 for reference
< sipa>
thnaks
< gmaxwell>
it worries me that I've never seen this complain before and now three in a few weeks, all with people testing 0.15rc
< gmaxwell>
complaint*
< luke-jr>
gmaxwell: could it be the Qt version bump?
< gmaxwell>
thats beyond my pay grade to speculate.
< cfields>
gmaxwell: all windows complaints?
< gmaxwell>
yes
< luke-jr>
a shame the users have done the workaround..
< gmaxwell>
it's a pretty bad failure mode, silently gone...
< luke-jr>
if we can get it reproduced again, it'd be nice to get the registry entries involved
< cfields>
i wonder if they're all multi-monitor
< gmaxwell>
luke-jr: well I told one to do it because it was a hail mary... I had no idea it would actually fix it.
< luke-jr>
cfields: hmm, monitors of different sizes maybe?
< gmaxwell>
we can ask.
< cfields>
luke-jr: i was thinking: bitcoin-qt on monitor 2, shutdown, restart with only monitor 1
< luke-jr>
I could totally see different-sized monitors confusing this
< luke-jr>
cfields: am I wrong that we don't check for visible coordinates?
< luke-jr>
err, that we do*
< luke-jr>
(which would probably fail if the monitors are different sizes, due to blocked-off regions within the total dimensions)
< cfields>
luke-jr: no clue
< achow101>
Gmaxwell: I believe it's a registry problem
< achow101>
Since qsettings stores things in registry
< sipa>
achow101: i think the registry is just a storage medium
< gmaxwell>
did we just start doing this... or
< sipa>
i think bitcoin-qt has done that since forver, and nothing changed wrt to that now
< achow101>
Gmaxwell: I don't think so. It's been reported a few times before with older versions
< morcos>
i know i'm the cause of rc3, but i actually advocate for drawing the line somewhere
< luke-jr>
maybe 4) Polish translation update
< morcos>
it wasn't clear to me if we have to wait 2 weeks now until we have a new RC
< luke-jr>
morcos: IMO the positioning issue is sufficient to warrant rc4
< morcos>
if so i think none of those are maybe sufficient for RC4
< sipa>
morcos: let's discuss this when wumpus is available
< luke-jr>
especially if users are encountering it
< morcos>
ok. at the very least lets, note that its a question, instead of a conclusion that there will be RC4
< meshcollider>
yeah and first lets make sure the fix for the positioning issue actually solves the problem lol
< morcos>
luke-jr: but there is a workaround no?
< morcos>
how much valuable info do you lose my resetguisettings
< sipa>
meshcollider: indeed
< luke-jr>
morcos: not a nice one - you lose all your other GUI settings
< achow101>
morcos: all gui settings
< achow101>
it can also be fixed with regedt
< sipa>
achow101 said he'd open a bug with relevant information - let's discuss there when we have that
< luke-jr>
k
< sipa>
#topic full segwit support
< sipa>
i have a question: should we 1) automatically add witness redeem scripts for newly generated addresses, or 2) bypass the restriction that the redeemscript must be available for P2WPKH when the pubkey itself is available?
< morcos>
sipa: could you explain that a bit more thoroughly
< BlueMatt>
sipa: #1
< sipa>
the downside of 2) is that we'd accept segwit payments to converted-p2pkh-to-p2wpkh addresses (which i think is a bad property), but that it significantly reduced the overhead of adding a key
< meshcollider>
I'd say 1 is better
< BlueMatt>
i guess drawback of 1 is you cant receive via segwit to old addresses?
< BlueMatt>
I'm ok with that
< sipa>
BlueMatt: i would call that an advantage
< BlueMatt>
agreed
< sipa>
ideally we don't accept anything to an address that wasn't given out
< sdaftuar>
^ that
< gmaxwell>
we should really try to avoid accepting an address that we'd never issue if at all possible, it's very dangerous if people think they can do that and have it work.
< meshcollider>
I guess there'd still be a way to manually do it if you wanted to though right?
< luke-jr>
indeed, if we accept segwit to non-segwit addresses, people might get a false impression it's supported, and lose money
< Chris_Stewart_5>
is #1 really that expensive?
< sipa>
alternatively, we can also have a boolean in the key meta data saying that the "corresponding" address is segwit
< sipa>
in which case we bypass the need-redeemscript property, but just for keys with that flag set
< sipa>
but at the cost of extra complexity
< morcos>
sipa: is hte issue wiht 1) bloat?
< luke-jr>
but when generating an address, it should only automatically add it if the user wants a witness address; we need to support at least P2SH-wrapper addresses until Bech32 adoption is widespread..
< sipa>
morcos: yes, just bloat
< gmaxwell>
manually sure, it's like importing a key.
< sipa>
luke-jr: that brings us to another question - my view is that we shouldn't support choosing on a per-address basis whether it should be segwit or not; just a wallet-wide flag that you now want segwit
< luke-jr>
sipa: I like the key metadata approach; that lets us refuse non-segwit payments to segwit keys
< sipa>
the reason for that is for interaction with hd auto topup
< gmaxwell>
^ I think so to, wallet wide because of recovery.
< instagibbs>
i think wallets make a lot more sense if by default you do one or the other, and support importing otherwise
< morcos>
sipa: +1 on no per-address choosing
< luke-jr>
sipa: then nobody can reasonably use p2wpkh until support for bech32 is universal? :/
< gmaxwell>
importing one shows that violate the wallet wide is fine.
< morcos>
the whole point of segwit (well one of them) is we think thats the right way to do transactions)
< BlueMatt>
sipa: we could also do that on-disk, but in-memory keep the bloat-y version?
< sdaftuar>
how will the migration to bech32 work?
< instagibbs>
luke-jr, is there any reason we cant return multiple address types? thinking out loud :)
< sipa>
luke-jr: i think we'll be forced to support bech32 as an optional thing and treat bech32 and its p2sh embdeed version identially
< luke-jr>
also, doesn't this mean you can't upgrade existing wallets?
< gmaxwell>
sdaftuar: send to it first, and when ~everyone can send it it, we start generating addresses.
< instagibbs>
if we're doing a hard switchover, we can break api a bit?
< luke-jr>
instagibbs: eww :(
< gmaxwell>
sipa: ugh.
< sipa>
gmaxwell: there's no way we can switch over an entire wallet to bech32
< sipa>
at least not the first ... year?
< gmaxwell>
certantly not today!
< gmaxwell>
yes sure. and
< instagibbs>
luke-jr, elaborate the ew
< luke-jr>
sipa: that's why per-address is useful
< sipa>
luke-jr: but per-address is inherently incompatible with hd auto topup
< gmaxwell>
luke-jr: per-address is not backup durable.
< luke-jr>
hmm
< luke-jr>
what if we use separate HD chains for each type?
< gmaxwell>
why are we expanding scope to recieve BIP173 addresses. I think we should not make this scope expansion now.
< morcos>
gmaxwell: what did you not like, having bech32 and p2sh both supported together?
< sipa>
i think we have two options (which apply to both segwit/legacy and to p2sh/bech32): treat them as identical and accept payments to both, or switch over the wallet entirely
< BlueMatt>
<luke-jr> also, doesn't this mean you can't upgrade existing wallets? <-- yea. this. the discussion about hd-upgrade kinda devolved (I've been mia for a week so may be behind), but it seems to me with the current disk structure we need hd-upgrade before we can do segwit-upgrade unless we want to start forking the -upgradewallet stuff
< gmaxwell>
morcos: because then people can randomly turn your addresses to the other kind which you've never given out and pay you.
< BlueMatt>
yea, I think its unacceptable to do that cause we have shit like breaking uncompressed keys, so we really, really dont want to support people blindly converting addresses, sets a terrible understanding
< sipa>
my view is that we should treat bech32 and p2sh as identical - because, by design, they are identical - every segwit version is supposed to work in both
< luke-jr>
BlueMatt: even the ability to upgrade an existing HD wallet to a segwit+HD wallet would be nice
< gmaxwell>
the pre-segwit vs segwit version of that question is incredibly dangerous. p2sh-embed vs not, is perhaps less so.
< sipa>
but we should not treat segwit and p2pkh as identical
< morcos>
so we can do this in 2 stages? switch whole wallet to p2sh embedded segwit, and then in 6-12 mos switch whole wallet to bech32?
< gmaxwell>
sipa: we should have specified this as part of the segwit bips then. :( but okay, it could be done.
< BlueMatt>
sipa: I'm ok with that
< gmaxwell>
morcos: that was my expectation, and we will have to do is regardless at least in terms of what addresses we return.
< jtimon>
sipa: well, we could , for example switch over enterily for segwit/legacy but treat identical for p2sh/bech32, no?
< sipa>
jtimon: that's exactly what i'm suggesting
< BlueMatt>
morcos: i think sipa is advocating for (and I like) - switch wallet over to "segwit" and give users the addresses in p2sh-embedded form, but really thats a ui-level thing
< * jtimon>
nods
< BlueMatt>
and maybe a flag for "i gave this address out as version X"
< sipa>
however, the bech32 question is not very urgent now
< sipa>
while switchover the segwit is
< sipa>
i guess there are a) support both for our own keys b) use separate hd chains for segwit c) switchover wallet as a whole
< morcos>
BlueMatt: right so we'd be capable of receiving a bech32 payment, but we would not give those out for some time?
< sipa>
i think (c) is best
< BlueMatt>
morcos: yes
< instagibbs>
BlueMatt, and send I hope?
< BlueMatt>
sipa: wait, I'm confused...does c include a and b?
< luke-jr>
(c) will slow adoption
< sipa>
BlueMatt: they are 3 distinct possibilities
< achow101>
(c) works well with multi wallet
< sipa>
(c) means there is a wallet-wide flag that says "SEGWIT: YES", and if so, all new addresses are generated as segwit, and integrated with hd topup
< sipa>
but no separate chains for segwit or not
< instagibbs>
achow101, right, my ledger support will likely simply utilize multiwallet for crossover
< gmaxwell>
sipa: and if we wanted to convert an old wallet we could just import all the keys.
< BlueMatt>
sipa: ah, yes, back to my previous question...what form does that flag take
< sipa>
BlueMatt: i see
< BlueMatt>
sipa: cause its damn-dirty to add a flag like that and not use our versioning stuff
< achow101>
If we do that, we can also implement the optional features thing for wallets
< BlueMatt>
but using our versioning stuff means we need hd-upgrade
< sipa>
BlueMatt: i want to get rid of the version stuff and have feature flags
< BlueMatt>
which i think we need, but may delay things
< BlueMatt>
sipa: but exponential blowup of feature options :(
< sipa>
BlueMatt: yes...
< sipa>
so perhaps the question is: is there any reason why wallets shouldn't have that segwit flag (apart from backup reasons)
< * BlueMatt>
sees no reason to *not* use existing versioning, but only question is possible delay
< instagibbs>
BlueMatt, sorry delay how
< BlueMatt>
sipa: yes, like anything else in the wallet, if user doesnt say -upgradewallet, we'd prefer to not break their backward compat
< BlueMatt>
instagibbs: because if we use the versioning stuff, hd-upgrade must happen first
< morcos>
Someone should write up a more comprehensive wallet plan. The only thing that's clear is we need to support any kind of wallet that has existed in the past
< instagibbs>
Ah, user delay, noted
< meshcollider>
well it would only happen when they choose to upgrade to segwit only, so you can just give them a warning then right
< jtimon>
sipa: what if the payer can't pay to segwit ? I think that's why luke wants to be able to genreate legacy for receiving, no?
< instagibbs>
morcos, some people will be in person in a few days.... would be a good time to review such a doc
< morcos>
But we don't need to support any possible combination.. So we shouldn't have segwit non-HD chain split wallets for example
< sipa>
jtimon: every wallet can send to P2SH
< morcos>
so lets make sure there is no way to create that
< BlueMatt>
morcos: we've generally supported opening new wallets with old versions as long as they are un-upgraded, too
< jtimon>
sipa: , oh, right, never mind
< gmaxwell>
for upgrading I think we'd set the flag, start exploring the segwit address chain from 0 and import all the old keys as whatever they are.
< sipa>
BlueMatt: yes, understood, but apart from that, is there any reason why someone would not want their wallet to construct segwit outputs?
< sipa>
*addresses
< BlueMatt>
morcos: yes, well then we need a "list of acceptable feature flag combinations" against which we check the wallet on load.... :/
< gmaxwell>
sipa: no, only wallets that are trying to stay unupgraded.
< morcos>
or for instance having segwit implies HD-chain split...
< BlueMatt>
sipa: not to my knowledge, no
< sipa>
so do we need a flag at all?
< sipa>
as opposed to just start doing it automatically in a new version
< BlueMatt>
only upgrade, to me
< BlueMatt>
well we'd need users to do a -walletupgrade
< sipa>
i don't think so
< BlueMatt>
errr, I'm not a fan of that
< meshcollider>
If you give them the setting somewhere in Qt then it can give them a compat warning when they go to upgrade it
< BlueMatt>
that'd be the first time we break opening new wallets with an old version by default (and have no way to not break it!)
< sipa>
maybe i'm missing something, but i see no failure scenario
< BlueMatt>
i guess if we do that we should switch to bdb 5
< gmaxwell>
if there is a reason we're unaware of the user can stay on the old version until we add support for whatever we want.
< luke-jr>
sipa: downgrading
< sipa>
luke-jr: elaborate
< sipa>
old versions will produce old addresses, and not add the witness redeem script
< luke-jr>
sipa: if I take my wallet, use it with 0.16, I should be able to use the same wallet with 0.12 again
< jtimon>
do we want to support downgrading wallets?
< luke-jr>
until I use -walletupgrade ofc
< BlueMatt>
sipa: its always been the case that you can pretty easily run your wallet with an old version as long as it is not upgraded
< BlueMatt>
with no significant feature loss
< sipa>
new versions will produce new address, add their redeemscript, and when downgrading, those addresses will keep working
< BlueMatt>
and definitely not with missing transactions
< sipa>
BlueMatt: i don't see a problem
< achow101>
So upgrade it with a version number
< luke-jr>
sipa: old versions don't know how to sign for segwit UTXOs..
< BlueMatt>
sipa: missing transactions
< sipa>
BlueMatt: ?
< sipa>
segwit address will work fine in older versions, if we use the redeemscript add construction
< BlueMatt>
if you receive a payment using segwit and then open that wallet with an old version, it will not be there
< sipa>
yes it will be
< sipa>
at least down to 0.13.0
< luke-jr>
sipa: and it will spend?
< sipa>
luke-jr: yes
< sipa>
luke-jr: the signing code for segwit works fine
< BlueMatt>
mm, fair, though will fail for native segwit, I suppose?
< sipa>
yes
< sipa>
but ignore bech32 for now
< jtimon>
dow we want to support wallet downgrade beyond 0.13 ?
< luke-jr>
sipa: okay, so how does this all work for people who actively do not wish to use segwit transactions?
< sipa>
jtimon: that's a good question
< BlueMatt>
ohhhhhhhhhh, wait, errrrrr, wont everythin break anyway cause it wont deserialize segwit-formatted txn in the wallet pre-0.13.0?
< sipa>
BlueMatt: yes
< BlueMatt>
lol, ok, well we're not fixing that one
< BlueMatt>
so I dont care
< morcos>
we're talking about 2 different things... are you saying if i open an 0.12 wallet in 0.16, then i can't reopen it in 0.12
< sipa>
i guess BlueMatt answered jtimon's question
< morcos>
that sounds like a terrible idea
< gmaxwell>
I think it's fine if use of segwit makes the wallet 0.15.1+
< sipa>
morcos: that's already the case...
< morcos>
if you upgrade it, then yeah of course you don't need to be able to go backwards
< BlueMatt>
gmaxwell: I'm not ok with that, I think
< luke-jr>
sipa: why?
< morcos>
sipa: huh?
< gmaxwell>
wut
< sipa>
morcos: because of what BlueMatt says
< morcos>
i don't think so at all
< BlueMatt>
gmaxwell: I'm fine with it being 0.13.0, which it already is
< morcos>
hmm.
< sipa>
you can receive a segwit transaction that pays you (via legacy output)
< morcos>
i don't think so
< BlueMatt>
gmaxwell: do we have any need to make it 0.15.1?
< sipa>
that will be stored as a segwit txn in your wallet
< gmaxwell>
I bet it doesn't work for 0.13.0
< BlueMatt>
do we get any features from that, really?
< sipa>
which 0.12 won't open
< morcos>
oh maybe
< morcos>
durn
< luke-jr>
ugh
< meshcollider>
Not if you just open the wallet without generating new addresses though ?
< gmaxwell>
BlueMatt: how about testing resources if nothing else... but also you'll corrupt your wallet, when it doesn't know how to extend the keypool.
< luke-jr>
does that mean if 0.12 receives a segwit tx, it will break newer??
< sipa>
meshcollider: there can be a segwit address that pays you via a plain old p2pkh address
< BlueMatt>
gmaxwell: if the wallet is un-upgraded it wont break?
< BlueMatt>
gmaxwell: obviously if its an hd wallet old versions will refuse to open
< gmaxwell>
BlueMatt: if it's unupgraded it's not giving out segwit addresses.
< BlueMatt>
gmaxwell: though testing resources are obviously always an issue
< BlueMatt>
gmaxwell: sipa was proposing that it give out segwit addresses even if it is unupgraded
< luke-jr>
what happens right now, if we have a segwit tx in the wallet without witness data? :/
< gmaxwell>
BlueMatt: how the heck does this even work for HD chains.
< instagibbs>
6 minutes
< meshcollider>
sipa
< sipa>
gmaxwell: it will work fine, unless you downgrade at the same time as you recover
< BlueMatt>
gmaxwell: hd doesnt matter...wait, sipa was saying no new hd chain, maybe we should reopen that part of the discussion :p
< meshcollider>
isnt that only the case if you gave out the address
< sipa>
meshcollider: no
< sipa>
19:53:19 < sipa> meshcollider: there can be a segwit address that pays you via a plain old p2pkh address
< sipa>
^ in that case you don't ever generated a segwit address
< meshcollider>
Can anyone generate that themselves?
< jtimon>
in any case, is the question adding a new wallet version or not?
< BlueMatt>
lol, sounds like we've got a lot to discuss next week
< BlueMatt>
homework: everyone go farmiliarize yourself with wallet
< BlueMatt>
ALL OF WALLET =D
< achow101>
I'm horribly confused now
< sipa>
meshcollider: no, but the sender can be segwit enabled while you aren't
< luke-jr>
BlueMatt: every time I do that, I get the inclination to throw it all away and start from scratch :x
< cfields>
heh, we desperately need to break this discussion up into chunks
< BlueMatt>
cfields: yes
< instagibbs>
can someone whip up a table or something of concerns
< instagibbs>
action item?
< luke-jr>
kanzure has one
< luke-jr>
just needs updating
< achow101>
Kanzure: add to list?
< meshcollider>
oh I see yeah, true
< instagibbs>
oh, for that too, if you're in SF next week
< cfields>
kanzure: and update it so that it's backwards compatible to the last list, please.
< achow101>
Lol
< sipa>
a) how to deal with 0.13.1 through 0.15.0 receiving a segwit transaction and then downgrading to 0.12.x or below
< sipa>
b) how to switch to generating segwit addresses
< sipa>
c) how to switch to generating bech32 addresses
< jtimon>
I won't go this time :( have productive discussions there, and fun too!
< BlueMatt>
d) how to deal with 0.15.1 downgradingg to 0.13.1 (in both hd and non-hd modes)
< BlueMatt>
e) whether to use a new hd chain for segwit addresses
< sipa>
e falls under b
< BlueMatt>
err, ok
< gmaxwell>
BlueMatt: please just don't support that. It makes no sense to back and revalidate segwit wallet support in old versions.
< gmaxwell>
we will not manage to adequately test it and it will sharply constrain our implementation choices.
< sipa>
any last minute short topic?
< BlueMatt>
gmaxwell: then we should explicitly break it...but, anyway, lets have this discussion next week
< gmaxwell>
and lead to weird corner case bugs.
< instagibbs>
sipa, activate schnorr?
< sipa>
#endmeeting
< lightningbot>
Meeting ended Thu Aug 31 20:00:02 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< jtimon>
gmaxwell: besides, is downgrading such an important use case?
< BlueMatt>
if we dont support downgrading, I very, very strongly support switching to bdb5 for wallet
< gmaxwell>
instagibbs: schnorr is cancled.
< instagibbs>
gmaxwell, ah shite.
< chainhead>
aggregated signatures
< gmaxwell>
Yea, AggSig.
< luke-jr>
BlueMatt: what does bdb5 have over 4?
< meshcollider>
eventually we want to remove bdb entirely right
< sipa>
nothing we care about, except likely being slightly longer supported in OSes
< achow101>
luke-jr it's actually in package managers by default
< BlueMatt>
luke-jr: not having every goddamned distro build with the backward-compat-break option?
< esotericnonsense>
is there a specific goal in mind with regard to downgrading? a 'support period' as such? it seems to me that defining that would be useful. say, 1 major version back, otherwise all bets are off.
< BlueMatt>
luke-jr: its purely a make-distros-keep-our-wallets-sane thing
< luke-jr>
sipa: that's never applied to wallets though
< sipa>
0.12 is EOL
< luke-jr>
wallets were supposed to remain compat to 0.3 even :/
< sipa>
we still stupport wallet files from 0.3.0 (and even lower, i think)
< BlueMatt>
luke-jr: well the second segwit activated that broke - we can now put transactions in the wallet (today) that have witnesses and those versions will fail to deserialize, I presume
< BlueMatt>
er, yes, but loading them should still be fine
< luke-jr>
BlueMatt: sounds like a bug :(
< sipa>
downgrading that far back is probably broken in multiple other ways too
< luke-jr>
BlueMatt: why don't we store them stripped?
< sipa>
luke-jr: iirc, no
< luke-jr>
sipa: ?
< sipa>
if i recall correctly, we do not store wallet transactions stripped
< luke-jr>
"why?" is not a boolean question :p
< * sipa>
lunch
< esotericnonsense>
i would be curious to know how many users actually express a desire to downgrade significantly (i.e. beyond some sort of emergency 'we found out .15.1 is broken, go back to .15' scenario). but I feel that I'm interrupting and so will shuffle off.
< jtimon>
so our goal is to make 0.15.1 wallets dongradeable to 0.12 ? that's I think too ambitious...
< BlueMatt>
esotericnonsense: I think its really just an "emergency-need-to-downgrade" support thing
< BlueMatt>
which, realistically, just means you need to be able to downgrade to the latest stable of the previous version
< morcos>
we have to stop using confusing technology
< morcos>
we don't support any downgrading now do we
< morcos>
we just support not-upgrading
< BlueMatt>
jtimon: I dont think thats possible, 0.13.1 may be possible, but, indeed, is also ambitious
< morcos>
or at least we used to
< BlueMatt>
morcos: i think you can create an old-version wallet
< luke-jr>
morcos: we've always supported downgrading 0.X wallets back to 0.X
< morcos>
why are you calling downgrading it
< BlueMatt>
morcos: we theoretically try to avoid breaking downgrading if you did not explicitly upgrade your wallet
< BlueMatt>
(with -walletupgrade)
< morcos>
if you take 0.10 wallet and run it in 0.14, it doesn't get upgraded automatically does it, its still a 0.10 wallet
< luke-jr>
morcos: that's the goal, yes
< BlueMatt>
correct
< morcos>
so you don't have to downgrade it to use it in 0.10
< luke-jr>
morcos: apparently we broke that in 0.13.1
< morcos>
you never upgraded it
< BlueMatt>
grrr, terminology
< morcos>
yes i understand
< morcos>
but stop saying downgrade
< BlueMatt>
you downgraded your node
< morcos>
that implies you take a bech32 segwit hd-split schnorr sig aggregation wallet with tons of txs and convert it back to some old format
< luke-jr>
it sounds like a simple fix would be to just store txs stripped in the wallet
< sipa>
luke-jr: a reason to not stre wallet txn as stripped: if you rebroadcast an unconfirmed tx, it needs to include the witness
< morcos>
not that you were using an old format wallet in a new piece of software but not using any of those features
< jtimon>
BlueMatt: right, to 14.2 sounds reasonable " just means you need to be able to downgrade to the latest stable of the previous version"
< luke-jr>
sipa: does this break right now, if we received the tx with 0.12?
< * BlueMatt>
-> food
< GAit>
jnewbery: want me to just squash or rebase too?
< BlueMatt>
to-be-continued next week :)
< luke-jr>
BlueMatt: kk
< luke-jr>
I need to get back to something too, so next week sgtm
< meshcollider>
next week is sf right? will you guys have a meeting summary online or something for those that aren't going
< gmaxwell>
sipa: can you propose a BIP modification to say that anything that accepts p2sh embedded segwit should also accept the non-embedded form or something
< gmaxwell>
I can buy your argument that it's reasonable to support both forms.
< gmaxwell>
But it's weird if if its totally adhoc.
< luke-jr>
seems a bit late for that
< luke-jr>
do other wallets support it?
< gmaxwell>
::sigh::
< gmaxwell>
it's bad if its inconsistent.
< gmaxwell>
"I converted this address, and it worked, I converted that address and the funds vanished into space"
< luke-jr>
people shouldn't try to "convert" addresses anyway. :/
< gmaxwell>
indeed, which is why I think it's bad to add both key types.
< kanzure>
yep, wallets should only be expected to receive whatever was given
< kanzure>
(by getnewaddress etc)
< gmaxwell>
but we do end up with another migration problem with the later switch to bech32, which would be sad.
< sipa>
luke-jr, gmaxwell: unfortunately, that means we'll be forced to have a separate chain for bech32 addresses
< sipa>
perhaps that's the best solution
< instagibbs>
sipa, in the case of migrating again?
< luke-jr>
probably more future-proof too
< gmaxwell>
sipa: that is what I expected us to do
< gmaxwell>
we'll also have this same thing for future sig versions like a v1 with aggsig.
< gmaxwell>
as an aside the masterkey lines in our dumps should include flags for this stuff
< gmaxwell>
but I coudl also buy the parallel for v0 and p2sh embedded... it would have some migration advantages.
< gmaxwell>
(I'm worried that we shouldn't do anything to delay supporting this though.)
< morcos>
I'm not sure I understand why you have to switch to a new chain. Why can't you just "upgrade" your old chain to support a new address type.. or is the idea that once you support bech32 you'll no longer support the p2sh version
< morcos>
or that a new bech32 chain won't i mean
< sipa>
morcos: if we want to be strict, and only accept payments to bech32 when a bech32 addr was given out, and p2sh/p2wpkh when that was given out, they need to be separate chains, or you need to drop support for one of them entirely
< gmaxwell>
so if you are adding keys for both types, it means that people can 'convert' your addresses and have it work, which is a funds loss risk when they do it someplace where it doesn't work.
< sipa>
otherwise you can't know when rescanning which is one
< gmaxwell>
We could take a position that we'll always do dual p2sh embedded and native (at least for the forseeable future) in parallel... but that will still be a risk for every wallet that isn't us, and for bitcoin core users with explicitly imported keys, when something thinks it can convert.
< gmaxwell>
and we cannot do dual-addresses for pre-segwit+segwit because of things like uncompressed keys.
< gmaxwell>
Because as sipa points out p2sh embedding would work for _all_ segwit, we could realistically do dual support there.
< morcos>
I don't know. I guess I'm not 100% sold on the philosophy. It kind of seems to me that if someone does pay you to an address you didn't give them.. You're going to want to find some way to recover those funds no?
< morcos>
I mean I get why we don't want to encourage a culture of loosey-goosey hide the key in your backyard or however you put it gmaxwell
< gmaxwell>
morcos: thats kind of the issue... then your keys are embedded in an HSM and it's virtually impossible for you to do that... or to do so you need to do dangerous crap with private keys,
< gmaxwell>
So basically if this is supported as a reasonable and customary practice, then it creates an effective obligation to do it.
< morcos>
But to me there is a tenous link btwn the fact that the wallet would be smart enough to accept it and people actually being encouraged to do it
< instagibbs>
also doesn't this basically require someone to understand bech32, but decide to wrap in p2sh?
< morcos>
You could even add some sort of flag or warning on txs that were received using unexpected address types
< luke-jr>
if someone pays to an address I didn't give them, they burned the bitcoins. they didn't pay me.
< instagibbs>
most cases should be the sender simply not understanding the bech32?
< gmaxwell>
morcos: things that work are what people do, this is the lesson from history -- in bitcoin but also in every internet protocol. If we make it work we need to be ready to support it.
< gmaxwell>
People deciding what to do don't read specs, they try them out and they do whatever works.
< gmaxwell>
I think we could reasonably support p2sh-embedded and segwit duality. Sipa points out that it'll work universally.
< morcos>
well perhaps we could solve the problem i'm more concerned with in a different way by making different paths in the HD wallet
< morcos>
so the same key never was used for both
< morcos>
but you don't have to change master keys
< gmaxwell>
morcos: you do that by using a different path.
< morcos>
the issue i don't like is invalidating peoples backups
< gmaxwell>
Which is what other wallets are also doing for segwit suppot, fwiw.
< morcos>
so when we're talking about switching to a new chain for bech32, we'r ejust talking about a new path?
< sipa>
morcos: yes
< morcos>
that seems much more reasonable to me
< gmaxwell>
I assume but the backup is somewhat invalidated because the backup doesn't have the metadata to tell it what paths are used. This is also true for the hdsplit.
< sipa>
and perhaps if we plan to do that, we should do the same for p2sh-p2wpkh
< morcos>
ignore my objections then... seems like we should do a quick online survey of other wallet providers and assess whether most have expected that p2sh wrapped == bech32, that is if you accept one you accept the other
< sipa>
there is a distinction in that bech32 or not is probably something we do want to do on a per-key basis
< morcos>
and unless thats nearly universal, we shouldn't introduce it
< luke-jr>
gmaxwell: backups can be expected invalidated when -walletupgrade is used
< gmaxwell>
Esp for BIP173; the transition can't really be done abruptly unless its seriously delayed.
< instagibbs>
morcos, most are doing bip49 I think, which only is about nested
< morcos>
luke-jr: why? shouldn't you just be able to -upgrade your backup
< gmaxwell>
while if we do dual embedded and native then probably I can use bech32 on day one, e.g. my exchange supports it, so I have it pay me via a 173 address.
< instagibbs>
not sure what they're expecting to do with bech32
< sipa>
morcos: that doesn't work for upgrading to hd, to hd split, nor for adding new chains
< gmaxwell>
morcos: just the backup doesn't know where and when the pathing change happened.
< luke-jr>
morcos: hmm, it might work in this case, but IMO we shouldn't *expect* it to
< gmaxwell>
i suppose you could replay the same actions and have it work, but it's tricky and easy to screw up.
< gmaxwell>
e.g. backup your wallet on day 0 with no segwith, use 100,000 keys... upgrade it later... use another 10000 keys... erase wallet... now how do you reproduce this...
< gmaxwell>
you need to know to expand the keypool 100,000 keys, then switch...
< luke-jr>
gmaxwell: if segwit uses a new path/chain, this would just work I guess
< morcos>
well, i was assuming the path branch happened at the beginning, but maybe that was a bad assumption
< morcos>
in which case you'd just look ahead whatever the number of keys is on all paths
< gmaxwell>
yes, it should happen at the beggin... how do we know to go to 100,000...
< gmaxwell>
okay, thats a more keypools solution.
< luke-jr>
gmaxwell: start monitoring both chains at their beginning, and extend each as needed
< morcos>
the same way we know to go to 100k if we never did anything and you backedup your wallet with nothing in it
< gmaxwell>
perhaps... this has some other tradeoffs.
< morcos>
i don't understand how you don't have to do that anyway
< gmaxwell>
e.g. say we have 4 segwit versions, and so now we have 10 keypools.
< gmaxwell>
but only one of them has ever been used.
< morcos>
so what happens when you upgrade your wallet? don't you have to still keep keypools for your old chains anyway?
< morcos>
or are you suggesting the upgrade is the user saying, i have no pending payments forever in the future for any address i've ever given out
< gmaxwell>
yep. that was my thinking.. just import all the keys that are already used.
< luke-jr>
gmaxwell: if we're worried about it, we should just stop using keypools?
< adiabat>
not to complicate things further, but my wallet software supports bech32 / p2wpkh but ignores nested completely
< gmaxwell>
it's not even clear to me that we should support upgrading.
< gmaxwell>
for something that changes your key paths perhaps we should only support that for new wallets. If you want old keys in it, you can import.
< sipa>
gmaxwell: there is no 'import'
< gmaxwell>
importmulti
< sipa>
gmaxwell: the effect of an import is exactly the same as just adding a key
< sipa>
gmaxwell: i mean that every key we generate is implicitly 'imported'
< gmaxwell>
import doesn't have anything to do with maintaining a keypool.
< gmaxwell>
yes, it's that PLUS keypool management.
< luke-jr>
time to reboot, bbl
< sipa>
gmaxwell: i just mean that is what we already doing
< sipa>
gmaxwell: saying "switching to a new keypool and treating all the existing keys as imported" is exactly the same as "switching to a new keypool"
< gmaxwell>
yes, but it's distinct from following all keypools which is what I was discussing with morcos.
< morcos>
gmaxwell: yeah i think if i understand what sipa is saying, once you've made the decision to start giving out addresses on a new chain, you no longer need to maintain a keypool for your old chain
< sipa>
ok, s/switching to/adding/g in my above statement
< sipa>
still same thing - the 'importing' thing is besides the point
< morcos>
youv've already got the current keypool as keys in your wallet, and its exactly as if you imported
< gmaxwell>
morcos: right thats an option, but it's not 'backup durable'
< morcos>
why not?
< gmaxwell>
because you restore a backup then don't do the upgrade or do instantly before scanning all the keys
< morcos>
hmm.. i see, its just more complicated to get it right i suppose
< gmaxwell>
and when we do the upgrade to we trigger a full (pruning incompatible multihour long) rescan
< morcos>
but the advantage of supporting 2 keypools is we can start issuing bech32 earlier
< morcos>
which seems a big gain
< gmaxwell>
Or we could just do dual-embedded and bech32.
< morcos>
hmm
< gmaxwell>
in which case it's just one keypool and we could use bech32 all the time, but there is a risk of people converting.
< gmaxwell>
For rapid bech32 deployment that is best by far.
< morcos>
but also means we can't ever easily stop supporting the old style
< gmaxwell>
because it lets you immediately use bech32 when you have a counterparty that supports it.
< gmaxwell>
morcos: right, we could for newer wallet 3 years from now, except for the risk of 'conversion'
< sipa>
just in case that isn't clear: if you currently use addwitnessaddress, you'll accept both p2sh and native-witness outputs to that address
< morcos>
and if you don't you'll accept neither?
< sipa>
yes
< morcos>
well i do think it makes sense to think about this comprehensively in the context of legacy -> hd -> hd-split -> p2shsegwit -> bech32 -> future witness versions
< morcos>
there are different issues for each of those
< morcos>
but i think we want the design to be something where people don't get confused understanding how it works
< gmaxwell>
we really cannot do legacy and segwit doppleganging due to the uncompressed key issue.
< morcos>
Imagine that each of those is considered a different path or whatever, and we introduced some concept of whether a path is active or not (meaning we are still potentially giving out addresses on it) and once it's not active we don't need to maintain keypools anymore, we just have the historical keys
< sipa>
gmaxwell: easy enough with a bit in the metadata
< gmaxwell>
I think we can realistically do native + embedded doppleganging and support it forever.
< sipa>
or just the addwitness approach
< gmaxwell>
bit in the metadata is a backup disaster.
< morcos>
When you upgrade you need to specify which ones you are still active on.. And there is a way to deactivate old types
< sipa>
gmaxwell: it would also be added during hd topup
< gmaxwell>
sipa: I don't understand how that works.
< gmaxwell>
I mean where do the bits come from
< sipa>
gmaxwell: once your wallet is segwit-enabled, you add it for every newly generated key (including auto topup)
< sipa>
it means you can't recover with an old version anymore, but that's inevitable
< gmaxwell>
but then you recover an old backup with a new wallet... will it then set the bit for everything
< sipa>
ah, i see
< gmaxwell>
I'm sorry, I think this is all confused; or I am all confused. Esp since we don't even really know if we are recovering or not at any point in time.
< morcos>
gmaxwell: I think when you said "that's a more keypools solution", I took that as a negative, but i'm not sure why it has to be a negative. Have a keypool for every path / type of address. Who cares?
< morcos>
I don't think it causes any wallet bloat if you're comparing it to importing your keys to a new wallet as you upgrade. It it causes in-memory bloat, that seems easy enough to optimize away, by leaving most of unlikely to be used keypools on disk.
< gmaxwell>
two keypools, technically. ends up being a megabyte of keypool for each or whatever with our current inefficient storage...
< gmaxwell>
(change and not change, is the two pools)
< morcos>
Yes, but don't you get that anyway, if you import to each version in turn as you're importing all the prior keypool keys
< gmaxwell>
We could do that. Also slower accepting blocks, because you now have a lot more keys to scan... but thats something that generally may need to be optimized.
< gmaxwell>
morcos: if you do that, but most users eventually will have just started eventually with vXX and won't have any old ones
< morcos>
yes, and so then you don't create the old ones
< gmaxwell>
so then there is key metadata which says the oldest kind to build, and we build all later ones.
< tloriato>
Hello. I was testing the development of some ideas using the BitcoinCore wallet and the CLI's commands, and I came across the move command. Unfortunately the Docs states that "move will be removed in a later version of Bitcoin Core. " Does another command fulfils its duty? I've read the documentation on the official webpage and came out with empty hands. I don't want to develop an architecture around a command that will be e
< sipa>
tloriato: the whole accounts subsystem is deprecated
< sipa>
there won't be a replacement for move, as there would be nothing to observe its effect
< sipa>
addresses will be able to have labels, and you'll be able to query for payments to specific labels, but labels don't have their own balance
< tloriato>
i see it
< tloriato>
well, thanks for the answer and clarification
< tloriato>
i'd assume that the parameter to GetNewAddress would be the label instead of account when that happens?