< kanzure>
i am unsure of the interactions between MIN_FINAL_CHANGE and the targeted fee rate and whether that should be the assumed fee rate for eventually spending the change output.
< kanzure>
oh, "achow101's implementation uses a 1008 block feerate estimate" for the change output amount check before decide burn to miner fee?
< achow101>
kanzure: the 1008 block feerate estimate is for the cost of change. It's the maximum of the feerate and the min relay fee
< kanzure>
"and now you're also throwing away the change output itself,so you are potentially overpaying the newly needed fee by double the cost of creating the change" <--- why not redo the final size estimate and calculate new fee, then redo coin selection for potentially fewer inputs? keep list of n best options, break after grinding for i dunno 10k rounds.
< gmaxwell>
because complex hairball.
< gmaxwell>
Achow's stuff doesn't have that issue, the branch and bound thing assumes always that there will be no change.
< kanzure>
is there a writeup of a wishlist for simulators in this area, or is that also hairball land?
< kanzure>
there was already some area in core that attempts to redo coin selection by increasing the amount and trying again; some of this could probably be consolidated.
< kanzure>
i have this weird case where i have to use two "change" outputs, one of them i can burn to miner fee if it's too small, the other one i need to go back and pick other inputs until i can piggyback. anyway, lots of questions around utxo results after actual usage, i will have to write a simulator at some point.
< kanzure>
(can't help but think of this like iterative rocket equation calculation until elon musk is sure that his rocket will land at zero velocity with zero fuel left, except for coins we're not targeting total fee exhaustion of course.)
< fanquake>
Would anyone object to #8550 going into 0.15 ?
< gmaxwell>
Thats the graph that doesn't break things up by feerate?
< gmaxwell>
those sorts of graphs have had really bad effects in social media.
< gmaxwell>
People flood with minrelay fee txn to push the numbers up. and then spam with OMG MEMPOOL FULL SELL BITCOINS NOW
< gmaxwell>
Since rate breaking up graphs became more common those minrelay floods seem to have stopped.
< fanquake>
Well we can adjust the graph to break the txs up and display however we'd like, but I think having that info available to node operators would be a plus.
< morcos>
fanquake: that PR is based on #8501 which seems like hasn't gotten work in quite some time
< morcos>
in any case I don't think it's making it for 0.15 at this point
< fanquake>
morcos Fair enough. Will concentrate on other PRs. Need to put aside some time to look through all your fee work.
< jtimon>
sipa: something seems wrong with how can I access the CTxOut of an input with AccessByTxid(/Coin in.prevout.hash but without needing in.prevout.n?
< jtimon>
I bet I'm missing something, but not sure what
< jtimon>
oh, there's no in.prevout.n anymore, nevermind
< jtimon>
wait...I'll think more about this, I know how to fing the PR that's relevant
< gmaxwell>
jtimon: you cannot. The database doesn't efficiently support that access anymore, it shouldn't generally be needed.
< jtimon>
gmaxwell: my point is...how can it even guarantee the input looked for is the one that is gotten without passing n? it seems like all instances of AccessByTxid should be replaced wiht inputs.AccessCoin(tx.vin[n].prevout) or a similar wrapper
< jtimon>
summary: AccessByTxid seems completely uinsafe to me at this point unless I'm missing something
< jtimon>
which is not uncommon
< sipa>
jtimon: AccessByTxid is only for finding height/coinbase
< sipa>
obviously you can't use it to find an actual output
< jtimon>
oh, I see, that's what I was missing
< sipa>
it's also extremely slow
< jtimon>
it still seems weird that you have access to some arvitrary txout from it though
< sipa>
i guess
< wumpus>
please don't do anything on transifex, especially with the "0.15.x" resource, I was running the script to copy over the translation strings from 0.14 and something weird happened, not sure why but it looked like someone overwrote the translation strings (even though it's locked)
< jtimon>
it shouldn't retun a full coin class, or something, anyway, thank you for the missing piece
< sipa>
seems very reasonable to document that
< sipa>
gmaxwell: finding CTxOuts is totally doable though... you just need AccessCoin not AccessByTxid
< sipa>
(Coin contains a CTxOut and fCoinbase and nHeight)
< bitcoin-git>
bitcoin/master 191d12b Wladimir J. van der Laan: qt: First translations update for 0.15
< wumpus>
transifex copy was successful this time - copied translations (+metadata) from 0.14 to 0.15, and updated 0.15 resource with new messages, set to auto-update from master (don't forget to change when 0.15 branches off), unlocked - should be good to go
< morcos>
achow101: instagibbs: Murch: I have a couple questions about branch&bound, effective value and change
< Murch>
okay
< morcos>
So I see how we are accounting for output size via output_fee and input_size via nInputBytes
< morcos>
But how are we accounting for fees paid on the fixed part of a tx
< morcos>
Won't we necessarily fail if we get too close to an exact match?
< morcos>
Second question: Why are we using the longest possible estimate for the creation of change?
< Murch>
We first get an estimate for the fee rate. Since we know which outputs we want to create for recipients, we can calculate the amount of fee for the outputs.
< Murch>
We add that and the cost of the transaction overhead to the target
< Murch>
So, when we select inputs, we can deduct the cost of the inputs from each that we select and thus we have accumulated all fees necessary.
< morcos>
The transaction overhead piece is the part I was missing
< morcos>
maybe i just missed it
< Murch>
Do you mean specifically in the code? I think that achow101 was missing something there, and we discovered the bug in review recently. I'm not sure if he already fixed it.
< morcos>
ok. yeah thats what i meant
< morcos>
i'm mostly asking questions about his code
< Murch>
Re 2nd Q: BnB only creates transactions without change outputs. Since we can account for the fees of the overhead and outputs in advance, and the fees for the inputs on the fly, we're good. So I don't get what you mean with "creation of change".
< morcos>
I mean the cost of change_
< morcos>
we're calculating it using a smart fee estimate for 1008 blocks
< morcos>
when in reality it'll cost us whatever fee we're using for this transaction creation
< Murch>
Well, actually, this is the part that I'm not comfortable with yet.
< Murch>
Obviously, the only amount that we're clearly saving is the cost of creating a change output.
< Murch>
However, in my simulation, I got a cost reduction by allowing the larger window of "cost of input+output". I was assuming _fixed feerates_ though.
< morcos>
ah i see, i'd missed that
< morcos>
well what i'm saying is something different again
< Murch>
I think that it would need some experimentation to determine whether it is actually a net-benefit to have the larger window and thus save the change more often or not.
< morcos>
actually waht we might want is something similar to what i did in #10712
< morcos>
but my point was the change_feerate we are using is possibly too low
< Murch>
One of the Trezor people implemented BnB for BitcoinJS this week, and came to the conclusion that "cost of change = change * current feerate" has greater savings.
< morcos>
maybe that is a good feerate to use for assumption of spending the change output in the future
< Murch>
So I would suggest that we go with that first.
< morcos>
might it make sense to cache the best result found so far and keep searching
< morcos>
and have two thresholds, one which is good enough to stop searching further, and one which is good enough if its the best thing we came to at the end of the exhaustive search (or hitting max tries)
< morcos>
In any case we're possibly willing to just throw away the amount of fees determined by change_feerate right? i'd be very hesitant to just use 1008 for that
< * instagibbs>
reading backlog
< morcos>
I've seen that number as high as 100 sat/byte, not sure poeple would want to throw away that much, at least not without looking for a better exact match
< Murch>
My gut feeling is that this would increase the size of the selected input sets. That would both increase the variance of the input set size, and perhaps reduce the utxo pool more quickly, perhaps reducing overall effectiveness of BnB.
< morcos>
sorry, what would?
< Murch>
More experiments would inform us, I guess.
< Murch>
finding the "best solution".
< Murch>
Also would probably increase the search times a lot
< Murch>
mh
< Murch>
sorry, I'm too slow
< morcos>
yeah i was a bit concerned about that
< Murch>
So 1) for the window of determining the exact match, I would use the same fee rate as for the transaction * size of a change output.
< morcos>
Yes even plus the dust threshold of the output itself
< morcos>
as done in 10712
< instagibbs>
Murch, im sorry you mean don't consider a future spend of it?
< Murch>
2) I would go with the first solution instead of the best solution, because it reduces variance in input set size, reduces computation time, and will probably be more conducive to finding many exact matches.
< instagibbs>
oh sorry, you mean use same feerate for both output size and future input
< morcos>
I'll defer to your judgement on that
< morcos>
no instagibbs i think he means don't consider future input
< instagibbs>
due to the bitcoinjs experiments?
< morcos>
but i'm suggesting we should consider future input via GetDustThreshold
< Murch>
instagibbs: I'm concerned that we're overestimating the saved cost
< morcos>
In addition to current fee rate times size of creating output
< Murch>
So, my experiments for my thesis assumed a fixed fee rate for the whole cycle. This allows me to be sure of the saved cost of "input + output"
< Murch>
Fixed fee rate is not a valid assumption IRL, so it's hard to estimate the saved cost for the input.
< Murch>
We see frequent changes in fees in a range of factor 50.
< instagibbs>
Yes, but if fees really do move up on longer-term I don't see why we'd ignore it
< Murch>
Now, currently there are four implementations of BnB that I'm aware of: My science project, Core, BitcoinJS, and what I'm working on for BitGo right now.
< instagibbs>
more aggressive non-change making has privacy advantages on top that I don't think we should ignore
< morcos>
I think we have to distinguish between giving up and generating the change (in which case htere is a cost) and saying ok well this solution isn't close enough that we're willing to just discard the difference
< Murch>
Karel implementing the BitcoinJS one, has done some more experiments and informed me that just using the "output as cost of change" resulted in lower total fees.
< Murch>
instagibbs: Smaller window might cause larger input sets, so it might actually work towards that end. It's hard to tell. ;)
< instagibbs>
Murch, what about how many change outputs are made vs
< morcos>
and potentially trying again
< instagibbs>
and how much are you saving
< instagibbs>
not asking for an answer right here, just think it's important to consider
< Murch>
instagibbs: I don't know. Maybe I was overestimating that effect myself, as I asked Karel whether the rate dropped significantly by making the window smaller.
< Murch>
instagibbs AFAIU, it didn't though.
< morcos>
This all depends on assumptions about the distributions of the utxos in the pool. It's going to be different for different people. In some cases making window larger will just cause you to waste money
< Murch>
instagibbs: Yes definitely need to consider that. Also the average input size and whether it exhausts our smaller inputs too quickly to do later exact matches.
< morcos>
B/c you would have found a smaller exact match. In other cases a larger window will allow you to find an exact match when you otherwise wouldn't have
< morcos>
Do we have any good data sets for a large number of users to evaluate this on at all
< Murch>
morcos "…wasting money…" exactly.
< instagibbs>
not public ones
< instagibbs>
:P
< Murch>
instagibbs: yep.
< Murch>
I'm gonna do a little more experiments in the coming week to evaluate the algorithm for our internal use. I also have another project though, so I can't give you a timeline. I might have more information on some point though.
< Murch>
Can't share the data of course. ;)
< morcos>
Does the KnapsackSolver still try to find an exact match if BnB fails?
< morcos>
Is that worth doing still?
< instagibbs>
likely not worth it
< morcos>
I'm just trying to figure out whether we cna assume we'll probably have change if BnB fails
< instagibbs>
I think we should toss all of that and assume we get change
< instagibbs>
we can definitely get data for that once we have BnB being used...
< morcos>
If so, then we definitely at least want to use the current fee rate times the size of the change out put, plus dustthreshold of output size
< morcos>
Whether we want to do that on a first pass or maybe do two passes or something, I don't know...
< instagibbs>
"use" in what sense, sorry
< morcos>
for the cost_of_change
< instagibbs>
k
< instagibbs>
remind me what the second term is accomplishing?
< morcos>
Also I think we need a lot more reasoning about how we intermingle the (BNB, knapsack) ordering with the (MinConf=6, MinConf=1, MinConf=0 (various chain lengths) ..) ordering
< instagibbs>
just a small window on top?
< morcos>
You can't create a change output smaller than DustThreshold anyway
< instagibbs>
I just need to read the branch again, ignore my q
< morcos>
so if you found a match where once you paid for the fees required to create the change output (at the feerate this tx is using) you only have < dust left for the change, then you'd just eliminate the change later anyway. So why not look for an exact match
< morcos>
Right now it looks to me at a cursory glance like we first try using BNB on all of the minConf orderings... So potentially we are going to create a long chain of unconfirmed txs just b/c they don't generate change (luckily that's anti-self-reinfocing)
< morcos>
That doesn't seem to me necessarily what the user wants? But I don't know what the right order is.
< morcos>
It would be good if there was a lot more documentation about the logic in that PR. The old code was already way under documented, lets not repeat that mistake
< instagibbs>
that block of SCMC could have a comment header explaining the steps
< Murch>
sorry, something came up. I agree that the KnapsackSolver should not try to find an exact match anymore. It's also not good at it anyway.
< Murch>
In most cases when it would find an exact match, it would throw it out because it can't pay the fees in subsequence.
< bitcoin-git>
bitcoin/0.14 ef810c4 practicalswift: [trivial] Fix a typo (introduced two days ago) in the default fee warning...
< spudowiar>
To confirm, UniValues can only be appended to, not modified, right?
< instagibbs>
seems that way
< spudowiar>
Copying the object to modify it seems wrong
< spudowiar>
I might copy the function from core_write to CWallet and modify it for my usecase
< spudowiar>
There are quite a few neat changes I could make then
< wumpus>
correct - you shouldn't need to edit univalue object, either consume them or generate them
< wumpus>
they're meant for being used on the interface, not meant as a lasting data representation format
< instagibbs>
morcos, interesting to note that a successful BnB on a long chain ends that chain for the user. Once we're using effective value everywhere you can do two quick SelectCoins calls, but then you still have that judgment call of which is better.
< morcos>
instagibbs: yeah its not obvious to me what the right outcome is, but i think we explicitly need to think about it. I think we'd do better creating change from confirmed outputs before extending a chain, at least until we do something smart with whole chain fee control.
< morcos>
but for instance we might prefer to create no-change from 1-confirm inputs before creating change from 6-confirm inputs
< instagibbs>
Indeed that would be better, but I'd still like to revisit BnB if we cant do knapsack w/ confirmed
< instagibbs>
Throw-away idea: Let the transaction construction loop happen once, never accept the first result, then you can compare the two,
< instagibbs>
the first will always fail anyways
< morcos>
sipa: style ruling please, this time i think i checked developer notes first. If I have a static class member like CWallet::fallbackFee should that be g_fallback_fee or m_fallback_fee?
< sipa>
not a constant?
< morcos>
well its a command line argument
< sipa>
i guess technically that would be a member field, but personally i very much think we should avoid non-const static members, and make them globals instead
< wumpus>
it's an interesting case
< morcos>
ok, i'm happy to do that. just global but declared in wallet.h ?
< sipa>
sounds good to me
< wumpus>
not convinced a global is better
< sipa>
static class members effectively are globals
< sipa>
they're just abusing the class as a namespace
< wumpus>
at least the class provides some kind of scoping
< morcos>
thats why my guess was CWallet::g_discard_rate
< wumpus>
well it's better than throwing everything into the global namespace
< sipa>
i guess the right approach is to actually have good namespacing
< wumpus>
sure
< wumpus>
but now it could collide with something e.g. outside of wallet, it's not clear it's for the wallet
< wumpus>
m_fallback_fee could be anything, also a mempool thing
< gmaxwell>
Keep in mind that we avoid spending third party unconfirmed inputs for security reasons; and our own for privacy (otherwise the change is immediately distinguishable)
< morcos>
so what am i doing then?
< morcos>
the existing similar variables are static class members for now
< sipa>
if it's a static member variable, call it CWallet::m_fallback_fee
< wumpus>
better to keep it consistent and make this one too, morcos, I'd say
< wumpus>
and yes call it m_fallback_fee
< morcos>
ok, i'll do that for now.. if we want to , we can clean up all of them later
< sipa>
ack
< morcos>
btw, i think it would be nice to have helper functions for all these command line arguments
< sipa>
yes...
< morcos>
so we could sort of declare the argument, its help string, its min and max value, etc.. all in one place
< wumpus>
we have a PR that improves argument handling IIRC
< wumpus>
would be nice post-0.15
< wumpus>
would be nice to finally be able to close #1044
< wumpus>
spudowiar: no, he means a compile-time dependency
< spudowiar>
Oh, gotcha, sorry :)
< spudowiar>
I just realised :)
< wumpus>
spudowiar: core_io etc do not include wallet.h at all ,they don't link against the wallet stuff
< spudowiar>
I didn't realise it was in bitcoin-common
< sipa>
spudowiar: if the function takes a wallet argument, it becomes code that cannot exist without the wallet code being present too
< morcos>
gmaxwell: very simple, but i fear there are too many outstanding other wallet fee PR's to bother with that for now (i built it on top of them since it interacts)
< sipa>
spudowiar: in general, the wallet is intended to be separated off at some point, and even if it isn't, it's good practices to reduce dependencies between modules
< wumpus>
anyhow, functionality that should be present when buildilng without the wallet relies on TxToUniv
< gmaxwell>
morcos: I like.
< morcos>
instagibbs: Murch: achow101: It's this discard rate idea that I'd use to set your window in BnB as well. In addition to the cost of creating the change at the current fee level.
< achow101>
discard rate?
< achow101>
(sorry, I've missed most of the conversation here)
< wumpus>
cfields: could you take a look at #10508 - it is a tests PR, but involves some small build system changes, would be nice if you could take a look
< instagibbs>
seems reasonable, user can always protect super-dustRelayFee change if they decide to
< Murch>
morcos: That would be a good price estimate for the input cost of the saved change output.
< morcos>
Murch: yes that's what i'm saying... and after #10712 its the calculation that is used to say oh wow we're paying way too much fee, lets add change
< morcos>
I'm actually not 100% convinced 10712 is a good idea.. It will lead to a bit more utxo bloat. The true fix is to be smarter about never creating small change which requires effective fee rates.
< kanzure>
is the sentiment that the 1008 block estimatesmartfee for change output size minimum threshold is a bad thing due to utxo bloat ?
< instagibbs>
kanzure, right now the wallet is just really bad at targeting good change size outputs or exact matches
< instagibbs>
so it kind of lands in the middle
< kanzure>
in another 'wallet' (not core) (it's not a wallet) i was going to ask for two fee rates, one for the non-change outputs and another for an estimation of the minimum size of any possible change output below which to burn to miner fee.
< instagibbs>
right, that's the above idea
< instagibbs>
discardRate, where it's ok to drop it
< kanzure>
if all outputs were required to be multisig p2sh then we could insist that everyone just transfers their change in the same output, and they can worry about spending it later. </joking, not practical>
< Murch>
morcos: I don't think I have a good overview of what's going on yet, but generally I'd suggest that we aim for clearcut scenarios:
< Murch>
1) Try to create no change output (use BnB)
< Murch>
2) If fail: Try to create change output greater than min_change
< Murch>
3) small number of cases that don't fit in any other bucket: If change output is too small to keep discard.
< Murch>
[…], discard.
< morcos>
Murch: 100% agree, but the real issue is once we have effective_value we can do both those things. even part 2 requires a good effective value.
< Murch>
what do you mean with "effective_value"?
< kanzure>
instagibbs: unfortunately we edge up to this fundamental tradeoff between discardRate, utxo bloat minimization, and folks losing money because small outputs are essentially unspendable (some always, some intermittently). essentially, certain payment amounts-- from certain inputs-- are simply not workable. if 10 more seconds of coin selection computation could solve this for a user, i thi...
< kanzure>
...nk that's worth the face value of the output.
< morcos>
An additionaly issue however is I don't think its the best idea in the world to add tons of inputs that have barely positive effective value when we are payin a high fee rate, but thats possibly a later improvement
< Murch>
morcos: I think you're worrying too much about utxo bloat. In my simulation, BnB + RandomSelection as fallback had a much lower average UTXO Pool size than Core selection.
< Murch>
BnB + Core as a fallback should be even smaller.
< gmaxwell>
I don't think there is any bloat concern on BNB. Thats part of why we're doing it first.
< morcos>
The big question is how often does BnB find an answer
< morcos>
I have no insight into that
< instagibbs>
Murch, could you just do dumb fallback
< instagibbs>
I think you did right?
< Murch>
morcos: In my simulation with 12k outgoing payments, I found ~39% exact matches with BnB and ~0.6% with Core.
< morcos>
If murch's simulation is on a larger than typical spendable utxo set then it may overestimate the benfit we gain from BnB
< Murch>
morcos: Saying we have no idea, seems a bit of a stretch.
< morcos>
12k outgoing payments from what utxo set per payment?
< morcos>
is each one from the actual utxo set that that payment was made from?
< morcos>
is that one big utxo set
< morcos>
sure, i think BnB will make a huge difference for commercial applications with large utxo sets
< Murch>
morcos: It's a sequence of 36k payments in total, 12k outgoing, 24k incoming.
< morcos>
but all to the same utxo set?
< gmaxwell>
morcos: he has a feed of input and output payment amounts simulate the wallet (e.g. how it's utxos evolve over time)
< Murch>
yes
< gmaxwell>
its*
< morcos>
so thats far from typical
< sipa>
morcos: but perhaps very significant on the overall utxo set
< achow101>
isn't that dataset from moneypot's payments?
< gmaxwell>
morcos: well it's actual for at least one user, we don't know how it represents everyone but users like this are a non-trivial amount of the total network behavior.
< sipa>
morcos: as in, perhaps a large portion of the actual network comes from large player's wallets
< morcos>
sipa: unknown. what % of utxos belong to a big wallet vs small
< sipa>
morcos: yes, i don't know either
< Murch>
I've also consolidated the incoming payments 4 to 1, to make a scenario with 6k incoming payments and 12k outgoing payments. It still did great on UTXO set reduction
< Murch>
achow101: Yes, the same
< morcos>
gmaxwell: don't get me wrong. i'm very in favor of doing BnB. its certainly not hurting small wallets
< gmaxwell>
Im not sure of the thrust of the discussion here, but I do not see how the BnB could be anything worse than a small improvement.
< bitcoin-git>
bitcoin/master 928c681 Matt Corallo: Use "replaceable" instead of "optintorbf" in createrawtransaction....
< bitcoin-git>
bitcoin/master fb915d5 Matt Corallo: Use "replaceable" instead of "optIntoRbf" in fundrawtransaction....
< bitcoin-git>
bitcoin/master 73c942e Matt Corallo: Use "replaceable" instead of "rbfoptin" in bitcoin-tx....
< gmaxwell>
morcos: okay good!
< morcos>
all i'm arguing is we can't look at this increase to 39% exact matches and assume its going to have a huge effect on the overall utxo set which is made up of many smaller sets
< sipa>
morcos: agree - we don't know the impact
< bitcoin-git>
[bitcoin] laanwj closed pull request #10698: Be consistent in calling transactions "replaceable" for Opt-In RBF (master...2017-06-replaceable-rpc-args) https://github.com/bitcoin/bitcoin/pull/10698
< Murch>
morcos: Coin selection in a wallet that has significantly more outgoing payments than incoming (i.e. most end-user cases) is trivial.
< morcos>
so we still have to be worried about utxo bloat in the event BnB fails
< gmaxwell>
oh sure, I think it's fair to say that we don't know how much of an improvement it will be in aggregate.
< morcos>
This is what i'm talking about
< instagibbs>
eh, economic node activity likely follows a power law
< gmaxwell>
Just that it will do no harm and at least in some cases help a lot.
< Murch>
morcos: Coin selection in a wallet that has more incoming payments than outgoing should do very well with BnB.
< instagibbs>
maybe wont matter for small wallets you're right
< morcos>
balancing users wasting their small utxos when they are getting very little to negative effective value from it vs at some point needing to aggregate those to avoid bloat
< morcos>
Murch thought i was over worrying about bloat
< Murch>
morcos: Well, but UTXO set being generally split over more different wallets is not something we can influence on the coinselection level in the first place. That's on the adoption level.
< gmaxwell>
morcos: well in achow101's implementation he is using (and would switch to exactly) something just like the dustfee metric you just linked to, I don't think we have to worry about waste if using that.
< morcos>
gmaxwell: the case i'm talking about is if we don't succeed in BnB, then what change do we aim for. We certainly would like to get higher than something governed by the discard rate.
< gmaxwell>
morcos: okay so BnB is basically unrelated.
< instagibbs>
I think we're agreeing here. For fallback we should pick something significantly higher if possible
< morcos>
gmaxwell: yes
< Murch>
Yeayeah
< morcos>
while we're on the subject
< morcos>
gmaxwell and everyone did you see my above question about how (bnb, knapsack) should be intermingled with (selectcoinsminconf (different params))
< morcos>
at what point do you prefer no change but less confirmations or longer unconfirmed chain
< gmaxwell>
morcos: yes, I have some agreement though there are privacy implications. We don't spend 6 conf or less from third parties for security reasons. And we try to avoid spending our own at less than 6 conf to avoid blowing up any privacy change has.
< gmaxwell>
But I think we should BNB longish chains for example.
< gmaxwell>
because that ends them.
< gmaxwell>
but obviously not overlong (24+) ones.
< morcos>
but not before creating a tx which doesn't spend unconfirmed ?
< gmaxwell>
OH on this subject. We need to consider the feerate of unconfirmed parents as part of their effective rate.
< gmaxwell>
And CPFP them.
< morcos>
actually, maybe it wouldn't be that hard to be kind of smart about it.
< morcos>
yes , well before that step, you could only consider extending chains which pay a feerate at least as high as the one you are paying
< gmaxwell>
For example if you managed to make a very low fee payment A, then make payment B with better fee settings. If B spends from A it needs to CPFP A up to B's target feerate.
< gmaxwell>
Indeed.
< gmaxwell>
I've seen some users screwed with this FWIW.
< morcos>
i don't think you'd necessarily want to automatically CPFP a chain if you had other options
< gmaxwell>
They made a payment with a very low rate shortly after startup, then the next day they made another payment that paid a reasonable rate, but was a child.
< instagibbs>
you'd need to avoid bumping twice, by detecting if they are cousins in a chain
< gmaxwell>
I think you might want to automatically CPFP any input that you set the same confirmed target or lower on.
< gmaxwell>
instagibbs: well they won't be unless we're making multiple change outputs or paying ourselves.... but yes...
< gmaxwell>
this suggests that perhaps we should be tracking what the fee settings were for those transactions.
< morcos>
goodness, this is going to get complicated.
< gmaxwell>
Hurray!
< gmaxwell>
(this means we're starting to understand all that we don't know about the problem space)
< instagibbs>
next we need to throw a general purpose optimizer at it
< rhavar>
pretty sure that's the only sane solution if you want to automatically do CPFP and stuff
< gmaxwell>
nah.
< gmaxwell>
Other than having a bunch of conditionals I think this stuff isn't *that* gnarly.
< instagibbs>
didn't close my statement with /s
< gmaxwell>
The hurestic would be that you first consider unconfirmeds that either are at the target or higher feerate or at the current target or lower... ... but always CPFP things up to the current destination rate.
< gmaxwell>
and use the CPFP impact in the EV calculations.
< rhavar>
I'm not sure it's a horrible idea to just write the whole thing up as a general purpose constrain solving problem
< rhavar>
and just not include a constrain solver :P
< rhavar>
have it as a plugin or something
< gmaxwell>
rhavar: I've done this previously, but the actual solving isn't really that big a deal.
< rhavar>
For casual wallet users, they're only going to have a couple of inputs and outputs -- you can pretty much brute force the space
< rhavar>
And for commercial users, they can actually install some shared object plugin that calls out to a proper library
< gmaxwell>
it's not magic in any case.
< rhavar>
That's what I'm doing right at this moment, I pretty much brute force solutions with a cost metric
< gmaxwell>
I think there is a AMSL statement of a toy coinselection problem that I wrote floating around out there somewhere.
< rhavar>
and it does a *decent* job
< rhavar>
I'll have a robust minizinc implementation pretty soon, if anyones interested
< gmaxwell>
rhavar: sure and thats what it does internally, it's just using a different cost design than you. (A dumb one that is optimizing under assumptions like addresses are never reused) :)
< gmaxwell>
unfortunately, last I checked there are no sutable solvers that we can distribute... but I wouldn't have any issue with having an interface to call out to something.
< rhavar>
I guess what I mean, is if you actually support loading a .so plugin or something -- you can pretty much not worry about it
< gmaxwell>
rhavar: except like, you know, we need to worry about that 99% of users (including commercial ones) that aren't going to solve this for themselves, and not better than we will. :)
< gmaxwell>
but no issue supporting loadable things for people who want them.
< morcos>
also the magic is in the metric isn't it
< rhavar>
I think it makes perfect sense for casual users too
< rhavar>
Just instead of using a proper constrain solver, they can use a brute forcer
< rhavar>
which is pretty trivial to write
< jtimon>
meeting?
< sipa>
soon!
< gmaxwell>
rhavar: you perhaps overestimate how far you can go with bruteforce... :P once you have more than two dozen inputs it starts becoming intractable.
< gmaxwell>
(and my wallets have a lot more than two dozen inputs, and I'm just some guy)
< rhavar>
I'm using a brute force search with ~2k inputs right now, and it does a *decent* job
< rhavar>
not perfect though, that's why I'm paying for the minizinc impl
< rhavar>
(I obviously don't search the entire space, I time out after 10 minutes)
< wumpus>
#startmeeting
< lightningbot>
Meeting started Thu Jul 6 19:00:44 2017 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
< jonasschnelli>
proposed topic: multiwallet endpoint vs json item
< sipa>
LO
< wumpus>
topics?
< achow101>
hi
< cfields>
hi
< kanzure>
hi.
< wumpus>
jonasschnelli: yeah, apparently we have to discuss that again, with all the competing PRs
< jonasschnelli>
heh. Yes
< wumpus>
jonasschnelli: though in principle we settled on endpoint a few weeks ago
< morcos>
begging for review... lots of fee/wallet/estimate stuff that needs to make 0.15
< morcos>
i already have 3 on high priority... sheepish grin
< wumpus>
yes, high priority for review will as usual be first topic
< gmaxwell>
morcos: We should do the things.
< jonasschnelli>
We set endpoints, but some where also in favor of the JSON item solution
< wumpus>
#topic high priority for review
< BlueMatt>
PSA: if you're running master, be very careful not to swap -txindex on your db: the check to prevent you from doing so is broken and you could corrupt your chainstate
< jonasschnelli>
It's to big and will re-focus during early 0.16
< jtimon>
maybe put #8498 in project 8 ?
< gribble>
https://github.com/bitcoin/bitcoin/issues/8498 | Near-Bugfix: Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
< luke-jr>
wumpus: the arguments for endpoint seem strong IMO
< instagibbs>
morcos, doesn;t help that they're an unconfirmed chain of PRs :)
< morcos>
instagibbs: i know! :) high priority review ones arent though
< sipa>
we need a chain length limit on PRs
< luke-jr>
guess we're not on that topic yet
< wumpus>
jtimon: is that high priority to get into 0.15?
< wumpus>
luke-jr: next topic
< BlueMatt>
wumpus: I think 10179 is ready(ish) for a merge, which makes my high-prio of 10652 cleaner
< jtimon>
wumpus: I haven't benchmarked, but it's an optimization and now also a "near bugfix"
< morcos>
BlueMatt: i was just rereviewing that, but don't wait on me, i'm out of sync these days and doing all posthumous review
< wumpus>
BlueMatt: agree
< sipa>
#10179
< gribble>
https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub
< BlueMatt>
yes, second PSA: never shy away from postumous review! the feeling that its not contributing to moving things forward is wrong, if you think something got merged without enough acks, just review it!
< morcos>
or if you want to be sure to understand the new code!
< BlueMatt>
well, that too
< Murch>
Or if you want to understand the code in the first place! :)
< wumpus>
:)
< wumpus>
ok, so anything that needs to be added to project 8?
< morcos>
i have other things needed for 0.15, but they are dependent on the ones i already have in 8
< morcos>
also i already have 3
< wumpus>
ok, just tag them for 0.15 then, don't need to be in that project
< wumpus>
#topic RPC interface for multiwallet (again)
< jtimon>
wumpus: doesn't that qualify for priority?
< instagibbs>
can someone give an overview of what people are thinking on interface for multiwallet... i missed this
< wumpus>
jtimon: if the gain is unclear, I don't think so
< jonasschnelli>
Again we should decide wether we use Endpoints of JSON objects for multiwallet switch... helps to continue on PRs
< wumpus>
instagibbs: please read the current PRs:
< sipa>
wumpus: can we have #10571 #10579 in 0.15?
< gribble>
https://github.com/bitcoin/bitcoin/issues/10571 | [RPC]Move transaction combining from signrawtransaction to new RPC by achow101 · Pull Request #10571 · bitcoin/bitcoin · GitHub
< gribble>
https://github.com/bitcoin/bitcoin/issues/10579 | [RPC] Split signrawtransaction into wallet and non-wallet RPC command by achow101 · Pull Request #10579 · bitcoin/bitcoin · GitHub
< jonasschnelli>
The JSON object is simpler... less impect
< jtimon>
wumpus: I think it's clearly a gain
< jonasschnelli>
the endpoint approach may allow more in future...
< jtimon>
I don't understand the criterion then
< wumpus>
#10650 #10653
< jonasschnelli>
In the JSON object approach (where you choose the wallet bases on a JSON array item), I don't like that the actual switch in in the JSON layer.
< wumpus>
and if you forget it it defaults to the 'default wallet'
< wumpus>
that's just too easy to mess up
< jonasschnelli>
ryanofsky: thanks for clearing this up... wasn't aware, sry
< wumpus>
the endpoint makes sure you can be connected to only one wallet with one RPC connection
< wumpus>
jonasschnelli: right!
< ryanofsky>
i think we should just get rid of the concept of default wallet
< wumpus>
ryanofsky: on the long run, yes, but that's no option for 0.15
< luke-jr>
ryanofsky: definitely not in 0.15
< kanzure>
what about, if more than one wallet, then default wallet must be explicitly specified
< ryanofsky>
if there's more than one wallet, it should just be an error not to specify a wallet
< wumpus>
let's focus on what we want to do now
< luke-jr>
then you break all existing sw
< wumpus>
I think for 0.15 we should simply do the endpoint-based interface
< ryanofsky>
we can do that right now
< gmaxwell>
wumpus: what do you think about the concern that the endpoint stuff establishes a new API that we'll be stuck supporting but haven't given much thought to?
< ryanofsky>
no need to wait
< sipa>
i think as an end goal, endpoint-based selection is awesome, because it prepares for process separation
< wumpus>
gmaxwell: the same is true for any RPC change
< jonasschnelli>
gmaxwell: we can mark it unstable?
< jonasschnelli>
v1 == unstable?
< jonasschnelli>
use / (v0) if you want stability
< sipa>
but if endpoints can't for example remove the non-wallet RPCs, that's sort of not really achieving that goal anyway
< wumpus>
gmaxwell: adding an argument to every wallet RPC call is also such a massive change
< sipa>
wumpus: with named args it's trivial, no?
< jonasschnelli>
I can work on splitting the RPC calls in wallet / nonwallet
< gmaxwell>
(I don't have strong opinions, just raising it)
< jonasschnelli>
if we agree on endpoints
< ryanofsky>
wumpus, are you saying 10661 is a massive change?
< sipa>
and it shouldn't be adding it to every RPC; just catch it in the rpc handler
< wumpus>
but the point is that it'd be something that has to be supported virtually forever
< luke-jr>
(only supporting the default wallet, per rpcauth user, seems the best for backward/forward compatibility still)
< wumpus>
and imo it's poorly thought out
< wumpus>
but I don't care deeply
< wumpus>
at this point we should simply make a choice
< wumpus>
if we don't make a choice today and stick with it
< gmaxwell>
I don't really think named arguments is a great thing. It would make support easier in some software in the short term.
< wumpus>
we can forget multiwallet for 0.15
< jonasschnelli>
ack!
< gmaxwell>
I think every criticism wumpus has on that one is spot on.
< * luke-jr>
suggests rpcauth-based default wallet, and we can figure out endpoints for 0.16
< wumpus>
gmaxwell: indeed - most RPC client libraries don't even support named arguments yet
< luke-jr>
that gives more time to think out API change
< wumpus>
gmaxwell: while changing the endpoint is easy, just change the URI
< wumpus>
luke-jr: please don't bring back a third option
< * BlueMatt>
always kinda assumed named args would allow us to add things like multiwallet/different number precision/etc in the future, as a simple add-on to every RPC without any massive code change everywhere
< wumpus>
luke-jr: that's not going to make it easier
< BlueMatt>
but, ok
< kanzure>
luke-jr: are wallets assigned per rpcauth user already?
< jonasschnelli>
no
< wumpus>
no
< kanzure>
uh..
< BlueMatt>
guess I had a different impression than everyone else, then
< luke-jr>
kanzure: there is no way to use multiwallet right now
< wumpus>
BlueMatt: it's possible, and not hard to implement, but just not the right choice for this IMO
< jonasschnelli>
What about using v1/wallet/y<filename> and mark it unstable (experimental?) for 0.15?
< ryanofsky>
BlueMatt, that was my impression too, it's the basis for 10661 & 10653
< jonasschnelli>
-y
< sipa>
i officially no longer have an opibion on approach
< wumpus>
jonasschnelli: sounds good to me
< BlueMatt>
ok, I mean I dont have a very strong impression, i just always thought that seemed natural
< luke-jr>
wumpus: we should have a per-user default wallet *regardless* of the other options; merging it first is a clean way to defer choosing between the others
< BlueMatt>
but, really, can we flip a coin?
< sipa>
< wumpus>
let's just go with endpoints for now
< jonasschnelli>
Somone disagree?
< midnightmagic>
how the heck did you send a blank line
< jonasschnelli>
Anyone
< gmaxwell>
I think we could say that the endpoint version totally unstable and will change to answer the concern that we're setting an api there premately.
< wumpus>
if no one cares deeply, let's just stick with the decision of a few weeks ago
< instagibbs>
gmaxwell, ack
< jonasschnelli>
gmaxwell: we could mark the whole multiwallet (incl. endpoint) as experimental in 0.15
< jonasschnelli>
And stable in 0.16
< wumpus>
midnightmagic: that wasn't a blank line, it as \x7f characters
< wumpus>
gmaxwell: yes, multiwallet is unstable in 0.15, +1
< wumpus>
gmaxwell: there's probably quite some things that need to change, still
< morcos>
no opinon on the issue, but ACK on making a decision.
< jonasschnelli>
ryanofsky: could you live with the endpoint solution?
< gmaxwell>
I think in general we should get into a practice of new API's being explicitly unstable in their first release. We've mulliganed quite a few times.
< wumpus>
gmaxwell: yes
< ryanofsky>
of course, yeah
< jonasschnelli>
okay. Let me finish the endpoint PR and hope it will make it into 0.15
< wumpus>
jonasschnelli: great!
< jonasschnelli>
/topic
< wumpus>
ryanofsky: thanks
< luke-jr>
jonasschnelli: can you do it on top of 7b73f24311639fdc79c22608c21e4bfcbc6d4243 ?
< jonasschnelli>
pr #?
< wumpus>
any other topics?
< wumpus>
remember morcos was saying something about fee PRs, but not sure it was aimed as a topic
< sipa>
wifi just fied in the BS office
< luke-jr>
jonasschnelli: it's part of #10615
< gmaxwell>
I just want to say that master continues to be mostly awesome and performing great. I'm really excited about this next release. (esp if we get our act togeather on multiwallet) :)
< gribble>
https://github.com/bitcoin/bitcoin/issues/10615 | RPC: Allow rpcauth configs to specify a 4th parameter naming a specific wallet (multiwallet RPC support) by luke-jr · Pull Request #10615 · bitcoin/bitcoin · GitHub
< jonasschnelli>
luke-jr: 7b73f24311639fdc79c22608c21e4bfcbc6d4243 pollutes server.h with CWallet... :/
< jonasschnelli>
(later)
< BlueMatt>
there are a bunch of fee PRs which I think are very useful, and we should try desperately to make progress on them for 15
< gmaxwell>
So yes, there are a number of fee/change handling PRs which are urgent for 0.15.
< morcos>
yeah i don't really have a topic, but i need some review
< morcos>
some are bug fixes
< gmaxwell>
But I don't know what to say beyond that since they're already on the high prio list.
< wumpus>
#topic fee PRs
< morcos>
some are RPC api finalization which would be good to get right
< jonasschnelli>
another topic proposal could be: txoutsbyaddress (it's marked with the 0.15 milestone)
< morcos>
I'm not sure if I broke it up in the easiest way possible for review, but am hesitant to try to reorganize this late in the game...
< wumpus>
jonasschnelli: bleh, server.h should definitely not get a CWallet reference, it's meant to be not specific to bitcoin, let alone wallet
< jonasschnelli>
wumpus: yes. I think the same
< luke-jr>
wumpus: jonasschnelli: I don't see a better alternative.
< morcos>
sounds like jonasschnelli also has his hands full with multiwallet and i think it would have been nice to get access to longer fee estimates in the GUI
< luke-jr>
(keep in mind not all calls will come from the RPC server)
< morcos>
but seems like that is not going to happen
< jonasschnelli>
gmaxwell: just not the conf target > 26+
< luke-jr>
jonasschnelli: that doesn't work for GUI or tests
< morcos>
no way to ask for non-conservative. but at least after one of my PR's it'll default to that if tx is replaceable
< luke-jr>
anyhow, later..
< jonasschnelli>
non-conservative would be simple (a checkbox?)
< jonasschnelli>
a slider with fix positions make little sense... sliders are ment to be linear
< jonasschnelli>
A dropdown could make more sense
< * luke-jr>
likes dropdown
< morcos>
jonasschnelli: yes sort of. the way it is implemented elsewhere is it defaults to the opposite of opt in rbf, but you coudl force it either way
< jonasschnelli>
luke-jr: names tend to bikeshed... but at least "conf-target in block | time"
< * luke-jr>
shrugs
< jonasschnelli>
Or maybe "time | blocks | feerate"
< jonasschnelli>
Ideally we would run coinselection when opening the dropdown to tell the (possible) absolute fee
< morcos>
jonasschnelli: please no
< morcos>
feerate selection first
< morcos>
then coin selection
< jonasschnelli>
heh.. I though somebody will complain. :)
< achow101>
coin selection needs fee rate..
< gmaxwell>
we can't realistically do that. We need the feerate to perform selection.
< morcos>
in the future coin selection may be different depending on feerate anyway
< gmaxwell>
(and will need it more in the future)
< jonasschnelli>
gmaxwell: do it for all options ... *duck*
< luke-jr>
>_<
< luke-jr>
coin selection can be slow, unless that's been optimised
< jonasschnelli>
Thoughs about the dropbox?
< sipa>
"The next Bitcoin-Qt version requires a 4k screen for coin selection"
< morcos>
in any case, i think _something_ simple would be ideal so users have access to longer than 25 confirms
< BlueMatt>
lol
< gmaxwell>
jonasschnelli: well we'd like to be able to do good selections which won't be instant. thats something that could be expiremented with later.
< jonasschnelli>
I'm happy to do it if it's general accaptable (the dropbown)
< BlueMatt>
sipa: ooo, I have those, sounds gogod!
< BlueMatt>
good
< gmaxwell>
well we do want multiple near term options because of market effects. e.g. 2,3,4,5,6,72,today,two days, three days, five days, 1 week... or something.
< jtimon>
what about a box with number of blocks instead of a dropbox ?
< morcos>
jonasschnelli: sure. something, anything. but recommend you build off my other PR #10706
< wumpus>
UTXO indexes would be nice, I'd love to have more query functionality for the UTXO set, we have to track that anyway with a full node
< jcorgan>
i've taken to using zmq to notify of new txes/blocks and the REST API to retrieve parsed info about them, for indexing externally
< morcos>
I've got to run, but someone please tag #10557 #10589 #10707 #10712 for 0.15
< gribble>
https://github.com/bitcoin/bitcoin/issues/10557 | Make check to distinguish between orphan txs and old txs more efficient. by morcos · Pull Request #10557 · bitcoin/bitcoin · GitHub
< jonasschnelli>
I guess one reason why some of the centralized services (like the BWS) still is based on 0.12.1 is because the indexes where never added to Core master branch
< gmaxwell>
wumpus: right. also if ever we support having pruned wallets (wallets that don't know their full history, but do have their full coins), the txout index is something we need for it... but not other indexes.
< wumpus>
oh sure morcos
< morcos>
(sorry)
< wumpus>
gmaxwell: yes!
< jonasschnelli>
tagged
< jonasschnelli>
gmaxwell: yes. For HD restores in pruned env. the utxo index is handy
< wumpus>
gmaxwell: would be very niec to instantly query the balance, if history isn't important
< gmaxwell>
rescan has become so slow for me at least that I'm kinda desperate for something like that.
< gmaxwell>
I've lost days of time waiting on rescans.
< wumpus>
next topic?
< sipa>
nope.
< gmaxwell>
wumpus: can you remind me of the 0.15 schedule?
< luke-jr>
eh, UTXO isn't a substitute for rescanning.. you'd miss historical txs
< gmaxwell>
luke-jr: thus pruned wallet, ... it's fine to not have historical txs if you know you don't.
< gmaxwell>
BlueMatt: thanks!
< wumpus>
luke-jr: what if you don't care about history, and just want balance + possibly spending?
< BlueMatt>
T-10 days to branch
< luke-jr>
gmaxwell: but you'd end up with *some* historical tx in this case, with no deterministic reason why some are missing
< gmaxwell>
luke-jr: you can also say that txes found on the blockchain aren't a replacement for having their metadata... :)
< luke-jr>
I suppose we could import them as all change-only or something? :/
< jonasschnelli>
luke-jr: you can scan in the background for the history in a very slow manner once you have done it via the UTXO set index
< wumpus>
BlueMatt: no, not to branch, to feature freeze
< sipa>
wumpus: it'd be incompatible with hardware wallets, before segwit
< luke-jr>
jonasschnelli: hmm, good idea
< sipa>
as you need the full crediting txn
< gmaxwell>
luke-jr: no you wouldn't: my suggestion is that a pruned wallet basically have a line shown in the UI where nothing is there before it except a move like ledger entry that shows the earlier balance.
< BlueMatt>
oh, sorry, yes, freeze
< wumpus>
BlueMatt: branch is 2017-08-06, so a month away
< gmaxwell>
luke-jr: I made an issue describing some ideas for that a while back.
< luke-jr>
gmaxwell: I see, makes sense
< achow101>
wumpus: can #10571 and #10579 be tagged for 0.15?
< gribble>
https://github.com/bitcoin/bitcoin/issues/10571 | [RPC]Move transaction combining from signrawtransaction to new RPC by achow101 · Pull Request #10571 · bitcoin/bitcoin · GitHub
< gribble>
https://github.com/bitcoin/bitcoin/issues/10579 | [RPC] Split signrawtransaction into wallet and non-wallet RPC command by achow101 · Pull Request #10579 · bitcoin/bitcoin · GitHub
< gmaxwell>
There are other reasons why building such things are attractive... (e.g. UTXO based sync can't provide the data to give history...)
< wumpus>
achow101: sure
< achow101>
thanks
< gmaxwell>
ack
< sipa>
early lunch?
< wumpus>
ye fine with me
< wumpus>
#endmeeting
< lightningbot>
Meeting ended Thu Jul 6 19:49:54 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< spudowiar>
Hardware wallet support rebased and now supports change addresses properly
< jtimon>
so why is #8498 not to be tagged for 0.15 nor project 8 ? I want to understand the criterion
< gribble>
https://github.com/bitcoin/bitcoin/issues/8498 | Near-Bugfix: Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
< jtimon>
criteria
< wumpus>
jtimon: well if others think it should be tagged 0.15 or be high priority for review it's fine with me
< spudowiar>
There are a few things I need to tackle now
< spudowiar>
a) How the plugin can tell whether it is Testnet or Mainnet
< wumpus>
jtimon: I don't personally see it as urgent enough for that, it's an optimization without mentioning timings
< wumpus>
jtimon: but that's just my opinion
< jtimon>
I see, thanks
< spudowiar>
b) How the plugin can show a UI? (e.g. TREZOR Pin Matrix)
< spudowiar>
Maybe the plugin can just create a window itself
< wumpus>
spudowiar: la) aunch it with a flag/option that specifies which block chain
< wumpus>
spudowiar: b) yes, have it draw it itself
< jtimon>
well, the timinigs changed over time. now acceptToMemoryPool is not as redundant in fee calculation as it sued to be, and the number of places where checkinouts have been called from has been varying as well
< spudowiar>
c) Multiple devices (I'm going to add a device identifier parameter and a listhwwdevices method to the plugin)
< wumpus>
spudowiar: or have a command askPin() from the plugin that shows a simple Qt message box with a prompt
< spudowiar>
wumpus: It has a special GUI though
< spudowiar>
Hence why I chose that example
< wumpus>
spudowiar: if it has a special gui it should definitely draw it itself, bitcoin qt can't get into the UI delegation/embedding business
< jtimon>
but there's more checkinputs calls now than when I first coded the thing
< spudowiar>
d) Simplify usage (e.g. without command line parameters)
< spudowiar>
This would require to store the plugin name and the device identifier somewhere (probably bitcoin.conf to begin with)
< spudowiar>
e) Need to test multisig
< wumpus>
yes
< spudowiar>
yes to what?
< wumpus>
"This would require to store the plugin name and the device identifier somewhere (probably bitcoin.conf to begin with)"
< spudowiar>
ok
< jtimon>
#10195 and #10192 likely changed how much of an improvement #8498 is as well
< gribble>
https://github.com/bitcoin/bitcoin/issues/8498 | Near-Bugfix: Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
< spudowiar>
Not going to bother with listhwwdevices for now (that would be useful for a GUI to select the hardware device)
< wumpus>
jtimon: some kind of measurement would be nice, e.g. startup time with a large wallet
< wumpus>
spudowiar: agreed, auto-detecting devices would be something for later
< jtimon>
why a wallet? the improvements are in connectBlock and AcceptToMemoryPool
< jtimon>
wumpus: specially on the latter
< wumpus>
jtimon: I'm confused then - yes wallet would not be appropriate then
< spudowiar>
Oh, and I need to get rid of boost::process dependency
< instagibbs>
spudowiar, please do :P
< wumpus>
I would first focus on functionality
< jonasschnelli>
spudowiar: I think URI schema is the best option for the GUI
< spudowiar>
instagibbs: Find me an alternative :) I don't know C++
< wumpus>
then only when it works, on removing dependencies
< jonasschnelli>
call sign://
< wumpus>
too easy to get stuck in yak shaving dependencies
< jonasschnelli>
or bitcoinsign://
< spudowiar>
jonasschnelli: That overcomplicates things in my opinion
< instagibbs>
wumpus, it's a super new module, only reason I care
< jonasschnelli>
It's a clean separation... could work as a standard
< jonasschnelli>
Otherwise other non-core application could not tab in
< spudowiar>
jonasschnelli: How do you get the data back, anyway?
< instagibbs>
I don't feel like installing it to test his work, so I've rolled my own solution until then
< spudowiar>
jonasschnelli: Why can't non-Core applications use these plugins?
< jonasschnelli>
spudowiar: you plugin would call back bitcoincore://
< sipa>
why a URI...?
< spudowiar>
So, Core needs to register its own protocol now?
< spudowiar>
URIs are really unnecessary here
< jonasschnelli>
sipa: URI is the only think that works in sandboxed env
< sipa>
jonasschnelli: i don't understand
< jonasschnelli>
(Android, iOS, OSX [soonish])
< sipa>
?
< jonasschnelli>
In sandboxed enviroments, interprocess communication is impossible, expect over URIs
< spudowiar>
jonasschnelli: JSON-RPC is transport agnostic
< spudowiar>
jonasschnelli: So you can do JSON-RPC over URI if you want, on those platforms
< spudowiar>
I'd prefer not to register arbitrary protocols on my desktop though
< sipa>
ok, so use some wrapper URI on those platforms
< spudowiar>
Not when a perfectly suitable solution exists
< gmaxwell>
what sipa said
< sipa>
not as part of the IPC mechanism in general
< jonasschnelli>
that makes sense....
< jonasschnelli>
in my example: bitcoinsign://signtx?data=blabla&callback=bitcoincore the blabla elements could be the JSON/RPC layer
< spudowiar>
Yep
< spudowiar>
At the moment though, I require the JSON-RPC to be in roughly getrawtransaction format
< spudowiar>
I want to strip parts of that out, then write documentation for the protocol
< jonasschnelli>
I have a branch somewhere where the GUI can select watchonly unspents and create a hex-tx instead when signing... I once thought you may should be able to forware that unsigned hex tx (including utxos) to another app via URI-schema
< bitcoin-git>
[bitcoin] theuni opened pull request #10756: net processing: swap out signals for an interface class (master...no-net-signals2) https://github.com/bitcoin/bitcoin/pull/10756
< cfields>
BlueMatt: I was holding off on ^^ because I thought it might stomp on some of your other PRs, but after taking a look, I think it might actually make your life a little easier
< gmaxwell>
<3
< gmaxwell>
though it shows how much y'all have worn me down about C++ features that I'm cheering for inheretence. :)
< cfields>
heh
< gmaxwell>
my backtraces thank you, however.
< gmaxwell>
I believe this may be measurably faster too... when I ripped out signals and replaced it with direct function calls it was.
< gmaxwell>
the signals stuff has thread synchronization inside it.
< cfields>
yea. I should add that to the PR description.. that might actually be the nicest part of the change
< cfields>
gmaxwell: yea, i've owed you this PR for months now. Sorry for the delay.
< gmaxwell>
Don't worry, I didn't remember.
< cfields>
heh ok
< jamesob>
any thoughts on adding a Dockerfile to the repo? might make setting up a dev environment marginally easier
< jtimon>
I "had" to delete ~/.bitcoin yesterday until I learned how to cleanup things in docker (damm, should have just copied it to another disk), otherwise I could have seen the historic min fee and feerate per block that I've always wanted to see...
< jtimon>
I hope it's well tested, I rarely introduce new features and didn't run the coverage thing, but I think I cover all the new code
< jtimon>
not sure what the proper C++11 replacement for boost::split(plot_values, str_plot_values, boost::is_any_of(",")); would be
< sipa>
i don't believe c++11 has a replacement for that