< eklitzke>
if you notice anyone on github opening issues about corrupted chainstate databases (i know this is a fairly common issue for people to open), direct them to #12690 and ask them to compile bitcoin with the new leveldb error log messages which are in master now
< adiabat>
not sure if this is a known bug: in the qt UI, the IBD window thing says "blocks left: unknown (syncing headers)"
< adiabat>
but it's not; it's downloading blocks according to debug.log. Also the other fields in the UI seem fine and are updating
< adiabat>
the bottom progress bar also says "syncing headers (99.9%)" but if I mouse-over the bar it pops up the actual height it's gotten to
< adiabat>
I clicked hide, then clicked the sync icon to bring it back up and it works normally, showing blocks left. so whatever...
< eklitzke>
sipa: when you were doing the -fsanitize=address stuff, did you have to rebuild all of the libraries (boost, db4, etc.) with that flag enabled as well?
< eklitzke>
asking because when i enable it all of the autoconf tests for the third party libs fail
< luke-jr>
eklitzke: at least one of the sanitizers is documented to require it
< sipa>
eklitzke: i believe not
< sipa>
i think only one of the sanitizes needs it
< SWXRU2cjac>
THIS IS A FREENODE BREAKING NEWS ALERT!! Hitechcg AND opal ARE GOING AT IT RIGHT NOW WITH A LOT OF FIGHTING AND ARGUING WOW YOU DON'T WANT TO MISS THIS!! TYPE /JOIN ## TO SEE THE ACTION...AGAIN TYPE /JOIN ## TO SEE THE ACTION!!
< SWXRU2cjac>
THIS IS A FREENODE BREAKING NEWS ALERT!! Hitechcg AND opal ARE GOING AT IT RIGHT NOW WITH A LOT OF FIGHTING AND ARGUING WOW YOU DON'T WANT TO MISS THIS!! TYPE /JOIN ## TO SEE THE ACTION...AGAIN TYPE /JOIN ## TO SEE THE ACTION!!
< SWXRU2cjac>
THIS IS A FREENODE BREAKING NEWS ALERT!! Hitechcg AND opal ARE GOING AT IT RIGHT NOW WITH A LOT OF FIGHTING AND ARGUING WOW YOU DON'T WANT TO MISS THIS!! TYPE /JOIN ## TO SEE THE ACTION...AGAIN TYPE /JOIN ## TO SEE THE ACTION!!
< bitcoin-git>
[bitcoin] drewx2 opened pull request #12693: Remove unused variable in SortForBlock (master...unused-sort-for-block-param) https://github.com/bitcoin/bitcoin/pull/12693
< bitcoin-git>
[bitcoin] achow101 opened pull request #12694: Actually disable BnB when there are preset inputs (master...fix-preset-coins-bnb) https://github.com/bitcoin/bitcoin/pull/12694
< achow101>
eklitzke: I'll write a test for that later. It's 3 AM for me right now and I kinda want to sleep
< eklitzke>
fair
< bitcoin-git>
[bitcoin] kallewoof closed pull request #10386: [WIP] [wallet] Optional '-avoidreuse' flag which defaults to not reusing addresses in sends (master...feature-white-black-address) https://github.com/bitcoin/bitcoin/pull/10386
< bitcoin-git>
[bitcoin] kallewoof closed pull request #10877: [rpc] Verbose flags for chaining and scripting (master...verbose-flagging) https://github.com/bitcoin/bitcoin/pull/10877
< bitcoin-git>
[bitcoin] eklitzke opened pull request #12695: [build] Fix some strange behavior with --enable-debug (master...enabledebug) https://github.com/bitcoin/bitcoin/pull/12695
< GAit>
While doing some regtesting I've noticed that the mempool returned by getrawmempool is sometimes a subset of what I expect if core is processing a block while I ask, even if getbestblockhash returns the same blockhash prior and post getting the rawmempool. Is this known/expected behavior?
< bitcoin-git>
bitcoin/master 152b7fb James O'Beirne: [tests] Add a (failing) test for waitforblockheight...
< bitcoin-git>
bitcoin/master f98b543 James O'Beirne: Only call NotifyBlockTip when the active chain changes...
< bitcoin-git>
bitcoin/master 947c25e Wladimir J. van der Laan: Merge #12431: Only call NotifyBlockTip when chainActive changes...
< bitcoin-git>
[bitcoin] laanwj closed pull request #12431: Only call NotifyBlockTip when chainActive changes (master...jamesob/2018-02-prevent-bad-latestblock) https://github.com/bitcoin/bitcoin/pull/12431
< instagibbs>
murchandamus, any notion on whether single random draw should prune after hitting it's required value?
< instagibbs>
scan the list again, try to remove items and stay above thresh. I have a hunch that any gain of "more distributed" output values from pure draw are outweighed by spending an addition utxo *now*
< promag>
"Finalize and close translations for 0.15"
< wumpus>
that's correct
< promag>
oh
< promag>
ok sorry
< instagibbs>
how many hours/minutes until dev meeting? clock change in US...
< sipa>
instagibbs: 98
< wumpus>
$ date -u -> Thu Mar 15 17:22:05 UTC 2018 -> 1 hour and 8 minutes
< wumpus>
eh 38
< instagibbs>
Core wallet currently orders inputs based on the CInputCoin ordering, which orders in non-descending order via COutPoint... wouldn't it be better to shuffle for privacy?
< instagibbs>
or are other wallets doing this as well
< sipa>
instagibbs: it doesn't shuffle? :o
< instagibbs>
nope, it uses std::set which uses, I believe, the comparator of the underlying type
< instagibbs>
based on my internetting
< instagibbs>
either way it's definitely not shuffled
< Murch>
instagibbs, sipa: Wouldn't it be fine if they were sorted by outpoint?
< instagibbs>
Murch, I'm not particularly bothered, provided it's a standard thing wallets do. I assumed everything was shuffled
< Murch>
The thing that you would want to obfuscate, is the order in which they were picked, or when they come from different wallets and such. If they'd all be ordered by the outpoint that would still be obfuscated
< instagibbs>
it's saying "I'm a Core wallet!" unless others do it as well
< instagibbs>
Slowly hoping we can stop announcing that so much :)
< gmaxwell>
I'd rather it sorted by script pubkey.
< gmaxwell>
(ignoring the consideration of distinguishability vs other software)
< GAit>
please ignore my comment above re: bestblockhash and getrawmempool, looks like i got confused
< wumpus>
instagibbs: as there is no common standard between wallets how to do those things, I suppose all software is pretty distinguishable
< gmaxwell>
(the reason for scriptpubkey is that we came up with a compact encoding that is much smaller when spk's are reused in adjcent outputs)
< luke-jr>
gmaxwell: I would think that encoding would be detrimental since it encourages reuse
< instagibbs>
luke-jr, doesn't make anything cheaper for wallet though
< luke-jr>
at the very least, though, I'd think we should use a std::unordered_set
< luke-jr>
instagibbs: not relevant; it encourages reuse because many people DO care about the health of the network
< gmaxwell>
luke-jr: it doesn't encourage it.
< gmaxwell>
it's not like you pay lower fees due to it.
< instagibbs>
luke-jr, nuking your own privacy is a larger negative :/
< instagibbs>
imo
< luke-jr>
instagibbs: hopefully
< instagibbs>
own and others*
< luke-jr>
instagibbs: but some people already do it, so apparently not a large enough negative
< luke-jr>
there's no reason to optimise reuse; in the ideal situation, it does nothing useful; in the worse situation, it's harmful
< sipa>
how does it encourage reuse?
< instagibbs>
anyways, I'll open an issue, since I found it surprising
< Randolf>
Hello.
< sipa>
i think they should be randomly ordered
< sipa>
we can't guess what other wallets are doing, but a random sort will at least sometimes be indistinguishable from others
< wumpus>
yes, why not
< wumpus>
there seems to be no reason to *not* randomly order inputs, at least with SIGHASH_ALL
< luke-jr>
Randolf: hi (you're an hour early)
< Randolf>
I am?
< sipa>
timezones are confusing
< Randolf>
Oh, I'm in Pacific Time.
< sipa>
the meeting is 8pm UTC
< luke-jr>
everyone should switch to tonal time ;)
< sipa>
or 7pm?
< wumpus>
7
< Randolf>
I've heard that Decimal Time is quite good -- only 10 hours in a day, 100 minutes per hours, but the seconds are a bit strange.
< luke-jr>
Randolf: that's pretty insane, not good :P
< instagibbs>
I can PR, it seems strictly better to shuffle imo, I've never heard of this COutPoint ordering used anywhere else
< wumpus>
instagibbs: ok
< Murch>
Randolf: If your calendar supports it, you can just set the event's time to the 7pm UTC, and it'll automatically fix weird shifts like that for you. ;)
< sipa>
Protip: Iceland is always on UTC
< luke-jr>
Murch: (and put the meeting overlapping with picking kids up at school, it seems)
< jcorgan>
protip: if your calendar doesn't support UTC as a time zone, setting it to Iceland
< jcorgan>
heh
< sipa>
!hi5 jcorgan
< luke-jr>
sadly, Florida just voted to make DST permanent/year-round, so this is going to be a persistent issue for me :x
< gribble>
Error: "hi5" is not a valid command.
< sipa>
luke-jr: which apprently violates federal law
< Randolf>
Murch: Thanks.
< luke-jr>
sipa: ⁈
< Murch>
Getting rid of summertime would be great.
< jcorgan>
getting rid of timezones would be great
< sipa>
luke-jr: states apparently have a right to choose whether they adopt DST or not, but not what timezone they are in
< luke-jr>
sipa: so how does choosing to adopt DST year-round violate federal law? :P
< CubicEarths>
Obviously solution - Set the meeting for a block height :)
< luke-jr>
CubicEarths: that would be a bit chaotic, but maybe fun for a few meetings
< luke-jr>
(and probably super annoying after a few)
< jcorgan>
miner incentives to delay or advance core mtgs :)
< luke-jr>
perhaps I'll learn to go to school early with a laptop, and join the meeting from there ;)
< instagibbs>
yes you creeping around the school parking lot with a laptop wont sound any alarms
< luke-jr>
lol
< luke-jr>
I'll need to get the wifi password from them first anyway
< wumpus>
I wouldn't mind moving it an hour earlier or later, but whatever time is chosen it's probably inconvenient to someone, and for the people in aus/asia it won't help
< wumpus>
CubicEarths: I guess that would be 'fair' for some definitions of fair, but hard to plan for :)
< luke-jr>
if we had more eastern devs, it might be "fair" to just shift it by 8 hours every week
< wumpus>
yes
< sipa>
how about shifting by 7 hours every week? that would over long periods of time uniformly distribute meetings :)
< sipa>
(no, not a serious suggestion)
< luke-jr>
anyhow, topic for meeting hopefully we can cover in the first ~10 minutes: multiwallet GUI, finally getting it merged?
< luke-jr>
jonasschnelli: will you be here? XD
< bitcoin-git>
[bitcoin] instagibbs opened pull request #12699: Shuffle transaction inputs before returning from SelectCoins (master...shuffleinputs) https://github.com/bitcoin/bitcoin/pull/12699
< wumpus>
meeting?
< provoostenator>
hi
< wumpus>
#startmeeting
< lightningbot>
Meeting started Thu Mar 15 19:01:22 2018 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
< cfields>
feel free to remove banman for being non-rebased as well. I'll try to get to it today/tomorrow, but I *think* that's in fitting with the discussion we had
< achow101>
+1 on removing things that need rebasing
< wumpus>
cfields: ok, done
< wumpus>
can always be readded
< Randolf>
I suspect that more people would do rebasing if the web-interface provided an option for that.
< wumpus>
and yes, it's consistent and fair with BlueMatt
< wumpus>
Randolf: not all rebases are trivial one-click affairs
< Randolf>
That's true.
< wumpus>
other topics?
< meshcollider>
luke's multiwallet stuff
< Randolf>
There seems to be confusion about whether Lightning Network is supported in Bitcoin 0.16.
< sipa>
...
< BlueMatt>
wut
< wumpus>
I don't think discussing GUI multiwallet makes sense without jonasschnelli here, but if you really want to discuss it meshcollider we could
< sipa>
and if you mean Bitcoin Core, that has nothing to do with Lightning
< achow101>
Randolf: from who?
< wumpus>
lightning is 'supported' since segwit was activated
< meshcollider>
wumpus: true I'm indifferent, I'd just like to see a decision be made lol
< Randolf>
achow101: Well, I see the question come up from time-to-time in the #bitcoin channel, and in other IRC channels there's disagreement about whether it's fully implemented or at all.
< BlueMatt>
Randolf: please take this after the meeting
< Randolf>
I'm thinking that it may be good to indicate it prominently on the bitcoin.org web site or a readme file clearly.
< Randolf>
Okay.
< promag>
regarding multiwallet gui, IMO both still have some things to fix
< provoostenator>
I'd also like the multiwallet stuff to go forward, but does it get in the way of the RPC Interface stuff?
< wumpus>
luke-jr: I agree, at some point, if it's an improvement we should merge it, it doesn't have to do every single thing people can think of
< provoostenator>
luke-jr: and there's the issue of putting model state in UI elements, which makes future changes to the design needlessly hard
< promag>
provoostenator: oh I still have to see that one
< wumpus>
collaboration works better on the master branch
< wumpus>
but jonasschnelli had some problems with your approach
< provoostenator>
E.g. if I want to change the dropdown into a menu, I now need to figure out where to keep the wallet array. The other PR handles that more elegantly (but maybe it's worse in other ways, I can't tell).
< wumpus>
provoostenator: agree
< promag>
well the way I see both build on top of a weak support of multiwallets
< ryanofsky>
fwiw, i haven't looked at jonas pr yet, but i think luke's approach is fine. if jonas has improvements, no reason they can't go in later
< promag>
maybe we should focus also on #12587, adding signals etc
< instagibbs>
so, Branch and Bound was merged(yay), which leaves us with what to do when we cannot find an exact match. Current strategy is to fallback to Single Random Draw as per murchandamus thesis
< instagibbs>
with some *, if achow101 wants to continue
< achow101>
*current plannned strategy is to use SRD
< * Randolf>
notes that SRD means Single Random Draw
< instagibbs>
morcos had any idea to only SRD on coins smaller than target, to slightly bias towards spending smaller coins, for utxo set reasons
< instagibbs>
debate as to whether that is even necessary were had
< instagibbs>
pure SRD gives the wallet a nice distribution of utxos, which likely feeds to better BnB exact match rates
< instagibbs>
that's... about it.
< * jtimon>
wonders if that slight bias could hurt privacy somehow
< sipa>
i think SRD may be a privacy issue too
< instagibbs>
jtimon, might mark as Core, but right now we hoover tiny inputs
< instagibbs>
sipa, versus what?
< sipa>
ideally you end up with a strategy that is hard to distinguish from BnB
< jtimon>
instagibbs: yeah, I'm not opposed to the idea in principle, just launching questions to the air to see what other people think
< sipa>
instagibbs: if you have 80% of UTXO very close to 1 BTC, then SRD will be very likely to include such outputs, revealing something about your UTXO size distribution
< jtimon>
thanks!
< instagibbs>
sipa, with a 1BTC target? As oppose to BnB?
< instagibbs>
Maybe we can continue post-meeting...
< sipa>
yeah
< * instagibbs>
will continue post meeting
< wumpus>
ok, anything else for the meeting?
< wumpus>
#endmeeting
< lightningbot>
Meeting ended Thu Mar 15 19:33:01 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< instagibbs>
sipa, I see what you mean, but I don't think other strategies are much(any?) better
< instagibbs>
aside from quite advanced coin control
< sipa>
Randolf: Lightning is implemented by software like c-lightning and lnd. Bitcoin is implemented by software like Bitcoin Core and btcd. They are otherwise unrelated.
< sipa>
Particular implementations of lightning may or may not require you to run particular bitcoin implementations, but that depends heavily on what software is involved.
< meshcollider>
e.g. lnd requires either core or btcd, c-lightning and eclair both requires core
< sipa>
he left
< instagibbs>
BnB dodges this issue I think by 1) not having change 2) "over fitting" to the exact target. Once we have change, we have to make decisions about how big that change should be. Right now it's 1 bitcent, and ends up with Core hoovering tiny inputs to hit that.
< Murch>
sorry, I was at lunch.
< Murch>
BnB strictly only includes inputs smaller or equal than the target, so limiting SRD to inputs smaller than the target might make the input set more like a BnB one.
< achow101>
The fact that there will be change indicates that BnB wasn't used at all though
< achow101>
so I don't think it really needs to look like BnB
< instagibbs>
its more about revealing stuff about the rest of your utxo set
< sipa>
well ideally the chabge isn't recognizable
< instagibbs>
typically it will mean a single output
< sipa>
but leaking BNB or not isn't as bad as leaking information about your utxo size distribution
< Murch>
sipa: with random selection, change might be bigger or smaller than the sending output, isn't that good?
< sipa>
there are so many aspects to privacy, it's hard :)
< instagibbs>
if you can name a counter-proposal, it may help
< instagibbs>
not putting on spot, for my own imagination
< Murch>
Largest first would probably also work well in combination with BnB
< Murch>
:p
< instagibbs>
lol
< Murch>
FIFO is pretty decent all around, might leak less information about composition, but more about time distribution
< sipa>
i was wondering if you could do something like randomly adding change of a particular size, and then run BnB again
< instagibbs>
sipa, me too, if you can engineer a target
< sipa>
but there are so many design parameters, and risk becoming very slow
< Murch>
Then we'd waste all the computational work to hit something that is actually arbitrary
< instagibbs>
i think picking the "right" change size using expert rules is foolhardy unless we really know more
< sipa>
agree
< Murch>
instagibbs: I mean you could fill buckets of change sizes, to spread combination space for BnB
< sipa>
Murch: yes, absolutely - though it could run with lower computational limits or so
< Murch>
but that seems overly optimistic in regard to how well we understand the overall effect
< instagibbs>
Murch, wallet history to know targets, etc :)
< instagibbs>
like, maybe Coinbase needs a 2 BTC utxo for daily payments, i dont!
< instagibbs>
but again this leaks info if used
< instagibbs>
ack on hard :(
< sipa>
use change = your previous payment output amount :p
< Murch>
sipa then you leak a connection to a previous tx
< sipa>
oh. duh.
< Murch>
It's been suggested to target the average amount of the last x spents from the wallet, to make sure the change is a useful size
< Murch>
but then again we're reducing the UTXO pool's variance
< Murch>
which is bad for the combination space of BnB
< Murch>
so, yes, random leaks information about the distribution of the wallet's utxo pool, but only if you create a large transaction target in the first place
< Murch>
otherwise the information is spread out over potentially unrelated tx in the first place
< Murch>
and if you're creating a large transaction, you'd be leaking information about your wallet's utxo pool composition with most strategies
< instagibbs>
sipa, i thought you were joking :P
< Murch>
we could have multiple fallback algorithms that switch off. :p
< Murch>
But that would be terrible to analyze
< Murch>
Largest first smaller than the target might actually also be an option.
< Murch>
achow101: I've been thinking. We might want to do SRD first, since it is computationally much cheaper than BnB, and then use the waste result of the SRD as an upper bound for BnB.
< Murch>
only switch to the BnB result if it has less waste.
< achow101>
Murch: i don't think that it is likely for SRD to find a result that has less waste than BnB. Furthermore, that would mean we do both SRD and BnB for every selection
< Murch>
Yeah, but shuffle pick from front is super cheap.
< Murch>
*"Yeah, but shuffle, then pick from front is super cheap."
< instagibbs>
BnB being preferred has the nice property that it ends transaction chaining in the wallet for good
< Murch>
mhkay, just an idea
< instagibbs>
not saying it's a bad idea, just saying it was one considering iirc
< Murch>
In the end it's always a question of tradeoffs. Do you want to minimize costs, maximize privacy, minimize UTXO pool, …, or some mix thereoff. ;)
< Murch>
instagibbs: I think that SRD or FIFO would probably be a contender for decent mix of tradeoffs, and would use one of those as a baseline to compare against. But then suggest that we compare with concrete other proposals.
< sipa>
Murch: i wonder if a good strategy (perhaps only when the utxo set is large) is use a pre filtering that trims the UTXO set down, biasing in favor of amounts that are further away from other amounts
< Murch>
That would definitely help BnB, and also spread out selection for SRD. Isn't that computationally intensive, though?
< sipa>
why would it?
< sipa>
it seems like it can be done with a single pass
< Murch>
How would you do it? Logarithmic bucketing?