< michagogo>
23:13:32 <gmaxwell> I've mused about the idea about having some shred wallet feature that creates some long timelocked spend of any remaining coins and gives them over to fees... then sends them off somewhere.
< michagogo>
23:14:25 <gmaxwell> because I am somewhat pained by all the utxo bloat created by people who end up with 0.00001 BTC in a wallet, in 100 inputs, and then they just delete the file because its effectively worthless.
< michagogo>
Isn't that already a thing?
< michagogo>
I mean, not built in to Core, but I could have sworn there was something that collects dust and bundles it into an ANYONECANPAY to a 0-value OP_RETURN
< gmaxwell>
yea, peter todd's dustbgone
< michagogo>
Yep, that's the one.
< michagogo>
BTW, something occurred to me earlier.
< michagogo>
In the latest update (Creators', I think it was called?) to Windows 10, they upgraded WSL
< michagogo>
And I think one of the big things, besides the fact that it now uses Xenial, is that you can now call Windows binaries from within bash.
< michagogo>
(Up until now, if you tried putting an exe into there it would try to execute it as a Linux binary and fail)
< achow101>
oh, they updated to xenial? then you can't cross compile from wsl
< michagogo>
achow101: yeah, for new installations
< michagogo>
Wait, really?
< achow101>
mingw on xenial does work IIRC
< michagogo>
So why can't you cross-compile?
< achow101>
well, not that it doesn't work, but rather that something (unknown) is different about it that makes it fail. there's an issue about it
< michagogo>
I was just confused because you said mingw on xenial does work -- do you mean that there's something different about one of the other packages that does it?
< michagogo>
Anyway, what occurred to me was this: Gitian supports virtualbox, right?
< achow101>
yes
< achow101>
I don't think it does it well though
< michagogo>
And I assume vboxmanage works the same (in terms of parameters etc) on Windows and Linux?
< achow101>
idk. I don't use virtualbox
< michagogo>
If that is the case, it should (I think?) now be possible to run Gitian from within WSL
< achow101>
couldn't you already do it with lxc?
< michagogo>
Can you?
< michagogo>
I would not expect LXC to work in WSL
< achow101>
i thought someone did it a while ago and it worked, but I don't quite remember. I haven't tried it myself
< cfields>
Any other gitian builders? Waiting for 1 more sig.
< achow101>
how many sigs do you need before doing the signed binaries?
< cfields>
achow101: usually 3. But usually wumpus is one of them. Since he uploads the tarballs, it's reassuring to know he's gotten the same result. So without his, I'd prefer to wait for another one/two before signing.
< cfields>
gitian builders: detached sigs pushed for 0.14.1
< michagogo>
Oops.
< michagogo>
I was running a while ! loop on the script to do the signed build
< michagogo>
But the script was returning failure, because the last step is opening a PR, which fails when there's already a PR open for the same branch
< chatter29>
hey guys
< chatter29>
allah is doing
< chatter29>
sun is not doing allah is doing
< chatter29>
to accept Islam say that i bear witness that there is no deity worthy of worship except Allah and Muhammad peace be upon him is his slave and messenger
< CubicEarth>
Should the default keypool size be increased to 1000?
< * wumpus>
still deleting "qa" directories everywhere :-)
< jonasschnelli>
#10244 and the IPC approach is in general good. Is there a broad agreement that we want to do this? I vaguely remember gmaxwell had some concerns. Just to give more clear feedback to ryanofsky
< gribble>
https://github.com/bitcoin/bitcoin/issues/10244 | [qt] Add abstraction layer for accessing node and wallet functionality from gui by ryanofsky · Pull Request #10244 · bitcoin/bitcoin · GitHub
< jonasschnelli>
I like it
< jonasschnelli>
But I remember when I also tried to write such *larger* PRs... they just have a hard time getting reviewed. Constant rebase and often great risks when merging.
<@wumpus>
yes it's difficult to do a big change like thta
<@wumpus>
I agree broadly, on the approach, the thing I didn't like about it is that he seemed to add more abstraction layers. What I don't like about that is that it reduces flexibility, hardcoding maybe stupid choices that were made years ago and made sense at the time, buried beneath layers of calls.
<@wumpus>
my original plan was to make ClientModel and WalletModel the abstractions for the core, they are currently thin wrappers around it, and could be extended/changed to support a remote core instance instead
<@wumpus>
along with using async notifications for replies from the core instead of synchronous calls and busy-waiting on locks
<@wumpus>
so I'm happy he's doing it, but it's not in the way that I would, maybe I should just let it go though
<@wumpus>
though I have a hard time reviewing code changes that aren't in line with my own thoughts
<@wumpus>
especially big ones
< sipa>
wumpus: is your concern that the result makes the client/walletmodel less independent?
< sipa>
in that it now requires the IPC abstraction below it
<@wumpus>
sipa: mostly that the code becomes even harder to work on
<@wumpus>
it seems like a double abstraction; walletmodel is already an abstraction for core's CWallet
<@wumpus>
now there's a class in between
< sipa>
seems like a reasonable concern to me... WalletModel and ClientModel don't really have much responsibility anymore
<@wumpus>
the best way to implement remote-anything is not necessarily changing every direct call into a IPC call
<@wumpus>
it's the easiest way to do it, but it's not how modern GUIs are written, ther shouldn't be anything waiting on replies from a remote process inside the GUI thread/loop
<@wumpus>
even for local implementation this is wrong, and it means that while e.g. sending a transaction while cs_main is busy, the GUI hangs instead of being able to show a moving progress indicator
<@wumpus>
I'm much more concerned with that than IPCing everything
< sipa>
seems reasonable, i don't know much about that
<@wumpus>
so this is kind of calcifying the state of the code that I'm already not happy with
< jonasschnelli>
wumpus: I completely agree with you.
< jonasschnelli>
I think it will be very hard to change the GUI towards that goal with the current review strategy
<@wumpus>
jonasschnelli: I tried to talk to him about this, but he seemed convinced that my concern had nothing to do with his changes, and that this can be done first.
< jonasschnelli>
Also the split between Wallet and Node (walletmodal / clientmodel) makes really sense.
< jonasschnelli>
*model
<@wumpus>
and he's not entirely incorrect but I in my opinoin it makes sense to combine those two things; if the core and GUI communicate with asynchronous messages, goign from there to an IPC protocol is much easier
< jonasschnelli>
I often though maybe rewrite the GUI from the ground up,...
<@wumpus>
and will likely be less complex
<@wumpus>
I'm not sure that is a good idea
< jonasschnelli>
Yeah.. me neither... :)
<@wumpus>
a rewrite is a lot of work to get feature parity, all bugs are born new, etc
<@wumpus>
usually I prefer incremental improvements unless something is total junk, which I think the code is not
< sipa>
wumpus: that was my concern as well... it seems crazy to me that introducing an IPC layer somehow helps making things more asynchronous
<@wumpus>
sipa: not the way he's doing it, at least
<@wumpus>
in the way I always imagined doing it, it would
< jonasschnelli>
What design would make sense for asynchronous messages?
<@wumpus>
jonasschnelli: qt signals/slots? same way that RPCConsole and RPCThread communicate for example
< jonasschnelli>
So each node call would run in its own thread? Or would that be a single node-/wallet-interaction thread with queue?
<@wumpus>
signals could be generated by something that listens to a network pipe or local implemantion, most of the GUI code could be oblivous to that
<@wumpus>
one thread should be OK usually
<@wumpus>
long running operations could be their own thread
< jonasschnelli>
Yes. I think this would be much better... would also require context sensitive activity-inidcators instead of a GUI freeze when communication is in progress
<@wumpus>
right
< jonasschnelli>
reminds me that the RPCConcole/RPCThread missed a activity indicator...
<@wumpus>
heck, even changing the cursor to a bee as on the Atari ST is better than hanging the GUI
< jonasschnelli>
(things like topupkeypool 1000)
< jonasschnelli>
wumpus: haha
<@wumpus>
yes, indeed, it doesn't provide feedback that something is in progress
< jonasschnelli>
Yes. We could start with a general activity indicator ("node communication happening") in the status bar
< jonasschnelli>
I hope we can convince ryanofsky
<@wumpus>
in any case showing a modal dialog is still possible
<@wumpus>
if you don't want the user to do other things
<@wumpus>
that's not hanging the GUI; the dialog could e.g. show an animation, or have an abort butotn, or react to the user in other ways
< jonasschnelli>
Yes. The modal non blocking concept is much better
< jonasschnelli>
Blocking the GUI must be avoided any time
< jonasschnelli>
Otherwise users will force quite
< jonasschnelli>
force kill
<@wumpus>
right. And some OSes will blank out the window
< bitcoin-git>
[bitcoin] sipa opened pull request #10248: Introduce CHashVerifier to hash while deserializing (master...chashverifier) https://github.com/bitcoin/bitcoin/pull/10248
<@wumpus>
so on the other hand I like that people actively work on the code and dno't want to discourage it, just because I have some different ideas
<@wumpus>
I know I myself have way too many ideas and too little time to work on them
< sipa>
i think the ultimate goal of having the GUI being independent from the core, and being able to start/stop it separate is very nice
< sipa>
if that requires an extra abstraction layer somewhere, fine
< sipa>
but i'm not convinced why that abstraction layer can't be clientmodel/walletmodel
< jonasschnelli>
I think one of ryanofsky arguments is that the client-/walletmodal also contains "business" logic and not only communication handling.
< jonasschnelli>
Which I partially can follow
< sipa>
well that business logic should move to the core
< sipa>
some of it can likely be merged with some code in rpcwallet
< luke-jr>
Random reddit user suggests that txindex should default to on when pruning is disabled.
< sipa>
...?
< luke-jr>
he was annoyed he had to reindex to setup an Electrum server because txindex defaulted to off, and argues that the txindex is small compared to the full blockchain
<@wumpus>
no, txindex should not be default-enabled at any time. By far most people don't use it, while it forces an overhead on everyone.
<@wumpus>
if it's an annoyance to have to enable it once, well yes maybe, but there's a cost to everything
<@wumpus>
can be kind of annoying if people have some far-out usage of a program and then expect everyone to accomodate to them
< luke-jr>
I suspect there are more Electrum users than Core-wallet users, sadly
<@wumpus>
of course there are more electrum users, that doesn't invalidate anything I said, most people running bitcoin core do not run an electrum server
< luke-jr>
everyone using Electrum should be running their own Electrum server
<@wumpus>
in any case people shoudl be encouraged to use an index on the utxo set intead of on the whole block chain
< luke-jr>
need an index on the whole blockchain to get tx history
<@wumpus>
there should be no need to index the whole block chain ,ever, unless you're doing chain analysis stuff, in which case you're working with such big data thatdoing an extra reindex should be peanuts
< sipa>
there is something crazy in this reasoning: bitcoin core is too annoying to use due to resource usage, thus people switch to electrum, but then complain they can't easily run their own server... which is even more expensive than just running a full node wallet in the first place
< sipa>
also, doesn't electrum full history require addrindex even?
< sipa>
not just txindex?
< luke-jr>
sipa: many people use Electrum because it supports hardware wallets
< luke-jr>
and yes, Electrum's server generates that itself from the txindex
< sipa>
then why can't it create the txindex too?
< luke-jr>
not sure.
< sipa>
luke-jr: well electrum uses a model that makes it inherently more expensive to run a server
< luke-jr>
sipa: depends whether storage/indexing costs more than doing the filtering on-the-fly for bloom nodes
< sipa>
stupid "the blockchain is my wallet model"
<@wumpus>
it could, but remember that it is python code, so doing it there would be slower than having core handle it
< luke-jr>
indexes are far less expensive than filtering for each user
< sipa>
luke-jr: my view is that end users shouldn't need the full blockchain
< sipa>
much less so a fully indexed one
< sipa>
i understand electrum has advantages, but there must be ways to get them without burdening full nodes further
< luke-jr>
sipa: not sure that's avoidable
< sipa>
finding your old transactions should be a rare occurence
<@wumpus>
if everyone ran their own trusted full node to connect to, they could run some remote wallet UI instead of electrum on the light device side
< luke-jr>
bloom allows you to get the data from other nodes, and use your own only for consensus, but those other nodes could omit info
< luke-jr>
wumpus: yes, in theory
<@wumpus>
luke-jr: everyoen running their own electrum server is theory just as well
<@wumpus>
luke-jr: and a much more overhead one at that
< sipa>
without the 'blockchain is my wallet' model, you could easily outsource the recovery of old transactions to a third party
<@wumpus>
why would everyone need to index everyone's transactions?
<@wumpus>
that's just crazy
< sipa>
to be clear: my concern is indirect... txindex is indeed that not much of a resource problem, but the fact that something 'requires' you to have txindex, implies it requires you to have the full blockchain readily available
< sipa>
an ecosystem that relies on everyone having the full blockchain available is disaster for scalability
<@wumpus>
in any case txindex is already supported, people can use it if they want
<@wumpus>
what I find deplorable is that people want to burden everyone with it just to avoid a one-time overhead for them
< sipa>
maybe instead we should make pruning default :)
< instagibbs>
^ was about tot say that
<@wumpus>
but I guess it fits well in the weird idea that many people have about the block chain, that it's some infinite externality that one can just keep dumping into, without relevant cost to anyone
< luke-jr>
sipa: IMO wait until we can IBD from pruned nodes
< sdaftuar>
sipa: lets merge that default to motivate block download improvements for 0.15 :)
< sipa>
luke-jr: fair point
< luke-jr>
I did make a pruning GUI for Knots, but it uses rwconf :x
<@wumpus>
I think offering a choice in the initial GUI dialog would already be nice
< sipa>
agree
<@wumpus>
(the one that lets you choose the data directory and shows an estimate of disk usage)
<@wumpus>
I wouldn't like pruning to be the default because it means throwing away downloaded data by default. Prefer to err on the safe side.
<@wumpus>
it's trivially possible to go from non-pruned to pruned, the other way around can be expensive
<@wumpus>
sure if e.g. the machine has little disk space it can be encouraged strongly to prune
<@wumpus>
just think it should not be enabled silently
< luke-jr>
IIRC the hold-back from Core was that you wanted the GUI to use an enum for each setting
< luke-jr>
but maybe just a minimal approach that only uses it for pruning would be okay before that's done
< jonasschnelli>
The idea of the branch above was to ask about pruning / dbcache / folder (what we have right now)
<@wumpus>
from a sandboxing point of view I don't really like the idea of daemons writing their own configuration files
<@wumpus>
but if it doesn't write bitcoin.conf, well, it's not too bad
< luke-jr>
right, that's why it was an independent file
< sipa>
maybe we should go back to serializing settings into wallet.dat
<@wumpus>
right
< * sipa>
ducks
<@wumpus>
also having it in per-network directory cleverly avoids the problem of a testnet and mainnet instance trying to write the file at the same time
< luke-jr>
ok, so plan: 1) split the rwconf low-level from the GUI Options changes, and PR the former only; 2) in a second PR (?), do just the pruning Intro stuff?
<@wumpus>
yes, I hope adding yet another settings source doesn't introduce a mess, it makes it harder to reason what takes precedence
<@wumpus>
ok, will build
< luke-jr>
thx
< jonasschnelli>
Using lambdas in Qt is a no go? Right? It would break Qt4 compile compat?
< sipa>
you cant use c++11 lambdas?
< jonasschnelli>
You can... but I think if you are using Qt "connect" mechanism and directly embed a lambda it will no longer be compilable with Qt4...
< jonasschnelli>
sipa: Yeah. But I think Qt4 can't handle something like connect( inProject, &myProject::signal, [=] () {}); because it's a marco or a MOC function (or whatever Qt uses to decompose)
< sipa>
oh
< Chris_Stewart_5>
sipa: Since we are talking about lambdas, if i use '[&]' on lambdas that will force the arg to be passed by reference, instead of being copied correct?
< ryanofsky>
i kind of like the coding convention where you use unnamed [&] capture for any lambda called synchronously, and named capture [&var1, var2] or [] for any lambda called asynchronously
< ryanofsky>
asynchronously meaning the closure is copied and called after the lambda expression goes out of scope
< bitcoin-git>
[bitcoin] jonasschnelli opened pull request #10251: Add balances cache / GUI: use a signal instead of a poll thread (master...2017/04/gui_rm_locks) https://github.com/bitcoin/bitcoin/pull/10251
< sipa>
ryanofsky: that's a nice convention
< sipa>
ryanofsky: is there a way to construct a non-copyable lambda?
< ryanofsky>
in c++14 there definitely is because you can capture by move
< ryanofsky>
so the resulting closure is move-only
< ryanofsky>
otherwise i don't think you can do it directly without std::bind or something
< sipa>
which incurs a performance penalty?
< ryanofsky>
yeah, probably minor unless lambda is capturing a lot of variables or a very big struct by copy
< bitcoin-git>
[bitcoin] jtimon closed pull request #10227: Make functions in validation.cpp static: (master...b14-chainparams-validation-static) https://github.com/bitcoin/bitcoin/pull/10227
< bitcoin-git>
[bitcoin] jonasschnelli opened pull request #10254: [Qt] Remove shutdown poll timer and replace it with a signal (master...2017/04/qt_shutdown) https://github.com/bitcoin/bitcoin/pull/10254
< bitcoin-git>
[bitcoin] jimmysong opened pull request #10255: [test] Add test for listaddressgroupings (master...test_listaddressgroupings) https://github.com/bitcoin/bitcoin/pull/10255
< morcos>
gmaxwell: i think in practice, the 3 calculations are often quite close.
< morcos>
but i wouldn't be concerned if it turned out the target/2 one dominated
< morcos>
you can look at each data point (tx and how long it took to get confirmed) as an independent event in which case you would expect the 60% n/2 and 85% n estimates to be roughly the same
< morcos>
but also you know they aren't really independent and at the extreme end of that , all you care about is the current tx congestion and what fee rate is being cleared down to
< morcos>
the way i looked at it is i could kind of have the best of couple differnet worlds by combining (taking the max) of different estimates
< morcos>
the n/2 estimate allows you to react more quickly b/c lets you look at the more recent history for more of the estimates so you'll react more quickly to changing conditions
< morcos>
but its important to also have an estimate with a 95% threshold so that you do know that your tx is going to be confirmed within a reasonable time frame.. you don't know if some of the data points were accelerated via special arrangement or CPFP'ed or had other special characteristics
< morcos>
also, if you have a long history of 100% of things being concerned, it can take quite some time for estimates to drop
< morcos>
man this is hard to walk through on IRC, it kind of sounds so handwavy
< sipa>
haha
< gmaxwell>
sipa: is there a reason that the CBlock doesn't cache its hash and redblockfromdisk couldn't use something like CHashverifier to populate it on construction?
< BlueMatt>
gmaxwell: not so useful given hash is only 80 bytes...
< BlueMatt>
but, it should cache in some way, indeed
< gmaxwell>
yea, I guess the hashverifier doesn't really matter for performance due to the size. I keep thinking it hashes the whole block I dunno why readblock from disk is so slow since it doesn't.
< luke-jr>
does it hash every transaction?
< BlueMatt>
gmaxwell: we have a benchmark for that because its so damned slow
< instagibbs>
why is it so damned slow
< BlueMatt>
instagibbs: you're allocating a metric shitload of objects on the heap
< gmaxwell>
oh because of that. right. why do I keep forgetting the allocations.
< gmaxwell>
Luke's revised text for the 0.14.2 changes is confusing. :(
< gmaxwell>
er 0.14.1
< gmaxwell>
I wish text weren't so hard.
< gmaxwell>
Apparently it has people thinking that 0.14.1 change to relay blocks to non-segwit nodes. :( I can see how it's easy to read that way, but didn't notice it before.