< wallet42>
@sipa: can we imagine a -utxomove option where coins get renamed to s[txid][nout] instead of deleting them from chainstate? Ideally also append the spend block height to the value. It would allow a lot of my analysis to not require a second database.
< wallet42>
or maybe call it -stxodb=1
< sdaftuar>
wallet42: that sounds interesting, but would be terrible for performance -- it would prevent an optimization we currently do where coins that are created in our memory cache, which are fully spent, never need a disk access
< sdaftuar>
wallet42: so i wouldn't expect that to be something we implement in the repo
< wallet42>
interesting..
< wallet42>
we'll it's obviously not for general consuption
< wallet42>
also maybe having another leveldb where these spend coins are stored instead of bloating the chainstate db
< wallet42>
sdaftuar: what's you estimate for manhours for someone who is familiar with the code for such a change?
< sdaftuar>
wallet42: no idea, sorry
< wumpus>
i wouldn't expect that to be upstreamed, but yeah you can do the patch for yourself, for better or worse lots of analysis is easiest to do by changing the node code
< sipa>
wallet42: i'd rather spend time on creating a project that implements tgat as a separate process than trying to hack it into bitcoind
< wallet42>
so how to approach this, zmq block only works after IBD
< wallet42>
i would love to not have to much duplicate data
< wumpus>
it's a one line change to make zmq notifications work before IBD
< wumpus>
certainly faster than hacking how the database works...
< sipa>
wallet42: i'd make a python project that uses basic p2p implementation
< wallet42>
will bitcoind then wait with processing the blocks and publishing to zmq when a buffer is full?
< wallet42>
so get multiple blocks in binary via INV from a local node
< sipa>
if your bitcoind is pruned, there is no duplication
< sipa>
with manual pruning you can inform bitcoind when you're ready processing a block
< wumpus>
no, bitcoind will never block on zmq notifications
< wumpus>
adding that wouldn't be too difficult either if zmq supports it
< wallet42>
wumpus: python will much slower to process blocks than bitcoind
< wumpus>
pypy is nice
< wallet42>
sipa: what about an rpc that i can use to query the blockindex and then access the *.blk files ondisk?
< wumpus>
and python is much more flexible as to formulating queries and analysis etc then hardcoding things into c++
< wallet42>
wumpus: love pypy, unfortunatly wasn't able to compile leveldb bindings for pypy3.6
< wumpus>
then don't use leveldb
< wumpus>
wallet42: you can look at contrib/linearize, it pretty much does that, though not through an RPC
< wumpus>
it builds its own mapping from block hash to location on disk
< wumpus>
it only uses rpc to get the list of block hashes that is the current chain
< sipa>
wallet42: you can use the rest interface to request blocks in binary
< sipa>
probably faster than rpc
< sipa>
p2p protocol is probably even faster
< wumpus>
that, too, though there's still a deserialize/serialize involved in that case, in principle rest could send the data directly from disk but it doesn't
< wallet42>
sendfile linux?
< wumpus>
maybe
< wallet42>
okay thanks for the pointers!
< wumpus>
in any case, usually optimization work is wasted if all you do is a one-time analysis
< wumpus>
just use a big machine
< wumpus>
hw is cheap custom software is not
< wallet42>
i want to build a in-memory blockchain database. instead of 1 big machine, have multiple machines indexed different parts of the chain
< wallet42>
clusterdb
< wallet42>
then have a map/reduce approach to queries
< wumpus>
it's a pity bitcoin's block chain is so easy to analyse isn't it
< bitcoin-git>
[bitcoin] Thoragh opened pull request #11533: Docs: Update WSL installation for Fall Creators update (master...Update-WSL-installation-for-Fall-Creators-update) https://github.com/bitcoin/bitcoin/pull/11533
< sdaftuar>
;;tblb 3600
< gribble>
The expected time between blocks taking 1 hour and 0 seconds to generate is 1 week, 3 days, 4 hours, 31 minutes, and 14 seconds
< sdaftuar>
;;tblb 7200
< gribble>
The expected time between blocks taking 2 hours and 0 seconds to generate is 51 years, 15 weeks, 4 days, 23 hours, 13 minutes, and 38 seconds
< bitcoin-git>
[bitcoin] sdaftuar opened pull request #11534: Evict outbound peers if tip is stale (master...2017-10-stale-tip-eviction) https://github.com/bitcoin/bitcoin/pull/11534
< PatBoy>
thx to all your great work guys ! (sry for writing spam here)
< wumpus>
*cries* more travis failures on master
< wumpus>
"Please upgrade to at least gpg 2.1.10 to check for weak signatures"
< karelb>
Travis is getting very unreliable recently
< karelb>
But you get what you pay for :)
< wumpus>
we'd pay for better travis support if it was possible!
< wumpus>
however there's no paid option for open source projects
< karelb>
Oh :(
< karelb>
That's unfortunate
< cfields>
wumpus: i have a ticket open atm about my suspended account (suspected of mining, i assume). I'll ask again when I hear back.
< wumpus>
cfields: thanks, hopefully they'll resolve that any time soon. mining. lol
< cfields>
wumpus: i assume the recent monero js mining stuff has made things difficult for them :(
< achow101>
what does "language: minimal" mean? I can't seem to find docs about it
< cfields>
achow101: it just selects their most minimal base image
< achow101>
oh
< cfields>
i can't find wtf a usable gnupg2 is installed on the big generic image, though :\
< wumpus>
it makes no sense
< arubi>
last time I tried tracking these things, I ended up in https://quay.io/organization/travisci . most updated seems to be te-main, you can follow tag -> sha of specific commit -> packages to see versions and stuff. eventually I ended up using their docker infrastructure to just run vanilla docker images
< cfields>
wumpus: wait, i can actually reproduce the error locally
< cfields>
ah, jeez
< cfields>
so many red herrings
< cfields>
one of sipa's old keys expired last night
< cfields>
and verify-commit doesn't play nice with that
< * cfields>
punts to BlueMatt
< BlueMatt>
argh
< cfields>
BlueMatt: for travis, i think we can just verify the commit range rather than the whole branch?
< cfields>
that would solve this issue at least
< BlueMatt>
no
< BlueMatt>
if verify-commits is broken, master needs to be force-pushed
< cfields>
oh wait nm, these aren't PRs
< BlueMatt>
thats kinda the whole point
< BlueMatt>
if a key is expired, however, you can just un-expire the key
< BlueMatt>
and then verify-commits will magically work again
< bitcoin-git>
[bitcoin] TheBlueMatt opened pull request #11539: [verify-commits] Allow revoked keys to expire (master...2017-10-verify-regsig-expsig) https://github.com/bitcoin/bitcoin/pull/11539
< cfields>
BlueMatt: i'm confused. I get different strings out of gpg
< cfields>
(looking at git verify-commit --raw <sha>)
< cfields>
i see KEYREVOKED/KEYEXPIRED/SIGEXPIRED
< cfields>
oh wait, REVKEYSIG/EXPKEYSIG take the place of VALIDSIG
< BlueMatt>
no they dont?
< cfields>
ok then confused. Could you paste your output of verify-commit --raw ?
< BlueMatt>
cfields: you should get a VALIDSIG line AND a REGKEYSIG/EXPKEYSIG line, you will however, not get a GOODSIG line (which is what git wants to see)
< cfields>
ok, gotcha
< BlueMatt>
are you getting confused between the --weak-digest sha1 pass and the one missing it?
< cfields>
no, i'm good. I was thinking different gpg versions might use different terms.
< cfields>
i was just missing the EXPKEYSIG output, only saw "KEYEXPIRED"