< provoostenator>
fanquake: can you add an instruction on how to lock the Ubuntu Docker image to a specific hash? (as we do with Debian VM images, though not with LXC base images)
< bitcoin-git>
bitcoin/master b7df96f Chun Kuan Lee: refactor: Drop boost::this_thread::interruption_point and boost::thread_interrupted in main thread
< bitcoin-git>
bitcoin/master b8b0b8c Wladimir J. van der Laan: Merge #14480: refactor: Drop boost::this_thread::interruption_point and boost::thread_interrupted in main thread...
< bitcoin-git>
[bitcoin] laanwj closed pull request #14480: refactor: Drop boost::this_thread::interruption_point and boost::thread_interrupted in main thread (master...drop-boost-thread-import) https://github.com/bitcoin/bitcoin/pull/14480
< bitcoin-git>
bitcoin/master 5c40e7b marcoagner: test: allows test_runner command line to receive parameters for each test
< bitcoin-git>
bitcoin/master 9390217 MarcoFalke: Merge #14795: test: allows test_runner command line to receive parameters for each test...
< bitcoin-git>
[bitcoin] MarcoFalke closed pull request #14795: test: allows test_runner command line to receive parameters for each test (master...fix_test_runner_parameters) https://github.com/bitcoin/bitcoin/pull/14795
< bitcoin-git>
[bitcoin] laanwj closed pull request #14884: Travis: enforce Python 3.4 support through linter (master...2018/12/python-3-4) https://github.com/bitcoin/bitcoin/pull/14884
< bitcoin-git>
[bitcoin] MarcoFalke opened pull request #14893: 0.17 [Backport 14890] rpc: Avoid creating non-standard raw transactions (0.17...Mf1812-17Backports) https://github.com/bitcoin/bitcoin/pull/14893
< jamesob>
#proposedmeetingtopic I'd like to revisit making some headway on one of the outstanding "add address index" PRs (e.g. #14035). Many companies end up using dubious peripheral systems (e.g. Bitcore) simply because bitcoind doesn't offer an address index, and I think there's a case to be made that the engineering costs & risk associated with maintaining such an index in core is outweighed by the practical risk many
< jamesob>
companies assume in having to run less-than-ideal indexing software. Would like to also discuss the alternative of having a "blessed" peripheral program that does indexing should an in-core index prove objectionable.
< jcorgan>
i used to maintain the addrindex patch originally written by sipa, before btcdrak took it over, then abandoned. this was an index over addresses referenced in all TXOs in the ledger, not just UTXOs
< jcorgan>
imho it was a simple, straightforward indexing option, like txindex, worked well, and the objections to merging it were mostly philosophical and somewhat political.
< * sipa>
believes it's inadvisable for people to build solutions that rely on a fully indexed blockchain to be available (as opposed to having wallet software that only scans for what matters); having it available in core would be misunderstood as it being a reasonable solution for production systems (while it should only be used for debugging)
< sipa>
i think that having an easily-accessible index may be preferable to everyone using the same centralized crappy one anyway, but there are also issues of scope creep etc
< sipa>
so a well-maintained separate index software package would be great, i think
< jcorgan>
i see our differences of opinion have survived the six years since #2802 ;-)
< sipa>
also, the software behind blockstream.info is open source now
< jcorgan>
er, five years
< gmaxwell>
jamesob: considering that we have not made much progress towards escaping the initial sync time which are toxic for the continued survival of the network, I don't think that work on that would be a good allocation of resources. In particual, if companies have a commercial interest in that functionality, then they can support creating it.
< gmaxwell>
jamesob: I think in general the project should be moving away from expecting that the user is going to immediately download the full blockchain history, rather than moving in the direction of making it more expensive to store and do.
< gmaxwell>
jamesob: having further clarification that utxo-only indexing actually satisfies any application would certantly be useful, since I think utxo oriented indexing at least stands at an intersection of interests.
< bitcoin-git>
bitcoin/master d6b3790 Chun Kuan Lee: tests: check readability of cookie file
< bitcoin-git>
bitcoin/master 2753285 MarcoFalke: Merge #14788: tests: Possible fix the permission error when the tests open the cookie file...
< bitcoin-git>
[bitcoin] MarcoFalke closed pull request #14788: tests: Possible fix the permission error when the tests open the cookie file (master...fix-win-test-perm-deny) https://github.com/bitcoin/bitcoin/pull/14788
< jamesob>
sipa: the only issue i can see with "having wallet software that only scans for what matters" is that in many cases users don't know what matters ahead of time and so the potential for constant rescanning probably makes amortizing that work into an index seem attractive
< sipa>
jamesob: then they're doing it wrong, is my point :)
< jamesob>
sipa: I agree with what you're saying, but I think the intuitive thing is to just build a big 'ol index and so that's what most engineers who're pressed for time end up doing
< jamesob>
so if we want to pitch the less intuitive "watch for what you want" angle, we should probably talk about building an example system that does that and tooling that makes it straightforward
< sipa>
jamesob: maybe that's something optech can do :)
< sipa>
and i agree that tooling is definitely lacking there (but the same is true for indexing, with everyone having their own solutions)
< jamesob>
heh, would love to. Maybe we can also gather some of the specific usecases as gmaxwell was saying
< jcorgan>
i moved to my own system of querying REST to deserialize blocks since genesis, indexing what i need, then listening on zmq for block notifications once caught up. it works well and lmdb makes a fantastic index store, but a lot more complicated than just rpc'ing addrindex and txindex when needed.
< jamesob>
it sounds like BIP-0157/0158 would go a long way towards the scan approach which is also something I'd like to talk about at some point... is anyone aware of any outstanding concerns with these BIPs and any reason we shouldn't be pushing forward on jimpo's related PRs?
< jamesob>
jcorgan right - having to add another RPC/auth mechanism besides core's kinda sucks. One interesting idea I've had is a service that would basically just wrap and enrich Core's API as well as do things like provide a more granular auth scheme (eg macaroons)
< sipa>
jamesob: i think bip157/158 are great
< jamesob>
sipa: that's good to hear. I'll allocate some time to review jimpo's work in the next week.
< sipa>
jamesob: with just 158, post-descriptor-based-wallet we could even use it for local wallet rescans
< jamesob>
yeah, definitely
< sipa>
(not right now, as we can't generate the set of all sPK's to watch for)
< bitcoin-git>
[bitcoin] jnewbery closed pull request #14888: Fix createrawtransaction multi op return - issue #14868 (master...fix-createrawtransaction-multi-OP_RETURN) https://github.com/bitcoin/bitcoin/pull/14888
< arch>
I apologize in advance if this was already answered somewhere, and will state in advance. Curious on a couple patch mgmt/versioning pieces with bitcoin-core.
< arch>
Since Berkeley DB (bdb) for database management was switched over to LevelDB in release 0.8 to reduce blockchain synchronization time, why does bdb remain in bitcoin/tree/master/depends/packages?
< arch>
To follow, if there are elements of bdb 4.8.30 from 5/1/2010 still necessary, why not update to Berkeley DB 12.1.6.0.19 - being the last version available under the Sleepycat Licence, and not currently identified with known vulnerabilities? Thank you for your review, and hopefully as well for a response.
< sipa>
arch: only the blockchain database is leveldb; wallet files are still BDB
< sipa>
arch: and BDB has terrible compatibility guarantees; once opened with version 4.9, you're unable to open it with 4.8 again for example
< arch>
sipa: Thank you. Other than compatability, is there any reason why it could not be upgraded to the last version indicated above?
< sipa>
arch: i think the API isn't compatible anymore with version 6+, but you can upgrade to 5.3
< sipa>
note that such binaries will not be interoperable with release binaries
< gmaxwell>
Also version 6+ has a less liberal software license.
< arch>
Yes, aware the software license became a problem after 12.1.6.0.19.
< gmaxwell>
sounds to me like someone else forked pre v6.0.20 and gave it a really high version number or something.
< gmaxwell>
in any case, our reason is 99% compatiblity. Becoming incompatible with wallet files isn't really acceptable, and we're not aware of any issues with bdb 4.8 that impact our use of it
< arch>
Since many of those CVE's included 12.1.6.0.35, and much of the 11.x platform, it was noted it specifically did not include 12.1.6.0.19 (last Sleepycat). As it was possible the much earlier versions may be susceptible to attack but undocumented, the hypothesis is that 12.1.6.0.19 may eliminate the 'unknown' vulnerability pieces and continue to remain compliant with open license model. This is something beyond me, but offering
< arch>
Thanks
< gmaxwell>
arch: 'vulnerabilities' are not really a concern in our usage of it.
< gmaxwell>
it isn't exposed to the network.
< arch>
fmaxwell: Would there be any possibility of localized attacks if there are potentially unknown vectors of attack on bdb? If not, then perfect. If yes, then perhaps further exploration is warranted.
< arch>
Sorry gmaxwell:
< gmaxwell>
arch: bitcoin is an unprivledge process, and if you get a wallet file from an untrusted source all bets are off in any case.
< gmaxwell>
(aside, a while back I fuzz tested bdb 6.something with the same cases that cause oob memory access in data files in bdb 4.x and they hadn't fixed them.
< gmaxwell>
)
< gmaxwell>
And, again, at the end of the day it isn't compatible.
< gmaxwell>
"Sorry, you upgraded, all your funds are gone"
< arch>
gmaxwell: Yes, that would be the ultimate problem.
< cluelessperson>
if my skillset is python, how might I contribute with that?
< wumpus>
cluelessperson: improve the functional tests!
< jamesob>
cluelessperson: could also take a look at adding any missing features to python-bitcoinlib
< wumpus>
almost all the code that's not c++ in the bitcoin core repository is python
< cluelessperson>
wumpus: would you be able to help me get started?
< cfields>
jonasschnelli: I went ahead and pushed the win sigs, feel free to tag whenever you push the mac ones. Just be sure to sign the tag.