< bitcoin-git> [bitcoin] instagibbs opened pull request #9906: Disallow copy constructor CReserveKeys (master...noreservecopy) https://github.com/bitcoin/bitcoin/pull/9906
< bitcoin-git> [bitcoin] TheOnlyAymes opened pull request #9907: 0.14 (master...0.14) https://github.com/bitcoin/bitcoin/pull/9907
< bitcoin-git> [bitcoin] TheOnlyAymes closed pull request #9907: 0.14 (master...0.14) https://github.com/bitcoin/bitcoin/pull/9907
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/f7ec7cfd38b5...58861ad91b49
< bitcoin-git> bitcoin/master 6485466 Wladimir J. van der Laan: test: Report InitBlockIndex result...
< bitcoin-git> bitcoin/master 58861ad Wladimir J. van der Laan: Merge #9904: test: Fail if InitBlockIndex fails...
< bitcoin-git> [bitcoin] laanwj closed pull request #9904: test: Fail if InitBlockIndex fails (master...2017_03_test_check_blkindex_result) https://github.com/bitcoin/bitcoin/pull/9904
< wumpus> jeremyrubin: well cloudabi executables are marked with a special "OS/ABI" type, so it's easy to identify them as such. Also it's recommended to use cloudabi-run to run them, which should not launch native executables
< gmaxwell> wumpus: nice blog post btw.
< wumpus> but yes it'd make sense to add a check to whatever processes submitted executables that it's really the right kind of executable
< wumpus> gmaxwell: thanks :)
< gmaxwell> wumpus: how is the kernel security for it implemented? assuming you're using its special libc and whatnot, does it just use seccomp mode 1 on Linux?
< gmaxwell> ah, it's the Capsicum stuff.
< wumpus> gmaxwell: the preferred approach is to have it run on top of capsicum, this is what is used on freebsd at least. There's also an emulator, which may use seccomp, not sure.
< gmaxwell> it's a little sad that dns requests have to be external, since libc vulnerablities in dns handling is one of the things that bothers me more about our common attack surface. E.g. DNSseeds start returning a glibc dns 0day.
< wumpus> well it could be implemented differently
< wumpus> you could have something that passes in a ready-to-go UDP file descriptor for communicating to your preferred DNS server
< gmaxwell> oh. hm yea, it's true.
< wumpus> then use an internal DNS implementation
< gmaxwell> or even a mini dns resolver process that just uses a non-sandboxed udp shim.
< wumpus> (e.g. libevent's)
< wumpus> exactly
< wumpus> my initial usecase will be to have all outgoing network traffic go through tor, so it won't be doing any DNS lookups locally at all, only at the exit nodes I guess
< gmaxwell> wumpus: for mlock, would it be possible for an external process to create an anonymous mmap, mlock it, then pass the descriptor over a unix domain socket to the sandboxed process?
< wumpus> gmaxwell: yes, that's possible
< jonasschnelli> What I once started is to use sipas crawler (bitcoin-seeder) and fill everything into a db, then use that db from djbdns (which is still one of the best dnsd IMO).
< wumpus> gmaxwell: it's a bit scary though as the external process will be able to keep a reference, and have all your key data :)
< gmaxwell> yes, though it could presumably also ptrace your process. and hopefully it wouldn't keep a reference. :)
< wumpus> ideally we'd not want to store any key data in the process at all; e.g. interface to yet another module for that, which could be hw
< gmaxwell> yea, indeed, should just be a signer process that has an interface like a hardware wallet. Sometimes it's an actual hardware wallet, other times it's a sandboxed process, SGX module, or whatnot.
< wumpus> right
< wumpus> jonasschnelli: to make a DNS server that can only look up bitcoind hosts?
< wumpus> jonasschnelli: oh I get it, of course, you want to run an alternative DNS server as DNS seeder
< jonasschnelli> wumpus: yes.
< jonasschnelli> The crawler is great,... but not the dnsd part
< gmaxwell> well we do use dns for things other than introduction.
< wumpus> do we?
< gmaxwell> sure, things like addnode can use it.
< gmaxwell> and it's not a bad idea there simply because ... IPs change.
< jonasschnelli> isn't that pure IPv4/v6?
< jonasschnelli> we could or we can?
< nemgun> Hello guys
< wumpus> it's possible to pass some of the connect-to command line options (proxy, tor, etc) as hostnames.... but meh, there's no use for that in cloudabi because it won't be able to connect to those anyway. The right wya to do that would be to pass in a fd to a pipe or such...
< wumpus> as for RPC you can do the lookups at the client side
< gmaxwell> wumpus: ah, well I thought it would eventually be good to be able to support the complete feature set (except for some antiquated features like the notify interfaces which might be better done another way).
< wumpus> well my goal with this experiment would be to make something better, sometimes that means making decisions to drop some functionality or do it some other way
< wumpus> it'd be possible to emulate the complete old feature set on top of the sandbox, but that's not very interesting
< wumpus> anyhow if addnode stores a name and does periodic DNS lookups, to track IP changes over time, there's something to be said that bitcoind needs to be able to initiate DNS lookups
< gmaxwell> it does (well it will relookup again on a reconnect, e.g. after a disconnect under the old name...)
< wumpus> simplest would be to use the "RESOLVE" [F0] SOCKS5 extension for that. This means the actual lookup can be done from another process (which could be sandboxed too)
< wumpus> this also means that if the DNS resolver process is compromised, bitcoind itself isn't
< wumpus> there's a bit of a bootstrapping problem here, ideally there would be a command 'instantiate a graph of cloudabi processes and connect them such and such'
< wumpus> btw verify-commits.sh is broken again
< wumpus> looks like it croaks on MarcoFalke's key
< jonasschnelli> hmm...
< bitcoin-git> [bitcoin] ryanofsky opened pull request #9908: Define 7200 second timestamp window constant (master...pr/timewin) https://github.com/bitcoin/bitcoin/pull/9908
< bitcoin-git> [bitcoin] ryanofsky opened pull request #9909: Add FindEarliestAtLeast test for edge cases (master...pr/faltest) https://github.com/bitcoin/bitcoin/pull/9909
< paveljanik> when running with -blocksonly, do we want to load and dump the memory pool?
< bitcoin-git> [bitcoin] ian-kelling opened pull request #9910: Docs: correct and elaborate -rpcbind doc (master...docs-rpcbind) https://github.com/bitcoin/bitcoin/pull/9910
< wumpus> paveljanik: does it hurt to do so? if not, better not to couple those options
< bitcoin-git> [bitcoin] laanwj pushed 10 new commits to master: https://github.com/bitcoin/bitcoin/compare/58861ad91b49...eb281842b7df
< bitcoin-git> bitcoin/master eca550f Luke Dashjr: RPC/Wallet: Pass CWallet as pointer to helper functions
< bitcoin-git> bitcoin/master d77ad6d Luke Dashjr: RPC: Do all wallet access through new GetWalletForJSONRPCRequest
< bitcoin-git> bitcoin/master 2e518e3 Luke Dashjr: Move nWalletUnlockTime to CWallet::nRelockTime, and name timed task unique per CWallet
< bitcoin-git> [bitcoin] laanwj closed pull request #8775: RPC refactoring: Access wallet using new GetWalletForJSONRPCRequest (master...multiwallet_prefactor_rpc) https://github.com/bitcoin/bitcoin/pull/8775
< paveljanik> wumpus, dumping is fast, but loading is very slow and CPU consuming. And it consumes memory if there was a large mempool.dat. But I do not care enough ;-)
< wumpus> I think there should be an option to bypass mempool loading/writing. And possibly it could parameter-interact with disable-blocks mode, as long as it's clearly documented.
< wumpus> or maybe an option to disable the mempool completely? I don't know.
< wumpus> I don't care enough either. It seems optimizing for something that is an incedigly rare edge case
< wumpus> if you want to optimize actual bottlenecks, please pick my sha256 or crc32 optimization experiment :-)
< paveljanik> Well, I'll first optimise my blocksonly gateway to the world ;-)
< wumpus> both optimizations that I mention are very relevant to validating blocks, too
< paveljanik> do you have the PR numbers? ;-)
< wumpus> no PRs yet,but you can find the branches linked in the README here: https://github.com/laanwj/bitcoin
< paveljanik> how do you satisfy multiarchitecture/multiOS with yasm?
< wumpus> dunno?
< wumpus> is yasm platform specific?
< wumpus> as for architectures, use a general interface, and compile on the assembly appropriate for the architecture. For specific instruction sets it makes sense to do run-time detection.
< wumpus> on x86 you can use the CPUID instruction for that (see for example the SDL2 implementation: https://github.com/spurious/SDL-mirror/blob/master/src/cpuinfo/SDL_cpuinfo.c), on ARM it's somewhat more difficult as the instruction to detect what instruction sets are available is privileged. So applications resort to OS-specific parsing of /dev/cpuinfo, or just trying out the instructions and catching SIGILL
< wumpus> openssl, most notably, uses the latter approach, which is why you'lll always have to skip a SIGILL when running bitcoind in gdb on ARM
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/eb281842b7df...ba80a684cfef
< bitcoin-git> bitcoin/master 90a1d9a Ian Kelling: Docs: add details to -rpcclienttimeout doc
< bitcoin-git> bitcoin/master ba80a68 Wladimir J. van der Laan: Merge #9903: Docs: add details to -rpcclienttimeout doc...
< bitcoin-git> [bitcoin] laanwj closed pull request #9903: Docs: add details to -rpcclienttimeout doc (master...docs-client-timeout) https://github.com/bitcoin/bitcoin/pull/9903
< jonasschnelli> What's the plan in how we solve the current travis problem in master?
< jonasschnelli> Anyone working on that?
< wumpus> I don't really know how to solve it
< wumpus> do we need to add a new key somewhere?
< wumpus> would prefer to leave it up to BlueMatt
< jonasschnelli> wumpus: https://github.com/bitcoin/bitcoin/issues/9898 isn't solved yet. right?
< jonasschnelli> Ah. But right, the issue on master is verify-commits.
< jonasschnelli> I though it was the travis OSX issue
< wumpus> oh I had forgot about that one
< wumpus> seems it went away out of itself though? pulls are passing
< wumpus> it's just master, where verify-commits is run, where things fail
< jonasschnelli> Okay. I'm happy 9898 is gone...
< bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/ba80a684cfef...75d012e8c7d0
< bitcoin-git> bitcoin/master 9de90bb Pavel Janík: Do not shadow variables (gcc set)
< bitcoin-git> bitcoin/master ad1ae7a Pavel Janík: Check and enable -Wshadow by default.
< bitcoin-git> bitcoin/master 75d012e Wladimir J. van der Laan: Merge #8808: Do not shadow variables (gcc set)...
< jonasschnelli> I think it would be great if #9294 could get a review from on of the bip32 authors (ping sipa, gmaxwell)
< gribble> https://github.com/bitcoin/bitcoin/issues/9294 | Use internal HD chain for change outputs (hd split) by jonasschnelli · Pull Request #9294 · bitcoin/bitcoin · GitHub
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/75d012e8c7d0...67c5cc19178d
< bitcoin-git> bitcoin/master 09fe346 Russell Yanofsky: Avoid -Wshadow warnings in wallet_tests...
< bitcoin-git> bitcoin/master 67c5cc1 Wladimir J. van der Laan: Merge #9828: Avoid -Wshadow warnings in wallet_tests...
< bitcoin-git> [bitcoin] laanwj closed pull request #9828: Avoid -Wshadow warnings in wallet_tests (master...pr/multishadow) https://github.com/bitcoin/bitcoin/pull/9828
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/67c5cc19178d...90cb2a218e58
< bitcoin-git> bitcoin/master f36bdf0 Johnathan Corgan: Enable host lookups for -proxy and -onion parameters...
< bitcoin-git> bitcoin/master 90cb2a2 Wladimir J. van der Laan: Merge #9774: Enable host lookups for -proxy and -onion parameters...
< bitcoin-git> [bitcoin] laanwj closed pull request #9774: Enable host lookups for -proxy and -onion parameters (master...hostname-lookups) https://github.com/bitcoin/bitcoin/pull/9774
< bitcoin-git> [bitcoin] jonasschnelli closed pull request #8764: [Wallet] get rid of pwalletMain, add simple CWallets infrastructure (master...2016/09/wallet_pointer) https://github.com/bitcoin/bitcoin/pull/8764
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/90cb2a218e58...53c300fb525a
< bitcoin-git> bitcoin/master e5b449c Ian Kelling: Docs: correct and elaborate -rpcbind doc...
< bitcoin-git> bitcoin/master 53c300f Wladimir J. van der Laan: Merge #9910: Docs: correct and elaborate -rpcbind doc...
< bitcoin-git> [bitcoin] laanwj closed pull request #9910: Docs: correct and elaborate -rpcbind doc (master...docs-rpcbind) https://github.com/bitcoin/bitcoin/pull/9910
< bitcoin-git> [bitcoin] paveljanik opened pull request #9911: WIP: Wshadow: gcc 4.8.5 fixes (master...20170303_Wshadow_streams) https://github.com/bitcoin/bitcoin/pull/9911
< bitcoin-git> [bitcoin] sdaftuar opened pull request #9912: Optimize GetWitnessHash() for non-segwit transactions (master...2017-03-optimize-witness-hash) https://github.com/bitcoin/bitcoin/pull/9912
< gmaxwell> What are the barriers on getting SSE2 SHA2 in use? The speedup for initial sync should be far from trivial now.
< gmaxwell> (plus lowering latency of everything else we do)
< sipa> what is the speedup factor?
< luke-jr> why not SSE4 or AVX?
< gmaxwell> If I'm not misremembering SSE2 was 95% of the speedup of SSE4 and AVX was not faster on the hardware we tested (and in fact AVX was slower on AMD). But just replace SSE2 in my comment with whatever made sense.
< gmaxwell> sipa: before jonasschnelli benchmarked IBD with the change and it was 5% faster. IIRC... but now with assume valid a MUCH larger percentage of the time is in hashing.
< gmaxwell> So "more than 5%", though I don't know how much more. It will only grow after your UTXO changes.
< sipa> does it need parallel hashing?