< sipa> cfields: clang does not like my code :(
< cfields> sipa: yea, needs a few little fixups...
< cfields> just template the param, and move the alignof() to the front of the definition
< cfields> (I assume you knew that, just verifying that it makes clang happy :)
< sipa> or just get rid of those helper functions
< sipa> they don't give much abstraction anyway :)
< cfields> heh
< cfields> sipa: this includes the optim from #13400. Was that intended?
< gribble> https://github.com/bitcoin/bitcoin/issues/13400 | sha256: small speedup for sse4 path. by theuni · Pull Request #13400 · bitcoin/bitcoin · GitHub
< sipa> oh?
< sipa> no
< sipa> i don't see that
< cfields> looking again, maybe order has me confused.
< phantomcircuit> sipa, the only caller of CCoinsViewDB::BatchWrite seems to be CCoinsViewCache::Flush which calls clear() on cacheCoins after
< phantomcircuit> but BatchWrite is erasing each entry itself from cacheCoins passed by reference it seems
< sipa> phantomcircuit: correct
< sipa> it's also potentially creating new entries in the parent cache
< sipa> so to compensate for that memory usage, it also erases on the fly from the other one
< gmaxwell> sipa: now that you've done intrensics you should be able to specialize the 64byte double sha2
< gmaxwell> pretty easily?
< phantomcircuit> sipa, right i see the pr, it's trying to avoid peak memory usage effectively being doubled by caching things twice
< gmaxwell> (at least the 32-byte specialized version should get heavy use... because of all the places we use double sha2...)
< sipa> gmaxwell: yup, one thing at a time
< phantomcircuit> does std::unordered_map::clear() even do anything on an empty map?
< sipa> no
< phantomcircuit> ok so that call in Flush() is effectively a noop but makes it clear that it's empty i guess
< cfields> sipa: hmm, come to think about it, the slowdown occured on AMD when Round() was done on SIMD instructions. Maybe it doesn't pay the same price on integers?
< cfields> Sigma0/Sigma1, that is.
< sipa> what slowdown?
< cfields> sipa: nm, I'll comment on the PR
< sipa> cfields: fixed, hopefully
< sipa> cfields: the new Round function in https://github.com/theuni/bitcoin/commit/d69a5164f914c6c2945d8f32134faa0da87795f5 makes the benchmark go from 3.68 ms to 4.16 ms
< cfields> sipa: Mind giving https://github.com/theuni/bitcoin/commit/4ee6fbb8b7525988783030eb0799bbc7293d50a0 a try? That's a little less dumb, doesn't force a dependency.
< sipa> (for me, on i7-7)
< sipa> sure
< cfields> sipa: I'm mostly curious to see if it's quicker on AMD.
< sipa> i'll try the different versions on a Ryzen system too if you want
< sipa> then again, on those systems we'll likely use SHA-NI instead :)
< cfields> heh, right
< cfields> you see what I'm getting at though, right?
< sipa> cfields: yeah, i'll benchmark on other systems
< bitcoin-git> [bitcoin] lucash-dev opened pull request #13443: Removed unused == operator from CMutableTransaction. (master...remove-CMutableTransaction-equals) https://github.com/bitcoin/bitcoin/pull/13443
< bitcoin-git> [bitcoin] edsgerlin opened pull request #13444: depends: bump openssl to 1.0.2o (master...patch-1) https://github.com/bitcoin/bitcoin/pull/13444
< kallewoof> Maybe I missed it.. do we ask someone to remove the needs rebase tags after we rebase stuff or...?
< sipa> kallewoof: DrahtBot will remove them when it gets around
< kallewoof> Oh, it is delayed? Gotcha
< bitcoin-git> [bitcoin] ken2812221 opened pull request #13445: build: Reset default -g -O2 flags when enable debug (master...debug_cflags) https://github.com/bitcoin/bitcoin/pull/13445
< sipa> kallewoof: it's a bot, it goes through all PRs one by one, it takes a while
< sipa> MarcoFalke runs it
< bitcoin-git> [bitcoin] ccdle12 closed pull request #13067: [WIP] Unit test sub-directories - Continued (master...PR-fixes-ccdle12) https://github.com/bitcoin/bitcoin/pull/13067
< jonasschnelli> Is that a pay to pubkey unspent?
< bitcoin-git> [bitcoin] murrayn opened pull request #13446: Build: remove non-distribution files/directories during make distclean. (master...distclean) https://github.com/bitcoin/bitcoin/pull/13446
< rafalcpp> wumpus: hm, why are symlinks prohibited from being in Tree-SHA512? they seem to normally work, they are shown as blobs, their git-sha1 is identical for symlinks with same symlink target, and so is sha512 that we get from them
< rafalcpp> BlueMatt: why we disallow symlinks? (it's your commit be908a69 - Fail merge if there are any symlinks)
< rafalcpp> it was decided in #9871 to disable them because bitcoin doesn't need them so no need to wonder if we handle them correctly
< gribble> https://github.com/bitcoin/bitcoin/issues/9871 | Add a tree sha512 hash to merge commits by sipa · Pull Request #9871 · bitcoin/bitcoin · GitHub
< wumpus> rafalcpp: basic security precaution, we could go into detail analysing the consequences of symlinks (relative, absolute, inside tree, outside tree, etc) or just blanket disallow them. As we don't require them, just disallowing is safer.
< wumpus> jonasschnelli: could be!
< wumpus> we don't really *want* symlinks ending up in the repository
< wumpus> `(they're also not compatible with some OSes)
< rafalcpp> wumpus: perhaps support for them could exist with --allow-symlinks defaulting to false? Other projects besides bitcoin could benefit from tree sha512
< wumpus> yes- it could be a git setting that defaults to false. But I don't think it's urgent.
< bitcoin-git> [bitcoin] MarcoFalke pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/7c32b414b632...b22115d9a3b0
< bitcoin-git> bitcoin/master 419a198 practicalswift: docs: Add a note about the source code filename naming convention
< bitcoin-git> bitcoin/master e567713 practicalswift: Do not use uppercase characters in source code filenames
< bitcoin-git> bitcoin/master b22115d MarcoFalke: Merge #13312: docs: Add a note about the source code filename naming convention...
< bitcoin-git> [bitcoin] MarcoFalke closed pull request #13312: docs: Add a note about the source code filename naming convention (master...lowercase-filenames) https://github.com/bitcoin/bitcoin/pull/13312
< fanquake> I guess one slightly annoying limitation of the linters is that upstream changes can't be tested in /bitcoin
< wumpus> yes
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/b22115d9a3b0...5315660baef0
< bitcoin-git> bitcoin/master fa8071a MarcoFalke: qa: Log as utf-8
< bitcoin-git> bitcoin/master 5315660 Wladimir J. van der Laan: Merge #13440: qa: Log as utf-8...
< bitcoin-git> [bitcoin] laanwj closed pull request #13440: qa: Log as utf-8 (master...Mf1806-qaLogUtf8) https://github.com/bitcoin/bitcoin/pull/13440
< fanquake> wumpus See #13444, I don't think that's something we want to do?
< gribble> https://github.com/bitcoin/bitcoin/issues/13444 | depends: bump openssl to 1.0.2o by edsgerlin · Pull Request #13444 · bitcoin/bitcoin · GitHub
< wumpus> fanquake: will have a look
< wumpus> fanquake: so the only thing affected by the OpenSSL version in depends is pretty much qt nowadays, right?
< wumpus> I tend to agree with you that it is unnecessary
< wumpus> (assuming that there has been no security issue that necessitates it)
< fanquake> wumpus Looking at the release notes, basically every major change in the 1.0.2 series is a CVE
< wumpus> seems like a nightmare to keep track of
< wumpus> maybe "update to most recent version" isn't that bad an idea, periodically, I don't know...
< fanquake> tbh I prefer the "drop the requirement for OpenSSL entirely" idea :p
< fanquake> There have been a few shots at that in the past, but never merged for various reasons.
< wumpus> for the core it's doable, but dropping the requirement from the GUI doesn't seem feasible to me, besides replacing it with some alternative SSL library (qt supports some) but I'm not sure that's better...
< wumpus> e.g. on windows, qt can use the native SSL support, so wouldn't strictly need OpenSSL - but that doesn't sound too great to me either...
< fanquake> Especially given the seeming lack of people we have testing anything on Windows
< wumpus> that, too
< wumpus> also the GUI code does some juggling with certificates that doesn't go through the Qt crypto API, so is OpenSSL specific
< wumpus> it doesn't feel really worth working on
< wumpus> in a bizarre twist of fate, I was working on a deprecation plan for payment requests, which would have solved this problem once and for all, but then bitpay announced they will *only* support it from then on.
< fanquake> heh
< fanquake> I remember that
< fanquake> #11622
< gribble> https://github.com/bitcoin/bitcoin/issues/11622 | build: Add --disable-bip70 configure option by laanwj · Pull Request #11622 · bitcoin/bitcoin · GitHub
< wumpus> that took all of the wind out of my sails in that regard
< wumpus> yes that one
< fanquake> I noticed that currently 1/3 of all PRs "need a rebase"
< wumpus> is:pr is:open -label:"needs rebase"
< fanquake> that indeed
< wumpus> or let's go nag some people, at least if it's PRs likely to be merged otherwise
< wumpus> it looks like most conflicts were caused by build system changes, as well as the txindex refactor
< fanquake> wumpus: wouldn't dare nag anyone :p
< wumpus> :p
< wumpus> any opinions on when we should tag 0.16.1 final?
< fanquake> Did anything happen for the assert on Windows?
< wumpus> not that I know of - rc2 was just a translation fix
< wumpus> you mean #13358? that doesn't seem to be a regression in 0.16.1
< gribble> https://github.com/bitcoin/bitcoin/issues/13358 | Assertion failed Error file chain.cpp 102 · Issue #13358 · bitcoin/bitcoin · GitHub
< fanquake> wumpus I thought there was another one on shutdown, but have just installed rc2 onto Windows 10 and no longer see it
< fanquake> cfields You might be interested, some new output I'm seeing after upgrading to Command Line Tools 10.0 https://0bin.net/paste/2iaLr12c+Q-4rlmS#lexGszZqOKn07pwYKQvTKYGgCBTkDPKGnlCo5PjbGTV
< fanquake> wumpus #12337
< gribble> https://github.com/bitcoin/bitcoin/issues/12337 | 0.16 Shutdown assertion · Issue #12337 · bitcoin/bitcoin · GitHub
< fanquake> However seems like that doesn't need to hold up a 0.16.1
< jnewbery> Review beg for #13066. We haven't run successfully run the extended tests or verify-commits in Travis for over two months and that would fix it
< gribble> https://github.com/bitcoin/bitcoin/issues/13066 | Migrate verify-commits script to python, run in travis by ken2812221 · Pull Request #13066 · bitcoin/bitcoin · GitHub
< wumpus> jnewbery: will take a look
< wumpus> kind of forgot about that one
< jnewbery> great. Thanks!
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/5315660baef0...ca2a23387be6
< bitcoin-git> bitcoin/master fa7a6cf MarcoFalke: policy: Treat segwit as always active
< bitcoin-git> bitcoin/master ca2a233 Wladimir J. van der Laan: Merge #13120: policy: Treat segwit as always active...
< bitcoin-git> [bitcoin] laanwj closed pull request #13120: policy: Treat segwit as always active (master...Mf1805-segwitGenesisPolicy) https://github.com/bitcoin/bitcoin/pull/13120
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/ca2a23387be6...fa4b9065a829
< bitcoin-git> bitcoin/master e5b2cd8 Chun Kuan Lee: Use python instead of slow shell script on verify-commits
< bitcoin-git> bitcoin/master fa4b906 Wladimir J. van der Laan: Merge #13066: Migrate verify-commits script to python, run in travis...
< bitcoin-git> [bitcoin] laanwj closed pull request #13066: Migrate verify-commits script to python, run in travis (master...verify-commits) https://github.com/bitcoin/bitcoin/pull/13066
< BlueMatt> wumpus: errr, did anyone else even review that?
< BlueMatt> else besides jnewbery, that is
< wumpus> yes, I reviewed it a long time ago
< BlueMatt> hmm, can you comment to that effect? I only see a concept ack from you on there
< wumpus> checked that it still was more or less the same
< wumpus> you didn't lookv ery well, there's an utACK from me in that topic too
< wumpus> apr 27
< BlueMatt> ah, ok, still wish we'd gotten more than 2 reviews with one pretty old on something thats designed to be usable to verify repo consistency
< wumpus> well at least it makes checking feasible again
< jnewbery> post-merge reviews welcome :)
< wumpus> no one was checking because the script was unusably slow
< BlueMatt> jnewbery: I dont feel comfortable reviewing security python, sadly
< BlueMatt> yes, indeed, better than it not getting run, which is apparently what was happening :(
< wumpus> FWIW, I'm 100% more comfortable reviewing python than shell script
< BlueMatt> heh, yes, I understand I'm like the only one left who prefers bash to python
< wumpus> there's just too much icky edge cases in shell script for me to be comfortable about it
< BlueMatt> lets just move everything to C :p
< wumpus> (like oh no, you forget to use the right quoting, now everything with spaces will escape.. and similar things)
< wumpus> feel free to rewrite it in C :p
< wumpus> it'd likely be even faster
< BlueMatt> lol, then it'd *actually* be usably fast
< BlueMatt> maybe I'll rewrite it in rust :p
< wumpus> hehe
< wumpus> yes that'd be cool
< * BlueMatt> has way too many rust projects now
< BlueMatt> I mean I've got a whole multi-daemon pool server and proxies in rust
< jnewbery> well, if anyone else knows python well enough to review #13066, I'm sure BlueMatt will appreciate your postmerge ACKs
< gribble> https://github.com/bitcoin/bitcoin/issues/13066 | Migrate verify-commits script to python, run in travis by ken2812221 · Pull Request #13066 · bitcoin/bitcoin · GitHub
< wumpus> yes, it's always good to have more review, also of code already merged
< bitcoin-git> [bitcoin] ken2812221 opened pull request #13447: travis: Increase travis_wait time while verifying commits (master...patch-1) https://github.com/bitcoin/bitcoin/pull/13447
< bitcoin-git> [bitcoin] practicalswift opened pull request #13448: Add linter: Make sure we explicitly open all text files using UTF-8 encoding in Python (master...lint-python-utf8-encoding) https://github.com/bitcoin/bitcoin/pull/13448
< promag> #13111 is ready for review
< gribble> https://github.com/bitcoin/bitcoin/issues/13111 | Add unloadwallet RPC by promag · Pull Request #13111 · bitcoin/bitcoin · GitHub
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/fa4b9065a829...a607d23ae82e
< bitcoin-git> bitcoin/master 57ba401 Pieter Wuille: Enable double-SHA256-for-64-byte code on 32-bit x86
< bitcoin-git> bitcoin/master a607d23 Wladimir J. van der Laan: Merge #13393: Enable double-SHA256-for-64-byte code on 32-bit x86...
< bitcoin-git> [bitcoin] laanwj closed pull request #13393: Enable double-SHA256-for-64-byte code on 32-bit x86 (master...201806_dsha256_i386) https://github.com/bitcoin/bitcoin/pull/13393
< promag> MarcoFalke: how about a "dormant" label, automatically added for something not updated for X days (either github or git)?
< bitcoin-git> [bitcoin] instagibbs opened pull request #13449: [WIP] support new multisig template in wallet for Solver, signing, and sign… (master...largemultisig) https://github.com/bitcoin/bitcoin/pull/13449
< luke-jr> #12859 seems to not be in 0.16.1 rc2?
< gribble> https://github.com/bitcoin/bitcoin/issues/12859 | Bugfix: Include for std::unique_ptr by luke-jr · Pull Request #12859 · bitcoin/bitcoin · GitHub
< luke-jr> wumpus: ^
< wumpus> luke-jr: it was also not tagged as such
< * luke-jr> doesn't do the tagging XD
< wumpus> you could have mentioned it, at least
< luke-jr> just did
< * luke-jr> wonders how wide this affects
< luke-jr> seems like Ubuntu doesn't have Qt 5.10 yet, but Debian testing does
< luke-jr> dunno how to check Fedora
< luke-jr> Arch has 5.11
< bitcoin-git> [bitcoin] practicalswift opened pull request #13450: Add linter: Enforce the source code file naming convention described in the developer notes (master...lint-filenames) https://github.com/bitcoin/bitcoin/pull/13450
< bitcoin-git> [bitcoin] instagibbs opened pull request #13451: expose CBlockIndex::nTx in getblockheader (master...expose_nTx) https://github.com/bitcoin/bitcoin/pull/13451
< luke-jr> #13120 should probably be backported too
< gribble> https://github.com/bitcoin/bitcoin/issues/13120 | policy: Treat segwit as always active by MarcoFalke · Pull Request #13120 · bitcoin/bitcoin · GitHub
< luke-jr> perhaps simplified
< bitcoin-git> [bitcoin] instagibbs opened pull request #13452: have verifytxoutproof check the number of txns in proof structure (master...actuallyverifytxoutproof) https://github.com/bitcoin/bitcoin/pull/13452