< bitcoin-git> [bitcoin] jnewbery opened pull request #18955: net processing: requeue transaction GETDATA requests more frequently (master...2020-05-requeue-txrequest) https://github.com/bitcoin/bitcoin/pull/18955
< meshcollider> wumpus: re #18700 what do you mean by not quite ready yet?
< gribble> https://github.com/bitcoin/bitcoin/issues/18700 | Fix locking on WSL using flock instead of fcntl by meshcollider · Pull Request #18700 · bitcoin/bitcoin · GitHub
< bitcoin-git> [bitcoin] fanquake pushed 5 commits to master: https://github.com/bitcoin/bitcoin/compare/eb2ffbb7c134...7a5767423f37
< bitcoin-git> bitcoin/master 047ceac Amiti Uttarwar: [net processing] ignore tx GETDATA from blocks-only peers
< bitcoin-git> bitcoin/master e257cf7 Amiti Uttarwar: [net processing] ignore unknown INV types in GETDATA messages
< bitcoin-git> bitcoin/master 2f03255 Amiti Uttarwar: [test] test that an invalid GETDATA doesn't prevent processing of future m...
< bitcoin-git> [bitcoin] fanquake merged pull request #18808: [net processing] Drop unknown types in getdata (master...2020-04-getdata) https://github.com/bitcoin/bitcoin/pull/18808
< bitcoin-git> [bitcoin] jnewbery closed pull request #18955: net processing: requeue transaction GETDATA requests more frequently (master...2020-05-requeue-txrequest) https://github.com/bitcoin/bitcoin/pull/18955
< bitcoin-git> [bitcoin] andrewtoth reopened pull request #18941: validation: Persist coins cache to disk and load on startup (master...persist-coinscache) https://github.com/bitcoin/bitcoin/pull/18941
< luke-jr> would be nice if Travis sent an email to the PR creator when it failed :x
< bitcoin-git> [bitcoin] practicalswift closed pull request #18432: util: Make our stringstream usage locale independent (master...cpp-locale) https://github.com/bitcoin/bitcoin/pull/18432
< bitcoin-git> [bitcoin] practicalswift closed pull request #18450: util: Use locale independent ToString(…) instead of locale dependent strprintf(…) for low-level string formatting (master...locale-independence) https://github.com/bitcoin/bitcoin/pull/18450
< bitcoin-git> [bitcoin] practicalswift closed pull request #18539: Avoid using locale-dependent boost trim functions in RPCAuthorized(…) and bitcoin-tx (master...avoid-locale-dependent-trim) https://github.com/bitcoin/bitcoin/pull/18539
< elichai2> jonatack: https://github.com/bitcoin/bitcoin/pull/18949#issuecomment-626901015 that's the main reason why I suggested this will be a thing anyone can add themselves, I don't think we have good metrics to choose reviewers ourselves
< bitcoin-git> [bitcoin] fanquake opened pull request #18956: build: enforce minumim required Windows version (7) (master...window_enforce_runtime_version) https://github.com/bitcoin/bitcoin/pull/18956
< bitcoin-git> [bitcoin] meeDamian opened pull request #18957: Add a link from ZMQ doc to ZMQ example in contrib/ (master...zmq-doc-link) https://github.com/bitcoin/bitcoin/pull/18957
< bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/7a5767423f37...49d237ce3230
< bitcoin-git> bitcoin/master 89fea68 fanquake: build: don't pass -w when building for Windows
< bitcoin-git> bitcoin/master 49d237c fanquake: Merge #18928: build: don't pass -w when building for Windows
< bitcoin-git> [bitcoin] fanquake merged pull request #18928: build: don't pass -w when building for Windows (master...dont_try_suppress_all_warnings_win) https://github.com/bitcoin/bitcoin/pull/18928
< bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/49d237ce3230...ad1b7b18177a
< bitcoin-git> bitcoin/master d97fac4 Damian Mee: Add a link from ZMQ doc to ZMQ example in contrib/
< bitcoin-git> bitcoin/master ad1b7b1 fanquake: Merge #18957: Add a link from ZMQ doc to ZMQ example in contrib/
< bitcoin-git> [bitcoin] fanquake merged pull request #18957: Add a link from ZMQ doc to ZMQ example in contrib/ (master...zmq-doc-link) https://github.com/bitcoin/bitcoin/pull/18957
< jonatack> gleb: interesting about bitpeers. seems there is the original one in go and a rewrite in python. which one did you try?
< bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/ad1b7b18177a...0f2fa599aee0
< bitcoin-git> bitcoin/master 83da576 Jon Atack: net: use CMessageHeader::HEADER_SIZE, add missing include
< bitcoin-git> bitcoin/master 0f2fa59 fanquake: Merge #18931: net: use CMessageHeader::HEADER_SIZE, add missing include
< bitcoin-git> [bitcoin] fanquake merged pull request #18931: net: use CMessageHeader::HEADER_SIZE, add missing include (master...netcpp-use-CMessageHeader_HEADER_SIZE) https://github.com/bitcoin/bitcoin/pull/18931
< jonatack> elichai2: right. I suppose it's worth trying. I like your idea of subscribing to labels you're interested in.
< bitcoin-git> [bitcoin] MarcoFalke pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/0f2fa599aee0...4fa31570144e
< bitcoin-git> bitcoin/master 3333cb9 MarcoFalke: fuzz: Pass down MAKEJOBS to test_runner
< bitcoin-git> bitcoin/master fa35c34 MarcoFalke: Remove unused ci configs that have been moved elsewhere
< bitcoin-git> bitcoin/master cbd6611 Russell Yanofsky: Set LD_LIBRARY_PATH consistently in travis tests
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #18929: ci: Pass down LD_LIBRARY_PATH and MAKEJOBS to fuzz test_runner (master...2005-fuzzMakejobs) https://github.com/bitcoin/bitcoin/pull/18929
< bitcoin-git> [bitcoin] MarcoFalke pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/4fa31570144e...e45fb7e0d276
< bitcoin-git> bitcoin/master 9ccaaba Jim Posen: [init] Add -peerblockfilters option
< bitcoin-git> bitcoin/master f9e00bb Jim Posen: [net processing] Message handling for getcfcheckpt.
< bitcoin-git> bitcoin/master 2308385 Jim Posen: [test] Add test for cfcheckpt
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #18877: Serve cfcheckpt requests (master...2020-04-serve-getcfcheckpt) https://github.com/bitcoin/bitcoin/pull/18877
< bitcoin-git> [bitcoin] practicalswift reopened pull request #18450: util: Use locale independent ToString(…) instead of locale dependent strprintf(…) for low-level string formatting (master...locale-independence) https://github.com/bitcoin/bitcoin/pull/18450
< bitcoin-git> [bitcoin] fanquake pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/e45fb7e0d276...8da1e43b63cb
< bitcoin-git> bitcoin/master 25d8264 tryphe: p2p: add MAX_FEELER_CONNECTIONS constant
< bitcoin-git> bitcoin/master e3047ed fanquake: test: use p2p constants in denial of service tests
< bitcoin-git> bitcoin/master 8da1e43 fanquake: Merge #18910: p2p: add MAX_FEELER_CONNECTIONS constant
< bitcoin-git> [bitcoin] fanquake merged pull request #18910: p2p: add MAX_FEELER_CONNECTIONS constant (master...add_max_feeler_connections) https://github.com/bitcoin/bitcoin/pull/18910
< bitcoin-git> [bitcoin] dongcarl opened pull request #18958: guix: Make V=1 more powerful for debugging (master...2020-05-guix-improvements) https://github.com/bitcoin/bitcoin/pull/18958
< bitcoin-git> [bitcoin] laanwj merged pull request #18945: build: Ensure source tarball has leading directory name (0.20) (0.20...2020_11_0.20_tarball_prefix) https://github.com/bitcoin/bitcoin/pull/18945
< bitcoin-git> [bitcoin] jnewbery opened pull request #18960: [indexes] Add compact block filter headers cache (master...2020-05-cfcheckpts-cache) https://github.com/bitcoin/bitcoin/pull/18960
< jb55> I was thinking about hacking BroadcastTransaction to only send to tor peers (when configured) as an alternative to the less granular onlynet=onion. surprised this isn't implemented? looks easy?
< luke-jr> sounds fragile
< jb55> reasoning: I want low block receive/relay latency but still want transaction broadcast privacy
< jb55> luke-jr: fragile in the sense of broadcast failing if there are no available tor peers or something?
< gleb> jb55: I think it's a good idea if implemented well, but the details need discussing. For example, if you're one of the very few nodes using that feature, a handful of peers can cluster your transactions together.
< luke-jr> jb55: in the sense that a non-tor peer might find a way to make you leak the info
< gleb> a handful of tor sybil peers :)
< luke-jr> even non-tor peers
< luke-jr> if everyone relays that tx except your IPv4 address…
< jb55> yeah so it would need to still relay it eventually, I was just thinking of hiding the initial source broadcast which might be easier to trace?
< gleb> You probably want a new circuit for every relay then? Depends on how widely this is used, tor network may be unhappy :p
< jb55> well I'm just thinking about tackling the easiest things first: hide initial broadcast but relay as usual like it some other person's tx. tor sybil attacks seems like a harder problem?
< gleb> jb55: tor sybil is not something abstract, they are probably already in place. And i'm saying this issue might make transactions *less private* after your idea. IP will be probably hidden (if done well), but then other issues arise. It's just not that trivial in my mind, so we need to be careful. I would ask other opinions before implementing,
< gleb> maybe a mailing list post?
< gleb> Like, would you relay to all your tor peers? Maybe relaying to just one works better in this case, since Tor relay part is already less "reliable"
< gleb> Probably you even want to relay to a *random* tor peer in the net?
< sipa> ideally you open a new tor connection, relay the tx, and disconnect
< sipa> any other communication you do with them lets them correlate the tx with your other traffic
< jb55> let's say I send a tx to a tor peer, they would have some custom code that correlates encrypted bitcoin-p2p traffic with transactions coming out of exit nodes somehow?
< jb55> trying to understand how likely this is...
< jb55> I guess if I connected to a tor peer that controlled exit nodes as well, and could correlate that way?
< nothingmuch> FWIW JoinMarket does has opt in tor broadcasting which connects to new random node: https://github.com/JoinMarket-Org/joinmarket/pull/662 and Wasabi picks node that is already connected, transmits to it and then disconnects from it:
< andrewtoth_> EPS also does what sipa describes https://github.com/chris-belcher/electrum-personal-server/pull/124
< andrewtoth_> mostly copied from JoinMarket
< bitcoin-git> [bitcoin] tarboss opened pull request #18961: remove assert in walletcontroller & run setparent in gui-qt main thread (master...master) https://github.com/bitcoin/bitcoin/pull/18961
< phantomcircuit> sipa, i'm thinking about adding a filter index for script pushes to support fast wallet rescans for legacy wallets, any thoughts on that
< phantomcircuit> aj, ^ same question
< sipa> phantomcircuit: unless you really need support for raw multisig (which i think the wallet doesn't even support anymore), you could use the bip158 filter
< sipa> and in descriptor wallets it should be even easier (as for those, the set of IsMine outputs is exactly equal to an explicitly constructed set)
< phantomcircuit> sipa, for non-legacy wallets using the existing bip158 "basic" filter (which is the entire script) is definitely preferable
< sipa> for legacy wallets too
< sipa> you can enumerate all watched addresses for those, actually
< sipa> it's just trickier
< sipa> since 0.17 bare multisig is not automatically watched anymore; you need to explicitly import it
< phantomcircuit> sipa, for older wallets my issue with constructing the scriptPubKey for each key is two fold, one i cannot at all support raw multisig, which possibly someone actually used, and two that will significantly effect the false positive rate of the filter
< sipa> you can also go the other way... build a separate ephemeral bloom filter for just the watched scriptPubKeys, and then iterate through blocks matching every sPK in it agains the filter
< sipa> that doesn't have the same I/O savings as using an explicitly constructed filter, but it's certainly logistically a lot easier
< sipa> and probably a lot faster than the current rescanning already
< sipa> the bare multisig problem doesn't exist anymore since 0.17
< phantomcircuit> sipa, why was support for raw multisig changed? significant performance issue?
< sipa> in fact, quite possibly that's how the wallet should work always
< phantomcircuit> actually found the #13002 pr
< sipa> just maintain a bloom filter of its watched sPKs, and match things against it
< gribble> https://github.com/bitcoin/bitcoin/issues/13002 | Do not treat bare multisig outputs as IsMine unless watched by sipa · Pull Request #13002 · bitcoin/bitcoin · GitHub
< phantomcircuit> sipa, my problem with that for legacy wallets is that we don't really have a way to list all of the sPKs for the wallet except to walk the private keys and calculate every sPK we have ever supported generating addresses for
< phantomcircuit> or am i getting that really wrong
< sipa> yes
< sipa> walk all the private keys, public keys, scripts, watched things, ...
< phantomcircuit> which will end up being a list that's substantially larger than you'd want
< sipa> how so?
< sipa> it is exactly what you want
< sipa> if you do anything else, you risk missing things
< phantomcircuit> possibly i add a new GCS filter index and se it in the wallet only when it's available?
< phantomcircuit> im not sure the performance of walking the chain to build an in memory bloomfilter would be great either honestly
< sipa> not the chain; the wallet
< sipa> ah, for a GCS filter, yes
< phantomcircuit> sipa, oh so build a filter of the wallet and then walk the chain checking against the filter
< yevaud> sipa: unfortunately walking all the permutations, while required for a legacy wallet, drives up the false positive rate substantially.
< phantomcircuit> i think we're talking about slightly different things here
< sipa> yevaud: you can set the FP rate to whatever you want
< sipa> phantomcircuit: you're talking about walking the chain (once) to build a GCS filter, and then when rescanning match every block's filter against the wallet
< phantomcircuit> i was talking about adding a permanent gcs filter index, in addition to the "basic" one and then using the MatchAny method to do high speed checking
< phantomcircuit> sipa, yes
< sipa> i'm talking about at rescan time building a bloom filter once from the wallet, and then walking the (full) chain and matching all sPKs in it against that filter
< sipa> and then suggesting that maybe that's how the wallet should always do output detection
< phantomcircuit> which i would expect to be faster than building a bloom filter and then walking the chain, since presumably there's going to be many more items in the chain filter than the wallet filter
< phantomcircuit> or maybe i have that wrong and i should benchmark it?
< yevaud> the way the filters are defined, aren't they salted with the block hash?
< sipa> phantomcircuit: matching against a bloom filter is way faster than a gcd filter
< sipa> *gcs
< phantomcircuit> yes the MatchAny function accepts an ElementSet which is just a list of byte vectors basically and i expect would be generally faster than parsing the blocks for scriptpubkeys
< phantomcircuit> sipa, hmm
< sipa> matching a set against a gcs filter is literally decoding all the elements in the gcs filter in order, and seeing if they're in your set
< sipa> actually, this may be faster than a bloom filter - it depends on the relative sizes of the filter and the set
< sipa> a wallet filter/set would have 1000-10000ish elements, think (2000 keys by default, 3 address types for each?)
< luke-jr> wumpus: #18902 is still needed to avoid a regression, so cutting up #18818 didn't actually save any time :/
< gribble> https://github.com/bitcoin/bitcoin/issues/18902 | Bugfix: Only use git for build info if the repository is actually the right one by luke-jr · Pull Request #18902 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/18818 | Fix release tarball generated by gitian by luke-jr · Pull Request #18818 · bitcoin/bitcoin · GitHub
< phantomcircuit> sipa, there's: P2PK, P2PKH, P2WPKH (though i dont see why the wallet even cares about witnesses in this context?), P2SH, P2WSH (same thing), and all of the watch only things
< sipa> phantomcircuit: the wallet supports segwit?
< phantomcircuit> sipa, im looking at the IsMineInner function now and there's distinction between witness and not-witness
< phantomcircuit> im not sure why, i think you did this probably so maybe you can help me understand?
< sipa> what distinction exactly?
< sipa> oh, the requirement that we don't match P2WPKH unless the P2SH-P2WPKH would be acceptable as well?
< phantomcircuit> well the solver returns both TX_SCRIPTHASH or TX_WITNESS_V0_SCRIPTHASH and the handling of them is slightly different
< sipa> the first is P2SH, the second is P2WSH
< sipa> for P2SH-P2WSH it will return P2SH, and the caller will then recurse into it
< bitcoin-git> [bitcoin] jnewbery opened pull request #18962: net processing: Only send a getheaders for one block in an INV (master...2020-05-limit-block-inv) https://github.com/bitcoin/bitcoin/pull/18962
< phantomcircuit> sipa, is TX_WITNESS_V0_SCRIPTHASH valid with a P2SH... or is it not even possible to trigger that with the solver
< phantomcircuit> s/with/within/
< sipa> phantomcircuit: yes, that's P2SH-P2WSH, and it's the default address format for multisig even
< sipa> or was for a long time
< sipa> maybe it's native P2WSH now
< bitcoin-git> [bitcoin] dongcarl opened pull request #18963: [WIP] rebase: Call ProcessNewBlock() asynchronously (master...2020-05-async-pnb) https://github.com/bitcoin/bitcoin/pull/18963
< bitcoin-git> [bitcoin] promag opened pull request #18964: rpc, wallet: Scan mempool after import* (master...2020-05-fix-18954) https://github.com/bitcoin/bitcoin/pull/18964
< promag> fanquake: is it me or you have your boots swapepd (in GH picture)?
< fanquake> promag: there's every chance I've got em on backwards there
< phantomcircuit> sipa, im seeing Lloop1_35 in `perf report` output with a debug build any idea what that is?
< promag> fanquake: :)
< sipa> phantomcircuit: possibly something in the avx2/sse4/shani SHA256 code
< sipa> if so it's just a goto target that the perf measurement tool misinterprets as a dymbol namr
< sipa> *symbol name
< midnight> Seriously, thanks for your graphs sipa. Please never stop making them. Or, alternatively, please give us lots of notice if you decide to shut them down. :)
< sipa> the code is open source
< phantomcircuit> sipa, ooh it's sha256
< phantomcircuit> looks to be a specific loop in the sha256 code that it's decided to annotate like that
< sipa> yeah, the name looked familiar
< phantomcircuit> sipa, right and the performance profile of the debug build is substantially different as to be useless
< jb55> sounds about right
< sipa> phantomcircuit: haha yes
< phantomcircuit> sipa, this seems to be spending most of it's cpu time running sha256
< phantomcircuit> i guess no matter what we want block data for we check the merkle tree?
< midnight> sipa: what's alist.h from?
< sipa> something i wrote in early 2000s i think :p
< midnight> where do I get it. :-)
< sipa> midnight: it boggles my mind that i've had this site on github since 2013, and never committed that file... and kept the site running despite moving it 2 or 3 times since
< sipa> pushed now
< sipa> maybe...