< bitcoin-git>
bitcoin/master 2f03255 Amiti Uttarwar: [test] test that an invalid GETDATA doesn't prevent processing of future m...
< bitcoin-git>
[bitcoin] fanquake merged pull request #18808: [net processing] Drop unknown types in getdata (master...2020-04-getdata) https://github.com/bitcoin/bitcoin/pull/18808
< bitcoin-git>
[bitcoin] jnewbery closed pull request #18955: net processing: requeue transaction GETDATA requests more frequently (master...2020-05-requeue-txrequest) https://github.com/bitcoin/bitcoin/pull/18955
< bitcoin-git>
[bitcoin] andrewtoth reopened pull request #18941: validation: Persist coins cache to disk and load on startup (master...persist-coinscache) https://github.com/bitcoin/bitcoin/pull/18941
< luke-jr>
would be nice if Travis sent an email to the PR creator when it failed :x
< bitcoin-git>
[bitcoin] practicalswift closed pull request #18450: util: Use locale independent ToString(…) instead of locale dependent strprintf(…) for low-level string formatting (master...locale-independence) https://github.com/bitcoin/bitcoin/pull/18450
< bitcoin-git>
[bitcoin] practicalswift closed pull request #18539: Avoid using locale-dependent boost trim functions in RPCAuthorized(…) and bitcoin-tx (master...avoid-locale-dependent-trim) https://github.com/bitcoin/bitcoin/pull/18539
< bitcoin-git>
[bitcoin] fanquake opened pull request #18956: build: enforce minumim required Windows version (7) (master...window_enforce_runtime_version) https://github.com/bitcoin/bitcoin/pull/18956
< bitcoin-git>
[bitcoin] meeDamian opened pull request #18957: Add a link from ZMQ doc to ZMQ example in contrib/ (master...zmq-doc-link) https://github.com/bitcoin/bitcoin/pull/18957
< bitcoin-git>
bitcoin/master 89fea68 fanquake: build: don't pass -w when building for Windows
< bitcoin-git>
bitcoin/master 49d237c fanquake: Merge #18928: build: don't pass -w when building for Windows
< bitcoin-git>
[bitcoin] fanquake merged pull request #18928: build: don't pass -w when building for Windows (master...dont_try_suppress_all_warnings_win) https://github.com/bitcoin/bitcoin/pull/18928
< bitcoin-git>
bitcoin/master d97fac4 Damian Mee: Add a link from ZMQ doc to ZMQ example in contrib/
< bitcoin-git>
bitcoin/master ad1b7b1 fanquake: Merge #18957: Add a link from ZMQ doc to ZMQ example in contrib/
< bitcoin-git>
[bitcoin] fanquake merged pull request #18957: Add a link from ZMQ doc to ZMQ example in contrib/ (master...zmq-doc-link) https://github.com/bitcoin/bitcoin/pull/18957
< jonatack>
gleb: interesting about bitpeers. seems there is the original one in go and a rewrite in python. which one did you try?
< bitcoin-git>
[bitcoin] dongcarl opened pull request #18958: guix: Make V=1 more powerful for debugging (master...2020-05-guix-improvements) https://github.com/bitcoin/bitcoin/pull/18958
< bitcoin-git>
[bitcoin] laanwj merged pull request #18945: build: Ensure source tarball has leading directory name (0.20) (0.20...2020_11_0.20_tarball_prefix) https://github.com/bitcoin/bitcoin/pull/18945
< jb55>
I was thinking about hacking BroadcastTransaction to only send to tor peers (when configured) as an alternative to the less granular onlynet=onion. surprised this isn't implemented? looks easy?
< luke-jr>
sounds fragile
< jb55>
reasoning: I want low block receive/relay latency but still want transaction broadcast privacy
< jb55>
luke-jr: fragile in the sense of broadcast failing if there are no available tor peers or something?
< gleb>
jb55: I think it's a good idea if implemented well, but the details need discussing. For example, if you're one of the very few nodes using that feature, a handful of peers can cluster your transactions together.
< luke-jr>
jb55: in the sense that a non-tor peer might find a way to make you leak the info
< gleb>
a handful of tor sybil peers :)
< luke-jr>
even non-tor peers
< luke-jr>
if everyone relays that tx except your IPv4 address…
< jb55>
yeah so it would need to still relay it eventually, I was just thinking of hiding the initial source broadcast which might be easier to trace?
< gleb>
You probably want a new circuit for every relay then? Depends on how widely this is used, tor network may be unhappy :p
< jb55>
well I'm just thinking about tackling the easiest things first: hide initial broadcast but relay as usual like it some other person's tx. tor sybil attacks seems like a harder problem?
< gleb>
jb55: tor sybil is not something abstract, they are probably already in place. And i'm saying this issue might make transactions *less private* after your idea. IP will be probably hidden (if done well), but then other issues arise. It's just not that trivial in my mind, so we need to be careful. I would ask other opinions before implementing,
< gleb>
maybe a mailing list post?
< gleb>
Like, would you relay to all your tor peers? Maybe relaying to just one works better in this case, since Tor relay part is already less "reliable"
< gleb>
Probably you even want to relay to a *random* tor peer in the net?
< sipa>
ideally you open a new tor connection, relay the tx, and disconnect
< sipa>
any other communication you do with them lets them correlate the tx with your other traffic
< jb55>
let's say I send a tx to a tor peer, they would have some custom code that correlates encrypted bitcoin-p2p traffic with transactions coming out of exit nodes somehow?
< jb55>
trying to understand how likely this is...
< jb55>
I guess if I connected to a tor peer that controlled exit nodes as well, and could correlate that way?
< nothingmuch>
FWIW JoinMarket does has opt in tor broadcasting which connects to new random node: https://github.com/JoinMarket-Org/joinmarket/pull/662 and Wasabi picks node that is already connected, transmits to it and then disconnects from it:
< bitcoin-git>
[bitcoin] tarboss opened pull request #18961: remove assert in walletcontroller & run setparent in gui-qt main thread (master...master) https://github.com/bitcoin/bitcoin/pull/18961
< phantomcircuit>
sipa, i'm thinking about adding a filter index for script pushes to support fast wallet rescans for legacy wallets, any thoughts on that
< phantomcircuit>
aj, ^ same question
< sipa>
phantomcircuit: unless you really need support for raw multisig (which i think the wallet doesn't even support anymore), you could use the bip158 filter
< sipa>
and in descriptor wallets it should be even easier (as for those, the set of IsMine outputs is exactly equal to an explicitly constructed set)
< phantomcircuit>
sipa, for non-legacy wallets using the existing bip158 "basic" filter (which is the entire script) is definitely preferable
< sipa>
for legacy wallets too
< sipa>
you can enumerate all watched addresses for those, actually
< sipa>
it's just trickier
< sipa>
since 0.17 bare multisig is not automatically watched anymore; you need to explicitly import it
< phantomcircuit>
sipa, for older wallets my issue with constructing the scriptPubKey for each key is two fold, one i cannot at all support raw multisig, which possibly someone actually used, and two that will significantly effect the false positive rate of the filter
< sipa>
you can also go the other way... build a separate ephemeral bloom filter for just the watched scriptPubKeys, and then iterate through blocks matching every sPK in it agains the filter
< sipa>
that doesn't have the same I/O savings as using an explicitly constructed filter, but it's certainly logistically a lot easier
< sipa>
and probably a lot faster than the current rescanning already
< sipa>
the bare multisig problem doesn't exist anymore since 0.17
< phantomcircuit>
sipa, why was support for raw multisig changed? significant performance issue?
< sipa>
in fact, quite possibly that's how the wallet should work always
< phantomcircuit>
actually found the #13002 pr
< sipa>
just maintain a bloom filter of its watched sPKs, and match things against it
< phantomcircuit>
sipa, my problem with that for legacy wallets is that we don't really have a way to list all of the sPKs for the wallet except to walk the private keys and calculate every sPK we have ever supported generating addresses for
< phantomcircuit>
or am i getting that really wrong
< sipa>
yes
< sipa>
walk all the private keys, public keys, scripts, watched things, ...
< phantomcircuit>
which will end up being a list that's substantially larger than you'd want
< sipa>
how so?
< sipa>
it is exactly what you want
< sipa>
if you do anything else, you risk missing things
< phantomcircuit>
possibly i add a new GCS filter index and se it in the wallet only when it's available?
< phantomcircuit>
im not sure the performance of walking the chain to build an in memory bloomfilter would be great either honestly
< sipa>
not the chain; the wallet
< sipa>
ah, for a GCS filter, yes
< phantomcircuit>
sipa, oh so build a filter of the wallet and then walk the chain checking against the filter
< yevaud>
sipa: unfortunately walking all the permutations, while required for a legacy wallet, drives up the false positive rate substantially.
< phantomcircuit>
i think we're talking about slightly different things here
< sipa>
yevaud: you can set the FP rate to whatever you want
< sipa>
phantomcircuit: you're talking about walking the chain (once) to build a GCS filter, and then when rescanning match every block's filter against the wallet
< phantomcircuit>
i was talking about adding a permanent gcs filter index, in addition to the "basic" one and then using the MatchAny method to do high speed checking
< phantomcircuit>
sipa, yes
< sipa>
i'm talking about at rescan time building a bloom filter once from the wallet, and then walking the (full) chain and matching all sPKs in it against that filter
< sipa>
and then suggesting that maybe that's how the wallet should always do output detection
< phantomcircuit>
which i would expect to be faster than building a bloom filter and then walking the chain, since presumably there's going to be many more items in the chain filter than the wallet filter
< phantomcircuit>
or maybe i have that wrong and i should benchmark it?
< yevaud>
the way the filters are defined, aren't they salted with the block hash?
< sipa>
phantomcircuit: matching against a bloom filter is way faster than a gcd filter
< sipa>
*gcs
< phantomcircuit>
yes the MatchAny function accepts an ElementSet which is just a list of byte vectors basically and i expect would be generally faster than parsing the blocks for scriptpubkeys
< phantomcircuit>
sipa, hmm
< sipa>
matching a set against a gcs filter is literally decoding all the elements in the gcs filter in order, and seeing if they're in your set
< sipa>
actually, this may be faster than a bloom filter - it depends on the relative sizes of the filter and the set
< sipa>
a wallet filter/set would have 1000-10000ish elements, think (2000 keys by default, 3 address types for each?)
< luke-jr>
wumpus: #18902 is still needed to avoid a regression, so cutting up #18818 didn't actually save any time :/
< gribble>
https://github.com/bitcoin/bitcoin/issues/18902 | Bugfix: Only use git for build info if the repository is actually the right one by luke-jr · Pull Request #18902 · bitcoin/bitcoin · GitHub
< phantomcircuit>
sipa, there's: P2PK, P2PKH, P2WPKH (though i dont see why the wallet even cares about witnesses in this context?), P2SH, P2WSH (same thing), and all of the watch only things
< sipa>
phantomcircuit: the wallet supports segwit?
< phantomcircuit>
sipa, im looking at the IsMineInner function now and there's distinction between witness and not-witness
< phantomcircuit>
im not sure why, i think you did this probably so maybe you can help me understand?
< sipa>
what distinction exactly?
< sipa>
oh, the requirement that we don't match P2WPKH unless the P2SH-P2WPKH would be acceptable as well?
< phantomcircuit>
well the solver returns both TX_SCRIPTHASH or TX_WITNESS_V0_SCRIPTHASH and the handling of them is slightly different
< sipa>
the first is P2SH, the second is P2WSH
< sipa>
for P2SH-P2WSH it will return P2SH, and the caller will then recurse into it
< bitcoin-git>
[bitcoin] jnewbery opened pull request #18962: net processing: Only send a getheaders for one block in an INV (master...2020-05-limit-block-inv) https://github.com/bitcoin/bitcoin/pull/18962
< phantomcircuit>
sipa, is TX_WITNESS_V0_SCRIPTHASH valid with a P2SH... or is it not even possible to trigger that with the solver
< phantomcircuit>
s/with/within/
< sipa>
phantomcircuit: yes, that's P2SH-P2WSH, and it's the default address format for multisig even
< promag>
fanquake: is it me or you have your boots swapepd (in GH picture)?
< fanquake>
promag: there's every chance I've got em on backwards there
< phantomcircuit>
sipa, im seeing Lloop1_35 in `perf report` output with a debug build any idea what that is?
< promag>
fanquake: :)
< sipa>
phantomcircuit: possibly something in the avx2/sse4/shani SHA256 code
< sipa>
if so it's just a goto target that the perf measurement tool misinterprets as a dymbol namr
< sipa>
*symbol name
< midnight>
Seriously, thanks for your graphs sipa. Please never stop making them. Or, alternatively, please give us lots of notice if you decide to shut them down. :)
< sipa>
the code is open source
< phantomcircuit>
sipa, ooh it's sha256
< phantomcircuit>
looks to be a specific loop in the sha256 code that it's decided to annotate like that
< sipa>
yeah, the name looked familiar
< phantomcircuit>
sipa, right and the performance profile of the debug build is substantially different as to be useless
< jb55>
sounds about right
< sipa>
phantomcircuit: haha yes
< phantomcircuit>
sipa, this seems to be spending most of it's cpu time running sha256
< phantomcircuit>
i guess no matter what we want block data for we check the merkle tree?
< midnight>
sipa: what's alist.h from?
< sipa>
something i wrote in early 2000s i think :p
< midnight>
where do I get it. :-)
< sipa>
midnight: it boggles my mind that i've had this site on github since 2013, and never committed that file... and kept the site running despite moving it 2 or 3 times since