< bitcoin-git> [bitcoin] adamjonas opened pull request #20668: doc: warn that incoming conns are unlikely when not using default ports (master...121520-non-standard-port-doc) https://github.com/bitcoin/bitcoin/pull/20668
< fanquake> wumpus / achow: do you have the links to any of the codesigning/inspection tools you wrote handy
< achow101> fanquake: https://gist.github.com/achow101/fef2415d99965de66ac083b54b83df6e is the tool we ended up with. I'm writing a full codesigning and inspection tool at https://github.com/achow101/signapple
< fanquake> achow: thanks
< sipa> achow101: how is that going?
< fanquake> I see lots of blobs
< achow101> sipa: I've reversed engineered the internal requirements thing. It's a mini scripting language with opcodes and stuff
< achow101> considered writing an AST representation of it but decided that was too much effort
< sipa> sounds like fun & an amazing timesink :)
< achow101> I think I can have it sign in a few days
< fanquake> achow101: is the goal here to replace codesign/codesign_allocate ?
< achow101> it fully verifies, including stuff that we don't currently deal with in our signing, like whatever entitlements are
< achow101> fanquake: yes
< sipa> p
< fanquake> q
< bitcoin-git> [bitcoin] MarcoFalke opened pull request #20669: [0.21] final rc4 backports (0.21...2012-rc4) https://github.com/bitcoin/bitcoin/pull/20669
< wumpus> from what i understood the entitlements are mostly used on iOS where it describes extra privileges of the application
< fanquake> yea we'd only need them on macOS if we wanted to use things like NFC, location services, apple pay etc
< sipa> apple pay integration, hmm!
< jonasschnelli> achow101: nice work!
< wumpus> yea desktop OSes tend to be much more lenient with regard to application permissions, though i was surprised recently that flatpak has a fine-grained system for controlling what an application is allowed to access, even some capability based thing to deny access to the entire home directory but allow access only to files selected by the user, kind of clever
< jonasschnelli> Yeah. I like this feature in modern macOS. Each application has to ask permission to access certain directories of the filesystem. Same for camera access, location, etc..
< jonasschnelli> On the other hand,.. it comes with a lot of control handed over to Apple (unnecessary). Like the whole notarization process.
< wumpus> that's one issue with any security feature in proprietary OSes, they always ends up as a way the vendor can assert control, even if the feature itself was a good idea
< wumpus> they always want to be the one controlling the locks and that makes it more like a jail (as Bunnie would say)
< wumpus> achow101: "It's a mini scripting language with opcodes and stuff" can you compile DOOM for it?
< jonasschnelli> heh
< sipa> yes, but does it run linux?
< wumpus> achow101: but nice work
< wumpus> sipa: finally a way to run linux on a mac !
< sipa> i mean, there is a linux system compiled to JS that runs in the browser right?
< jonasschnelli> OMG
< sipa> wow
< sipa> even works on my phone
< wumpus> jonasschnelli: risc-v is clearly the new webassembly
< jonasschnelli> windows98 with minesweeper: http://copy.sh/v86/?profile=windows98
< sipa> wth
< sipa> if my phone's cpu is fast enough to run a browser, in which an emulated operating system can run, and really all i do with it is using it as a pocket watch with wikipedia and irc functionality, i'm doing something wront
< wumpus> that one is veeery slow here
< wumpus> sipa: i'm sure it drains the battery like crazy, i guess that's the main bottleneck to doing more CPU heavy things on a phone
< wumpus> does anyone maybe have a better idea how we can avoid introducing a circular dependency here between coinstats and validation: https://github.com/bitcoin/bitcoin/pull/19806#pullrequestreview-552964012
< jonasschnelli> I don't see other options than a separate implementation unit
< wumpus> I think it makes logical sense too, snapshot handling is adjacent to but not part of validation
< wumpus> woohoo, appveyor cooked up a new reason to fail "The object or library file 'C:\Qt5.9.8_x64_static_vs2019\lib\Qt5Core.lib' was created by a different version of the compiler than other objects like 'x64\Release\bitcoin-qt\main.obj'; rebuild all objects and libraries with the same"
< wumpus> don't know if it's general or only on the 0.21 branch
< vasild> jonatack: I2P! :)
< vasild> What is the reason we don't gossip host names, e.g. bitcoinnode.foo.org:8333?
< wumpus> we really don't want DNS lookups in the P2P code
< vasild> wumpus: Why? For performance reasons?
< wumpus> also working with dynamically-sized strings in any network protocol has a large potential to introduce bugs
< wumpus> security/privacy reasons mostly, it's harder to do with tor
< vasild> I see
< wumpus> I don't really see much of an advantage in it personally, DNS is useful for humans, but for a addresses sent over a network protocol it seems to me it adds an unnecessary level of indirection
< vasild> yeah, maybe some benefit could be that a hostname could resolve to a few IP addresses or if the address changes over time, then addrman entries need not be updated
< vasild> I asked because in I2P we do need to do a lookup - the .b32.i2p address must be resolved to a "destination" (a few 100s of bytes) and then we connect to that "destination"
< wumpus> i didn't know that when writing the BIP
< wumpus> should we have gossipped destinations directly?
< wumpus> though we can fit 512 bytes i guess the size does make it impractical
< hebasto> wumpus: to fix appveyor on 0.21 the baclport of the recent changes seem required: #20489, #20506
< gribble> https://github.com/bitcoin/bitcoin/issues/20489 | CI msvc: only build vcpkg dependencies for release (not debug) to reduce build times by sipsorcery · Pull Request #20489 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/20506 | ci: AppVeyor fixes for Visual Studio 2019 16.8.1 image by sipsorcery · Pull Request #20506 · bitcoin/bitcoin · GitHub
< hebasto> should I take care of it?
< vasild> wumpus: I think it is fine as it is, gossiping destinations directly seems an overkill to me
< vasild> but it opens the opportunity to allow i2p hostnames, e.g. zzz.i2p in addition to .b32.i2p addresses
< vasild> also, lontivero mentioned in https://github.com/bitcoin/bitcoin/pull/20254#discussion_r543586946 that even .b32.i2p addresses may have different length :-X
< vasild> eventually, maybe we would update bip155 to mandate the i2p addr size is allowed to be either 32 or 35 bytes
< wumpus> fwiw, I would prefer not to broadcast hostnames, keep the gossiping to (low level) network addresses, preferably fixed length
< vasild> wumpus: right
< wumpus> right--the current protocol mandates a specific I2P address scheme, same as we do for TorV1/V2, if there's a different one in the future then we'll need to add a new address type
< vasild> i2p-exotic, length=35
< wumpus> I intentionally didn't leave any extensibility within one address type, to make it straightforward to handle
< bitcoin-git> [bitcoin] MarcoFalke opened pull request #20671: Replace boost::optional with std::optional (master...2012-stdOpt) https://github.com/bitcoin/bitcoin/pull/20671
< hebasto> wumpus: are you still seeing an issue with appveyor on the 0.21 branch?
< jonatack> vasild: hey, sorry, was reviewing. keeping each address types simple sgtm.
< vasild> jonatack: yeah, so on IP2 I am doing final self re-review and test ci runs to ensure it is still green and will open a PR with proper I2P support which will supersede #20254
< gribble> https://github.com/bitcoin/bitcoin/issues/20254 | Add I2P support using statically configured destinations by vasild · Pull Request #20254 · bitcoin/bitcoin · GitHub
< jonatack> vasild: nice, will look out for that
< vasild> I wonder if it makes sense to write a primitive SAM proxy server in python in order to test it in the functional tests... maybe yes
< vasild> so far I used a hackish unit tests that kind of starts the "entire" bitcoind by calling CConnman::Start() and requires a running i2p server - not acceptable
< bitcoin-git> [bitcoin] MarcoFalke closed pull request #16168: DevelopOmniLayer (master...develop) https://github.com/bitcoin/bitcoin/pull/16168
< bitcoin-git> [bitcoin] MarcoFalke pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/dff0f6f753ea...b440c33179e7
< bitcoin-git> bitcoin/master ed73f8c practicalswift: net: Move eviction node selection logic to SelectNodeToEvict(...)
< bitcoin-git> bitcoin/master 685c428 practicalswift: test: Add unit testing of node eviction logic
< bitcoin-git> bitcoin/master fee8823 practicalswift: Assert eviction at >= 29 candidates. Assert non-eviction at <= 20 candidat...
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #20477: net: Add unit testing of node eviction logic (master...unit-test-node-eviction-logic) https://github.com/bitcoin/bitcoin/pull/20477
< bitcoin-git> [bitcoin] jonasschnelli pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/b440c33179e7...6af8a6232b2d
< bitcoin-git> bitcoin/master 267f259 Hennadii Stepanov: depends: Drop workaround for a fixed bug in Qt build system
< bitcoin-git> bitcoin/master 6af8a62 Jonas Schnelli: Merge #20650: depends: Drop workaround for a fixed bug in Qt build system
< bitcoin-git> [bitcoin] jonasschnelli merged pull request #20650: depends: Drop workaround for a fixed bug in Qt build system (master...201214-xcb) https://github.com/bitcoin/bitcoin/pull/20650
< wumpus> hebasto: let's see how #20669 goes
< gribble> https://github.com/bitcoin/bitcoin/issues/20669 | [0.21] final rc4 backports by MarcoFalke · Pull Request #20669 · bitcoin/bitcoin · GitHub
< wumpus> vasild: we don't have testing for socks5 either, but agree it'd make sense
< wumpus> vasild: oh wait we do lol
< vasild> I will at least try to do it and hope it does not end in too many lines of python code
< vasild> test/functional/test_framework/socks5.py class Socks5Server() ...
< vasild> 160 lines
< vasild> btw, I noticed that we do send(sock, buf, size) and expect the return value to be "size" which is not robust
< vasild> also, the socket is non-blocking
< wumpus> true, it can't be generally relied on, e.g. if the system call is interrupted
< vasild> I guess it mostly works because the socks5 proxy is usually on localhost where the connection is "fast and stable" :)
< Jackielove4u> Does anybody here have rc3 running ?
< jonasschnelli> Jackielove4u: yes. Why?
< wumpus> hey, didn't we create an issue for rc test reports this time?
< Jackielove4u> my rc3 node does not accept any incoming connections
< wumpus> looks like we forgot
< fanquake> #20555
< gribble> https://github.com/bitcoin/bitcoin/issues/20555 | v0.21.0 testing · Issue #20555 · bitcoin/bitcoin · GitHub
< Jackielove4u> i am trying to connect my rc3 client to my rc3 node
< wumpus> oh it's just not pinned
< fanquake> Yea I just pinned it
< Jackielove4u> with connect= parameter
< wumpus> thanks :)
< Jackielove4u> the connection gets dropped instantly
< Jackielove4u> can i get a ip from any of you guys
< Jackielove4u> to try to connect to you
< Jackielove4u> i was able to get it fixed though by reverting this commit: https://github.com/bitcoin/bitcoin/pull/20612/commits/9e806887a8f9ef63431b28d7dfd0470aa663dd02
< Jackielove4u> not sure if this is the expected behaviour (rc3 not able to connect to rc3)
< wumpus> strange, that commit is supposed to *fix* disconnections by not sending sendaddrv2 to nodes we're not sure support it
< Jackielove4u> Can anybody replicate? or give me your node ip so i can try to connect to yours
< vasild> Jackielove4u: did you try to connect rc2 with rc3?
< Jackielove4u> no i did not
< Jackielove4u> i can try
< vasild> rc3 <-> {rc3, <=0.20.1} should be fine
< vasild> Jackielove4u: run both nodes with -debug=net and show both logs here (by pastebin)
< MarcoFalke> Indeed rc3 <-> rc3 should be fine
< vasild> this is strange: https://bpa.st/HMPA why disconnect due to ping timeout (20 minutes), 2 minutes after we have received some messages from the peer?
< vasild> 1. should we disconnect in this case, given that the connection is obviously not dead and 2. where is the pong message in this particular case?
< bitcoin-git> [bitcoin] hebasto opened pull request #20673: depends: Use more legible qmake commands in qt package (master...201216-subdir) https://github.com/bitcoin/bitcoin/pull/20673
< jnewbery> vasild: the peer didn't respond to a ping message. We should disconnect. It also looks like it hasn't sent you a verack (and possibly not a version either)
< vasild> jnewbery: I removed the non-relevant messages
< vasild> just left the first one to show when the connection starts and addrv2 to show that something has arrived
< jnewbery> vasild: can you provide full logs? Very difficult to debug issues from partial logs
< jnewbery> but to answer your question, yes we always disconnect a peer that doesn't respond to a ping within 20 minutes. See InactivityCheck() in net.cpp
< MarcoFalke> Something's clearly wrong with the connection if they don't respond with pong
< jnewbery> Jackielove4u: I assume you also dropped the subsequent commit https://github.com/bitcoin/bitcoin/pull/20612/commits/9e806887a8f9ef63431b28d7dfd0470aa663dd02 as well? rc2 and rc3 don't play nice together with that commit, but rc3 <-> rc3 should be fine (and we have big problems if not)
< Jackielove4u> rc3 <-> rc3 without dropping that commit is not connecting for me
< vasild> jnewbery: here is the full log since you asked https://bpa.st/BE5Q, but I did not intend to debug this (yet), just ask if the behavior looks reasonable. You already answered the first question.
< vasild> I dont have the logs of the other side with -debug=net
< aj> jnewbery: re 19858 ; that log snippet is quite trimmed. https://0bin.net/paste/wnUMx2lG#XKm3fF9Wp8dOX03IKYazBTZaCDYN6CegUK-VW5rWjeJ is a more recent example
< jnewbery> argh. why do people mess with logs??
< Jackielove4u> If anybody is willing to share their node ip (that is running rc3), i can try to connect to yours and see if i get a connection
< vasild> jnewbery: in my case I left the few relevant lines to make it easier to do a high-level overview, which would be more tedious with 700+ lines of log
< Jackielove4u> rc3 <-> <=0.20.1 works fine
< vasild> Jackielove4u: did you run with -debug=net?
< Jackielove4u> not yet, my kid is yelling that his nerf gun is stuck and need to fix that first
< vasild> :-D
< Jackielove4u> its lockdown in the netherlands
< Jackielove4u> so i have to deal with kids now also
< jnewbery> vasild: there's something wrong with that peer. It's also not _sending_ pings every two minutes as we'd expect from a Bitcoin Core node
< vasild> hmm, both peers do not send pings every 2 minutes
< jnewbery> I wonder if there's a problem with that connection. The getdata/block round trip time is often over a minute
< vasild> jnewbery: that's over i2p and is for some reason very slow
< vasild> I think I overcooked the settings on one of my i2p routers
< wumpus> is it a very slow CPU? i think it's possible for networking to hang on block validation if it takes very long
< wumpus> oh, i2p
< vasild> cpu should be fine (not very slow)
< Jackielove4u> nerf gun fixed
< bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/6af8a6232b2d...a023094fc417
< bitcoin-git> bitcoin/master 3b064fc Michael Dietz: test: run mempool_expiry.py even with wallet disabled
< bitcoin-git> bitcoin/master a023094 MarcoFalke: Merge #20276: test: run mempool_expiry.py even with wallet disabled
< jnewbery> vasild: is it possible that your router is dropping packets? I don't know anything about how i2p works. Does it have reliable delivery?
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #20276: test: run mempool_expiry.py even with wallet disabled (master...mempool-expiry-miniwallet) https://github.com/bitcoin/bitcoin/pull/20276
< vasild> huh
< jnewbery> aj: ah I see you added fRelay logging. Great. Then the peer is misbehaving by sending those invs. I'd guess badly implemented spy nodes. I've seen issues before where peers mysteriously ignored fRelay.
< vasild> it has "stream" and "datagram" options, let me reread the docs..
< Jackielove4u> upon testing i saw that my bitcoin core rc3 client (run on a desktop) and my schildbach wallet (run on android) does not connect to my rc3 bitcoin core node (run on a server)
< Jackielove4u> bitcoin core rc3 client (run on a desktop) and my schildbach wallet (run on android) does connect to bitcoin core 0.20.x and lower
< aj> jnewbery: yeah; as per the comment i added -- i forgot to redact my ip before pasting that log snippet, then noticed my ip wasn't in the log; so those peers are definitely not pristine bitcoin core, so don't think it's our bug on the sending side either
< aj> Jackielove4u: what's a bitcoin core rc3 client?
< vasild> jnewbery: "Virtual streams are guaranteed to be sent reliably and in order..." (from https://geti2p.net/en/docs/api/samv3)
< Jackielove4u> with client i mean qt
< aj> jnewbery: also, i think that's the first example of that code actually catching a spy-ish node in the wild, yay!
< Jackielove4u> with node i mean bitcoind
< MarcoFalke> Jackielove4u: This sounds like an issue unrelated to rc3
< aj> Jackielove4u: sounds like the client or the node is still at rc2 rather than rc3 to me
< Jackielove4u> both are certainly on rc3
< aj> Jackielove4u: run both with -debug=net and check for a line about sendaddrv2 just prior to the disconnection
< MarcoFalke> Yes, without the debug log we can't debug this any further
< sdaftuar> aj: i've tried to debug this same issue (where a node appears to send us inv's after we start a blocks-only connection) many times over the years, and never been able to replicate using known bitcoin core software on both ends.
< sdaftuar> would be good to write up your analysis on github maybe so that we have notes for the next time we look into something like this
< sdaftuar> (since it's one of those things where we can never quite be sure whether a peer is lying or we have a bug!)
< jnewbery> If you consider BIP37 and optional standard, then it's not really a violation of the protocol to send tx INVs after receiving a version with fRelay = false
< bitcoin-git> [bitcoin] laanwj pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/a023094fc417...2d4eeaf9c588
< bitcoin-git> bitcoin/master c175690 Jarol Rodriguez: doc: Update for FreeBSD 12.2, add GUI Build Instructions
< bitcoin-git> bitcoin/master 2d4eeaf Wladimir J. van der Laan: Merge #20601: doc: Update for FreeBSD 12.2, add GUI Build Instructions
< bitcoin-git> [bitcoin] laanwj merged pull request #20601: doc: Update for FreeBSD 12.2, add GUI Build Instructions (master...master) https://github.com/bitcoin/bitcoin/pull/20601
< jnewbery> *an optional
< aj> jnewbery: i think if we're wanting a no-tx connection (to conserve b/w eg) then disconnecting is still the right thing to do if we receive txs on it?
< jnewbery> aj: right. Perhaps we should also disconnect peers that don't provide fRelay in their version message to make it explicit that support for fRelay is required
< aj> jnewbery: hmm, i guess i'm not logging whether these peers are sending frelay or not
< jnewbery> that'd be interesting. If they sent it but ignored the value that we sent
< bitcoin-git> [bitcoin] MarcoFalke pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/2d4eeaf9c588...9dbcd3710521
< bitcoin-git> bitcoin/master fa8e15f MarcoFalke: test: pep8 wallet_multiwallet.py
< bitcoin-git> bitcoin/master fab48da MarcoFalke: test: Fix intermittent wallet_multiwallet issue with got_loading_error
< aj> jnewbery: that's what i'd guess they're doing. not seeing much advantage in disconnecting 0s-15s earlier though
< bitcoin-git> bitcoin/master 9dbcd37 MarcoFalke: Merge #20569: test: Fix intermittent wallet_multiwallet issue with got_loa...
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #20569: test: Fix intermittent wallet_multiwallet issue with got_loading_error (master...2012-testWalletInt) https://github.com/bitcoin/bitcoin/pull/20569
< jnewbery> or maybe we should deprecate fRelay and always use a feefilter message with MAX_MONEY
< aj> what happened with the p2p v2 messages? did it ever even get a proper bip?
< jonasschnelli> aj: soon
< jonasschnelli> I'm currently working on the new AEAD (with internal rekeying)
< jonasschnelli> Which should end up in the BIP
< jonasschnelli> Discussion is somewhat still ongoing https://gist.github.com/jonasschnelli/c530ea8421b8d0e80c51486325587c52 (but I guess its ready to work into the BIP)
< jnewbery> aj: what percentage of additional block-relay-only peers are you seeing this quick disconnect from? Does this limit the effectiveness of the additional block-relay-only peer mechanism?
< * jonasschnelli> fears scope creep in v2
< aj> jnewbery: handwavy guess, maybe 10%?
< bitcoin-git> [bitcoin] MarcoFalke opened pull request #20674: fuzz: Call SendMessages after ProcessMessage to increase coverage (master...2012-fuzzSendMessages) https://github.com/bitcoin/bitcoin/pull/20674
< aj> jnewbery: i changed the timer to 20s (which doesn't really work because of the 30s min lifetime), which is the only reason i'm seeing lots
< sdaftuar> jnewbery: i think depending on your addrman, you might see a similar percentage of connections never get instantiated (because you have stale addresses that you're trying to connect to)
< sdaftuar> jnewbery: but i think that is fine; if we eventually are able to find honest peers with putting low load on the network, that's much better than where we were before
< sdaftuar> s/instantiated/connected/
< aj> jnewbery: ~12.5% -- 366 "inv sent in violation of protocol" disconnections versus 2559 "disconnecting extra block-relay-only peer" messages. there's also a bunch of "i tried making a b-r-o extra conn, but they basically immediately disconnected me examples, which i haven't looked into at all
< jnewbery> sdaftuar: does that depend on whether we're making progress through all the addresses in our addrman? If we're eclipsed then maybe our peers will be able to churn our new tables and we'll only ever connect to bad peers
< Jackielove4u> jnewbery: you were right
< Jackielove4u> rc2 and rc3 dont work nicely along
< Jackielove4u> it seems my node was indeed still on rc2
< jnewbery> Jackielove4u: phew. That's good news :)
< Jackielove4u> sorry for wasting your times
< sdaftuar> jnewbery: we pull from both the new table and tried table in making these connections
< sdaftuar> if someone takes over the whole addrman, it's game over of course
< sdaftuar> i guess if somehow our only honest addrman addresses were in the same netgroups as our existing outbound connections, we would never finnd them
< sdaftuar> but that seems like a pretty small effect given that we only have 10 peers
< aj> Jackielove4u: that's okay, the "i told you so" seratonin release makes it all worthwhile ;)
< jnewbery> I guess a more general point is that we select with replacement, so if there are only a few honest peers in our addrman, then it'll take a long time to pick them
< jnewbery> the new table is 4x the capacity of the tried table. I'd expect in reality the number of addresses are much more skewed towards the new table, but that's just a guess
< sdaftuar> jnewbery: i think we choose from new vs tried with 50% probability, don't we?
< jnewbery> oh you're right!
< bitcoin-git> [bitcoin] laanwj pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/9dbcd3710521...3f205808a5f2
< bitcoin-git> bitcoin/master cd03513 Wladimir J. van der Laan: init: Signal-safe instant shutdown
< bitcoin-git> bitcoin/master 3f20580 Wladimir J. van der Laan: Merge #20605: init: Signal-safe instant shutdown
< bitcoin-git> [bitcoin] laanwj merged pull request #20605: init: Signal-safe instant shutdown (master...2020_12_instant_shutdown) https://github.com/bitcoin/bitcoin/pull/20605
< jnewbery> sdaftuar: do you have a sense of what proportion of connection attempts fail after being selected in ThreadOpenConnections() ?
< sdaftuar> maybe half-ish
< sdaftuar> i just did a quick check to see how many successful block-relay only connections i've had, and it's about half what you'd expect if every connection succeeded and we connect to one every 5 minutes
< sdaftuar> but i'm sure it depends a lot on your addrman. the node i just checked is on both tor and ipv4, i'm sure there are quirks with how tor addresses are relayed, etc
< aj> jnewbery: could add logging for IsFeelerConn() -> fDisconect=true and cmpare that to how often we see "Making feeler connection to" log entries
< jnewbery> aj: good idea
< jnewbery> I wonder if we should do the IsDiscouraged() check earlier. If you end up with some discouraged addresses in your addrman, I guess they'll just stay there and you'll sometimes select them but never be able to connect
< bitcoin-git> [bitcoin] MarcoFalke pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/3f205808a5f2...69f1ee1922e3
< bitcoin-git> bitcoin/master 6d3af3a Ivan Metlushko: wallettool: pass in DatabaseOptions into MakeWallet
< bitcoin-git> bitcoin/master 345e88e Ivan Metlushko: wallettool: add param to create descriptors wallet
< bitcoin-git> bitcoin/master 173cc9b Ivan Metlushko: test: walettool create descriptors
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #20365: wallettool: add parameter to create descriptors wallet (master...wallet_tool_create_descriptors) https://github.com/bitcoin/bitcoin/pull/20365
< bitcoin-git> [bitcoin] laanwj pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/69f1ee1922e3...ad3d4b3929ab
< bitcoin-git> bitcoin/master 83f8821 Pieter Wuille: refactor: add IsAddrCompatible() to CNode
< bitcoin-git> bitcoin/master 37fe80e Pieter Wuille: Only consider addrv2 peers for relay of non-addrv1 addresses
< bitcoin-git> bitcoin/master ad3d4b3 Wladimir J. van der Laan: Merge #20661: Only select from addrv2-capable peers for torv3 address rela...
< bitcoin-git> [bitcoin] laanwj merged pull request #20661: Only select from addrv2-capable peers for torv3 address relay (master...202012_torv2_relay_only_addrv2) https://github.com/bitcoin/bitcoin/pull/20661
< MarcoFalke> Backport of this ^ conflicts with 7fabe0f359ae16ed36ce4ca2c33631d038c21448
< MarcoFalke> Should 7fabe0f359ae16ed36ce4ca2c33631d038c21448 also be backported?
< MarcoFalke> #19763
< gribble> https://github.com/bitcoin/bitcoin/issues/19763 | net: dont try to relay to the address originator by vasild · Pull Request #19763 · bitcoin/bitcoin · GitHub
< vasild> it would be easy to resolve, in case it is decided not to backport 19763
< sdaftuar> MarcoFalke: hm I'm not sure it's worth backporting #19763 myself. if we are concerned about addr propagation though, i would imagine that (after #20661) #20561 would be a bigger impact to get torv3 addresses to propagate quicker
< gribble> https://github.com/bitcoin/bitcoin/issues/19763 | net: dont try to relay to the address originator by vasild · Pull Request #19763 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/20661 | Only select from addrv2-capable peers for torv3 address relay by sipa · Pull Request #20661 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/20561 | p2p: periodically clear m_addr_known by sdaftuar · Pull Request #20561 · bitcoin/bitcoin · GitHub
< sdaftuar> as without 20561, self-announcements go out far less frequently
< sdaftuar> curious what others think though
< sipa> agree
< MarcoFalke> I don't feel strongly what to backport and what not, I guess one question is how much are v3 addresses considered to be supported in 0.21.0
< MarcoFalke> Maybe it is best to leave it for 0.21.1
< sdaftuar> i didn't quite understand how all this works -- is it the case that tor users who upgrade to 0.21 will suddenly switch to having a v3 address?
< sdaftuar> (sorry if this badly misunderstands how bitcoind and tor work together)
< sipa> sdaftuar: if they use tor autoconfig yes
< sdaftuar> thanks -- that lines up with my own experience, i didn't know if i had done something unusual
< luke-jr> only v3?
< luke-jr> or also?
< sdaftuar> luke-jr: yes, as far as i can tell
< luke-jr> not sure that's a good idea
< sipa> only v3
< luke-jr> well, I guess they can still connect to v2 peers?
< sipa> yes
< sipa> as long as tor v2 isn"t disabled network-wide
< sipa> in 10 months or so
< luke-jr> wen bitcoin-dnsseed support v3? :P
< sipa> luke-jr: makes little sense, as they can't be exposed through dns
< luke-jr> sipa: can be exposed via web :x
< luke-jr> pretty sure they can be on DNS too tho
< luke-jr> just needs to use a different record type
< sipa> luke-jr: then we need actual DNS client functionality, and can't just use OS resolvers
< luke-jr> Tor won't do it for us?
< sipa> tor has no DNS resolving functionality as far as i know
< bitcoin-git> [bitcoin] laanwj pushed 5 commits to master: https://github.com/bitcoin/bitcoin/compare/ad3d4b3929ab...5b6f970e3fda
< bitcoin-git> bitcoin/master a07910a Antoine Riard: test: Makes wtxidrelay support a generic P2PInterface option
< bitcoin-git> bitcoin/master 06efb31 Antoine Riard: Add functional test test_txid_inv_delay
< bitcoin-git> bitcoin/master d3b5eac Antoine Riard: Add mutation for functional test test_preferred_inv
< bitcoin-git> [bitcoin] laanwj merged pull request #20171: Add functional test test_txid_inv_delay (master...2020-10-txid-delay-test) https://github.com/bitcoin/bitcoin/pull/20171
< sipa> there is a "connect me to this NAME" function that we use
< sipa> which does the resolving at an exit node and forwards the connection there
< sipa> but we never see the IP it resolved to
< wumpus> tor has DNS lookup support, however it only looks up one address, it cannot be used to get multiple results nor special records
< wumpus> (it's a socks5 extension which we don't use)
< wumpus> so indeed, DNS seeds are useless for it
< sipa> we could have seed nodes that are available for getaddr oneshot connections, exposed over hidden services
< wumpus> yes
< sipa> that may be a useful thing to add to my seeder
< sipa> PRs welcome :)
< sdaftuar> sipa: hm if we're relying on that we should revisit some of the issues we discussed in the last few months about how addrfetch connections work
< sdaftuar> actually, i think it's only an issue if the addrfetch peer you connect to is running bitcoin core, so maybe not an issue if your seeder is custom sofwtare with different behavior
< sipa> sdaftuar: when you run with onlynet=tor all DNS seed requests are converted into addrfetch connections to the DNS seeds
< sipa> which means it's doing an extra indirection
< sipa> this works fine, but it means seed requests do go outside of the tor network
< sdaftuar> when you say "we could have seed nodes that are available for getaddr one shot connections" - does that mean that we have hardcoded addresses available for that, or are you saying that there's a dns resolution that happens so that we would connect to a random bitcoin core node for the addrfetch behavior?
< sipa> my proposal is adding hardcoded addrfetchable hidden service addresses, which act as a seeder (and only support addrfetch), instead of as a full node
< sdaftuar> ah ok, that makes sense
< sdaftuar> i assume your seeder doesn't (eg) announce it's own address to the node that connects to it before responding to a getaddr request
< sdaftuar> as i think that behavior in bitcoin core causes problems with the addrfetch logic
< sipa> you can't connect to my seeder; its only publicly reachable interface is DNS
< sdaftuar> oh so "PRs welcome" meant we need to add quite a bit of functionality? :)
< sipa> yes :D
< sdaftuar> haha ok
< sipa> it does have support for bitcoin P2P connections though (as it uses those for crawling)
< sipa> but it doesn't listen
< sdaftuar> pretty sure if i tried to add listening support I'd end up letting your seeder get dosed in some way or other
< sipa> yeah... maybe it's better to do it as a patch to bitcoin core itself
< wumpus> sipa: i guess the first thing would be to make the crawler understand addrv2 and crawl torv3's, so that we have them in the list for hardcoded seeds
< sdaftuar> that seems straightforward. we basically just need a tool to manage the addrman a little more directly?
< sipa> yeah
< sipa> sdaftuar: my seeder does more than addrman, it keeps more statistics about connections and uptime
< sipa> but apart from that it's pretty similar
< sdaftuar> i was imagining that your seeder would run along side it, rather than be integrated into it?
< sdaftuar> but i don't know how feasible or practical that is
< Kiminuo> I have, out of frustration, changed CI tests from Ubuntu 18.04 to Ubuntu 20.04 and also bumped Clang (https://github.com/bitcoin/bitcoin/pull/19245/commits/cf948ffac6fc950066cd8975485a37a90d15ae07) but CI did not notice the change (https://github.com/bitcoin/bitcoin/runs/1562869764). Does CI require some sort of cache invalidation to do the build I want?
< Kiminuo> The idea being that I'm struggling with Ubuntu 18.04 & std::filesystem so when I upgrade to 20.04 for my https://github.com/bitcoin/bitcoin/pull/19245 I can at least move it an inch forward to have green tests.
< Kiminuo> And then I can attempt to lower dependencies back (clang, Ubuntu).
< Kiminuo> I have tested locally using https://github.com/bitcoin/bitcoin/tree/master/ci#configurations before pushing that commit and it was green for Win64 test.
< sipa> Kiminuo: i'm very confused by the travis output
< sipa> where are all the travis jobs?
< sipa> (unrelated to your PR)
< sipa> i just don't know where to look anymore
< Kiminuo> moved to cirrus?
< sipa> oh?
< sipa> oh it seems there are more cirrus jobs now
< Kiminuo> https://github.com/bitcoin/bitcoin/runs/1562869764 I can see tests on the left, can you?
< sipa> maybe there is just one travis job left, and that's why it isn't showing a list?
< Kiminuo> https://github.com/bitcoin/bitcoin/pull/20658 i think this PR moves completely testing from Travis to Cirrus CI
< Kiminuo> but that's just my impression, I just check bitcoin core PRs but not in-depth
< sipa> seems i completely missed that :)
< wumpus> yes, travis is being phased out
< wumpus> they are stopping the free tier
< sipa> i saw they're moving to travis-ci.com, but travis-ci.com does have a free option it seems?
< sipa> anyway, no objecting to moving
< wumpus> i thought more than just the name changed but i don't know the exact details
< bitcoin-git> [bitcoin] laanwj pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/5b6f970e3fda...427f8c2cff01
< bitcoin-git> bitcoin/master 8989577 Dmitry Petukhov: Fix length of R check in test/key_tests.cpp:key_signature_tests
< bitcoin-git> bitcoin/master 427f8c2 Wladimir J. van der Laan: Merge #20248: test: fix length of R check in key_signature_tests
< bitcoin-git> [bitcoin] laanwj merged pull request #20248: test: fix length of R check in key_signature_tests (master...fix-test-key-signature-low-r) https://github.com/bitcoin/bitcoin/pull/20248
< sipa> Kiminuo: the failing build you link so is a mingw build; it's not compiled with gcc or clang
< luke-jr> eh?
< luke-jr> mingw = gcc
< sipa> at least not the usual gcc or clang packages; it's based on the gcc codebase of course
< Kiminuo> sipa, but how come that it builds for me locally with: FILE_ENV="./ci/test/00_setup_env_win64.sh" ./ci/test_run_all.sh ... meaning, in my Docker it seems to work but not on Cirrus CI
< sipa> perhaps you have a more recent mingw-w64?
< Kiminuo> but that's not possible, right? Docker is environment agnostic
< sipa> ok, then i have no clue, sorry :)
< sipa> i don't know anything about docker
< Kiminuo> well, CI tests run in Docker
< Kiminuo> so I'm very confused by this
< sipa> ping MarcoFalke?
< sipa> http://mingw-w64.org/doku.php/download makes me believe the latest mingw-w64 release is based on GCC 8.3.0
< sipa> but it's also possible i'm misinterpreting that, and mingw is really just some patchset/tooling that can be built with any underlying GCC version
< Kiminuo> yeah
< bitcoin-git> [bitcoin] vova557 opened pull request #20675: Vova557 (master...patch-11) https://github.com/bitcoin/bitcoin/pull/20675
< Kiminuo> sipa, I will need to find somebody who would like to finish that PR 19245. C++ building, linking and Bitcoin build system is not something I have adequate knowledge of. It takes me ages to move it an inch forward. I kind of did what I could. Can this be possibly discussed at a dev meeting?
< sipa> Kiminuo: sure; if you just want to leave up for grabs, no discussion necessary... but mentioning it in the meeting may help moving it forward
< Kiminuo> Another approach would be some "mentored approach". That is probably less ideal for me because I can help with programming side but not build side. And this PR needs the later.
< Kiminuo> right
< bitcoin-git> [bitcoin] sdaftuar opened pull request #20676: Replace m_tx_relay/nullptr checks in net_processing.cpp (master...2020-12-refactor-m_tx_relay) https://github.com/bitcoin/bitcoin/pull/20676
< jnewbery> sdaftuar: > we basically need a tool to manage the addrman a little more directly
< jnewbery> What are the requirements? We can add addresses to addrman using addpeeraddress. Shouldn't be too difficult to enhance that a bit more
< sdaftuar> jnewbery: i dunno, i might have misunderstood what sipa had in mind!
< sdaftuar> but the ability to add and remove addresses from addrman was what i was imagining was needed
< sipa> well, one option is to just integrate the dns seeder (excluding the dns serving feature and/or crawling, but at least the database part) into bitcoin core, and
< sipa> but that's probably not the right way
< sipa> another is to have a way of specifying on the command line "serve getaddr requests from data provided by $SCRIPT" or so, which would then be able to query a crawler
< sipa> and in either case, adding support for addrv2 to the crawler is probably a first step
< sdaftuar> i assume this is the code: https://github.com/sipa/bitcoin-seeder ?
< sipa> yes
< bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/427f8c2cff01...ae9ee5bdb18c
< bitcoin-git> bitcoin/master ea36a45 John Newbery: [net] Make p2p recv buffer timeout 20 minutes for all peers
< bitcoin-git> bitcoin/master ae9ee5b MarcoFalke: Merge #20651: net: Make p2p recv buffer timeout 20 minutes for all peers
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #20651: net: Make p2p recv buffer timeout 20 minutes for all peers (master...2020-12-recv-buffer-timeout) https://github.com/bitcoin/bitcoin/pull/20651
< bitcoin-git> [bitcoin] MarcoFalke closed pull request #20675: Vova557 (master...patch-11) https://github.com/bitcoin/bitcoin/pull/20675
< Kiminuo> @MarcoFalke, Hi, could you mark this https://github.com/bitcoin/bitcoin/pull/19245 PR with "up for grabs", "looking for help" or something in similar vein?
< Kiminuo> I have just modified the PR description to add as much as I know about the issue.
< bitcoin-git> [bitcoin] laanwj pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/ae9ee5bdb18c...4acbcfa97d20
< bitcoin-git> bitcoin/master 22437fc Carl Dong: build: Run libdmg-hfsplus's DMG tool in make deploy
< bitcoin-git> bitcoin/master 0df9819 fanquake: build: Replace genisoimage with xorriso
< bitcoin-git> bitcoin/master 7587d11 fanquake: build: remove cdrkit package from depends
< bitcoin-git> [bitcoin] laanwj merged pull request #20470: build: Replace genisoimage with xorriso (master...2020-11-xorriso) https://github.com/bitcoin/bitcoin/pull/20470
< jeremyrubin> Has anyone dealt with mysterious undeclared identifier for template functions defined within a .cpp unit? E.g., I have template<T> foo.... defined but for some reason function bar trying to call foo says no such identifier?
< jeremyrubin> I'm able to compile the same code on one machine and not the other
< jeremyrubin> maybe a build cache bug... let me distclean
< andytoshi> is this in uint256.cpp? this has templates in a .cpp file and then "explicit instantiations" below that you need to add if you add any functions
< andytoshi> or more generally, if you have templates which live outside a header
< jeremyrubin> hmm yeah it was actually in script/interpreter with init.cpp
< jeremyrubin> maybe the clang on this VM can't instantiate a local method if it isn't explicit
< jeremyrubin> but I feel like something that is local only and not visible from header shouldn't need explicit inst?
< sipa> jeremyrubin: what's the actual code?
< sipa> i would be very surprised if any modern compiler has a bug w.r.t. this
< sipa> it's possible it's something that depends on what C++ version you're using, and some compilers permit stuff from future versions
< jeremyrubin> let me wait on a distclean rebuild... maybe some issue with that
< jeremyrubin> this VPS is slow so will take a few
< bitcoin-git> [bitcoin] sdaftuar opened pull request #20677: doc: Remove shouty enums in net_processing comments (master...2020-12-remove-shouty-enums) https://github.com/bitcoin/bitcoin/pull/20677
< bitcoin-git> [bitcoin] laanwj pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/4acbcfa97d20...1811e488d53b
< bitcoin-git> bitcoin/master 281cf99 practicalswift: Do not run functions with necessary side-effects in assert()
< bitcoin-git> bitcoin/master 5021810 practicalswift: Make CanFlushToDisk a const member function
< bitcoin-git> bitcoin/master 1811e48 Wladimir J. van der Laan: Merge #20575: Do not run functions with necessary side-effects in assert()...
< bitcoin-git> [bitcoin] laanwj merged pull request #20575: Do not run functions with necessary side-effects in assert() (master...side-effect-free-assert) https://github.com/bitcoin/bitcoin/pull/20575