< bitcoin-git> [bitcoin] fanquake pushed 5 commits to master: https://github.com/bitcoin/bitcoin/compare/ca1886056325...8387f832d693
< bitcoin-git> bitcoin/master daf5553 Suhas Daftuar: Avoid calling CAddrMan::Connected() on block-relay-only peer addresses
< bitcoin-git> bitcoin/master 4fe338a Suhas Daftuar: Call CAddrMan::Good() on block-relay-only peer addresses
< bitcoin-git> bitcoin/master e8b215a Suhas Daftuar: Refactor test for existing peer connection into own function
< bitcoin-git> [bitcoin] fanquake merged pull request #20187: Addrman: test-before-evict bugfix and improvements for block-relay-only peers (master...2020-10-addrman-block-relay) https://github.com/bitcoin/bitcoin/pull/20187
< bosch-0> Yo
< bosch-0> Want to have a read of my project before I submit it? https://github.com/Bosch-0/Project-Muggle/blob/master/README.md
< bosch-0> If ya not too busy
< S3RK> question about fuzzing. we use regtest chain params, but in corups for src/test/fuzz/descriptor_parse.cpp I see xprv keys. As a result fuzzing don't cover code paths with valid keys for bip32 decriptors. I tried to add initial seeds with tprv, but fuzzer doesn't detect any new edges covered. What do I miss?
< bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/8387f832d693...5174b534da57
< bitcoin-git> bitcoin/master c2cf8a1 practicalswift: fuzz: Check for addrv1 compatibility before using addrv1 serializer on CSe...
< bitcoin-git> bitcoin/master 5174b53 MarcoFalke: Merge #20289: fuzz: Check for addrv1 compatibility before using addrv1 ser...
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #20289: fuzz: Check for addrv1 compatibility before using addrv1 serializer/deserializer on CService (master...fuzzers-service_deserialize-addrv2) https://github.com/bitcoin/bitcoin/pull/20289
< bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/5174b534da57...218fe60d91a9
< bitcoin-git> bitcoin/master 28f8cb1 practicalswift: fuzz: Fix DecodeHexTx fuzzing harness issue
< bitcoin-git> bitcoin/master 218fe60 MarcoFalke: Merge #20290: fuzz: Fix DecodeHexTx fuzzing harness issue
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #20290: fuzz: Fix DecodeHexTx fuzzing harness issue (master...fuzzers-decode_tx-fixup) https://github.com/bitcoin/bitcoin/pull/20290
< MarcoFalke> the meeting topics link in the IRC topic of this channel is no longer being updated. I created a wiki entry to collect meeting topics: https://github.com/bitcoin-core/bitcoin-devwiki/wiki/General-IRC-meeting
< MarcoFalke> jonatack: Oh thanks. Didn't know this existed
< jonatack> MarcoFalke: same, I saw achow101 mention it a couple weeks ago
< MarcoFalke> Well, could make sense to adjust the link for that then ;)
< MarcoFalke> anyone know who can edit the IRC topic? ping wumpus sipa
< bitcoin-git> [bitcoin] jnewbery opened pull request #20291: [net] Consolidate logic around calling CAddrMan::Connected() (master...2020-11-consolidate-addrman-connect) https://github.com/bitcoin/bitcoin/pull/20291
< bitcoin-git> [bitcoin] jonatack closed pull request #20288: script, doc: contrib/seeds updates (master...contrib-seeds-fixups) https://github.com/bitcoin/bitcoin/pull/20288
< bitcoin-git> [bitcoin] MarcoFalke opened pull request #20292: test: Fix intermittent feature_taproot issue (master...2011-testFixes) https://github.com/bitcoin/bitcoin/pull/20292
< bitcoin-git> [bitcoin] MarcoFalke opened pull request #20294: ci: Run more ci configs on cirrus (master...2011-ciCirrus) https://github.com/bitcoin/bitcoin/pull/20294
< bitcoin-git> [bitcoin] Sjors opened pull request #20295: rpc: getblockfrompeer (master...2020/11/getblockfrompeer) https://github.com/bitcoin/bitcoin/pull/20295
< wumpus> MarcoFalke: I can, what would you like to change?
< bitcoin-git> [bitcoin] laanwj pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/218fe60d91a9...95bde34a7186
< bitcoin-git> bitcoin/master 36e875b RandyMcMillan: contrib: Add new versions to makeseeds.py and update gitignore
< bitcoin-git> bitcoin/master 6866259 Wladimir J. van der Laan: net: Hardcoded seeds update for 0.21
< bitcoin-git> bitcoin/master 95bde34 Wladimir J. van der Laan: Merge #20237: net: Hardcoded seeds update for 0.21
< bitcoin-git> [bitcoin] laanwj merged pull request #20237: net: Hardcoded seeds update for 0.21 (master...2020_10_seeds_update) https://github.com/bitcoin/bitcoin/pull/20237
< wumpus> ok
< jnewbery> Hi folks. Reminder that we have a P2P irc meeting in just under half an hour. Proposed topics are here: https://github.com/bitcoin-core/bitcoin-devwiki/wiki/P2P-IRC-meetings#03-nov-2020
< gribble> https://github.com/bitcoin/bitcoin/issues/03 | Encrypt wallet · Issue #3 · bitcoin/bitcoin · GitHub
< MarcoFalke> thanks
< sdaftuar> hi
< jnewbery> #startmeeting
< jnewbery> #bitcoin-core-dev P2P Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball kvaciral ariard digi_james
< vasild> hi
< jnewbery> amiti fjahr jeremyrubin lightlike emilengler jonatack hebasto jb55 elichai2
< phantomcircuit> hi
< emzy> hi
< aj> hi
< jnewbery> that ping thing is like a blockchain. It only gets longer, never shorter
< ariard> hi
< lightlike> hi
< wumpus> hi
< jonatack> hi
< kanzure> hi
< gribble> https://github.com/bitcoin/bitcoin/issues/03 | Encrypt wallet · Issue #3 · bitcoin/bitcoin · GitHub
< jnewbery> Does anyone have any more topics that they want to propose now, or any general updates they want to give?
< sdaftuar> i just wanted to review beg #19858
< gribble> https://github.com/bitcoin/bitcoin/issues/19858 | Periodically make block-relay connections and sync headers by sdaftuar · Pull Request #19858 · bitcoin/bitcoin · GitHub
< sdaftuar> i've started to now page that PR out of my memory, so would like to make progress before i totally forget what ti is :)
< jnewbery> I think we haven't branched 0.21 yet, so we're still in feature freeze, right?
< wumpus> right
< aj> branch is due pretty soon though?
< sdaftuar> i assume so -- but i think if we collect acks now it could merge shortly after?
< amiti> hi
< luke-jr> kinda curious about the status of p2p encryption
< luke-jr> but jonas isn't here
< ariard> noted, I'll reack it
< jonatack> #18242 was just rebased and it builds cleanly
< wumpus> yes, feature freeze is only about what is merged, not what is reviewed :)
< gribble> https://github.com/bitcoin/bitcoin/issues/18242 | Add BIP324 encrypted p2p transport de-/serializer (only used in tests) by jonasschnelli · Pull Request #18242 · bitcoin/bitcoin · GitHub
< aj> "2020-11-01 split off 0.21" per 18947
< jonatack> (would be good to move it forward for 0.22)
< jnewbery> wumpus: while you're here, what's your expectation for when 0.21 will be branched?
< wumpus> there's still quite a list of PRs tagged with 0.21.0, these either need to be merged or removed from the milestone first https://github.com/bitcoin/bitcoin/pulls?q=is%3Aopen+is%3Apr+milestone%3A0.21.0
< wumpus> but I hope some time next week
< jnewbery> thanks!
< wumpus> next general meeting we should go over the list
< jnewbery> ok, onto the topics
< jnewbery> #topic addrman file versioning
< vasild> jnewbery: how did you suddenly come to the realization that old versions are not guaranteed to fail the parsing, out of the blue?
< jnewbery> #19954 meant that in future it won't be possible to do forwards-compatible changes to the peers.dat format. #20284 is suggested as a way to allow that again.
< gribble> https://github.com/bitcoin/bitcoin/issues/19954 | Complete the BIP155 implementation and upgrade to TORv3 by vasild · Pull Request #19954 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/20284 | addrman: ensure old versions dont parse peers.dat by vasild · Pull Request #20284 · bitcoin/bitcoin · GitHub
< jnewbery> vasild: I hadn't reviewed 19954 before
< jnewbery> I started looking at the addrman code this weekend
< vasild> actually in the future it will be possible to make forward-compatible changes, but the format version must not be incremented
< sdaftuar> forwards compatible? do you mean backwards compatible?
< sdaftuar> i'm confused why we can't write future code to parse whatever we need to
< vasild> backward compatible
< jnewbery> sdaftuar: I mean forwards compatible, i.e. you can upgrade, make changes to peers.dat and then downgrade and still parse peers.dat
< sdaftuar> that really sounds like backwards compatible to me, but thanks for clarifying what you mean
< jnewbery> I believe we generally want to allow people to downgrade at least one version, right? So people can back out upgrades
< vasild> the current code supports maximum format=3 and if it sees format=4 it will refuse to parse it
< hebasto> hi
< jnewbery> I think backwards compatible would mean that we're able to read old versions of peers.dat, which obviously we can always guarantee
< jnewbery> forwards compatible means that we're flexible with future peers.dat formatting
< jonatack> jnewbery: yes, i was reporting issues with that (upgrading, then downgrading) in 19954 iirc
< jnewbery> vasild: right, so the current code on master is not forwards compatible with formats > 4
< vasild> >= 4
< jnewbery> sorry yes, >= 4
< vasild> yes, if changes are made to the format it must stay at format=3 if we want 0.21 to parse it
< vasild> (as of the current master)
< luke-jr> it would be annoying if users lost their addrman upgrading or downgrading
< vasild> pr20284 changes that
< jnewbery> luke-jr: they already lost part of their addrman upgrading to 0.20 because of https://github.com/bitcoin/bitcoin/pull/16702#discussion_r515608294, but I guess no-one noticed
< luke-jr> :/
< vasild> pr20284 makes it possible to say in peers.dat "this is format=5, but anybody who can read format=3 can also read this one"
< luke-jr> maybe the filename should just be changed if that's not the case, in the future?
< jnewbery> vasild: yes, that's the functionality we want before the 0.21 release. Just want to make sure people are happy with repurposing nKeySize for versioning.
< sdaftuar> jnewbery: sorry if this is a digression, but i might be misunderstanding the issue in #16702 -- it looks like the version check there just causes potential rebucketing, not losing addresses?
< gribble> https://github.com/bitcoin/bitcoin/issues/16702 | p2p: supplying and using asmap to improve IP bucketing in addrman by naumenkogs · Pull Request #16702 · bitcoin/bitcoin · GitHub
< wumpus> I think it's a hack but also a clever way of doing so, and he also introduces a real versioning mechanism for next time, so it's only for once
< sdaftuar> so the issue that does appear to be there is that downgrading after running 0.20 may be a problem
< jnewbery> sdaftuar: it means that entries in new tables can only appear in one new table (or even zero if there's a collision)
< wumpus> definitely don't have any alternative proposals
< jnewbery> the point of the serialization format is that entries can appear in multiple new tables
< vasild> luke-jr: right, filename change is another option
< sdaftuar> wumpus: we could rename the peers.dat file for 0.21, and migrate data from the old file to the new one?
< wumpus> sdaftuar: that's a good suggestion too
< sdaftuar> jnewbery: hmm i need to read that more carefully then.
< jnewbery> sdaftuar: that would definitely prevent being able to keep your addrman data over a downgrade
< wumpus> though it'd require updating a lot of documentation all over the place describing bitcoin core's files
< wumpus> everyone is used to "peers.dat"
< luke-jr> how often does anyone mess with peers.dat directly?
< wumpus> I have no idea
< vasild> btw, I did consider the file name change, but preferred the versioning inside the file, instead of using the filesystem for versioning
< wumpus> yes, I slightly prefer this as well
< sdaftuar> i think it's okay if once in a while we have to give users annoying instructions for downgrading.
< luke-jr> IMO it depends on compatibility
< luke-jr> if downgrading doesn't lose the address book, same filename is good; otherwise, a new name avoids losing the old addrman for the old ver
< jnewbery> what would the annoying instructions be?
< sdaftuar> we could add a tool that either converts formats, or an RPC or something
< luke-jr> ^
< wumpus> sdaftuar: as I understand older versions will just ignore the incompatible peers.dat, and create a new empty one. But yes I agree. Downgrading is quite a rare event anyhow.
< luke-jr> ew
< sdaftuar> what did we do the last time we changed block file name conventions?
< wumpus> sdaftuar: no one is going to go through that much work for their peers table
< luke-jr> sdaftuar: iirc hard linked
< sdaftuar> luke-jr: that doesn't sound backwards compatible?
< luke-jr> why not?
< jonatack> I must have lost my peers.dat a couple dozen times while testing the addrv2 PRs. within the next year, anyone still using tor v2 will be forced to upgrade to 0.21 anyway.
< sdaftuar> oh we hardlinked new and old filenames together, i see
< wumpus> I'm not sure it makes sense to extensively describe alternatives here, we have vasild's work, unless someone really strongly disagrees with it let's review it and get it merged asap
< jonatack> unless they don't use tor
< jnewbery> I don't think there's any need to change file names. It should be perfectly possible to have internal versioning for changes that are forwards compatible and not forwards compatible. I think that's what 20284 does, but I haven't reviewed it yet.
< wumpus> assuming this really needs to go in for 0.21.0-otherwise there's no hurry
< aj> afaics, for forwards compatible changes, we just don't bump the version number?
< sipa> argh, timezones
< vasild> I started a 0.20.1 node on my new peers.dat and it parsed it without emitting errors, so it is not just "theoretical"
< vasild> surely it parsed everything as garbage
< wumpus> it's definitely not theoretical
< vasild> I don't know why during testing of that change, some weeks ago I always got some read/parse error, which fooled me that old versions fail to parse always
< jonatack> vasild: same
< vasild> :-D
< jnewbery> aj: I think there needs to be versioning to show that there are semantic changes, even if they are forwards compatible
< wumpus> garbage addresses would be kind of nasty, especially if they might also be gossiped and thus pollute the rest of the network too
< luke-jr> jnewbery: point being if it isn't forwards compatible, the rename keeps the old file for the old version if you downgrade
< jnewbery> luke-jr: that's a good point
< jnewbery> So the action is to review 20284, unless anyone has anything important to add
< jnewbery> #topic I2P support, some background at vasild/bitcoin/wiki/I2P-connectivity (vasild)
< jonatack> When you lose your peers.dat, a new one doesn't seem to take very long to get up to a decent size (40-50k peers)
< jnewbery> jonatack: it probably takes longer for your tried tables to get populated
< sdaftuar> jonatack: the whole network losing their peers.dat in the event of downgarde might be a scary idea though (imagine a bug in 0.21, say)
< sipa> jonatack: that may be deceptive though; just because you have a lot of rumoured IP addresses, they aren't necessarily good
< jonatack> good points, thanks
< aj> jnewbery: if you bump the version from 5 to 6 (eg), and 6 is "unknown" it will be treated as non-supported, so won't be forward compatible. you need major/minor versions if you want to do that
< jnewbery> aj: I think that's effectively what 20284 does
< sipa> can we imagine any backward but not forward compatible change?
< vasild> aj:
< vasild> 16:15 < vasild> pr20284 makes it possible to say in peers.dat "this is format=5, but anybody who can
< vasild> read format=3 can also read this one"
< aj> vasild, jnewbery: i believe you're both mistaken... it just supports a range: "this code supports versions 3..5". introducing version 6 won't make old code support it
< jnewbery> sipa: after #19954, all changes are not forward compatible (because 0.21 onwards will reject any version greater than the one they know)
< gribble> https://github.com/bitcoin/bitcoin/issues/19954 | Complete the BIP155 implementation and upgrade to TORv3 by vasild · Pull Request #19954 · bitcoin/bitcoin · GitHub
< aj> sipa: backward compatible is easy: if version == 4: ... elif version == 5: ...
< jnewbery> aj: +1
< sipa> jnewbery: yes, i mean semantically
< sipa> is there any change we can imagine to peers.dat that would remain readable by new versions?
< aj> sipa: readable by old code, you mean?
< sipa> oh
< * sdaftuar> is super confused
< sipa> do i have my forward/backward mixed up?
< aj> new code, old peers.dat == backwards compatible
< sdaftuar> sipa: i thought john has his forward/backward mixed up, for what it's worth. and maybe aj too.
< aj> old code, new peers.dat == forwards compatible
< vasild> awkward
< sdaftuar> vasild: +1
< aj> "the format is ___ compatible" is how i'm interpreting it
< sipa> the goal here is to make sure that new versions can keep reading old files?
< jnewbery> sdaftuar: wikipedia seems to agree with our definitions: Backward compatibility (sometimes known as backwards compatibility) is a property of a system, product, or technology that allows for interoperability with an older legacy system, or with input designed for such a system
< vasild> no
< jnewbery> where the input here is peers.dat
< vasild> sipa: new code can always read old files
< jnewbery> old peers.dat, new addrman is backwards compatibility
< jnewbery> new peers.dat, old addrman is forwards compatibility
< sipa> ok so this is about new files being readable by old code? can we imagine a change where that's useful?
< sdaftuar> sipa: it's about being able to downgrade
< sdaftuar> and not losing all your data
< jnewbery> i.e. you right the code in a way that it can handle future changes to format that you don't know about, which is potentially important for downgrades
< sipa> sdaftuar: yes, my question is: can we imagine any change to peers.dat where that's even feasible?
< luke-jr> sipa: if you don't use Tor, this change is?
< sdaftuar> sipa: well it depends on what's on the table? eg if we wrote out the old format for the addresses that are supported by the old format in a file that is readable by old software, then sure
< jnewbery> yes, for example the change from 0 to 1 and 1 to 2 both allowed old versions to read new files
< sipa> jnewbery: ah, indeed!
< sipa> thanks
< vasild> sipa: you authored that! :-)
< luke-jr> hmm, we could in theory even split a peers-torv3.dat for those, and keep peers.dat as-is and compatibel?
< sipa> vasild: it's early
< sdaftuar> luke-jr: interesting idea
< aj> or put the torv3 addresses as an add-on at the end of the file or something?
< wumpus> where does that stop, though... would we have peers-i2p.dat as well?
< jonatack> if anyone following along is wondering, we're talking about enum class Format in addrman.h::L269
< luke-jr> wumpus: why not?
< sipa> this seems hard
< luke-jr> is there a reason to throw it all in the same file?
< jnewbery> jonatack: well more generally about the Serialize and Unserialize methods for CAddrMan
< sdaftuar> wumpus: perhaps eventually files get merged, once old software no longer is a plausible downgrade target?
< sipa> peers.dat files aren't just lists of addresses
< sipa> they dump the tables and their layout too
< jnewbery> we could replace peers.dat with sqlite :)
< wumpus> I'm really not sure this is worth so much worrying about, if we're really worried about people downgrading why not make a backup at first run with 0.21.0?
< wumpus> then if people downgrade they can restore the backup and *shrug*
< sipa> if you split the file in two, you can't just merge them back without losing informatiom
< luke-jr> jnewbery: that might not be a bad idea
< sdaftuar> wumpus: yes i agree with that type of suggestion too, along the lines of "sometimes it can be annoying to downgrade"
< luke-jr> wumpus: how is that different from just using a new filename, except requiring an extra step to downgrade?
< wumpus> luke-jr: it keeps the filename the same for the bulk of the users
< wumpus> the non-downgrading path is likely be far most popular, or at least one'd hope :)
< wumpus> in any case, we needa solution here fairly quickly
< jnewbery> wumpus: I agree that it's more likely, but I think we should strive to make downgrades across a single version seamless, just in case we ship a bad bug and need people to roll back
< jnewbery> *upgrading is the more likely path than downgrading
< jnewbery> ok, we're running out of time and there are a couple more topics, so perhaps we should move on
< wumpus> I'm not sure last-minute changes lke 'sort peers.dat per network' or other complete changes to handling peers database
< sipa> wumpus: agree
< jnewbery> vasild: did you have anything you wanted to add about I2P support?
< ariard> jnewbery: I'm fine to report mine next time
< jnewbery> ariard: thanks. Maybe a good idea so people have some time to think about it before the meeting
< vasild> Anybody interested in I2P support - see https://github.com/vasild/bitcoin/wiki/I2P-connectivity and discuss here, I guess after the meeting or anytime
< sdaftuar> i've got a question about i2p (applies to any new exotic network, even tor stuff that we're doing) - what is our undestanding for how these addresses get gossiped to peers that support the network type?
< wumpus> jnewbery: yes, agree, it's kind of nasty that this issue comes up so last minute
< sipa> sdaftuar: i2p/cjdns do not get rumoured by the current code, at all
< vasild> sipa: no, they would get rumored if the node has connectivity to i2p
< vasild> or am I mistaken...
< wumpus> cjdns not because it uses IPv6 addresses in a range we consider local / unroutable?
< sipa> vasild: which isn't possible.on the current code :)
< sipa> see #20119
< gribble> https://github.com/bitcoin/bitcoin/issues/20119 | BIP155 follow-ups by sipa · Pull Request #20119 · bitcoin/bitcoin · GitHub
< sdaftuar> so the basic idea is that if we know how to reach a network, we'll rumor it to 2 peers or so
< sdaftuar> and if we don't, it still goes to 1 i think? is that correct?
< sipa> sdaftuar: there are classes now
< vasild> sdaftuar: yes
< jnewbery> sdaftuar: 1.5 now I think
< sipa> 1) reachable networks get rumoured 2x
< vasild> sdaftuar: see RelayAddress() in the case fReachable==true
< sipa> 2) unreachable ipv4/ipv6/torv2/torv3 get rumoured 1.5x
< jonatack> see #19728
< gribble> https://github.com/bitcoin/bitcoin/issues/19728 | Increase the ip address relay branching factor for unreachable networks by sipa · Pull Request #19728 · bitcoin/bitcoin · GitHub
< sipa> 3) unreachable others do not get rumoured at all
< vasild> #20254 makes i2p reachable
< gribble> https://github.com/bitcoin/bitcoin/issues/20254 | Add I2P support using statically configured destinations by vasild · Pull Request #20254 · bitcoin/bitcoin · GitHub
< vasild> btw, in I2P we can see the peer's address and be sure data comes from him (the address is a hash of his public key)
< sdaftuar> sipa: thanks. one more question, how do we decide what address to advertise ourselves as, if we're reachable in multiple ways?
< luke-jr> vasild: same for Tor? :p
< vasild> luke-jr: no, in Tor we don't see the address of the peer that connected to us
< luke-jr> oh, right
< sipa> "tor has no from address"
< luke-jr> XD
< wumpus> heh
< vasild> so in I2P we have P2P encryption by default :-)
< jnewbery> sdaftuar: that logic is in AdvertiseLocal() in net.cpp
< sipa> vasild: with public identities, though :(
< vasild> yes
< luke-jr> kinda by design in i2p?
< vasild> yes
< sipa> sdaftuar: among the local addresses compatible with that peer, the one that has received the most mentions in incomimg version messages
< sipa> which reminds me: we can't put torv3/i2p/cjdns in version messages, so those will never receive any mentions
< sdaftuar> sipa: it seems like for something like adding a new newtork type, sometimes we'll want to advertise our i2p address instead right?
< sdaftuar> sipa: oh taht seems important
< jnewbery> do we need a new version version? :)
< sdaftuar> jnewbery: i think we can just add data on to the end and no one will mind
< sipa> i'm not sure this is useful for those networks
< sipa> as there isn't any useful compatibility matrix for them
< jnewbery> that's time!
< jnewbery> #endmeeting
< sdaftuar> sipa: address rumoring isn't useful, or something else?
< jonasschnelli> oh.. I missed the meeting.
< wumpus> sipa: there was the idea to add that address in the 'sendaddrv2' message
< vasild> I guess sipa means the "voting"
< sdaftuar> ah
< vasild> or how many times the address receives "mentions"
< aj> zzz
< * luke-jr> gives jonasschnelli the mic
< jonasschnelli> I wanted to ask about BIP324's rekeying
< sipa> sdaftuar: scoring your local addresses using incoming version mentions isn't useful for these separate networls
< sipa> jonasschnelli: ongoing discussion, it seems!
< sdaftuar> yeah i was thinking that if we are trying to bootstratp a new type of address, we need to proactively advertise that address even to peers who don't understand it
< jonasschnelli> sipa: may you have 10 minutes for discussion the 1s rekey approach
< sdaftuar> so that eventually peers that do support it, receive it
< sipa> sdaftuar: that's a good point
< wumpus> sipa: (which would be easier to add new things to than the version message, and is relevant to the same functionality)
< vasild> sdaftuar: my thought exactly, but we decided to not relay i2p addresses by nodes that are not connected to i2p; at least in 0.21
< jonasschnelli> sipa: indeed... its ongoing. But assume a 1s rekeying interval would imply some sort of a rekey-message,.. right?
< sipa> jonasschnelli: my current thinking is that doing rekeying at the key stream level is super cheap and simple
< jonasschnelli> rekey on every ping/pong would be less complicated to implement (rather then time based)
< sipa> jonasschnelli: and i see no reason to have it at the application level as well if we do that
< sipa> jonasschnelli: i absolutely don't suggest rekeying every secomd :)
< jonasschnelli> sipa: at stream level would be something like basically rekey on after message?
< vasild> there is already one bitcoin node listening at yfsvsy467mt5xafaq7zaukkjyzehvmew445yaaejvrwpk53acejq.b32.i2p (irc gossip :-D)
< jonasschnelli> "after every message
< sipa> jonasschnelli: no, every X bytes
< sdaftuar> vasild: nice :)
< jonasschnelli> sipa: okay. What we have now.. but probably smaller than 1GB
< sipa> jonasschnelli: no!
< sipa> it's currently done at the layer above
< sipa> this would let you just build it inside chacha
< jonasschnelli> ah. Built into the AEAD
< ariard> jonasschnelli: I think you can pick up a value based on blocks frequency if you want something to happen every Y period of time?
< sipa> without having a bit to signal rekeying or so
< sipa> ariard: that's just as hard
< sipa> jonasschnelli: this would give you forward secrecy (also a weird term
< jonasschnelli> What if the byte limit hits in the middle of a message payload?
< sipa> for ~free, at the byte level if you wanted to
< sipa> jonasschnelli: yes, so what?
< sipa> the stream cipher does it automatically
< ariard> sipa: do you mean byte-accounting is hard to get right?
< sipa> ariard: no, byte-accounting is trivial
< jonasschnelli> sipa: hmm... but the mAC?
< sipa> jonasschnelli: the mac keys don't cycle
< sipa> only the encryption keys
< jonasschnelli> but the current MAC key is derived from the chacha20 stream
< sipa> jonasschnelli: yes that'll need to change
< jonasschnelli> sipa: hmm...
< sipa> have a separate stream where the mac keys are drawn from, for example
< sipa> but it means you can drop the signalling of rekeying etc
< sipa> and concerns about peers not doing it, or doing it too often
< sipa> as long as we don't worry about secrecy in the other direction (which the current bip324 writeup doesn't, afaik), i think this is very much preferable
< jonasschnelli> sipa: do the benefits of this justify further deviation from the "well tested" ChaCha20Poly1305@openssh?
< sipa> jonasschnelli: we're already deviating from it
< jonasschnelli> sipa: but in a pretty minor way
< sipa> jonasschnelli: maybe superficially... but openssh rekeys by redoing DH, no?
< jonasschnelli> sipa: what do you think are the benefits over a rekeying signal (where we can detect DoS or exceeded limits)?
< jonasschnelli> sipa: probably... haven't looked at their rekeying.
< jonasschnelli> but the AEAD has no rekeying (@openssh and @Bitcoin)
< sipa> so doing rekeying by redoing DH makes sense, as it gives you secrecy in both directions
< jonasschnelli> Yes. Our rekeying does a independent rekey per direction,... whenever the limits are reached.
< sipa> rekeying by hashing only gives you forward secrecy, not the other direction
< sipa> but as long as we only care about forward secrecy... doing the whole signalling thing is kind of overkill
< jonasschnelli> I see
< jonasschnelli> I see what you mean with direction noew
< jonasschnelli> *now
< sipa> you can get cheap forward secrecy at the byte level, instead of at the 1 GB level
< sipa> (by generating 4096 bytes of stream cipher ahead of time, rekeying, and then throwing the stream cipher bytes away as they get used)
< sipa> you can't do that with 1 GB of material, and can't do that if you don't know when the other party is going to reky
< sipa> anyway, my point is: not redoing DH is already a big conceptual departure from the existing protocol
< jonasschnelli> I think I understand this... I'm just unsure about the MAC construct... and concerned about complication of the implementation (with its risks)
< sipa> yeah
< sipa> to be honest, i habe difficulty remember the exact way the stream/keys are constructed now too
< sipa> this may simplify it...
< jonasschnelli> sipa: we rekey with SHA256(SHA256(session ID || old_symmetric_cipher_key))... wouldn't an attacker stealing the symmetric be unable to go in both directions?
< jonasschnelli> since the "session ID" requires the ECDH key
< sipa> jonasschnelli: if an attacker knows the old key, they can also apply the hashing
< jonasschnelli> sipa: the attacked doesn't know the session ID, right?
< sipa> well in this scenario they do!
< jonasschnelli> sipa: So the scenario is that ECDH is broken?
< sipa> jonasschnelli: or they broke into your machine and dumped its memory
< sipa> forward secrecy protects against that case... if an attacker does that, they *still* can't decrypt old messages
< jonasschnelli> okay.. got it
< sipa> but the other direction... you can't prevent an attacker who read your RAM from decrypting future messages, unless you redo DH
< jonasschnelli> including re-generation of emphemeral keys,.. right?
< sipa> yes, that's what DH does
< jonasschnelli> indeed. :)
< jonasschnelli> and you think that backward secrecy (probably called different) can be achived by altering the AEAD and including a deterministic rekeying?
< jonasschnelli> or has the AEAD-inner-rekeyng nothing to do with the secrecy directions?
< sipa> that only provides forward secrecy
< sipa> but it can do it at the byte level
< jonasschnelli> more elegant but same level of security as the current proposal?
< sipa> indeed
< jonasschnelli> okay... got it.
< sipa> perhaps we want some affordance for redoing DH too, but we can probably just disconnect and reconnect for that...
< jonasschnelli> I haven't seen BIP324 discussion about the missing backward secrecy of the current rekeying approach.
< sipa> i think it's less of a concern
< sipa> but i'll try to summerize
< sipa> on githun
< jonasschnelli> great,... thanks.
< jonasschnelli> I'd like to not loose to much momentum on the implementation. But also,... not urging to stop improving the protocol. :)
< jonasschnelli> However... thanks sipa for explaining!
< wumpus> vasild: here is already one bitcoin node listening at yfsvsy467mt5xafaq7zaukkjyze hvmew445yaaejvrwpk53acejq.b32.i2p (irc gossip :-D) <- good to know, i'll have a look at setting up I2P too
< jonatack> vasild: i think i'm connected to your i2p peer
< amiti> hi! does anyone actively use `-loadblock` / have current use cases?
< sipa> amiti: did yiu see my response on the issue?
< amiti> yeah, you said "I don't know if those see much use still", so I thought I'd ask more people :)
< sipa> amiti: right
< amiti> (for anyone wondering, #19594)
< gribble> https://github.com/bitcoin/bitcoin/issues/19594 | refactor: Make mapBlocksUnknownParent local, and rename it by hebasto · Pull Request #19594 · bitcoin/bitcoin · GitHub
< sipa> i mean: it was intended for supporting bootstrap.dat, and i think that's the only use case
< sipa> if there is anyone actively relying on that feature i don't know
< amiti> gotcha. ok maybe I could rephrase the question as- does anyone actively use bootstrap.dat?
< luke-jr> it's not even maintained anymore
< sipa> not by us
< sipa> though some sites provide them
< luke-jr> updated? O.o
< sipa> since the improved block download logic in 0.10 there is much less use for it, as just syncing from random peers is likely faster than downloading + processing in sequence
< sipa> https://flo.sh/download-bitcoin-blockchain-bootstrap/ apparently has a fairly recent one
< luke-jr> TIL
< luke-jr> I suppose it might be handy for people syncing offline PCs
< sipa> TIL also
< luke-jr> hmm, I should probably stop parsing my entire debug.log every hour
< luke-jr> it's getting behind XD
< amiti> do I understand correctly: the idea of bootstrap.dat / loadblock is that you input a file with the sequential blocks to process. in the current state, what are the differences from importing the datadir and reindexing? / would there be any advantage? (ofc one difference is that loadblock requires the blocks to be in order)
< sipa> i expect you can just rename a bootstrap.dat file to be blk00000.dat, and run -reindex
< sipa> amiti: i think part is that you don't want to encourage people copying entire datadirs, as chainstates don't get re-verified
< sipa> and anything that involves manually messing with the datadir is probably reserved for people who know what they're doing anyway
< amiti> oh I see, so loadblock does more verification than reindexing? ok I'm not super familiar with reindexing so I'll go learn more
< luke-jr> loadblock just treats the file as an untrusted peer
< sipa> amiti: no, it's exactly the same
< sipa> amiti: but if you copy a *chainstate* from someone (not just block files), they don't get re-verified (because your node wouldn't know it's operating on an imported copy)
< amiti> ahhhh I see
< sipa> still, i don't have a good answer for whether it's worth supporting -loadblock if it is a burden to maintain... i suspect not
< sipa> i also don't think it's a large burden, thoug
< * midnight> makes use of loadblocks still fwiw
< sipa> good to know!
< amiti> yeah, for me the first question is "is it still useful". then the next question is "is it a higher burden to maintain, or to remove?"
< sipa> i don't think i've personally used it for many years
< amiti> midnight: ooo, willing to tell us more about how/why you use it?
< midnight> sure. I use it as a way of explicitly migrating block data between machines; also occasionally to (expensively) explore early blocks and early large-block-number reorgs. Mostly as a way to ensure migrated block data is reverified with modern versions as my node instances are typically very long-lived.
< midnight> I'd be completely fine with just a network-based block injector if it exists.
< amiti> okay gotcha. thanks!
< midnight> \o I wouldn't be too broken-up about it being removed, as long as it's still possible for me to help seed my friends with a copy of current block data from my own nodes somehow.
< midnight> (so, I test it first before handing it to them)
< amiti> 👍