nanotube has quit [Ping timeout: 260 seconds]
<bitcoin-git> [bitcoin] Xekyo opened pull request #27021: Implement Mini version of BlockAssembler to calculate mining scores (master...add-mini-miner) https://github.com/bitcoin/bitcoin/pull/27021
Norrin has quit [Ping timeout: 255 seconds]
ghost43 has quit [Quit: Leaving]
ghost43 has joined #bitcoin-core-dev
MrFrancis has joined #bitcoin-core-dev
Norrin has joined #bitcoin-core-dev
andrew_mo_ has quit []
MrFrancis has quit [Ping timeout: 252 seconds]
MrFrancis has joined #bitcoin-core-dev
nanotube has joined #bitcoin-core-dev
halosghost has quit [Quit: WeeChat 3.8]
scg has joined #bitcoin-core-dev
brunoerg has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 252 seconds]
b_101 has joined #bitcoin-core-dev
PaperSword has quit [Quit: PaperSword]
b_101 has quit [Ping timeout: 248 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 252 seconds]
b_101 has joined #bitcoin-core-dev
___nick___ has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
___nick___ has joined #bitcoin-core-dev
___nick___ has quit [Client Quit]
b_101 has quit [Ping timeout: 248 seconds]
___nick___ has joined #bitcoin-core-dev
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 248 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 252 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 252 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 248 seconds]
b_101 has joined #bitcoin-core-dev
MrFrancis has quit [Ping timeout: 248 seconds]
ghost43 has quit [Remote host closed the connection]
ghost43 has joined #bitcoin-core-dev
Guest7541 has joined #bitcoin-core-dev
Guest7541 has quit [Client Quit]
cmirror has quit [Remote host closed the connection]
cmirror has joined #bitcoin-core-dev
Norrin has quit [Ping timeout: 255 seconds]
Norrin has joined #bitcoin-core-dev
achow101 has quit [Ping timeout: 255 seconds]
as2333 has quit [Quit: as2333]
achow101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 260 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 248 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 268 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 260 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 255 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 255 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 260 seconds]
b_101 has joined #bitcoin-core-dev
cryptapus has quit [Remote host closed the connection]
cryptapus has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 260 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 268 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 252 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 268 seconds]
_andrewtoth_ has joined #bitcoin-core-dev
ghost43_ has joined #bitcoin-core-dev
qxs has quit [Ping timeout: 255 seconds]
ghost43 has quit [Remote host closed the connection]
andrewtoth_ has quit [Ping timeout: 255 seconds]
qxs has joined #bitcoin-core-dev
bitdex has joined #bitcoin-core-dev
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 255 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 268 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 260 seconds]
javi404 has quit [Remote host closed the connection]
javi404 has joined #bitcoin-core-dev
dermoth has quit [Ping timeout: 255 seconds]
robszarka has joined #bitcoin-core-dev
szarka has quit [Ping timeout: 252 seconds]
enyc has quit [Ping timeout: 264 seconds]
rszarka has joined #bitcoin-core-dev
robszarka has quit [Ping timeout: 248 seconds]
Guyver2 has joined #bitcoin-core-dev
robszarka has joined #bitcoin-core-dev
robszarka has quit [K-Lined]
szarka has joined #bitcoin-core-dev
rszarka has quit [Ping timeout: 248 seconds]
dermoth has joined #bitcoin-core-dev
dougefish has joined #bitcoin-core-dev
dermoth has quit [Ping timeout: 255 seconds]
dermoth has joined #bitcoin-core-dev
<bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/fdd363ebd917...102645280b94
<bitcoin-git> bitcoin/master 71383f2 fanquake: ci: avoid using -Werror for older compilers
<bitcoin-git> bitcoin/master 1026452 MarcoFalke: Merge bitcoin/bitcoin#27013: ci: avoid using `-Werror` for older compilers
<bitcoin-git> [bitcoin] MarcoFalke merged pull request #27013: ci: avoid using `-Werror` for older compilers (master...avoid_werror_older_release_compiler) https://github.com/bitcoin/bitcoin/pull/27013
PaperSword has joined #bitcoin-core-dev
Guyver2 has left #bitcoin-core-dev [Closing Window]
MrFrancis has joined #bitcoin-core-dev
hsmiths has quit [Ping timeout: 260 seconds]
<bitcoin-git> [bitcoin] fanquake closed pull request #26348: Make P2SH redeem script "IF .. PUSH x ELSE ... PUSH y ENDIF CHECKMULTISIG .. " standard (master...master) https://github.com/bitcoin/bitcoin/pull/26348
<bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/102645280b94...9dc50a5a0788
<bitcoin-git> bitcoin/master fad7af7 MarcoFalke: Use steady clock for logging timer
<bitcoin-git> bitcoin/master 9dc50a5 fanquake: Merge bitcoin/bitcoin#27005: util: Use steady clock for logging timer
<bitcoin-git> [bitcoin] fanquake merged pull request #27005: util: Use steady clock for logging timer (master...2301-log-steady-clock-🔆) https://github.com/bitcoin/bitcoin/pull/27005
ghost43_ has quit [Remote host closed the connection]
ghost43 has joined #bitcoin-core-dev
<bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/9dc50a5a0788...21138fe37724
<bitcoin-git> bitcoin/master fa47b28 MarcoFalke: refactor: Remove unused CDataStream SerializeMany constructor
<bitcoin-git> bitcoin/master 21138fe fanquake: Merge bitcoin/bitcoin#26992: refactor: Remove unused CDataStream Serialize...
<bitcoin-git> [bitcoin] fanquake merged pull request #26992: refactor: Remove unused CDataStream SerializeMany constructor (master...2301-ser-many-remove-đź’») https://github.com/bitcoin/bitcoin/pull/26992
sudoforge has joined #bitcoin-core-dev
Guest3492 has joined #bitcoin-core-dev
Guest3492 has quit [Client Quit]
Livestradamus8 has joined #bitcoin-core-dev
Livestradamus has quit [Ping timeout: 248 seconds]
Livestradamus8 is now known as Livestradamus
PaperSword has quit [Quit: PaperSword]
AaronvanW has joined #bitcoin-core-dev
<fjahr> sipa: Thanks a lot for taking a look! On performance/bottneck.py: Yeah, I am sure there is room for improvement in terms of performance, I only glanced at bottleneck.py briefly and saw that it uses a different approach (I use numpy and pandas) so I skipped digging deeper because I just wanted to get something out that sort of works for conceptual discussion. I will work on improving performance when we have agreement on
<fjahr> data sources and the release process steps.
<fjahr> And yeah, that was my understanding that we would work towards shipping a file with Bitcoin Core. Maybe in the first iteration it gets shipped with the release but it’s not used by default. However users could use the included file by just passing -asmap=1 and they don’t have to get the file from somewhere and provide it explicitly. I think that might be a good intermediate step. Of course they should still be able to
<fjahr> override it with their own file.
<fjahr> On the asmap file export for audit purposes: I didn’t spend much time thing about it but my first question would be, do we really need to make it that explicit? I am thinking of the ASMap file as code, it’s just not tracked in the repo. But the selected candidate file could still be pulled from the asmap-data repo and included in the reproducible build process, right? So those that run the build process will not get the
<fjahr> same result if the file is different, those that check the signatures can be sure the right file was used. And those that want to look at the raw file can download it from asmap-data and compare the hash. If we need it I would suggest the binary writes it out to default-asmap.dat in the datadir. But I am a developer and move stuff around in the datadir all the time, so maybe my opinion doesn’t align with most users and
<fjahr> for them an explicit export RPC is better. I think someone will write that RPC eventually anyway, might be a good first issue.
<fjahr> On how to use the multimap stuff: yeah, that would need a change in the bucketing logic and I haven’t worked on that yet. But the background is: Initially gleb already mentioned this as a potential issue and “stretch goal” in his write-up after coredev zurich, that there are big entities that have multiple ASNs and just bucketing for ASNs alone doesn’t catch that. Since the ROAs are signed we know that if there are
<fjahr> two valid ROAs for the same IP block but with different ASNs, then either this is the same entity that has multiple ASNs or these are two different entities that have very close business ties. Either way I think it makes sense to stick both ASNs in the same bucket in that case. But indeed it’s a nice to have for the start, but I wanted to name it as a clear additional win from using RPKI. In Kartograf I am currently
<fjahr> discarding the additional ROAs. Unfortunately there is no good tie-breaker when deciding between two different ROAs but we can just decide on something.
<fjahr> We could probably also solve this without changing the bucketing logic by writing synthetic consolidated ASNs into the asmap. But that is just a spontaneous idea, I will need to think about this more.
<fjahr> Downside, that would probably make it harder to audit the ASMap.
<bitcoin-git> [gui] hebasto merged pull request #704: Correctly limit overview transaction list (master...2023_01_FixOverviewTxList) https://github.com/bitcoin-core/gui/pull/704
<bitcoin-git> [bitcoin] hebasto pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/21138fe37724...526f67a5cabf
<bitcoin-git> bitcoin/master 08209c0 John Moffett: Correctly limit overview transaction list
<bitcoin-git> bitcoin/master 526f67a Hennadii Stepanov: Merge bitcoin-core/gui#704: Correctly limit overview transaction list
<bitcoin-git> [bitcoin] willcl-ark opened pull request #27025: github: Switch to yaml issue templates (master...yaml-issue-templates) https://github.com/bitcoin/bitcoin/pull/27025
MrFrancis has quit [Ping timeout: 248 seconds]
<bitcoin-git> [gui] hebasto merged pull request #695: Fix misleading RPC console wallet message (master...2023_01_FixNoWalletRPCConsoleMessage) https://github.com/bitcoin-core/gui/pull/695
<bitcoin-git> [bitcoin] hebasto pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/526f67a5cabf...ea41abade4b7
<bitcoin-git> bitcoin/master 576f7b8 John Moffett: Fix misleading RPC console wallet message
<bitcoin-git> bitcoin/master ea41aba Hennadii Stepanov: Merge bitcoin-core/gui#695: Fix misleading RPC console wallet message
dviola has joined #bitcoin-core-dev
ghost43 has quit [Quit: Leaving]
ghost43 has joined #bitcoin-core-dev
Norrin has quit [Remote host closed the connection]
brunoerg has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
MrFrancis has joined #bitcoin-core-dev
AaronvanW has quit [Remote host closed the connection]
brunoerg has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
MrFrancis has quit [Ping timeout: 252 seconds]
<sipa> fjahr: Oh you're talking about one entity having multiple ASNs? Or multiple IPs having the same ASN?
<sipa> In the first case, if it can be identified, it doesn't matter; just give them one number. The actual numbers don't matter, just whether IPs are in the same or different ASNs.
<fjahr> one entity having multiple ASN and each ASN announcing multiple IP blocks, with some overlap on the IP blocks between them
<fjahr> yeah, that would be the "synthetic" ASN then, where the ASN in asmap is not the same as it was in the ROA, just makes auditing/resoning harder I think, but I will try this out
<sipa> The only thing the actual choice of ASN matters for is when switching from one asmap to another, a remapping in addrman happens... if the more thing remain in the same ASN the less information will collide on update.
nanotube has quit [Ping timeout: 252 seconds]
<sipa> So if you use some way of choosing a synthetic ASN (probably from the set of real ones), it's better to do it in a way that does flip-flop too much.
<sipa> fjahr: You're right, perhaps being able to inspect the built-in asmap doesn't matter too much; it can be consider source code, and presumably can be found when looking at the source.
<sipa> I looked at your example PR with IP ASN mappings text file... I saw it contains a number of things the current binary format can't deal with (AS0, and multiple numbers separated by underscores or commas; also a few numbers that are too large). If I drop all of those (picking the first in case of _ and ,), I get an encoded file that's close in size to what my collection/bottlenecking logic does, though a very substantial portion of the IP ranges are mapped
<sipa> differently.
AaronvanW has joined #bitcoin-core-dev
Norrin has joined #bitcoin-core-dev
<fjahr> ok, I will play with that concept of joining ASNs a bit and see what the results would look like. I know about 10% of valid ROAs have another ROA that announces a different ASN for the same IP block but beyond that I need better statistics.
<fjahr> Ok, weird about the formatting stuff, I will take a look.
<fjahr> For the differences as well I will have to check where they are coming from. The example is partly RPKI and partly collector (CAIDA PFX2ASN), yours is only collectors so I would assume the differences are between the RPKI and collectors but let's see.
<sipa> Did you see my asmap-tool.py that can compute a diff of mappings (either in txt or binary form)?
<fjahr> Since I discarded a random ROA if there were multiple for the same IP block, that alone could explain multiple % of differences
<fjahr> I saw it yeah, I will use it, also asmapy has some diff logic that I wanted the test/review as well
<sipa> It was something like 10-20% of the internet was mapped differently.
<fjahr> It's a jungle out there :)
<sipa> yeah
<brunoerg> @fjahr: I created the diff tool on asmapy from the sipa's one, they're basically similar.
<fjahr> sipa: ah, what's the time delta between your file and mine? Did you use collector data from the same day?
<fjahr> brunoerg: ok
<sipa> i created mine yesterday
<sipa> now there is a % difference even between two consecutive days, with the same data sources and tooling
<sipa> but the difference i saw between yours and mine was way bigger than that
bitdex has quit [Quit: = ""]
<fjahr> ok, will investigate
Norrin has quit [Ping timeout: 255 seconds]
Norrin has joined #bitcoin-core-dev
<sipa> @fjahr Does your code do bottleneck analysis actually? (if all routes to an IP range X go through the same last two ASNs, you list the first of those as that range's AS rather than the last one)?
halosghost has joined #bitcoin-core-dev
yanmaani has quit [Ping timeout: 255 seconds]
virtu has joined #bitcoin-core-dev
virtu has quit [Changing host]
virtu has joined #bitcoin-core-dev
Norrin has quit [Ping timeout: 255 seconds]
Norrin has joined #bitcoin-core-dev
<fjahr> sipa: No, I don’t use a BGP dump right now so I don’t see that data. I am not sure yet if the bottleneck logic is really an improvement when using collector data since it can also be an attack vector. But I haven’t had enough time to form an opinion. I will compare datasets with and without bottleneck logic to see how much of the difference it explains. The ROAs authenticate the announcing ASN so it will probably be
<fjahr> the last one usually. Everything else being equal I am not sure what might be better: an authenticated source or and unauthenticated bottleneck. But I have a feeling that some of this will also be resolved via the multimap stuff, i.e. if one ASN is a bottleneck to another ASN, I think there is a high chance that these ASNs are the same entity or related and then (if they use RPKI) hopefully they also authenticate both ASNs
<fjahr> for the IP block that the one ASN is the bottleneck for. But this already on my list to explore further as well :)
<sipa> Ah, I see. That would be a big reason why we'd see differences between the resulting data.
<sipa> I'll redo but without the bottleneck logic, and see if that results in a smaller diff.
<sipa> Yeah, I'm not sure what's best either (unauthenticated bgpdump with bottleneck analysis, or authenticated data source that doesn't support that).
ghost43 has quit [Remote host closed the connection]
ghost43 has joined #bitcoin-core-dev
<fjahr> Thanks! Yeah, maybe even both things can be combined in some way that also requires more research.
yanmaani has joined #bitcoin-core-dev
yanmaani has quit [Remote host closed the connection]
yanmaani has joined #bitcoin-core-dev
brunoerg has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
<bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/ea41abade4b7...b3ef3291993d
<bitcoin-git> bitcoin/master fa486de MarcoFalke: ci: Cache package manager install step
<bitcoin-git> bitcoin/master b3ef329 MarcoFalke: Merge bitcoin/bitcoin#26976: ci: Cache package manager install step
<bitcoin-git> [bitcoin] MarcoFalke merged pull request #26976: ci: Cache package manager install step (master...2301-ci-cache-apt-đźš´) https://github.com/bitcoin/bitcoin/pull/26976
Yihen has quit [Ping timeout: 252 seconds]
<bitcoin-git> [bitcoin] fanquake opened pull request #27027: build: use _FORTIFY_SOURCE=3 (master...fortify_source_3) https://github.com/bitcoin/bitcoin/pull/27027
<bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/b3ef3291993d...c2028f98aec3
<bitcoin-git> bitcoin/master fa6986a MarcoFalke: ci: Print iwyu patch in git diff format
<bitcoin-git> bitcoin/master c2028f9 fanquake: Merge bitcoin/bitcoin#27012: ci: Print iwyu patch in git diff format
<bitcoin-git> [bitcoin] fanquake merged pull request #27012: ci: Print iwyu patch in git diff format (master...2302-ci-iwyu-git-diff-đź“–) https://github.com/bitcoin/bitcoin/pull/27012
AaronvanW has quit [Remote host closed the connection]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 248 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 252 seconds]
b_101 has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 260 seconds]
b_101 has joined #bitcoin-core-dev
MrFrancis has joined #bitcoin-core-dev
<instagibbs> with nanobench is there a way to do expensive prep outside of actual measured run? or do something like a "lap" functionality to see how long each section takes?
<instagibbs> read the docs and couldnt find anything like that
<fanquake> think you can just do it before calling bench.run()?
b_101 has quit [Ping timeout: 248 seconds]
<instagibbs> if it doesn't have to be redone yeah
b_101 has joined #bitcoin-core-dev
<sipa> i was going to suggest the same; if that's not what you mean, I'm not sure I understand the question
<instagibbs> If there's expensive prep that is blown away by each run, is there a way to do the expensive prep i nthe loop but not count towards the bench
<instagibbs> Alternative question: is there a cheap way to clone a CTxMemPool :)
<bitcoin-git> [bitcoin] MarcoFalke opened pull request #27028: ci: Cache more stuff in the ci images: msan, iwyu, pip, sdks (master...2301-ci-cache-apt-đźš´) https://github.com/bitcoin/bitcoin/pull/27028
<sipa> Create two benchmarks, one with just the prep step, and another with both the prep and the benchmark?
<instagibbs> true
b_101 has quit [Ping timeout: 248 seconds]
AaronvanW has joined #bitcoin-core-dev
<sipa> it may very well destroy all precision your benchmark has, though
b_101 has joined #bitcoin-core-dev
<sipa> because you'll be subtracting two very close numbers both with relative errors
SpellChecker has quit [Ping timeout: 255 seconds]
SpellChecker_ has joined #bitcoin-core-dev
<instagibbs> depends on how bad this scales, but noted
b_101 has quit [Ping timeout: 252 seconds]
<bitcoin-git> [bitcoin] hebasto closed pull request #23609: build: Add -fvisibility=hidden flag for macOS host (master...211126-reduce) https://github.com/bitcoin/bitcoin/pull/23609
roze_paul has joined #bitcoin-core-dev
<bitcoin-git> [bitcoin] fanquake opened pull request #27029: guix: consolidate to glibc 2.27 for Linux builds (master...glibc_2_27) https://github.com/bitcoin/bitcoin/pull/27029
<bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/c2028f98aec3...7753efcbcf3c
<bitcoin-git> bitcoin/master fab9f7d MarcoFalke: test: Use std::unique_ptr over manual delete in coins_tests
<bitcoin-git> bitcoin/master 7753efc fanquake: Merge bitcoin/bitcoin#27004: test: Use std::unique_ptr over manual delete ...
<bitcoin-git> [bitcoin] fanquake merged pull request #27004: test: Use std::unique_ptr over manual delete in coins_tests (master...2301-test-uniq-ptr-đź•´) https://github.com/bitcoin/bitcoin/pull/27004
sudoforge has quit [Ping timeout: 265 seconds]
qxs has quit [Remote host closed the connection]
qxs has joined #bitcoin-core-dev
<sipa> @fjahr I probably won't be around for today's IRC meeting.
FrancisMr has joined #bitcoin-core-dev
<fjahr> sipa: I am sure it won't be the last on this topic, thanks for the useful feedback so far!
MrFrancis has quit [Ping timeout: 248 seconds]
salvatoshi has quit [Ping timeout: 252 seconds]
as2333 has joined #bitcoin-core-dev
Guest7 has joined #bitcoin-core-dev
Guest7 has quit [Client Quit]
yanmaani has quit [Ping timeout: 255 seconds]
yanmaani has joined #bitcoin-core-dev
roze_paul has quit [Quit: https://quassel-irc.org - Chat comfortably. Anywhere.]
Norrin has quit [Remote host closed the connection]
qxs has quit [Remote host closed the connection]
qxs has joined #bitcoin-core-dev
Guest45_ has joined #bitcoin-core-dev
Talkless has joined #bitcoin-core-dev
Guest45_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
FrancisMr has quit [Ping timeout: 252 seconds]
FrancisMr has joined #bitcoin-core-dev
Guest45_ has joined #bitcoin-core-dev
pablomartin has joined #bitcoin-core-dev
hernanmarino has joined #bitcoin-core-dev
Guest45_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Guest45_ has joined #bitcoin-core-dev
b_101 has joined #bitcoin-core-dev
<MacroFake> hi
<warren> hi
<achow101> #startmeeting
<core-meetingbot> Meeting started Thu Feb 2 19:00:22 2023 UTC. The chair is achow101. Information about MeetBot at https://bitcoin.jonasschnelli.ch/ircmeetings.
<core-meetingbot> Available commands: action commands idea info link nick
<brunoerg> hi
<achow101> #bitcoin-core-dev Meeting: achow101 aj amiti ariard b10c BlueMatt cfields Chris_Stewart_5 darosior digi_james dongcarl elichai2 emilengler fanquake fjahr gleb glozow gmaxwell gwillen hebasto instagibbs jamesob jarolrod jb55 jeremyrubin jl2012 jnewbery jonasschnelli jonatack jtimon kallewoof kanzure kvaciral laanwj larryruane lightlike luke-jr maaku marcofalke meshcollider michagogo moneyball morcos nehan NicolasDorier paveljanik petertodd
<achow101> phantomcircuit promag provoostenator ryanofsky sdaftuar sipa vasild
<hebasto> hi
<fjahr> hi
<jon_atack> hi
<furszy> hi
<lightlike> hi
<achow101> There are two preproposed meeting topics: ASMap in the release process (fjahr), Self-hosted Gitlab instance (fjahr)
<achow101> anything else to add to the list?
<glozow> hi
<pinheadmz> hi
<achow101> #topic high priority for review
<core-meetingbot> topic: high priority for review
<MacroFake> for me plz: https://github.com/bitcoin-core/gui/pull/697 (Remove reindex special case from the progress bar label)
<larryruane> hi
<achow101> anything to add/remove/merge?
<kanzure> hi
<achow101> MacroFake: done
<MacroFake> thanks!
<achow101> #22693 and #25740 are both recently needs rebase
<gribble> https://github.com/bitcoin/bitcoin/issues/22693 | RPC/Wallet: Add "use_txids" to output of getaddressinfo by luke-jr · Pull Request #22693 · bitcoin/bitcoin · GitHub
<gribble> https://github.com/bitcoin/bitcoin/issues/25740 | assumeutxo: background validation completion by jamesob · Pull Request #25740 · bitcoin/bitcoin · GitHub
darosior has quit [Read error: Connection reset by peer]
darosior has joined #bitcoin-core-dev
<achow101> #topic ASMap in the release process (fjahr)
<core-meetingbot> topic: ASMap in the release process (fjahr)
<fjahr> Hi, first of all, sorry about the wordy write-up, I should have included a tldr; I just wrote a quick one for those that didn’t have the time to read it. The main points I am making are: 1. In order to ship an asmap file with the release we want to make sure it’s the best quality data. The main way to QA was assumed to be based on diffing files. I tried to find a systematical way to do QA via diffs so I don’t think it
<fjahr> is a reliable option. 2. Instead we should make the asmap creation process transparent and reproducible. The asmap file should be treated like a PR, it will be presented and reviewers can test and run QA on it as they like but that is not part of the release process or responsibility of maintainer. 3. Quality of the input data is even more important. I believe that for our purpose rpki > irr > collectors, in terms of
<fjahr> quality. So far only collectors were considered as an input source.
<fjahr> Thanks to everyone who provided feedback so far! There was some nice feedback from sipa here earlier today, it shows that in the process of joining the input data the devil is in the details and there is still more research to be done but I think it’s moving in the right direction. Also my code is buggy ^^
<achow101> doesn't getting asmap data require having your own AS?
<fjahr> I would like to answer any questions and take feedback on what is bothering people.
<sipa> hi, i'm half here
<fjahr> achow101: there are lots of open sources for data. the approaches we are discussing are all built on those.
<achow101> i should probably read the backscroll
<fjahr> But if you have any way to open a BGP session, you can collect the global feed youself and for some people that may be a preferred aproach
<fjahr> But most people probably won't be able to do it
<fjahr> Achow101: the "general knowledge" section in the write-up should be interesting :)
<sipa> As I understand it, there are 2 main ways of building the data: (a) from BGP routes (which requires being your own BGP node, or getting trusted data from someone dumping it) or (b) authoritatively from the assigmed mappings directly.
<brunoerg> > The main way to QA was assumed to be based on diffing files
<brunoerg> I agree, but what should we compare?
<fjahr> My opinion the diffing question: That’s probably the most controversial point among those that have dove deeper into the topic. In short: I just couldn’t find any way to develop a concise set of rules that could be giving a clear indication if a file is safe/unsafe and I tried really hard (TM). So I think if we tell people they can run a diff and there is a X% match on metric Y that means it’s safe, then we give
<fjahr> people a false sense of security at best, at worst the release process becomes a mess because people are unsure if the numbers mean the file is safe or not and then nobody ends up using it. I am happy to review if people have suggestions for this but if we wait for this then the process will drag on for a while IMO.
darosior has quit [Ping timeout: 248 seconds]
<sipa> FWIW, I concur that diffing isn't viable as a means of QA (or not the primary one at least), the differences from maps constructed from one day to another often have substantial changes already (more than you as a human could look at and judge "this looks reasonable" or not).
<fjahr> So, I think "we" as in as a part of the release process shouldn't compare anything IMO
<lightlike> fjahr: but what should a reviewer do in order to ack the PR?
<fjahr> But anyone can run experiments, that's what the review phase is for
<brunoerg> I made an experiment generating 2 files from 2 last Core's release date and there was A LOT of changes
<fjahr> lightlike: that's up to the reviewer, I mean I don't tell people how to review my PRs
<fjahr> of course there will be knowledge sharing and tools written for this
<fjahr> but people can not expect Bitcoin core maintainers to do that work for them
<sipa> As I understand it, @fjahr's suggesting is that we effectively establish a reproducible process for building the maps, so hopefully it suffices for some reviewers to actually reproduce it
<fjahr> brunoerg: yepp, and that's the problem
<sipa> Though what that process would be seems a bit unspecified for now?
<brunoerg> @fjahr: but in that PR you would tell how you generated that file in order to reviewers to follow the same step?
<fjahr> sipa: will need to be specified a lot more in detail, yes
<fjahr> brunoerg: from the same raw files, yes
<jamesob> achow101: thanks for the ping, I'll rebase AU momentarily
<brunoerg> cool
<fjahr> the raw files are also uploaded in the zip file
<achow101> that doesn't really solve the problem though, the raw files could just be malicious
<sipa> Maybe a bit more practical, do you expect that the txt range/asn map file will become part of the bitcoin core source repo? or just the combined binary dat file? or neither?
<sipa> binary file is roughly 1.2 MB right now
<fjahr> For the RPKI data the only way it could be malicious is there could be stuff missing
<fjahr> everything else can be malicious, yes, but that the general crux with BGP
<fjahr> so you can be Amazon or the NSA and still end up with a BGP hijack in your routing table
Guest45_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<sipa> RPKI is signed, right?
<fjahr> sipa: I think not in the source, I mean it's possible, but it seems more elegant to have the candidates in asmap-data seperately and only join it during build process
<fjahr> sipa: yes
<achow101> do we expect asmap to be in our release distributions?
<fjahr> The easiest way to explain it is: RPKI is SSL for BGP (very roughly)
<fjahr> achow101: that is kind of the goal
<sipa> @achow101 My hope is that eventually it'll end up being part of the binary, or a separately-distributed file that's part of the installation and used by default
<fjahr> *end goal, there may be intermittend steps where it's not the default etc.
<fjahr> The one downside of RPKI is it's not deployed everywhere so it can not be the only data source
<sipa> but we could use RPKI as primary, and only use other sources for ranges for which RPKI is not available?
<fjahr> yes, that is the approach I build with kartograf
Guest45_ has joined #bitcoin-core-dev
<fjahr> A few people have asked this: yes, I will also post this to the ML soon, just wanted to hear from this group first, particular potential pushback on the release process stuff
darosior has joined #bitcoin-core-dev
<fjahr> Any more questions? I am happy to answer at any time here or in DMs, I hope we can get as many people onboard on this problem space as possible to get more review :)
<achow101> thanks for reviving this project. I'll have to do some more reading before I can give an feedback
<fjahr> achow101: great, thanks!
<achow101> #topic Self-hosted Gitlab instance (fjahr)
<core-meetingbot> topic: Self-hosted Gitlab instance (fjahr)
<fjahr> Err, basically I just wanted to repeat what I said earlier: Just a quick announcement/invite if someone wants to get access and try stuff out, also happy to share admin access with the people I know, it’s a playground. Currently some key features are missing because it’s on the Community Plan but there are open source specific licenses and Mike Schmidt has been in contact with them already about an Enterprise license,
<fjahr> we’ll probably coordinate more on that soon. But I am a more focussed on ASMap for now tbh.
<fjahr> It’s here: gitlab.sighash.org, but requires registration to see anything currently. No need for an email confirmation so you can enter whatever you want. Just ping me because I have to confirm you. I haven’t figured out all the settings so hopefully this will change as well soon.
<fjahr> That's already it, I wasn't present at the last core dev but I read the notes on the topic so I thought it might be helpful
Guest45_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<MacroFake> fjahr: Would be good to answer the basic questions from the wiki
<achow101> we have a wiki page with some links to older discussions on this topic: https://github.com/bitcoin-core/bitcoin-devwiki/wiki/GitHub-alternatives-for-Bitcoin-Core
<fjahr> right, yeah, so if anyone wants to help me answer these questions, let me know :D
<jon_atack> achow101: i think the idea was to include the ASMap data in the distributed binary but not in the repository. discussion about this took place here: https://www.erisian.com.au/bitcoin-core-dev/log-2019-06-20.html#l-632
<jon_atack> (sorry, catching up)
<achow101> any other topics to discuss?
<jon_atack> achow101: mind adding https://github.com/bitcoin/bitcoin/pull/26283 to the blockers list?
<jon_atack> #26283
<gribble> https://github.com/bitcoin/bitcoin/issues/26283 | p2p: Fill reconciliation sets and request reconciliation (Erlay) by naumenkogs · Pull Request #26283 · bitcoin/bitcoin · GitHub
Guest45_ has joined #bitcoin-core-dev
<jon_atack> seems to be in final review phase, i plan to get to it
<achow101> jon_atack: done
<jon_atack> achow101: ty
<achow101> #endmeeting
<core-meetingbot> topic: Bitcoin Core development discussion and commit log | Feel free to watch, but please take commentary and usage questions to #bitcoin | Channel logs: http://www.erisian.com.au/bitcoin-core-dev/, http://gnusha.org/bitcoin-core-dev/ | Meeting topics http://gnusha.org/bitcoin-core-dev/proposedmeetingtopics.txt / http://gnusha.org/bitcoin-core-dev/proposedwalletmeetingtopics.txt
<core-meetingbot> Meeting ended Thu Feb 2 19:36:53 2023 UTC.
<MacroFake> fjahr: Could add a "Sign in with GitHub" to your Gitlab?
<fjahr> ser, didn't we want to leave GH? ;)
<fjahr> I will check :)
AaronvanW has quit [Quit: Leaving...]
<MacroFake> fjahr: Yeah, not sure if I was trolling
qxs has quit [Remote host closed the connection]
qxs has joined #bitcoin-core-dev
Guest45_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
AlissonM has joined #bitcoin-core-dev
AlissonM has quit [Client Quit]
brunoerg has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
Guest45_ has joined #bitcoin-core-dev
<MacroFake> Basic downside would likely be that someone (who?) would need to host the server, do sysadmin stuff, do anti-DoS, run the CI
scg has quit [Ping timeout: 260 seconds]
<MacroFake> If any of this breaks down, and the person is on holiday, the whole project will be on forced holiday
<bitcoin-git> [bitcoin] hebasto closed pull request #26763: ci: Treat IWYU violations as errors (master...221228-iwyu) https://github.com/bitcoin/bitcoin/pull/26763
<bitcoin-git> [bitcoin] hebasto closed pull request #26766: ci: Use clang-15 and IWYU v0.19 in "tidy" task (master...221228-clang15) https://github.com/bitcoin/bitcoin/pull/26766
brunoerg has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
Talkless has quit [Quit: Konversation terminated!]
<fjahr> MacroFake: Yeah, I have only started to think about this, I think one possible solution could be that multiple people run their own instances and they are syncing the main instance and if the main instance is attacked one of the old followers becomes the new main instance, but it sounds pretty dirty...
pablomartin4btc has joined #bitcoin-core-dev
<MacroFake> Is there support for syncing on GitLab?
<fjahr> I am not sure gitlab is the best solution but it was the easiest to set up and several people seemed eager to try, so I am just experimenting
<fjahr> yeah
pablomartin has quit [Ping timeout: 268 seconds]
<MacroFake> I mean all of the stuff (issues, comments, reviews, ....)
<MacroFake> Doing a git mirror is trivial
<fjahr> quite a lot, between GH and GL, between GL and GL self-hosted/enterprise
<fjahr> yeah all of it as far as I understood, I pull all of that from GH over to my GL instance
<fjahr> it's just stuck now on the state from a week ago because continuous sync is an enterprise license feature and we haven't gotten that yet
Guest45_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Guest45_ has joined #bitcoin-core-dev
Guest45_ has quit [Client Quit]
FrancisMr has quit [Ping timeout: 248 seconds]
b_101 has quit [Ping timeout: 252 seconds]
<fjahr> MacroFake: Given that the syncing between GH and GL seems pretty good, one simple step would be to configure an official mirror on GL (official, not self-hosted) that just keeps syncing. That would at least give a simple backup in case GH becomes hostile but GL doesn't (yet). It's from one central entity to another but nobody has to run infra. Not sure if this was already discussed or maybe someone is already doing this in
<fjahr> a private repo...
b_101 has joined #bitcoin-core-dev
<MacroFake> Running a backup sounds good. (There is a gh-meta backup, but it can't be browsed)
<MacroFake> Though, if GitHub deletes the repo, then GitLab might as well?
<fjahr> sure, it's not a great protection in case of a real purge
bugs_ has joined #bitcoin-core-dev
<MacroFake> Is there a service that hosts GitLab instances and promises to not delete them?
<fjahr> I think Tor does a similar think, I think GL is their main git host and they sync it to GH for visibility, seems to work well judging from outside
<fjahr> hehe, promises are meant to be broken
<fjahr> can we put the code on megaupload? %)
b_101 has quit [Ping timeout: 252 seconds]
<MacroFake> tor only mirrors the .git stuff on gh
b_101 has joined #bitcoin-core-dev
<fjahr> hm, ok, probably they don't people to start commenting in GH thinking it's were the action is. Would be interesting to check if the conversations can be mirror while at the same time participation can be prevented.
<sipa> I don't think our concern isn't really mirrorring the git data. That's easy. It's having a place for development, which means having issues/prs/...
<fjahr> hosting the code in ordinal inscriptions? *hides*
<fjahr> sipa: yeah the syncing of that works well between GH and GL, so at least there could be a quick switch
b_101 has quit [Ping timeout: 252 seconds]
halo has joined #bitcoin-core-dev
Guest5 has joined #bitcoin-core-dev
b_101 has joined #bitcoin-core-dev
___nick___ has quit [Ping timeout: 248 seconds]
halosghost has quit [Ping timeout: 264 seconds]
<fjahr> Maybe a self-hosted GL that follows GH with everything and can't be shut down like a repo on GL.com is the best trade-off, it could be kept private until its needed and then the hosting isn't that big of a challenge while it's not in use and it's not too bad if it goes down for a day. The sync can catch up.
<fjahr> The setup isn't so hard, but the license is needed. And the license stuff also makes me a bit suspicious that GL could shut down a self-hosted instance too, but didn't research that yet.
b_101 has quit [Ping timeout: 248 seconds]
b_101 has joined #bitcoin-core-dev
<MacroFake> which of the 13 gitlab legal entities would be responsible for the project? :sweat-smile: https://about.gitlab.com/company/visiting/
<fjahr> it's a 1 out of 13 multisig probably
salvatoshi has joined #bitcoin-core-dev
sudoforge has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 268 seconds]
b_101 has joined #bitcoin-core-dev
<MacroFake> fjahr: You could start your asmap repo on GitLab to start fetching experience
<MacroFake> I might move some of mine
b_101 has quit [Ping timeout: 268 seconds]
Norrin has joined #bitcoin-core-dev
<fjahr> MacroFake: the downside is it creates a bit bigger barrier for getting other people involved that are primarily on GH, but I will experiment and move something over I think
<MacroFake> fjahr: Well, if that is a reason, then we wouldn't want to move the other repos as well, unless they are already gone?
<fjahr> do you mean moving or mirroring? The mirroring allows to move when the mirrored repo is suddenly gone, no need to move before that.
<fjahr> I understand move as making it the primary place were development and discussion happens
<MacroFake> Yeah, but then it should be a read-only mirror to avoid confusion, unless there is a two-way sync?
<fjahr> there is even two way sync functionality but that sounds like asking for trouble
<MacroFake> heh, right
<fjahr> maybe we should play it through with a fake repo on GH, create some PRs, make som discussion, read-only mirror it on a self-hosted GL and then delete the GH and switch over to the GL mirror and continue development there
Guest45_ has joined #bitcoin-core-dev
b_101 has joined #bitcoin-core-dev
Guest45_ has quit [Client Quit]
<fjahr> I will try this out
Guest3312 has joined #bitcoin-core-dev
bugs_ has quit [Quit: Leaving]
Guest3312 has quit [Client Quit]
Guest2003 has joined #bitcoin-core-dev
pablomartin4btc has quit [Quit: Leaving]
Guest128 has joined #bitcoin-core-dev
halo is now known as alot
brunoerg has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 255 seconds]
Guest2003 has quit [Quit: Client closed]
AaronvanW has joined #bitcoin-core-dev
yanmaani has quit [Ping timeout: 255 seconds]
brunoerg has joined #bitcoin-core-dev
yanmaani has joined #bitcoin-core-dev
hernanmarino has quit [Quit: Leaving]
brunoerg has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
salvatoshi has quit [Ping timeout: 268 seconds]
b_101 has quit [Ping timeout: 252 seconds]
yanmaani has quit [Remote host closed the connection]
b_101 has joined #bitcoin-core-dev
Guest45_ has joined #bitcoin-core-dev
AaronvanW has quit [Quit: Leaving...]
yanmaani has joined #bitcoin-core-dev
Guest45_ has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
Zenton has quit [Ping timeout: 264 seconds]
Zenton has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 252 seconds]
b_101 has joined #bitcoin-core-dev
alot has quit [Quit: WeeChat 3.8]
b_101 has quit [Ping timeout: 260 seconds]
b_101 has joined #bitcoin-core-dev
Guest128 has quit [Quit: Client closed]
yanmaani has quit [Remote host closed the connection]
yanmaani has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 248 seconds]
b_101 has joined #bitcoin-core-dev
brunoerg has quit [Remote host closed the connection]
brunoerg has joined #bitcoin-core-dev
b_101 has quit [Ping timeout: 260 seconds]
b_101 has joined #bitcoin-core-dev