jespada has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
Victor_sueca has quit [Ping timeout: 248 seconds]
eval-exec has quit [Ping timeout: 260 seconds]
Guest36 has quit [Quit: Client closed]
Guyver2 has left #bitcoin-core-dev [Closing Window]
Victor_sueca has joined #bitcoin-core-dev
epic33 has joined #bitcoin-core-dev
epic33 has quit [Client Quit]
kevkevin has joined #bitcoin-core-dev
kevkevin has quit [Ping timeout: 245 seconds]
kevkevin has joined #bitcoin-core-dev
NakedKing has joined #bitcoin-core-dev
<NakedKing>
Hello, i'm trying to find best approach for my app which handles users incoming-outgoing transactions. this is generally a payment system. i want to dedicate a wallet (or a wallet kind) to my every user. i've a big performance problem. i dont store any info at my database. every time i process api calls. And unload all wallets and load relevant wallet makes slow my system
<NakedKing>
chatgpt suggested me something named HD Walllet, and suggested also $this->bitcoin->getHDWallet(); but there is no function like getHDWallet, i guess it was an hallusinotion
<darosior>
For next release's testing guide: it would be interesting to get more testing of the new PCP / NAT-PMP implementation against routers provided by large ISPs. At least in the US from what i read on the PR it was only tested against Verizon and Spectrum provided routers. It seems to be something non-too-technical hobbyists could help with.
<sipa>
good idea
<laanwj>
yes, agree
<sipa>
also, how do we feel about making it on by default?
<sipa>
maybe not for 29.x, but it'd be nice if we could get there
<darosior>
Yes, the unit tests from laanwj, potential more testing outside of us, definitely gives more confidence in doing this. Would be nice to have fuzzing coverage first. I think some of what laanwj did for unit tests could be reused there.
<sipa>
right
<darosior>
Also in testing of the rc it would be nice to know if people had to modify their router's configuration. For instance on Spectrum i had to manually enable PCP support on the router, which kind of kills the purpose.
<sipa>
darosior: did it actually use PCP, or fall back to NAT-PMP?
<darosior>
I assumed it used PCP because i saw no log of falling back to NAT-PMP whatsoever
<laanwj>
fuzzing would be nice, i'm really out of my depth there, but yes id assume a similar setup could be used to inject packets as in the unit tests
<dergoegge>
could be a starting point, never polished it...
<darosior>
I'll look into it, reviewed the unit test PR yesterday so it's still fresh in my mind
<laanwj>
thanks!
<_aj_>
laanwj: hey, you've automated nostr stuff, right? are there any tools/libs you can recommend?
<laanwj>
_aj_: yes, i added nostr notification support to the bot that posts merged PRs (https://github.com/laanwj/ghi), and i have some scripts to post the torrents and release notifications (https://github.com/laanwj/miscellany/tree/main/nostr) - i ended up rolling my own code to sign and broadcast notes because none of the python libraries were actively maintained / documented, for rust there's the nostr-sdk crate which is very usable
<_aj_>
laanwj: hmm, copying your handrolled code sounds more appealing than making/resurrecting my own at least
<_aj_>
laanwj: though i guess yours is write only, and doesn't subscribe to anything?
<darosior>
sipa, sdaftuar: question regarding your anti-DoS header sync work in the context of marcofleon's recent PR to remove checkpoints. We have this check to not send headers to peers unless our tip has enough work, to avoid an IDB node to be fed an invalid chain at checkpoint height and send that to peers which would ban it and it would basically
<darosior>
eclipse itself from the honest network. marcofleon points out this logic can be removed along with the checkpoints as since the anti-DoS header sync work we would only ever have headers to serve that have minimum chain work. That makes sense to me, but then it begs the question why this logic was not removed with the anti-DoS header sync work
<darosior>
itself? Any reason to keep it we'd be missing?
<laanwj>
_aj_: correct, it's write-only, that's the only thing i've ever needed to automate
jespada has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
<_aj_>
darosior: it wasn't removed then out of an abundance of caution, mostly iirc?
<laanwj>
if you also want to subscribe to relays it gets more complicated quickly and it makes sense to use a binding for the rust crate (it's available for python at least)
<sdaftuar>
darosior: that's not quite true, because an adversary can feed you the honest chain during pre-sync, and then a bogus one during redownload
<sdaftuar>
darosior: so you'd detect it quickly and disconnect but be left with headers that are possibly checkpoint-violating (if your node isn't enforcing checkpoints)
<darosior>
ha! thanks. Glad i asked
<_aj_>
sdaftuar: doesn't headerssync fail to pass through headers that don't match presync with high probability?
<sdaftuar>
_aj_: i thought it was like ~4 bits per 2000 headers that you had to match?
<sdaftuar>
oh times 5 or something
* sdaftuar
checks
<_aj_>
24 bits over 14k headers i think?
<dergoegge>
If I'm not mistaken, you wouldn't serve those checkpoint violating headers because they wouldn't pass `PeerManagerImpl::BlockRequestAllowed`? i.e. `pindex->IsValid(BLOCK_VALID_SCRIPTS)` would never be true I think
preimage has joined #bitcoin-core-dev
<sdaftuar>
just chatted offline about this... i think it should be very expensive for an attacker to put a node in a state where its active chain (ie block tip) violates the checkpoints, because i believe we don't download blocks unless they are part of a headers chain that has minchainwork, and we don't process unrequested blocks that are below minchainwork
<sdaftuar>
since we respond to getheaders requests by looking at our active chain (ie the most-work chain based on what blocks we have, not what's in our overall headers tree), i think it ought to be "safe" to remove that check...
<sdaftuar>
however, i dont' think the check costs us much either?
<Sjors[m]>
dergoegge: the BlockRequestAllowed check only happens if there's no locator. IIUC a typical bootstrapping node will start by sending us a GETHEADERS message with the locator pointing to the genesis block.
<dergoegge>
but then you serve from the active chain anyway
<dergoegge>
which won't be the checkpoint violating one
<Sjors[m]>
Right, we construct our reply by iterating ActiveChain().Next
<Sjors[m]>
It still seems a bit indirect to rely on the assumption that we don't download and verify blocks until min chainwork is reached.
<Sjors[m]>
The check indeed seems dirt cheap: m_chainman.ActiveTip()->nChainWork < m_chainman.MinimumChainWork()
jespada has joined #bitcoin-core-dev
Guest42 has joined #bitcoin-core-dev
Guest42 has quit [Quit: Client closed]
lattice has joined #bitcoin-core-dev
preimage has quit [Ping timeout: 245 seconds]
Guest55 has left #bitcoin-core-dev [#bitcoin-core-dev]
tstearns has joined #bitcoin-core-dev
bugs_ has joined #bitcoin-core-dev
fmenezes has joined #bitcoin-core-dev
Talkless has joined #bitcoin-core-dev
vasild has quit [Ping timeout: 264 seconds]
fmenezes has quit [Quit: Client closed]
<darosior>
Re turning on PCP by default, how much should we be concerned about potential bandwidth cost to users of default Bitcoin Core? Do we have stats of average cost of running a listening node? Has this been a concerned raised in the past back when UPnP was enabled by default?
kevkevin has quit [Remote host closed the connection]
kevkevin has joined #bitcoin-core-dev
kevkevin has quit [Ping timeout: 252 seconds]
yonson has joined #bitcoin-core-dev
jespada has quit [Quit: My Mac has gone to sleep. ZZZzzz…]
kevkevin has quit [Remote host closed the connection]
jespada has joined #bitcoin-core-dev
jespada has quit [Client Quit]
kevkevin has joined #bitcoin-core-dev
<bitcoin-git>
[bitcoin] yancyribbens opened pull request #31656: test: Add expected result assertions (master...add-expected-results-to-coin-grinder-tests) https://github.com/bitcoin/bitcoin/pull/31656
mudsip has joined #bitcoin-core-dev
mudsip has quit []
Cory64 has quit [Quit: Client closed]
Cory64 has joined #bitcoin-core-dev
<laanwj>
darosior arguably they'd already be running a listening node but it's not connectable due to their router; if listening is disabled, so will the port mapping. But yes, maybe one of the initial GUI questions should be whether to enable listening. On the other hand i doubt people concerned about bandwidth run bitcoin core at all
Talkless has quit [Quit: Konversation terminated!]
<laanwj>
it's true that bandwidth requirements were lower when UPnP was default enabled, as the block chain wasn't that big yet
<laanwj>
but that was never the reason to disable it, or afaik even discussed at the time, the only reason was miniupnp's insecurity
<darosior>
Yes i don't expect bandwidth to be a big concern, but it depends on the load and i have no idea what it is today for a listening node
<bugs_>
as more people leave the cities for the countryside more will become bandwidth concerned :-)
<sipa>
do we have statistics for total up/down network volume somewhere?
<laanwj>
if the bandwidth use is too large it shouldnt be dependent on enabling port forwarding, people running bitcoin core on computers with a public IP could just as well run into trouble because of bandwidth
<laanwj>
i just think it's seperate concerns
Guest89 has joined #bitcoin-core-dev
<lightlike>
seems like the potential bandwidth increase should be mentioned in the release notes. There are probably a number of nodes are currently implicitly listen-only (they know they are not reachable and therefore don't bother to set listen=0 even though they want it) - these would need to change their configuration.
<laanwj>
if you're concerned about bandwidth of incoming connections you should disable listening, not disable port forwarding
<laanwj>
sure, enabling it by default would need to be mentioned in the release notes
<sipa>
or enable the maxuploadtarget functionalityu
<laanwj>
exactly
<sipa>
absolutely worth mentioning in the release notes, but also, we cannot really predict to what extent people might be impacted by this
<sipa>
and no way to obtain data about it except trying it
<laanwj>
it might also be that having more connectable nodes reduces the bandwidth requirements for each one, as the load is spread out more; new nodes need to sync from somewhere, after all
<laanwj>
at least, if they're not all pruning nodes :)
<darosior>
Yes, if IBD nodes choose peers to download from randomly. The other reasonable thing for an IBD node to do would be to download from faster peers, which are most likely those in a datacenter and not the ones listening behind a router at home, so it wouldn't impact those.
<laanwj>
would you want to prefer nodes in datacenters? one of the reasons to do this in the first place is to be less dependent on the large datacenters
<sipa>
for IBD, i don't think it matters much
<laanwj>
right, sure
<darosior>
I was just trying to come up with reasons not to enable PCP by default besides the increased attack surface. I agree users concerned about bandwidth should leverage the bandwidth reduction options. It also seems unlikely that a user would 1) be concerned about bandwidth 2) be indirectly relying on its firewall instead of -listen=0 3) would see a
<darosior>
massive increase of his bandwidth usage.
<sipa>
for steady state, i think having a large decentralized corpus of reachable nodes is ideal
<darosior>
Yes, also for headers
<sipa>
it's also possible that there are just a ton of PCP-reachable nodes on crappy internet, and having those join the set of reachable nodes makes the IBD experience worse for new nodes (which could potentially be mitigated by improved IBD block-fetching logic)
<darosior>
But for downloading the bulk of the chain i think it'd be fine from the point of view of the IBDing node to choose to download from the fastest peers (us implementing that by default in Bitcoin Core might have undesirable effects though..)
<sipa>
darosior: the stall-detection effectively functions as a weak "download from fastest peers" mechanism
<lightlike>
I'm not sure if nodes should be too aggressive with peer selection during IBD, actively dropping all but the fastest peers. There is some benefit to distribute bandwidth / costs over the entire network, even if IBD is a bit slower for the individual node.
Guest89 has quit [Quit: Client closed]
ciaox has joined #bitcoin-core-dev
<laanwj>
i don't think IBD is limited by download speed for most users, but by validation speed, which is limited by CPU and database disk i/o, picking fast peers makes sense but not necessarily the fastest possible ones
Cory64 has quit [Quit: Client closed]
Cory64 has joined #bitcoin-core-dev
ciaox has quit [Changing host]
ciaox has joined #bitcoin-core-dev
<ciaox>
I agree with that. I think for most users, IBD bottlenecks more on validation speed rather than download speed—CPU and disk I/O seem to be the main culprits. So while picking fast peers is important, it doesn’t necessarily have to be the absolute fastest. A balance between speed and reliability would probably be more effective without