<roasbeef>
does that figure look accurate to ppl? prepping to run a script on my own nodes peer.json/dat rn
<roasbeef>
background is that in lnd 0.13 we started to "officially" support pruned nodes by fetching blocks we need from the p2p network if the node doesn't have it
<roasbeef>
we get those peers from the connected set of the pruned node
fulldecent has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
<roasbeef>
but we're seeing issues where many of their peers are actually _also_ pruned, so lnd can't get the blocks it needs
<roasbeef>
I recall there was something lofty proposal for nodes to start advertising their block range/ guessing that never fully took off?
brunoerg has joined #bitcoin-core-dev
roconnor has quit [Read error: Connection reset by peer]
brunoerg has quit [Ping timeout: 250 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 240 seconds]
Guest2954 has joined #bitcoin-core-dev
Guest2954 has quit [Client Quit]
earnestly has joined #bitcoin-core-dev
fulldecent has joined #bitcoin-core-dev
rex4539 has quit []
rex4539 has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
<sipa>
roasbeef: there were some ideas around that, but i think that's what eventually turned into the NETWORK_LIMITED proposal
brunoerg has quit [Ping timeout: 246 seconds]
<sipa>
roasbeef: do you mean nodes look like they're pruned but advertizing it incorrectly?
<roasbeef>
sipa: no I think they're advertising it correctly, just this situation where these pruned btcpay bitcoind nodes need to get blocks from peers, but turns out all their peers are actually pruned themselves
<roasbeef>
ran a script of one of my own btcd nodes, and out of the 37k peers in its peers.json, 1606 of them (4%) signal both witness and aren't pruned
<roasbeef>
this is just addrs it has, no guarantee all of them are actually live/reachable/listening
<sipa>
perhaps this is more a problem that too few of the addrs you learn are reachable/listening?
<sipa>
(rather than too few of those listening/reachable ones being non-pruned)
<roasbeef>
perhaps, but this isn't just my node, luke's stats (however they calcuated) show a similar percentage when taking into account all nodes he's ever seen
<roasbeef>
basically have bene trying to get to the bottom of this weird bug for btcpay server, and turns out it's that the nodes of all these users are only connected to other pruned nodes for the most part
<sipa>
interesting
<sipa>
so is there some sort of clustering behavior, where pruned nodes are more likely to be connected to other pruned nodes than average?
<roasbeef>
yeh not sure, perhaps? our code will also periodically poll getnodeaddresses as well to account for the fact that a node amy just have a set of poor connections
<luke-jr>
well, with 8 outbound connections, that's only ~12%, so if archive nodes are <12%, it's fairly possible you might just not have one?
<roasbeef>
but seems in practice that isn't helping out much either
<sipa>
my own long-running node seems to have mostly pruned peers as outbounds as well
<roasbeef>
the plot thickens...
brunoerg has joined #bitcoin-core-dev
ghost43 has quit [Remote host closed the connection]
<jeremyrubin>
We could add a rule to give inbound priority to unpruned nodes? It's not gameable since we can ask for a random piece of txdata as a challenge response game
<jeremyrubin>
E.g. oh you say you're a full node? Give me the nth txn in block at height x
grettke has joined #bitcoin-core-dev
<jeremyrubin>
Then you can spv check it and ban if not correct
<jeremyrubin>
It isn't bad overhead since this is for outbound connections from the full node, so presumably were ok making that more expensive to do
<jeremyrubin>
This can even be done on some basis (e.g. daily?) to ensure the node doesn't switch to pruned
<sipa>
you can't ask for arbitrary transactions
<sipa>
only full blocks
earnestly has quit [Read error: Connection reset by peer]
<luke-jr>
jeremyrubin: if I have a full node, I can pass on the challenge ;)
<luke-jr>
the real questionis why a node should insist on a peer being archival, unless it is in IBD..
earnestly has joined #bitcoin-core-dev
fearbeag has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 268 seconds]
brunoerg has joined #bitcoin-core-dev
<sipa>
right
brunoerg has quit [Ping timeout: 245 seconds]
<achow101>
are you sure they're all pruned? NETWORK and NETWORK_LIMITED aren't mutually exclusive
<sipa>
oh!
<achow101>
we advertise both without pruning, and just NETWORK_LIMITED if pruned
<sipa>
only 3 out of my 10 outbound peers don't have NETWORK
grettke has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
achow101 has quit [Quit: Bye]
achow101 has joined #bitcoin-core-dev
fearbeag is now known as seaner
sudoforge has quit [Quit: 404]
RDK has joined #bitcoin-core-dev
rex4539 has quit []
rex4539 has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
kcalvinalvin has quit [*.net *.split]
da2ce7 has quit [*.net *.split]
brunoerg has quit [Ping timeout: 245 seconds]
kcalvinalvin has joined #bitcoin-core-dev
da2ce7 has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 246 seconds]
brunoerg has joined #bitcoin-core-dev
cmirror has quit [Remote host closed the connection]
cmirror has joined #bitcoin-core-dev
RDK has quit [Quit: Leaving]
brunoerg has quit [Ping timeout: 250 seconds]
mikehu44 has quit [Ping timeout: 268 seconds]
yanmaani has quit [Ping timeout: 276 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 240 seconds]
grettke has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 268 seconds]
brunoerg has joined #bitcoin-core-dev
<jeremyrubin>
sipa: if you can request a block you can definitely request a txn?
<jeremyrubin>
luke-jr if you run archival it's low cost for you to do the challenge and it's only outbound peers (one you picked) doing the challenge also gets you a slot on that node you wouldn't otherwise get
<jeremyrubin>
A node should try to know how to reach one archival node to help new nodes join the network
brunoerg has quit [Ping timeout: 250 seconds]
kexkey_ has joined #bitcoin-core-dev
kexkey has quit [Ping timeout: 256 seconds]
jarthur_ is now known as jarthur
vysn has joined #bitcoin-core-dev
earnestly has quit [Ping timeout: 256 seconds]
brunoerg has joined #bitcoin-core-dev
fulldecent has quit [Quit: My MacBook has gone to sleep. ZZZzzz…]
bitcoin-git has left #bitcoin-core-dev [#bitcoin-core-dev]
brunoerg has quit [Ping timeout: 268 seconds]
goatpig has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 250 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 264 seconds]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 240 seconds]
<laanwj>
jnewbery: thanks
brunoerg has joined #bitcoin-core-dev
vincenzopalazzo has quit [Quit: Bridge terminating on SIGTERM]
devrandom has quit [Quit: Bridge terminating on SIGTERM]
kakolainen[m] has quit [Quit: Bridge terminating on SIGTERM]
ademan[m] has quit [Quit: Bridge terminating on SIGTERM]
awesome_doge has quit [Quit: Bridge terminating on SIGTERM]
cotsuka has quit [Quit: Bridge terminating on SIGTERM]
kvaciral[m] has quit [Quit: Bridge terminating on SIGTERM]
stick[m] has quit [Quit: Bridge terminating on SIGTERM]
Enki[m] has quit [Quit: Bridge terminating on SIGTERM]
Murch[m] has quit [Quit: Bridge terminating on SIGTERM]
siv2r[m] has quit [Quit: Bridge terminating on SIGTERM]
objdefined[m] has quit [Quit: Bridge terminating on SIGTERM]
RCasatta[m] has quit [Quit: Bridge terminating on SIGTERM]
merkle_noob[m] has quit [Quit: Bridge terminating on SIGTERM]
kvaciral[m] has joined #bitcoin-core-dev
vincenzopalazzo has joined #bitcoin-core-dev
devrandom has joined #bitcoin-core-dev
stick[m] has joined #bitcoin-core-dev
Enki[m] has joined #bitcoin-core-dev
Murch[m] has joined #bitcoin-core-dev
merkle_noob[m] has joined #bitcoin-core-dev
siv2r[m] has joined #bitcoin-core-dev
cotsuka has joined #bitcoin-core-dev
kakolainen[m] has joined #bitcoin-core-dev
objdefined[m] has joined #bitcoin-core-dev
awesome_doge has joined #bitcoin-core-dev
ademan[m] has joined #bitcoin-core-dev
RCasatta[m] has joined #bitcoin-core-dev
connecta has joined #bitcoin-core-dev
aitorjs has joined #bitcoin-core-dev
aitorjs has quit [Client Quit]
Murch[m] has quit [Quit: Client limit exceeded: 20000]
kvaciral[m] has quit [Quit: Client limit exceeded: 20000]
devrandom has quit [Quit: Client limit exceeded: 20000]
stick[m] has quit [Quit: Client limit exceeded: 20000]
vincenzopalazzo has quit [Quit: Client limit exceeded: 20000]
cotsuka has quit [Quit: Client limit exceeded: 20000]
siv2r[m] has quit [Quit: Client limit exceeded: 20000]
merkle_noob[m] has quit [Quit: Client limit exceeded: 20000]
Enki[m] has quit [Quit: Client limit exceeded: 20000]
jonatack has quit [Quit: Connection closed]
jonatack has joined #bitcoin-core-dev
connecta has quit [Ping timeout: 256 seconds]
bitdex has quit [Quit: = ""]
Guyver2 has joined #bitcoin-core-dev
fulldecent has joined #bitcoin-core-dev
goatpig has quit [Remote host closed the connection]
z9z0b3t1c has joined #bitcoin-core-dev
masta`` has quit [Read error: Connection reset by peer]
masta`` has joined #bitcoin-core-dev
<sipa>
jeremyrubin: getdata for transactions only works for recently announced ones
kvaciral[m] has joined #bitcoin-core-dev
vincenzopalazzo has joined #bitcoin-core-dev
devrandom has joined #bitcoin-core-dev
stick[m] has joined #bitcoin-core-dev
Murch[m] has joined #bitcoin-core-dev
Enki[m] has joined #bitcoin-core-dev
siv2r[m] has joined #bitcoin-core-dev
merkle_noob[m] has joined #bitcoin-core-dev
cotsuka has joined #bitcoin-core-dev
goatpig has joined #bitcoin-core-dev
Guyver2_ has joined #bitcoin-core-dev
<MarcoFalke>
Assuming I have a minisketch with capacity 1, two entries. And another one with no entries. Does Decode(1) on the difference return false or true?
<MarcoFalke>
I'd expect false, but it does return true in my fuzz test
Guyver2 has quit [Ping timeout: 256 seconds]
Guyver2_ is now known as Guyver2
<sipa>
MarcoFalke: is one of the entries 0?
<MarcoFalke>
The entries are 4193909242 and 1
<MarcoFalke>
the large one is f9f9f9fa in hex
brunoerg has quit [Remote host closed the connection]
<sipa>
there is always a probability that decoding succeeds even if the capacity is exceeded
<sipa>
the fuzzer may have found an input for which that is the case
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 245 seconds]
masta`` has quit [Read error: Connection reset by peer]
<jeremyrubin>
You can't get dos'd through this because it's only for outbound peers from the prover, and we only really need to do a challenge every now and again. You could of course just send the whole block but that seems a waste of bandwidth
<jeremyrubin>
You can limit how often you're willing to re verify
AaronvanW has quit [Remote host closed the connection]
<luke-jr>
why a merkle proof?
<luke-jr>
just send the txid?
<luke-jr>
or heck, it could just be a random 16 bytes from the block (though that forces a specific serialisation so maybe worse)
<luke-jr>
but *any* proof is going to be MITM-able, and why is there a desire to do so anyway?
sudoforge has joined #bitcoin-core-dev
<sipa>
you could do better actually
<sipa>
send a challenge
<sipa>
and a txid
<sipa>
and have them respond with hmac(key=challenge, msg=tx)
<sipa>
(still unconvinced this is desirable at all, just pointing out that one can prove to have a tx without building a merkle proof for it, if the challenger already has the tx as well)
<jeremyrubin>
The verifier runs a pruned node
<jeremyrubin>
They have headers but not the tx, that's why the challenge is the nth transaction at some block
<sipa>
ah, right
bitcoin-git has joined #bitcoin-core-dev
<bitcoin-git>
[bitcoin] ryanofsky opened pull request #23497: Add `src/node/` and `src/wallet/` code to `node::` and `wallet::` namespaces (master...pr/names) https://github.com/bitcoin/bitcoin/pull/23497
bitcoin-git has left #bitcoin-core-dev [#bitcoin-core-dev]
<jeremyrubin>
W.r.t. it being MITM-able, the challenge is forward able sure. But only to a dishonest archival node.
<jeremyrubin>
You can also do a fair coin flip type thing to select the block/txn so that it guarantees there's no possibility of forwarding the request to any honest node even w rate limit
<jeremyrubin>
But tbh this is too in the weeds.
<jeremyrubin>
The main question is is it desirable to ensure that every pruned node has a connection to at least one archival node?
<sipa>
i'm not sure that it is
<sipa>
(also not sure that it isn't)
gene has quit [Quit: gene]
AaronvanW has joined #bitcoin-core-dev
<jeremyrubin>
Is it desirable to ensure that a member of the graph of archival nodes willing to serve blocks is easily discoverable for new peers?
gene has joined #bitcoin-core-dev
<sipa>
nodes rumour peer addresses regardless of whether they're node_network or not
<sipa>
so unless there is some reason to believe there is a clustering effect that we want to counteract, i'm not sure that's a useful criterion
sudoforge has quit [Ping timeout: 256 seconds]
sudoforge has joined #bitcoin-core-dev
sudoforge has quit [Ping timeout: 264 seconds]
sudoforge has joined #bitcoin-core-dev
sudoforge has quit [Ping timeout: 264 seconds]
sudoforge has joined #bitcoin-core-dev
sudoforge has quit [Read error: Connection reset by peer]
gnaf has joined #bitcoin-core-dev
gnaf has quit [Client Quit]
AaronvanW has quit [Ping timeout: 260 seconds]
sudoforge has joined #bitcoin-core-dev
masta`` has joined #bitcoin-core-dev
Talkless has joined #bitcoin-core-dev
Guest78 has joined #bitcoin-core-dev
<Guest78>
าีั
<Guest78>
kuy
Guest78 has quit [Client Quit]
bitcoin-git has joined #bitcoin-core-dev
<bitcoin-git>
[bitcoin] theStack opened pull request #23498: test: remove unnecessary block rehashing prior to solving (master...202111-test-remove_superflous_block_rehashs_before_solving) https://github.com/bitcoin/bitcoin/pull/23498
bitcoin-git has left #bitcoin-core-dev [#bitcoin-core-dev]
bitcoin-git has left #bitcoin-core-dev [#bitcoin-core-dev]
brunoerg has quit [Ping timeout: 264 seconds]
karonto has joined #bitcoin-core-dev
brunoerg has joined #bitcoin-core-dev
Guest32 has quit [Quit: Client closed]
brunoerg has quit [Ping timeout: 256 seconds]
<achow101>
anyone doing taproot spends notice that sometimes taproot spends end up severely under the feerate target?
bomb-on has joined #bitcoin-core-dev
<sipa>
achow101: hmm, how?
<achow101>
sipa: seems like with some(?) script path spends it ends up way under the target feerate
<achow101>
I think it's that the estimation occurs on the key path, but then script path is actually used
<achow101>
just noticed this while trying to construct a script path spend
<achow101>
also trying to write a test, and the size esimation seems to be ending up 1 byte short too (key path spend fails, with estimated vsize 1 vbyte less than real vsize)
<sipa>
ah yes, for script path spend i can imagine it's wildly off
<achow101>
why would it be?
<achow101>
if the actual signer uses the script path, shouldn't the dummy signer figure that out as well?
<sipa>
hmm
<achow101>
I think the dummy signer thinks we have the privkey for the internal key
<achow101>
but for this particular construction, we don't
<sipa>
it's inherently difficult, because the role constructing the transaction may not know what path the signers will be using
z0d has joined #bitcoin-core-dev
sipsorcery has quit [Ping timeout: 268 seconds]
<achow101>
sure, but in the single signer context, it shouldn't be
<sipa>
true
Talkless has quit [Quit: Konversation terminated!]
brunoerg has joined #bitcoin-core-dev
brunoerg has quit [Ping timeout: 268 seconds]
Guyver2 has quit [Quit: Going offline, see ya! (www.adiirc.com)]