< bitcoin-git>
bitcoin/master 5e0cd25 Bruno Garcia: fix the unreachable code at feature_taproot
< bitcoin-git>
bitcoin/master cf26ca3 fanquake: Merge #21081: test: fix the unreachable code at feature_taproot
< bitcoin-git>
[bitcoin] fanquake merged pull request #21081: test: fix the unreachable code at feature_taproot (master...taproot-test-return) https://github.com/bitcoin/bitcoin/pull/21081
< fanquake>
sipa: did you want to go ahead and regenerate that test data ?
< sipa>
fanquake: at some point if nobody beats me to it :)
< sipa>
remind me in a week or so
< bitcoin-git>
[bitcoin] fanquake opened pull request #21107: test: remove type: comments in favour of actual annotations (master...just_use_typing_directly) https://github.com/bitcoin/bitcoin/pull/21107
< bitcoin-git>
bitcoin/master e9189a7 fanquake: build: more robustly check for fcf-protection support
< bitcoin-git>
bitcoin/master 8636288 fanquake: Merge #20720: build: more robustly check for fcf-protection support
< bitcoin-git>
[bitcoin] fanquake merged pull request #20720: build: more robustly check for fcf-protection support (master...more_robust_fcf_protection) https://github.com/bitcoin/bitcoin/pull/20720
< shesek>
what would you estimate to be a 'reasonable' upper bound for the time it should take bitcoind to shutdown cleanly?
< shesek>
say, with a large dbcache and lots of flushing to do; or on something without much memory/ram like a raspberry
< sipa>
i wouldn't be surprised if can find pathological situations (huge dbcache, slow spinning disk connected over USB2) where it takes an hour
< sipa>
that's a very rough guess though, haven't really soon anything like that; several minutes i've seen
< shesek>
I've personally seen 15-20 minutes, when I shutdown right after finishing IBD
< shesek>
I'm configuring an init system to forcibly kill bitcoind if exiting takes too long, was thinking to give it like 30 minutes but an hour works too
< sipa>
probably better to actually try rather than rely on my vague guess :)
< shesek>
its a (yet another) easy node setup thing, I can't tell what specs users are going to run this on so kinda hard to have a good estimate for this
< shesek>
its also a really edge-casey thing, I might never run into a long shutdown when I try
< sipa>
"look man, i'm really offended that you discriminate against full nodes running on fridges"
< shesek>
:)
< * fanquake>
can't wait for bitcoind on a toaster
< shesek>
what about startup? how long would you give `bitcoin-cli -rpcwait` to try before realizing its not going to happen?
< shesek>
would be useful if bitcoin-cli had a timeout argument btw. I'm using the `timeout` program as a workaround
< sipa>
if on the same kind of system, you crash during a very slow flush right after IBD, i think it may take even longer
< sipa>
-rpcclienttimeout ?
< shesek>
oh, I didn't realize this worked with -rpcwait
< shesek>
isn't this the timeout for HTTP requests, while -rpcwait will be waiting without doing an http request while bitcoind is warming up?
< sipa>
i don't know
< shesek>
what does rpcwait do exactly? waits for the cookie file to appear?
< fanquake>
it tries to construct a request, and if an exception is thrown, it waits
< shesek>
what is the worse that can happen if I use someone else's datadir (a la prunednode.today)?
< shesek>
obviously I can be led to follow an invalid chain and accept non-existing coins. but will bitcoind be internally consistent? could I inspect its state via the RPC and have any assurances of what I see? say, if I check that the hash of the tip and the utxo set matches a trusted source, should this give me confidence that the datadir is valid?
< shesek>
I guess there's also some potential for things like buffer overflows that might escalate into code execution, but probably not very likely?
< sipa>
shesek: if the data is entirely untrusted, i wouldn't vouch for lack of buffer overflows etc
< shesek>
and other than things like buffer overflow?
< shesek>
to what extend do the sanity checks that bitcoind performs on startup give me assurance against inconsistent state?
< sipa>
you really shouldn't work with untrusted local storage
< shesek>
well, I don't use these things myself, but some people do. btcpay has a fastsync-by-copying-the-datadir option, and prunednode.today is also gaining popularity, I think typically alongside specter
< shesek>
I'm trying to better understand the risk model this has
< sipa>
:(
< sipa>
tell them to not do that :(
< sipa>
i mean... if you're getting the software bundled from a particular source, then it doesn't matter that the rest of the data also comes from that source of course
< shesek>
not quite the same though, software is auditable by humans, a binary blob isn't
< sipa>
but it'd be a lot nicer if people would help out with getting assumeutxo to move forward instead of workarounds that introduce central points of trust
< shesek>
I agree, but these are not necessarily the same people... the btcpay devs are doing what they can within constraints to make full nodes more accessible
< shesek>
its a tricky thing though. introducing more accessible middlegrounds - be it SPV validation, assumevalid, assumeutxo, client-side block filtering, datadir snapshot fastsync, whatever - can win you some people that would otherwise use worse entirely centralized solutions, but on the flip side can also encourage people who would otherwise have a fully-verified full node to be lazy and go for the easier middleground
< shesek>
its really hard to say if introducing a middleground option is going to do more good or more harm
< shesek>
I'm contemplating whether I should support datadir-snapshot-based fastsync in my easy node setup thing (a bundle of bitcoind, bwt electrum server and a few other programs). I'm fairly certain that it would get more people to use it, people that would otherwise be using public electrum servers. a node initiated from someone else's datadir would be a step up for them, as unideal as it is >.<
< wumpus>
not sure if that needs to be a requirement for merge but i guess it's important to doing releases
< fanquake>
wumpus: I'll be following up
< wumpus>
shesek: technically, copying someone elses's data directory is much more efficient, the whole, social, difficulty is chosing the 'someone else' is this case
< wumpus>
shesek: these kind of proposals are as old as bitcoin itself but i don't think anyone ever solved the 'who to trust' issue, how can you have infrastructure that gives you a reasonable confidence in that a certain validation is correct, like you can probably just copy a friend's node and trust them but yo udefinitely don't want to YOLO it off the internet
< wumpus>
in any case jamesob 's "assumevalid" work provides the technical infrastructure to handle this case better
< wumpus>
fanquake: thanks, does it still make sense for me to upload the differing binaries?
< fanquake>
wumpus: sure, if it's not too hard. I'll do some comparisons
< bitcoin-git>
bitcoin/master 9913419 fanquake: test: remove type: comments in favour of actual annotations
< bitcoin-git>
bitcoin/master b401b09 MarcoFalke: Merge #21107: test: remove type: comments in favour of actual annotations
< bitcoin-git>
[bitcoin] MarcoFalke merged pull request #21107: test: remove type: comments in favour of actual annotations (master...just_use_typing_directly) https://github.com/bitcoin/bitcoin/pull/21107
< wumpus>
fanquake: it's trivial (just need to re-do some runs because the result has been overwritten, well anyway a good test for local determinsism) but was distracted by FOSDEM this weekend
< sipa>
how was it?
< sipa>
(fosdem)
< wumpus>
interesting! highlights for me were RISC-V/secure architectures (CheriBSD was interesting, a POSIX veriant using pointer capabilities for memory safety throughout), overlay networks and P2P (point of note for me was "yggdrasil" which is inspired by CJDNS but does some things differently), open source CAD tools, and update about openwifi (open software/hardware wifi)
< wumpus>
also something about standards, proposed architectures for libre Trusted Execution Environments / key storage modules, anyhow i live-tooted a few things on my mastodon https://x0f.org/@orionwl might be a few interesting links
< sipa>
wumpus: did you ever attend fosdem in person?
< wumpus>
plenty of times i intended to go but didn't or couldn't for reasons i guess it's off the table now
< bitcoin-git>
[bitcoin] fanquake opened pull request #21112: ci: use Focal for macOS and Win cross builds (master...actually_use_focal_in_cirrus_ci) https://github.com/bitcoin/bitcoin/pull/21112
< sipa>
wumpus: i did, but it's been a long time
< sipa>
2005 mayve
< sipa>
it's kind of underwhelming, but exactly what you expect from a free (as in beer) event organized by foss people :)
< emzy>
Will there be recordigs from FOSDEM? I missed most of Sunday.
< wumpus>
sipa: heh, that's also the idea i get from it, but i kind of like the ad-hoc ness, really aimed at fostering discussion and cooperation instead of 'here, this is my work, bye' that was mostly my impression from academic conferences
< wumpus>
emzy: yes the videos will be posted soon(TM)
< luke-jr>
wumpus: for some reason GitHub didn't tell me about it, merged thanks
< MarcoFalke>
Oh, it is the GitHub 500 time again
< wumpus>
MarcoFalke: strange! I thought I saw mismatches (as noted in my post), but must have looked wrong then
< MarcoFalke>
wumpus: Oh, maybe you are right. I didn't check *all* hashes, only a few
< MarcoFalke>
wumpus: Is this an issue with the bump or a pre-exisiting one?
< wumpus>
MarcoFalke: I have never noticed it before
< wumpus>
lol i was about to upload my results somewhere but have just wiped my gitian VM again because it seemed unnecessary now that they were deterministic
< wumpus>
*starts another build*
< wumpus>
no cache either this time
< MarcoFalke>
It seems intermittently non-deterministic. I wonder if it is also non-deterministic on the same machine
< wumpus>
the strangest thing for me was that some of the files matched but others didn't (even for the same OS), my impression was that especially tars were affected not zips, but that might be a by-effect
< dongcarl>
Haven't been following, but is there a non-determinism problem?
< MarcoFalke>
dongcarl: Yeah, at least starting with #21036
< MarcoFalke>
(currently unclear if this happened earlier)
< MarcoFalke>
diffoscope might tell
< bitcoin-git>
[bitcoin] amitiuttarwar opened pull request #21121: [test] Small unit test improvements, including helper to make mempool transaction (master...2021-01-unit-test-valid-tx) https://github.com/bitcoin/bitcoin/pull/21121