< kallewoof>
I'm a bit surprised at the concerns with people running distros from 2007 not being able to run the latest & greatest version of bitcoin core. Does such a person actually exist?
< kallewoof>
Also amazed at how long RHEL support lasts. :O
< mryandao>
enterprise would be running the oldest version of bitcoin possible
< kallewoof>
mryandao: Yeah, so even if RHEL 8 was released with C++14 support it wouldn't really matter cause its users would be running bitcoin core 0.3 or something, anyway...
< jonasschnelli>
sipa: Yes. Possible,... but seems like we re-inventing a description language like JSON. :)
< jonasschnelli>
I like such compact forms though,.. but they tend to fail once extended to the max
< jonasschnelli>
Maybe it makes more sense to use JSON as the container/desciption format rather then inventing a custom CSV
< jonasschnelli>
not sure though,.. the compact string format would be used at various places,.. like importmulti, etc.
< jonasschnelli>
We just need to make sure it works for all possible extensions
< sipa>
jonasschnelli: i'm worrier about introducing too many formats
< sipa>
jonasschnelli: my idea is that the "records" that eventually will constitute a wallet will consist of a) a descriptor of script(s) and (b) birthdate (c) watchonly or not (d) private key optionally
< jonasschnelli>
Yes. I think that is a very good idea..
< sipa>
the (a) part is something that should be shared with scanutxo and perhaps some other things
< jonasschnelli>
My concerns are more about the format
< sipa>
so it would be nice of it's a nice and self contained thing
< sipa>
it could be json
< sipa>
that was what i was thinking too... but it's kind of unwieldy too
< jonasschnelli>
Not sure what would make more sense...
< sipa>
so i'm considering a single string thing
< jonasschnelli>
If the use cases are beyond JSON RPC, then I think a "new" format may make sense
< jonasschnelli>
JSON is very extensible...
< sipa>
dump files
< jonasschnelli>
Yes...
< * jonasschnelli>
thinking...
< jonasschnelli>
The whole dump file is another things... I'm currently not sure what the purpose of a dump file is, and if it is "a backup", if it is the right format
< jonasschnelli>
But I kinda like a string based descriptor,... lets do that
< jonasschnelli>
sipa: have you done a short spec on it already?
< jonasschnelli>
Would it be k/v or fix order/index? Would the xpub ckd range also be possible to describe?
< sipa>
yes, it needs to be
< jonasschnelli>
What is "(c) watchonly or not"?
< sipa>
jonasschnelli: read my.gist
< sipa>
i've linked to it many times :)
< jonasschnelli>
heh.. okay. I shoud re-read you wallet gist, yes.
< sipa>
jonasschnelli: the idea is that watchonly should not be tied to whether or not you happen to ha e the private key
< jonasschnelli>
We should that probably stick to the channel topic. :)
< jonasschnelli>
hmm...
< jonasschnelli>
But I guess senseless for primitive scripts like P2WPKH?
< sipa>
no
< sipa>
for example if your private key is in a hardware wallet
< sipa>
that shouldn't be treated as watchonly
< sipa>
it is yours
< jonasschnelli>
Aha... you look at it from an overall perspective... I was looking from a pure "wallet" perspective
< sipa>
watchonly is for multisig things that you participate in, but don't want counted towards your balance
< sipa>
so my view is that private key or not should be independent from "counted towards balance" ("watch only") or not
< jonasschnelli>
So then there would be 3 categories, hot keys (default), non-private-key-keys (HWW) and watch only?
< sipa>
no, there are "yours" records and "non-yours records" ("watch only")
< jonasschnelli>
When I did a primitive PoC with Core and the Bitbox, I came to the conclusion that three categories may make sense...
< sipa>
in addition, you have private keys... in various places; some can be in your wallet, some are not
< sipa>
the wallet should not care about whether it has the private key or not
< jonasschnelli>
But since multiwallet, this is no longer necesarry I gues
< jonasschnelli>
*s
< sipa>
please, read my gist :)
< jonasschnelli>
Yes. I see. Makes sense to me...
< jonasschnelli>
Yes. I should read your gist every morning...
< jonasschnelli>
amen. :)
< fanquake>
wumpus got a 6.3 vm setup, will get through those bsd PRs
< wumpus>
fanquake: great!
< fanquake>
#13355 fixes running gmake check
< gribble>
https://github.com/bitcoin/bitcoin/issues/13355 | Fix "gmake check" under OpenBSD 6.3 (probably *BSD): Avoid using GNU grep specific regexp handling by practicalswift · Pull Request #13355 · bitcoin/bitcoin · GitHub
< wumpus>
thanks for checking, it seems about time to merge it
< bitcoin-git>
bitcoin/master db56755 practicalswift: Fix "gmake check" under OpenBSD 6.3 (probably *BSD): Avoid using GNU grep specific regexp handling
< bitcoin-git>
bitcoin/master 0b1c0c4 Wladimir J. van der Laan: Merge #13355: Fix "gmake check" under OpenBSD 6.3 (probably *BSD): Avoid using GNU grep specific regexp handling...
< bitcoin-git>
[bitcoin] laanwj closed pull request #13355: Fix "gmake check" under OpenBSD 6.3 (probably *BSD): Avoid using GNU grep specific regexp handling (master...openbsd-gmake-check) https://github.com/bitcoin/bitcoin/pull/13355
< fanquake>
Obviously 13294 was submitted without running any tests, seeing as they would have been broken at the time.. ? Bit scary given the changes
< aaa_>
join
< aaa_>
??
< Guest15263>
11111
< Guest15263>
31232321
< Guest15263>
adad
< Guest15263>
ad
< Guest15263>
d
< Guest15263>
d
< Guest15263>
d
< Guest15263>
d
< Guest15263>
d
< Guest15263>
d
< Guest15263>
d
< Guest15263>
d
< Guest15263>
d
< Guest15263>
d
< Guest15263>
d
< fanquake>
Please stop.
< Guest15263>
Why no one speaks?Is this a bitcore channel?
< fanquake>
This channel is for Core Development discussion. Generally there are always hundreds of people idling/watching. Actually discussion happens sporadically.
< fanquake>
*actual
< Guest15263>
soga
< wumpus>
fanquake: #13294 duplicates too much of the ifdef forest for my taste, Maybe empact's idea of just pre-declaring the function instead would result in less mess.
< gribble>
https://github.com/bitcoin/bitcoin/issues/13294 | Fix compiler warnings emitted when compiling under stock OpenBSD 6.3 by practicalswift · Pull Request #13294 · bitcoin/bitcoin · GitHub
< wumpus>
I think there should be a limit to how much logic to add just to avoid a compiler warning, we're at the point again where the avoidance of warnings becomes a goal in itself instead of an indication of potential problems.
< wumpus>
I mean at least in #13349 I also solved an issue, while removing the warning.
< fanquake>
wumpus I agree. Should find out what the test plan was to ensure that no behaviour changed for any of the platforms affected by that forrest, just to silence a warning.
< fanquake>
That feels like exactly the kind of place you'd subtly break something.
< wumpus>
woule be a very bad place to break something too, and hard todetect
< bitcoin-git>
[bitcoin] yuntai opened pull request #13365: RPC/REST/ZMQ: Set label with importprivkey only requested. #13087 (master...master) https://github.com/bitcoin/bitcoin/pull/13365
< bitcoin-git>
[bitcoin] giulio92 opened pull request #13366: Docs: Rename “OS X” to the newer “macOS” convention (master...osx-renaming) https://github.com/bitcoin/bitcoin/pull/13366
< jonasschnelli>
sipa: what do you think about "address:<addr>/b<timestamp_uint64>/w|p<pkey_wif>" or "script:<script_hex>" or "p2wpkh:<pub|xpub>/r0-2000/..."?
< jonasschnelli>
pub/xpub is autodetect, first char r | b | w | p is for (r)ange, (b)irthday, (w)atchonly, (p)rivatekey
< jonasschnelli>
we could avoid the first "key-char" and detect the type an the length and characteristics (8byte int == birthday, <num>–<num> == range, etc.)
< jonasschnelli>
But it smells after voodoo... so unsure
< gojo>
Hi, I'm experienced developer Blockchain, C/C++, reverse engineering, compilers, drivers, GPU/OpenCL, Cryptography, Python, NodeJS, Linux, Windows, Web, mobile, embedded and more. Urgent! I'm looking for important job!
< jonasschnelli>
gojo: no jobs here...
< TheCharlatan>
Picking up from yesterday's back compat question, I'm having the same problems with gcc-7 as ken2812221 in #12511 using same configure and environment flags as in the gitian linux descriptor. I'm not sure if there is a simple fix available for this, like currently with aliasing memcpy.
< jonasschnelli>
I like the backward compatible way (only >128bit use v2)
< jonasschnelli>
Not sure about the name addrv2 (versioning of protocol commands)
< jonasschnelli>
or it should be something like addr32
< wumpus>
yes the name is completely contingent, I don't care
< wumpus>
as for the backwards compatibility, I think I prefer a new network version that simply uses addrv2 only, this will allow moving away from legacy addr messages at some point completely
< wumpus>
I think the only advantage of the backward compatible approach is that it doesn't require a new network version, but I'm not sure how important that is
< jonasschnelli>
Yes. It is a cleaner cut
< wumpus>
we could even *not* rename the message, with the new protocol version, but that introduces some fragility I'm not happy with
< jonasschnelli>
I guess using a varint for services is probably an unnecessary optimization? It would save 33% for a IPv4 addr
< jonasschnelli>
(with the current used service bits)
< jonasschnelli>
AFAIK
< jonasschnelli>
wumpus: I guess not using a new command would probably make a lot of stupid light clients crash. :)
< wumpus>
jonasschnelli: that's an interesting proposal
< wumpus>
jonasschnelli: I like it, don't see a reason why not
< jonasschnelli>
Yes. I think its worth it.
< wumpus>
jonasschnelli: even beter, it's future-proof
< jonasschnelli>
Especially since the addrv2 is already varlen
< wumpus>
(I guess, in a way)
< jonasschnelli>
>8byte service bits?!... not sure if we ever reach that limit :)
< wumpus>
yes, decided on varlen because padding everything to 32 bytes would be crazy, and I like not wasting space for v4 addresses
< jonasschnelli>
Yes. Good point.
< jonasschnelli>
Another redundant byte is the std::vector<uint8_t> varlen byte
< jonasschnelli>
But I guess that serialization property we can't change
< jonasschnelli>
(since the length is indirectly defined by the networkID)
< luke-jr>
wumpus: it might be a good idea to include port in addr
< luke-jr>
hidden services don't have ports IIRC, and non-TCP protocols might not either - or might have 64-bit ports or smth
< luke-jr>
32-bit times seem to be dying, but we probably will end up with a new addr message before 2038 I guess
< luke-jr>
especially if services isn't revised now
< jonasschnelli>
makes sense to me: putting port into the addr part
< wumpus>
hidden services have ports
< wumpus>
luke-jr: we could use a VARINT for the time too
< wumpus>
luke-jr: another option would be to lower precision
< luke-jr>
hmm
< luke-jr>
or both
< luke-jr>
epoch 2009 with 1 hour resolution seems plenty
< luke-jr>
or 90 minutes to be tonal-correct :D
< wumpus>
luke-jr: jonasschnelli: changed them both to VARINT, thanks for the comments
< wumpus>
I've added the concern about lowering time precision as well as rolling the port into address into a "considerations" section, I'm not sure about those
< luke-jr>
well, like I said, it's unlikely we'll still be using this BIP in 2038 if it doesn't improve the services field, so 32-bit time probably is fine
< luke-jr>
and unsigned 32-bit time is actually even 2106, not 2038
< wumpus>
having always been angry at the people that decided to use two-digit years and gave us the Y2K nonsense, I'm very happy to make it 2038-proof
< gmaxwell>
wumpus: cool!
< gmaxwell>
So would this also accomidate I2P? I don't really recall what I2P needed, but I vaguely recall its addresses were 512 bits-ish... though it isn't unlikely that I'm mistaken.
< luke-jr>
I think that's the goal
< gmaxwell>
oh nevermind I see there is a section, don't see how I missed it on the first pass.
< wumpus>
gmaxwell: yep - I2P is 256 bits
< wumpus>
(could trivially extend the max length to 512 bits if that turns out to be needed for something, but I like the strict length requirement as it bounds the maximum size of addr messages)
< gmaxwell>
wumpus: some ideas circulating for nodes that store some fraction of the blockchain (e.g. some FEC slice, as in taek's post, or some range of blocks) need a bit of signaling to encode what is there. With 64 bits of 'services' it could be stuffed there, but I'm not sure if thats what you'd want?
< gmaxwell>
yes, I like strict length checking too.
< gmaxwell>
wumpus: so would we want to addrv3 for those things, user service bits, or add some opional field mechenism(s)?
< luke-jr>
gmaxwell: services currently apply an | to the bits, so not useful for multi-bit stuff sadly
< wumpus>
gmaxwell: that was brought up in the meeting yesterday as well, by sdaftuar , but I'm not sure what that would imply in practice
< wumpus>
e.g. gossiping block ranges directly would not be useful as the information is outdated by time someone gets it
< luke-jr>
wumpus: just a seed that nodes can deterministically calculate block numbers from
< gmaxwell>
wumpus: these wouldn't be outdated by construction.
< gmaxwell>
as luke-jr says.
< wumpus>
but if it's some other kind of descriptor, sure, it could be added
< gmaxwell>
(also, FEC slice proposals don't have that issue)
< wumpus>
how many bits per address?
< gmaxwell>
I haven't seen any ideas that need more than 32 bits, most probably need 8-16 bits. Perhaps just stick in a 32 bit field which has a "service flag specific interpertation" ?
< luke-jr>
there might be a fingerprinting risk here with too many bits, but too few might be an issue for storage too
< wumpus>
an optional field mechanism could work, but otoh, I'm a bit afraid of overdesign (also: it still has to be bounded)
< gmaxwell>
luke-jr: yes but leave that problem to the other proposals.
< luke-jr>
ah, true
< wumpus>
ah, an optional 32-bit field per service bit?
< luke-jr>
then I can make a 2k addr message.. :/
< gmaxwell>
If you care about space time field could be reduced to 16 bits easily. Turn it into a "time ago seen" quantized to 1 hour precision. (IIRC we quantize times to 2hrs regardless).
< gmaxwell>
wumpus: oh I wasn't thinking of that, I was just thinking of 32 bits whos meaning is defined as "depends on the service bits".
< gmaxwell>
(e.g. just to take specifying its content out of scope for this BIP)
< gmaxwell>
having a field per service bit would be nice but unfortunately would allow huge messages.
< luke-jr>
I suppose nodes could policy-drop the extra data if they don't comprehend it?
< gmaxwell>
wumpus: is there any particular reason why you have a potentially irrelevant port field, rather than encoding the port in the low bytes of the address as relevant for the service type?
< gmaxwell>
luke-jr: then you need a mechenism to encode which extra data is there.
< luke-jr>
gmaxwell: you need that anyway if it's optional?
< luke-jr>
and it wouldn't make sense to have extra data for most of the current service bits
< gmaxwell>
luke-jr: right well if we didn't care about size we could just define that every service bit set gets 32 bits of flags. :)
< gmaxwell>
if we want optional flags. I guess the best thing would just be a byte to include the count of them, then a byte "type" for each one where the type also encodes if the payload is 0/8/16/32 bits. (using the two MSB of the type to encode the length). And then bound the count of them so that the total is still reasonably sized.
< gmaxwell>
And then define nodes should strip ones they don't understand.
< gmaxwell>
or something along those lines.
< wumpus>
gmaxwell: yes, making the port optinal or rolling it into addr was mentioned by luke-jr, it's under "considerations", I'm not sure
< gmaxwell>
If that kind of mechenism existed there might be less argument to make service flags anything but 32 bits?
< gmaxwell>
wumpus: I don't care one way or another, it would save two bytes when not used, and perhaps eliminate some stupidity of what happens when it's non-zero for I2P or whatever doesn't use it.
< wumpus>
gmaxwell: at least currently all the mechanisms have a concept of port
< gmaxwell>
My memory of I2P is really bad then.
< wumpus>
32 bits "depending on the service bits" would be ok, although it leaves some difficulty if multiple service bits want to use it
< wumpus>
same here, I intend to run this by one of the I2P devs before publishing it anyhow
< gmaxwell>
Beyond node-flavors for striping, another thing we might want more payload for is alternative ports for other transports. E.g. if later we define a bitcoin-over-udp we might want to signal the ports for that instead of sending two addr messages for the two endpoints. I'm not currently remembering any other service flag things where we've wanted other payload.
< gmaxwell>
but I'm sure we could come up with other ones.
< gmaxwell>
I don't feel that strongly about any of this of course, if we're too limited it's not like it's a big deal to define an addr3.
< wumpus>
we already send multiple addr messages when listening to multiple interfaces
< wumpus>
but indeed there's a lot of reasons adding service-specfic information to gossiping would potentiall be useful, even though we have none at the moment
< gmaxwell>
yes though it would be really inefficient if almost every node were doing it.
< gmaxwell>
(sending a whole seperate addr just to give another 16 bit port number)
< wumpus>
yes
< wumpus>
there's risk of combinatorial blowup there
< wumpus>
that's a good argument for encoding the port separately though
< wumpus>
that extends better to having multiple ports
< gmaxwell>
one could use the above mechenism of a <type/length> byte followed by payload to do the port.
< wumpus>
yes
< wumpus>
that's a good idea, I think
< wumpus>
though it increases the average size, port is only 2 bytes, making it a 'data item' will by definition be larger
< gmaxwell>
Yes, though it's two bytes overhead assuming you used my above suggestion and had no other data items.
< gmaxwell>
One byte overhead if you had other data items.
< wumpus>
true, I just mean in practice, everything will be sending a port
< gmaxwell>
And I think if the data item mechenism existed service bits could stay 32 bits, saving a byte.
< gmaxwell>
or you keep the port field (or merge into the variable length addresses) for the primary port to get that savings.
< wumpus>
aren't service bits 64 bits right now?
< gmaxwell>
oh. hm! I thought they were 32bits!
< gmaxwell>
nope, 64.
< wumpus>
I thought so too yesterday
< wumpus>
that's why Jonas proposed turning it into a VARINT, which I did
< sipa>
they're 64 bits now
< wumpus>
I wonder if CJDNS should get its own network id, the reason I'm not sure is because they're explicitly IPv6 addresses
< gmaxwell>
I suspect it should, since you can't reach them unless you use CJDNS? so it's like onion embedded tor?
< sipa>
yeah, i good criterion for separate networks would be whether they need separate routing
< wumpus>
gmaxwell: that's true; the differece is that cjdns runs as a network interface, so from the viewpoint of the application it's simply IPv6 networking. OTOH it's possible to set up the firewall to do the same with TOr, so I'm not sure it's a useful distinction for this.
< wumpus>
so yes, better to just add a n ID, they're cheap
< wumpus>
I guess we could save 6 bytes per Tor v2 address by not doing the onioncat thing (otoh, who's going to care about that going forward)
< gmaxwell>
ah, just don't onioncat embed them. that would be nice. though who cares much about tor v2.
< bitcoin-git>
[bitcoin] MarcoFalke opened pull request #13367: qa: Increase includeconf test coverage (master...Mf1806-qaIncludeconf) https://github.com/bitcoin/bitcoin/pull/13367
< gmaxwell>
I assume that within a couple years we'll stop forwarding torv2 completely.
< sipa>
wumpus: for simplicity, i think it makes sense to put tor v2 in a separate class
< sipa>
it needs separate routing after all
< wumpus>
sipa: that's what I've done, I just kept the onioncat wrapping (so addresses are 16 bytes not 10)
< wumpus>
gmaxwell: exactly my point, we don't really want to encourage torv2 after this work we're doing to support torv3 which is better in any way
< echeveria>
wumpus: cjd is pretty much exclusively ipv6 in their own range.
< echeveria>
it's really kind of its own net because you can't reach any of the peers any other way.
< wumpus>
echeveria: indeed, there are no 'exit nodes'
< gmaxwell>
cfields: re: the 4/8-way sse/avx hash calculator, one of the reasons why txid computation is a little less interesting other than the complication, is that it's not in the critical path for HB block relay, but tree computation is.
< gmaxwell>
and so sipa's PR drops HB relay about 5ms per hop, which is a pretty large fraction of the total processing time.
< sipa>
wumpus: thanks, read the BIP draft now
< sipa>
it seems strange to use the onioncat embedded for tor v2, if there is a separate network type anyway (it just burdens clients to add/strip/check it )
< sipa>
varint for 64-bit services... it saves a bit, but they're already 64 bits, and varint can't actually encode anything longer than 64 bits
< promag>
should unloadwallet PR include UI support so that unloaded wallets are removed?
< gojo>
What tools you are using for debug view uint256?
< sipa>
it has a GetHex() method for converting to a string
< gojo>
I mean inside gdb UI
< gojo>
i use GetHex() method for debug txdb my fork bugs
< gojo>
I join my own live logging tool for structured logs for fix complex bugs
< gojo>
I can share my code if needed
< gojo>
I looking for another way with gdbgui React debug widget
< sipa>
i know uint256's internal view doesn't match the normal human readable form (your point 1), but i gave you the solution (use print ....GetHex())
< gojo>
i need for debug my fork step by step each line for 100k blocks
< sipa>
good luck then
< gojo>
thanks
< gojo>
i'm have solution that maybe helps someone
< gojo>
i made several forks for Dash, Komodo etc
< sipa>
off topic here
< gojo>
btc-core is not offtopic
< gojo>
i have a logging system for bitcoin-core
< gojo>
i want to share it for made life easier
< sipa>
ok
< gojo>
but maybe someone has better solution
< sipa>
feel free to open a pull request to improve logging, if you have useful code