<bitcoin-git>
bitcoin/master c6d1b8d Sebastian Falbesoner: gui: change example address from legacy (P2PKH) to bech32m (P2TR)
<bitcoin-git>
bitcoin/master c05c214 merge-script: Merge bitcoin-core/gui#808: Change example address from legacy (P2PKH) to ...
<bitcoin-git>
[gui] hebasto merged pull request #808: Change example address from legacy (P2PKH) to bech32m (P2TR) (master...example-addr_update_to_p2tr) https://github.com/bitcoin-core/gui/pull/808
vasild has quit [Remote host closed the connection]
vasild has joined #bitcoin-core-dev
<laanwj>
<Sjors[m]> "Oh I guess this effectively..." <- yes, that's what it effectively does
<laanwj>
socks5 (not even the TOR extensions) have no way to perform a DNS lookup returning multiple results, so "just connect to one using the remote OS' guess" is all it can do, or use built-in seeds
<laanwj>
using the local DNS would be a clear DNS leak
<laanwj>
i must admit that it confused me at first too, like "why is it connected to the DNS seed, would the DNS seed run a node"--so maybe it's not commented well enough
<lightlike>
I think the comment was added because multiple people (myself included) thought at some point that we'd connect to a node run by the DNS seed operator...
Zenton has quit [Read error: Connection reset by peer]
Guyver2 has left #bitcoin-core-dev [Closing Window]
<laanwj>
i've updated the google calendar linked on the bitcoincore.org site: removed the wallet meeting, added links, changed PR review meeting to show first wednesday of each month only
<laanwj>
if there's anything else there or on https://bitcoincore.org/en/meetings/ that isn't right let me know, i haven't paid attention for some time
<bitcoin-git>
[bitcoin] willcl-ark opened pull request #29903: Move bitcoin.conf to /share/examples dir in releases (master...conf-in-dst-share) https://github.com/bitcoin/bitcoin/pull/29903
<Sjors[m]>
For when there is no proxy, it calls that
<laanwj>
i'm happy to work on wayland support but it makes only sense to fix that after cmake and (i guess) qt6
<fanquake>
One of the problems with 22708 was that it added wayland and didn't remove X, given us twice us much stuff to maintain
<sipa>
Sjors[m]: in that case yes, but that is not used (or should not be used) in case we have a tor proxy
<laanwj>
using wayland is pretty easy to do in a static way, as long as you can load the GL library dynamially, which you should anyhow
<laanwj>
* library dynamially at run time, which
SpellChecker has quit [Remote host closed the connection]
<fanquake>
laanwj: I think it'd be good to have some sort of plan. There's a lot in flight here, CMake, Qt 6, the new GUI etc. It'd be good to know from someone working on that what is actually needed, what the timelines for each are etc, so we can avoid making a bunch of changes, just to potentially throw them all away soon after. i.e doing wayland with autotools & qt 5 vs with cmake and qt 6, and when we are dropping all the ancient x lib stuff
SpellChecker has joined #bitcoin-core-dev
<fanquake>
I'm hoping that all the new wayland deps, and itself all have cmake build systems, otherwise getting rid of autotools and friends will remain a pipe dream
<laanwj>
fanquake: agree, i personally think it's too early to drop X support entirely, i think we should add wayland support first, then see how it goes, then a release or so later drop X
<fanquake>
annoyingly a bunch of the x libs have opted for meson, instead of cmake, so if they stay around we'd need to add another build tool to the mix
<laanwj>
there's no way around maintaining both for a while i'm afaid
dviola has quit [Quit: WeeChat 4.2.2]
<laanwj>
in any case no hurry for the wayland part imo
<hebasto>
as CMake is expected to be landed into the master branch in August, it seems reasonable to handle stuff in the following order: CMake -- Qt6 -- Wayland
<laanwj>
yes it's the future but the future is ever far away
<laanwj>
anyone reasonable runs Xwayland, there are plenty of linux aplications that still require X, eg krita
<laanwj>
and it's not like we're doing high perf frame-synced embedded graphics stuf
<hebasto>
^ right
<fanquake>
I'm not even sure of the state of our x support. I think all the libs we currently use in depends have all been deprecated for some other "newer/more comprehensive" suite of x libs
dviola has joined #bitcoin-core-dev
<laanwj>
but this is what i meant, let's worry about wayland when all the other things are done, if it's just a matter of a checkbox on flatpak.org so be it 😄
the_mariner has joined #bitcoin-core-dev
<fanquake>
Yea. Plenty of other gui bugs/problems that'd be more useful to solve in the interim
<laanwj>
hebasto: sgtm
<laanwj>
<fanquake> "I'm not even sure of the state..." <- could be ! the last major library change i can think of was XCB, which was a long time ago,and maybe some XKB/input bus accessiblity stuff--at least the latter is shared by wayland, development on X (besides Xwayland) is effectively dead
Chris_Stewart_5 has quit [Ping timeout: 268 seconds]
Chris_Stewart_5 has joined #bitcoin-core-dev
pablomartin has quit [Ping timeout: 240 seconds]
<laanwj>
FWIW, i was (lightly) involved in the integration of wayland to Godot, there are definitely parallels with our GUI situation and games, in that they are binaries that need to run in an unpredictable, maybe even hostile environment. One think id like to do is dynamically load all dependencies at runtime. Anything to do with input, output, system integration. This reduces or even removes compile-time dependency on anything (headers can often be
<laanwj>
vendored).
<laanwj>
i will look into how to do this with Qt6, so we can know before we start integration
<laanwj>
that is, unless someone wants to work on a godot instead of qml based gui 😄
bitdex has quit [Quit: = ""]
jon_atack has joined #bitcoin-core-dev
jonatack has quit [Ping timeout: 255 seconds]
<sipa>
sounds fancy
<laanwj>
it has a great ui framework, not that different in concept to qml, also supports mobile well, some people actually use it for non-game apps! but one thing is that it's written around a real time frameloop so there's CPU/GPU usage even if there's no interaction or changes... so for a mostly static ui that's a bit of a bummer
<laanwj>
so unfortunately Qt, still is a slightly better choice...
oneeyedalien has joined #bitcoin-core-dev
<bitcoin-git>
[bitcoin] fjahr opened pull request #29904: refactor: Use our own implementation of urlDecode (master...2024-04-libevent-urldecode) https://github.com/bitcoin/bitcoin/pull/29904
jonatack has joined #bitcoin-core-dev
jon_atack has quit [Ping timeout: 256 seconds]
virtu has joined #bitcoin-core-dev
adil2 has joined #bitcoin-core-dev
adil2 has quit [Client Quit]
<fjahr>
Some very superficial analysis for pinheadmz's libevent topic, probably flawed but maybe it helps some that haven't interacted with libevent much: https://gist.github.com/fjahr/c0c339a4b5021ae815d5bcff42e9fc57 Posting now so people can take a quick look before we start.
<laanwj>
most of it is straightforward, the most annoying and hard to replace part of libevent that we use is the webserver
<laanwj>
we had tons of problems with boost::asio's http stuff, which prompted moving to libevent, which at least functionality wise worked a lot better, though there's some ugly hacks around its single thread support interaction w/ rpc
<laanwj>
that said i'm happy now that we never moved P2P networing to libevent
<pinheadmz>
yes, phew ;-)
<pinheadmz>
is there a simpler http server in cpp with MIT we can just vendor in?
<pinheadmz>
do we need a whole event loop? if so, even libuv is way better maintained
<laanwj>
i don't know, maybe, webservers are a hairy mess in general, although this one is for internal use only and ot supposed to be exposed to the internet so it's slightly less security critical at least...
<pinheadmz>
right, seems like the kind of thing we could vendor in or rewrite even
jarthur has joined #bitcoin-core-dev
<laanwj>
it's also not super performance criticial i'd say, though some people that do heavy use of the API might disagree here
<fjahr>
yeah, for our usecase I think most projects will be overengineered and we have to understand all the code anyway but I don't was to post spoilers before we start the topic :D
<sdaftuar>
I'm almost done rewriting my branch with the draft cluster mempool implementation, which I'll then rebase on master and push up to the draft PR.
<sdaftuar>
I believe Pieter is working on getting a version of the linearization code ready for its own PR. Once we're at that point, that would be the first item for people to review as part of the cluster mempool roadmap.
<willcl-ark>
Hi
<kanzure>
hi
<sdaftuar>
Also, I posted a summary of my research on data from 2023 to delving, summarizing the effects of cluster mempool on last year's data. If anyone has further thoughts/concerns about the effects, I'd love to hear feedback.
<sdaftuar>
I think that's it for me
<achow101>
is there an eta on that PR being opened?
<achow101>
#26606 is still the pr to review, and it's been getting some review since the session we had at CoreDev. I'm hoping that it can be merged in the next couple of weeks
<laanwj>
still planning to do some big endian tests with the migration wallet
<achow101>
sipa: added
<sipa>
it gives us an actual fully deterministic mode for the RNG for use in fuzz and tests, hopefully dropping the need for various "deterministic mode" things in the future
<sipa>
achow101: thanks
<achow101>
#topic libevent: replace? fork & maintain? leave it alone?
plank7 has joined #bitcoin-core-dev
<achow101>
(pinheadmz)
<pinheadmz>
yeah so i started working on adding unix socket support for rpc
<pinheadmz>
we pretty much only use libevent for http service, in the rpc server and bitcoin cli
<cfields>
concept ack :)
<pinheadmz>
and if you look at libevent repo, its all bitcoin core devs issues and prs anyway
<achow101>
my super naive thought is that since we already do the socket stuff for p2p handling, and http is a fairly simple and straightforward protocol (especially for our usage), it wouldn't be that complicated to implement it ourselves?
<fjahr>
Strongly in favor of removing libevent and replacing it with our own code. My frustrations with it are a few years old already (https://github.com/libevent/libevent/issues/1024) and there are many others that have felt the same before me and since. My mind went there very quickly also after reading the zx story, pretty much exact same symptoms can be observed in libevent. I don’t think we want to maintain a fork. I think
<fjahr>
implementing the libevent parts we actually use are not an easy project but sets us up much better for the future.
cryptapus has joined #bitcoin-core-dev
<achow101>
I could be totally off base though if anyone actually has experience with writing http servers
<laanwj>
concept ack on replacing libevent with something of our own, not with moving to another event library like libuv, would be a large chance of the same shit again
<pinheadmz>
does anyone know off hand a compact simple http server written in cpp with MIT license?
<cfields>
many many years ago I tried to port our net code to libevent rather than handling sockets ourselves and found it to be woefully underpowered and unmaintained back then.
<cfields>
imo it's a substantial liability at this point
<pinheadmz>
ok so sounds like favor is to inlude new code in the repo. easiest would be to find an existing implementation with compatible license and vendor it i suppose
<stickies-v>
at the previous coredev cfields and I had a look at potential options to replace the libevent webserver and https://github.com/uNetworking/uWebSockets seemed like a potential candidate, its C++17 and seems quite well maintained
<stickies-v>
it's apache 2.0 license tho
<laanwj>
a http server is doable if you can keep the scope small, internal-use only, no high perf requirements, it can't be nginx or apache
<pinheadmz>
right, and if we deprecate the REST interface i think we dont even need to support GET requests ;-)
<vasild>
Concept ACK on replacing the sockets stuff with our own. Maybe the http server as well.
<pinheadmz>
vasild p2p sockets are already our own right? is that what you mean
<sipa>
stickies-v: websockets sounds like about an order of magnitude harder than just http, actually...
<pinheadmz>
who did the work to replace boost::process ? wondering how that kind of thing goes, replacing a big dependency
<achow101>
sipa: I had a look at websockets a few months ago (for "serverless" payjoin I think) and it's actually not that complicated
<pinheadmz>
achow101 websockets still requires http...
<laanwj>
yes, websockets are more involved, but not much
<darosior>
Except for backward compat, why does JSONRPC has to be through HTTP at all?
<laanwj>
but we're not talking about changing the RPC protocol here
<sipa>
okay, i know too little about webstuff to really have an informed opinion
<sipa>
laanwj: yeah
<laanwj>
let's keep that orthogonal
<hebasto>
pinheadmz: work of still in progress; unused code has to be removed
<vasild>
pinheadmz: I mean to replace all libevent-sockets with our own code - talking to the socks5 proxy. Replacing the libevent http server would be a separate bigger task, maybe feasible as well.
<darosior>
(c-lightning for instance has its JSONRPC server directly without HTTP)
<achow101>
darosior: because it's always been that way :p
<darosior>
:)
<sipa>
maybe together with a move to UNIX-socket based RPC, we can drop the HTTP part?
<pinheadmz>
darosior super interesting idea
israt has quit [Ping timeout: 250 seconds]
<pinheadmz>
so users cant for example curl the api
<sipa>
(we'd keep the HTTP part for TCP-based JSON-RPC, of course)
<achow101>
sipa: I'd rather not. there's a ton of stuff that already uses HTTP, and switching from tcp to unix sockets as a client isn't particularly complicated
<fanquake>
what is the ton of stuff
<sipa>
achow101: right, but that stuff has to change anyway if it wants to use UNIX sockets
<cfields>
could be a proxy http app in between?
<laanwj>
agree with achow101, so much stuff is using http
<achow101>
fanquake: literally anything that's already an rpc client? e.g. lightning impls, various python, rust, other language libraries
<pinheadmz>
i think we have to support http forever right? its not just bitcoin-cli its the client lib written in every language
<pinheadmz>
ueaj
<pinheadmz>
s/yeah
<laanwj>
in retrospect it'd have been better to do like c-lightning, just JSON records over a pipe, but i don't think it's a good idea to change that now
<sipa>
it was just an idea, not a strong opinion
<laanwj>
all user software uses the current JSON RPC, if we abandon that, no one will upgrade. anymore basically :)
<pinheadmz>
ok so sounds like the move is to include a http.cpp in bitcoin core one way or another
<sipa>
my idea is that we'd have two interfaces, TCP+HTTP+JSON-RPC, and UNIX+JSON-RPC, and both remain supported forever
<pinheadmz>
doesnt need to be the best webserver ever but should handle long ques of rpc commands from multiple clients
<laanwj>
heck there's software that still requires the legacy wallet, i intend to help joinmarket port to the new one, interface drift is a pain even where it's really needed
<pinheadmz>
sipa well there is also a broader interest in not getting xz-ed by libevent
<laanwj>
@pinheadmz exactly
<_aj_>
pinheadmz: (can't get xz'ed if upstream never updates though?)
<pinheadmz>
lol there is that
<fanquake>
I think it's much less likely to happen via libevent than other deps
<fanquake>
and it basically already happened via boost process
<pinheadmz>
dont wanna stray but did boost get hacked ?
<fanquake>
compromising one of N random boost maintainers to get something into the 130mb tarball is much easier than getting something into libevent, that doesn't do new releases
<achow101>
pinheadmz: I think the next step would be to concretely propose a replacement then, as there seems to be agreement that replacing with something is the way to go.
<laanwj>
i was kind of disappointed in the maintenance state of libevent, back when i tried to add UNIX socket support
<pinheadmz>
laanwj i got chu fam
<darosior>
achow101: +1
<pinheadmz>
achow101 agreed
<laanwj>
@achow101 agree
<fanquake>
pinheadz: not hacked, when we re-enabled boost-process for windows it was doing random networking things, that never got picked up in review
<pinheadmz>
😬
<laanwj>
windows has this wacky winsock library that should only be initialized by us
<achow101>
anything else to discuss today?
<hebasto>
yeap
<fjahr>
If you read the timeline of the xz backdoor, it exactly sounds like the state of libevent now, if libevent started to do releases more often all of a sudden I would be very scared
<laanwj>
@fjahr libevent is an attractive target because of its use in tor
<fanquake>
Yea it's used all over the place. chromium, torrent libs etc
<laanwj>
that also makes that there are probably lots of security conscious people looking at it though
<vasild>
"if libevent started to do releases more often all of a sudden" -- or it has already been backdoored and things have settled now...
<laanwj>
heeh
<achow101>
#endmeeting
<pinheadmz>
how tf is tor and chromium doing unix sockets then! lol
<laanwj>
<pinheadmz> "how tf is tor and chromium doing..." <- i assume theyre not using the http part; the basic networking event handling works fine for any kind of socket, it's the web server/client that has the limitation
<jonatack>
"Cjdns running with Rust/Tokio based UDP Interface for the first time ever. Only thing left to do is port Pipe / PipeServer (i.e. unix socket / windows named pipe based interface) and then I can finally yank Libuv from the codebase." https://twitter.com/cjdelisle/status/1779990853844373820
<laanwj>
jonatack: interesting; i'd assume something like cjdns does actually need high performance networking, which is a good reason to use a library like libuv (eg to abstract various fancy OS-specific completion mechanisms), that said, if they're going the rust route there's plenty of other options
bugs_ has joined #bitcoin-core-dev
preimage has joined #bitcoin-core-dev
salvatoshi has quit [Ping timeout: 256 seconds]
szkl has quit [Quit: Connection closed for inactivity]
blockdyor has quit [Quit: My iMac has gone to sleep. ZZZzzz…]
cjd_ has joined #bitcoin-core-dev
cjd_ is now known as cjd
blockdyor has joined #bitcoin-core-dev
<cjd>
Hello Folks, I understand there was some discussion about removal of Libev from Core and the topic came up that I am working on getting Libuv out of cjdns - the choice to move away from Libuv is not exactly a negative for Libuv, but the way it has been used in the context of cjdns is not ideal at all.
<cjd>
1. We use an ancient fork of Libuv which has a patch to support watching a Windows HANDLE - so we can't easily update it
<cjd>
2. Cjdns uses an Allocator structure (i.e. talloc) and Libuv does not shutdown synchronously so it is the cause of asynchronous onFree hook in the allocator which created an enormous amount of complexity
<cjd>
3. We're not going to get any faster without moving to multi-threaded code, and Libuv is not really designed for multi-threaded - at least you can't share a uv_loop_t
<cjd>
4. Rust makes multi-threaded async code so much more pleasant, and Rust and C interop is not that bad, so it made sense to move the hot & async parts of the code to Rust
<laanwj>
@cjd thanks for the information, with libevent and us it's a similar situation, we're not exactly using it in an optimal way (nor for the main networking), and we'd require some custom patching to do what we want (unix sockets in http), and we use some hacks to work with the single-threaded nature of the event loop and worker threads which aren't quite nice (especially while using a C API from C++)
<cjd>
If you don't hate adding Rust to the build, I would take a look at using Tokio for the event loop and calling it through C functions
<laanwj>
that's for the recommendation--there's some intend to use rust more, but starting with some optional features, tools, and so on, we're not ready to use it for the core RPC, also i think we could do with something much simpler, the way we use libevent we don't quite need a heavy duty async framework
<cjd>
Oh, another thing about my application is I don't use Boost (obviously) so libuv is the only thing I use to bind to the OS, it's either that, or POSIX, or OS-specific headers. I've been burned by the latter enough that having something like the Rust stdlib to call through is convenient
<laanwj>
hah we're trying to get rid of boost as a dependency, i'm happy we're not using it for networking ! the main P2P code already uses OS-specific network functionality so it's a smaller step to also using it for RPC
<laanwj>
i mean the P2P code actually covers all the difficult/complex networking we do, a local RPC server should be straightforward in comparison
<laanwj>
if it wouldn't have been for needing a http server/client :)
<cjd>
I'm not such a fan of OS-specific code, but I get to write enough of it with the subtle differences between ... everything ... when it comes to TUN devices