< Talkless>
Could it be "controversial" to implement addrindex, so that other third party wallets (like Electrum) would not need intermediary server (like Electrumx) to make it work? It could access all info via Bitcoin RPC instead.
< bitcoin-git>
[bitcoin] hebasto closed pull request #18501: build: Fix boost detection on Ubuntu ARM 18.04 (master...20200402-boost-arm) https://github.com/bitcoin/bitcoin/pull/18501
< hebasto>
wumpus: done
< jonasschnelli>
Talkless: there where plenty of discussions about indexes in the past. AFAIK it looks like the tone among developers is to support an address index for the UTXO set but not a full including spent index.
< jonasschnelli>
Using and relying an address index does probably indicate conceptual issue
< jonasschnelli>
(like centralised validation)
< jonasschnelli>
With multiwallet and enforced watch-only support, I think using Bitcoin Core as a backand for personal wallets is possible/fine.
< jonasschnelli>
Some glue code is today necessary (to speak stratum/electrum, have hardened endpoint). Look at software like EPS (electrum personal server) from belcher.
< Talkless>
jonasschnelli: what centralized? If I have my own full node, and I want some software to access it's address index via RPC (I'm not about peer services), how's that centralzied?
< Talkless>
I know EPS and ElectrumX, the point is to get rid fo these extra hassle
< jonasschnelli>
Talkless: why would you need to index the whole blockchain if you just query a handful of addresses for your personal wallets?
< Talkless>
I run txindex of "whole" blockhain too
< Talkless>
It doesn't have to be only for my own wallet
< jonasschnelli>
Talkless: Ideally we get rid of those "in-between" servers. But there are no plans to add an hardened API that allows exposing to the broader internet (which is the main concern IMO)
< Talkless>
this feature could be used for self-hosted explorers, etc.
< jonasschnelli>
Talkless: so yes. That's why I said "centralised" validation.
< Talkless>
jonasschnelli: I'm about RPC, not peer services, again :)
< jonasschnelli>
RPC should not be exposed to the broad internet!
< Talkless>
yes
< Talkless>
I speak if you have your own full node, and your own Elecrum wallet (or whatever)
< Talkless>
you setup your Electum to conenct to your node via RPC credentials, and that's it!
< jonasschnelli>
I see the usecase for a personal block explorer. But IMO you should use third party software for this. There is no need to add complex indexing to the software that provides also the Bitcoin consensus protocol.
< Talkless>
now I need to setup ElectrumX or EPS or whatever in between
< Talkless>
This is not the consesus protocol, it's RPC! :)
< Talkless>
or am I missing something
< jonasschnelli>
EPS / ElectrumX are currently your endpoints (authentication, hardening, anti DDOS).
< jonasschnelli>
We can't have that in Bitcoin-Core
< jonasschnelli>
No. It's not consensus.
< jonasschnelli>
But if we add a such index (that has no broader use case), we would add it into the same software that also provides the consensus protocol. Thus. Developer resources, etc.
< Talkless>
jonasschnelli: does txindex has no broaeder use case too?
< Talkless>
i.e. txindex vs addrindex
< Talkless>
"EPS / ElectrumX are currently your endpoints" yes, and I am just thinking if it's possible to get rid of them
< Talkless>
And it seems we would need address index inside Core to do that
< jonasschnelli>
txindex is a relict and a development feature. I think no productive software should ultimatively rely on it
< Talkless>
Core also provides fee info via RPC, so with full set, no extra intermediaey local servers would b eneeded, just bitcoind + Electurm
< jonasschnelli>
Even if Bitcoin Core would have a such full address index,... I think it would be naive to also add the endpoint with all the required httpd hardening and authentication methods.
< Talkless>
no, no endpoint
< Talkless>
just same RPC we have now with auth
< Talkless>
just for local use
< jonasschnelli>
I totally get your point. But we have limited resources. We can't add everything into one package.
< Talkless>
not PUBLIC endpoint like peer servics, that is*
< Talkless>
if you want you can publish your RPC ports, if you into it.. :)
< jonasschnelli>
If it would be on the same RPC level, you still need an additional server that provides TLS, authentication, etc.
< Talkless>
jonasschnelli: of course, I understand, but if someone (me or whatever) would consider adding -addrindex, how it would be contraveral, "doable".
< jonasschnelli>
then the question remains, why not use EPS/ElectrumX anyways?
< jonasschnelli>
Talkless: I don't understand your last message
< Talkless>
It's extra software to trust, update, install, etc..
< jonasschnelli>
But as long as Core has no http server that can be exposed to the public, you won't get rid of that extra software/trust?
< Talkless>
jonasschnelli: I understand that there are limited dev. resources and I am not asking for some one to implement. Let's say hypothetically I would like to impelment it (or pay someone), how it would be reasonable to ask to merge it, how controversal that would be
< Talkless>
I don't know why you keep talking about http server :/
< jonasschnelli>
I can't say that for sure. But the past has shown that full indexes required to get rid of electrumX have little chance to get merged
< * jonasschnelli>
looking for past PRs
< Talkless>
I have LOCAL full node, LOCAL electrum wallet, I would like that my (and only my) local electrum wallet would connect DIRECTLY to that local bitcoin full node via RPC and could work just like that
< jonasschnelli>
run EPS. :)
< Talkless>
I'm not suggesting to add public peer service accessible for everyone, like ElecrurmX server does.
< jonasschnelli>
Or add the Core RPC protocol to electrum (so it could not only speak stratum/electrum it could also use Core directly to fetch data)
< Talkless>
it's still another software to maintain
< jonasschnelli>
Maybe it's simpler to tell electrum to speak with core
< Talkless>
jonasschnelli: electrum needs address info, which is provided by ElectrumX
< Talkless>
jonasschnelli: yes that's the idea, that electrum would speek to core, but it needs one extra index! :)
< jonasschnelli>
No on a conceptual layer
< fjahr>
@Talkless have you checked out previous attempts to add this index? See for example #14053 which is still active and bitcoin core reviews session https://bitcoincore.reviews/14053.html. If you haven't seen it I think it will provide more context on the issues.
< jonasschnelli>
Electrum needs a backand that can get transaction for a specific set of addresses
< Talkless>
No I haven't seen.
< jonasschnelli>
a full index is 99% wasted data for a personal wallet
< jonasschnelli>
That's why EPS works
< jonasschnelli>
EPS won't provide you a full index
< jonasschnelli>
It just makes Core speak "stratum" (the electrum protocol)
< Talkless>
jonasschnelli: it would be nice if EPS would be integrated as a "backend" withing Electrum, but now I need to setup extras softeware, it's just a hassle.
< Talkless>
jonasschnelli: you need to fiddlle with these xpubs and stuff, and I have multiple wallets...
< Talkless>
Anyway, I'll see that PR, thanks for that!
< jonasschnelli>
IMO a future version of a software acting like EPS could be a 500 line script that runs in a httpd-daemon process.
< jonasschnelli>
#14053 only adds an UTXO set address index (limited electrum feature set, no backup recivery with historical transaction).
< Talkless>
IMO it should be a swtich in Electum to select Electrum P2P or Bitcoin Core backend (which would use EPS within Electrum).
< Talkless>
jonasschnelli: alternative is just to wait until Bitcoin Qt has BIP39 support and easy HW wallet support, so I could just switch to Bitcoin Qt :)
< jonasschnelli>
needs a non trivial rebase I guess
< Talkless>
yeahm 2017..
< fjahr>
thanks, the different versions of the index are discussed in the PR discussions of 14053 and the review club so it will still be interesting to check those out, especially since this is the most up to date perspective
< jonasschnelli>
maybe 14053 could list some use cases. That would help a lot
< bitcoin-git>
[bitcoin] MarcoFalke merged pull request #18363: tests: Add fuzzing harness for HTTPRequest, libevent's evhttp and related functions (master...fuzzers-http_request) https://github.com/bitcoin/bitcoin/pull/18363
< wumpus>
achow101: what would you want to replace them with?
< achow101>
psbt
< achow101>
obviously we would need some more psbt rpcs
< sipa>
wumpus: can we not rewrite the signraw* RPCs by having them effectively wrap PSBT internally?
< sipa>
as in: convert to PSBT, invoke PSBT signing logic, convert back?
< wumpus>
I think documentation is most important in that case, to have a clear overview for people using the old raw-transactions workflow what to do now, this is going to trip up a lot of users I think
< wumpus>
sipa: yes, or that
< achow101>
sipa: I gave that some thought, but some things still don't quite make sense
< achow101>
wumpus: I think what it would really be is mark it as deprecated, but don't remove it for a while. we have some RPCs like that currently
< achow101>
so people currently using signrawtx don't get hurt, but we push new people to use psbts
< wumpus>
achow101: deprecated RPCs tend to be removed really quickly nowadays (like the next release), maybe call it something else then, like "did you know about this better system? here's documentation which you could consider" … then only really deprecate it much later :)
< MarcoFalke>
Yeah, sometimes I wish we had a deprecated RPC cycle that lasted longer than just one release
< sipa>
the problem is we really have no good way to reach users except the -deprecatedrpc mechanism
< MarcoFalke>
Something like "this is deprecated now and will be -deprecatedrpc when the release that deprecated it is EOL"
< sipa>
if we just say "look there is something better now" in release notes or so... probably nobody will notice
< wumpus>
but that's just me, every deprecation means *all* software using that API needs to be updated, and sometimes that's a pain, not all open source software gets maintained that actively and there's a large variance in versions people run
< MarcoFalke>
Yes, what wumpus said
< sipa>
yeah, in particular signraw*... i think will be a pain for people
< wumpus>
yes, release notes are not documentation, you can certainly point to it from there, also mention it in the help of the RPC
< MarcoFalke>
And if you use a library that is unmaintained for as long as it takes for a Bitcoin core release to go EOL, maybe you shouldn't use it
< sipa>
so i don't know what a good solution is
< wumpus>
if you need other ways to reach people well I can retweet things on the @bitcoincore twitter :p
< sipa>
send an alert message!
< sipa>
oh wait
< wumpus>
haha
< achow101>
when we removed signrawtransaction itself (replaced with signrawtransactionwithkey and signrawtransactionwithwallet), was there a lot of noise?
< wumpus>
I still mistype that one :(
< MarcoFalke>
autocomplete
< wumpus>
signrawtransactionwithwallet is soo long
< sipa>
achow101: moving to PSBT is a bigger workflow change though
< achow101>
definitely could have been named better
< MarcoFalke>
I use the gui :see-no-evil"
< wumpus>
but apart from that, no, not that much noise about it I've noticed
< achow101>
the main thing is that signrawtx doesn't work super well with descriptor wallets. and wrapping it with psbts pretty much results in a half-assed psbt workflow but in the context of raw transactions
< sipa>
achow101: what's the reason for that?
< wumpus>
a lot of people are aware of PSBT as a thing now, but I'm not sure they know it as potential replacement of the raw transactions API
< achow101>
sipa: the main symptom is with multisig. it's because in descriptor wallets, we're getting rid of the mixed watch only and non-watchonly in a single wallet
< achow101>
so if you have a multisig, you have one wallet with key(s) and another wallet that has the multisig as watchonly
< achow101>
with signrawtx, the wallet with the keys can't sign because it doesn't know the redeemScript
< sipa>
achow101: but the redeemscript is in the raw tx
< achow101>
so you have to go the multisig watchonly, do a signrawtx which (in theory) will just drop the redeemScript into the raw tx
< achow101>
then you go back to the wallet with the keys to sign
< sipa>
so in a convert-to-and-from PSBT, that would be retained
< achow101>
this process is the same thing as just using psbts, but hacking things into the raw tx format
< achow101>
it's still workflow breaking because you can't use a single wallet for keys and multisig and just do one command that does everything
< sipa>
achow101: imho it should be just possible to have the keys and descriptors in the same wallet
< sipa>
(but possibly in separate SPKmans)
< achow101>
that brings back the mixed watchonly thing though
< sipa>
how so?
< sipa>
this would be a wallet that does not watch payments to the individual keys, only to the multisig descriptor
< sipa>
but can sign with one of the involved keys
< achow101>
so that would be a multisig descriptor with private keys
< sipa>
yes
< achow101>
hmm
< achow101>
we'd need a way to export private keys and easily construct that descriptor
< sipa>
just not all private keys (because what's the point of a multisig with all private keys in one place...)
< bitcoin-git>
[bitcoin] ryanofsky opened pull request #18568: doc: Clarify developer notes about constant naming (master...pr/capconst) https://github.com/bitcoin/bitcoin/pull/18568
< instagibbs>
ideally internally everything unfinished tx related would be PSBT
< achow101>
sipa: that still does break existing multisig workflows as addmultisigaddress won't work anymore
< achow101>
but not as badly
< sipa>
achow101: i would hope addmultisigaddress doesn't exist for a descriptor wallet
< achow101>
it doesn't
< achow101>
i've just overloaded it in the rpc tests to be createmultisig -> impordescriptors