< gmaxwell> luke-jr: as a compromise to your tor bundling thing-- what about the idea just of a seperate bundled download being available?
< gmaxwell> Seems to me that would have some of the advantage without most of the downsides... and would help make a case for doing it more broadly?
< echeveria> gmaxwell: I'm uncomfortable with increasing the number of people using Tor. as you pointed out the attack surface is greatly increased versus running bitcoind. depending how its configured this will result in people listening who wouldn't otherwise expect it.
< echeveria> if the purpose *isn't* to listen on a HS by default, there's no increase in listening sockets. if it's the default configured this way, people will be listening when they never expected to be able to.
< echeveria> if it requires configuration to have it listening, why bundle it at all?
< echeveria> echo "apt install tor" > ./setup_hs.sh; chmod +x setup_hs.sh
< echeveria> magic.
< gmaxwell> echeveria: bitcoind listens by default, if you want it not to listen you have to turn it off...
< echeveria> gmaxwell: yeah, but most people are behind NAT at least on their home networks.
< echeveria> so actually listening without configuration would be a surprise
< gmaxwell> echeveria: like most other p2p software until two years ago bitcoind used upnp to automatically enable incoming connections.
< gmaxwell> it only stopped because of the non-stop CVEs in the upnp software.
< echeveria> gmaxwell: that's been gone for like, 2, 3 years right?
< gmaxwell> yes, and its been a disaster that it's gone-- the end result is a massive shift towards listeners on VPSes, god knows what percentage of them are spys&sybils.
< echeveria> I acknowledge there's been a shift in where peers are, but supplementing them with even more trivially sybil attacked HS peers is if anything more harmful.
< gmaxwell> Adding more honest peers does not make any of that worse.
< echeveria> if you're never making outgoing connections over Tor or HS, yeah.
< echeveria> if the quality of peers on the network is a concern, perhaps the raise from 5 outgoing peers to 8 was too conservative. if the concern is the number, or availability of listening sockets, then there's probably a bit more of an engineering challenge there.
< echeveria> (I've mentioned it before, but I'd like to see a separate "headers only" socket which you make as many connections to as possible, even hundreds of peers- but when I looked into actually doing that the engineering involved was ridiculous)
< gmaxwell> echeveria: we'll soon be able to realistically handle many times the current number of peers with little cost.
< gmaxwell> not just limited to headers.
< gmaxwell> (well 'soon' relatively speaking)
< echeveria> from using select()?
< gmaxwell> no though we don't use select anymore.
< gmaxwell> right now having more peers is a problem because of the extreme amount of bandwidth wasted by txn rumoring.
< gmaxwell> but that'll be largely eliminated.
< echeveria> right.
< echeveria> I'm not saying peer quality isn't a concern, to be clear. there's lots which I can identify as crappy, which means there's a lot more that I'm not most likely.
< bitcoin-git> [bitcoin] jonasschnelli opened pull request #15433: Use a single wallet batch for UpgradeKeyMetadata (master...2019/02/wallet_key_upgrade_batch) https://github.com/bitcoin/bitcoin/pull/15433
< aj> oh, whatever happened to supporting nat-pmp?
< gmaxwell> sounds like a great idea. No one has done it.
< aj> https://github.com/bitcoin/bitcoin/pull/12288 <-- there was a pr a while ago
< echeveria> gmaxwell: perhaps ipv6 will save all.
< gmaxwell> aj: ah, well-- see the history there.
< gmaxwell> (I'm not quite sure it why it was subtreeing a library for a protocol that consists of "send a struct with a few fields filled out to the network", but there you go)
< aj> getting the info to populate the fields is OS specific iirc, so using the library saves redoing all the windows/mac/linux code
< gmaxwell> IIRC you need your local port and address, and such, which we already have.
< gmaxwell> (as we need to stuff it into addr messages)
< aj> hmm, i think you need the gw address too?
< aj> ah, nat-pmp tells you your local port, but you need the gw address to send the query in the first place
< aj> local addr, not local port
< aj> ugh, not local addr, it tells you your external ipv4 address
< gmaxwell> yes, it tells you the external info.
< aj> but yeah, it's https://github.com/miniupnp/libnatpmp/blob/master/getgateway.c that's most of the code
< gmaxwell> fair enough.
< gmaxwell> aj: some context, the reason we don't have upnp anymore is that the miniupnp library's code which can only be described as obviously frightening. (XML parsing with a mixture C string functions and raw pointer throbbing, clearly written by someone whos never had their hand burned by doing that before), so I can only imagine that no one was really excited about taking a huge ball of code from
< gmaxwell> miniupnp.
< gmaxwell> perhaps the natpmp code is a lot better, no idea.
< aj> yeah
< aj> well, there's no xml which already counts as a lot better to me, but... yeah
< gmaxwell> absolutely.
< wumpus> luke-jr | wumpus: why would tor be illegal? <- because authoritarian politicians don't like it, probably because in unstable countries it has sometimes be used to plan uprisings and such, as in Egypt
< wumpus> (someone from Egypt once told me you could get in trouble for running Tor there, I don't know if it's still true, but where human lives are at stake it's kind of important to be careful)
< wumpus> and yes, some vps definitely kick you off for running tor, I've had to explain quite a few times I'm not running an exit node
< wumpus> also I've always believed the way forward would be to improve the bitcoin protocol itself; BIP150/151, Dandelion, as well as lightning onion routing
< wumpus> back in 2012 I'd probably have agreed with you to integrate Tor, but it seems overkill now
< wumpus> a heavy-handed, noisy measure
< wumpus> but yes as gmaxwell already says, it could be a separate bundle people could choose to download, I'm fine with that, but not as only option
< wumpus> and yes natpmp support would be good, who is going to pick up #12288 ?
< gribble> https://github.com/bitcoin/bitcoin/issues/12288 | [WIP][NET] Add NATPMP support. by annanay25 · Pull Request #12288 · bitcoin/bitcoin · GitHub
< gmaxwell> probably tor usage provides the most value when part of the network does it...
< wumpus> gmaxwell: how do you mean? you mean having tor part of the network code?
< wumpus> ohh I get it now, sorry
< wumpus> yes, I think even in general it's good to have part of the network on different transports
< wumpus> which reminds me I should pick up my new address message BIP again
< gmaxwell> oops yep, thats it.-- effectively having seperate network topologies means that to knock everything out an attacker has to hit all of them.
< provoostenator> Tor hidden services avoid the need for true inbound connections. We could try to do something similar? Somehow have peers introduce eachother. Or does require keeping that introducing node in the loop?
< wumpus> I'm not sure I understand; doesn't one of the nodes have to be listening for the other to connect to it?
< wumpus> whether the nodes are on clearnet, tor, i2p, or another overlay network doesn't change that
< gmaxwell> provoostenator: not really.
< provoostenator> How does Tor avoid this problem?
< wumpus> they use relays and onion routing
< gmaxwell> tor is a tunneling protocol... and tor nodes are all listening. you can connect to a HS running on a tor client behind a firewall because the client connects out to the tor network.
< provoostenator> So who's doing the listening? A smaller number of tor nodes?
< wumpus> yes, exactly, they're indirectly connected through relays
< wumpus> routing your traffic over a relay would make zero sense for the bitcoin protocol as all the data is public
< wumpus> well okay there's the onion routing proposal for transactions, but besides that
< provoostenator> It could make sense for authentical (RPC like) stuff, which is what Luke was trying to solve.
< provoostenator> But it seems a bit heavy.
< provoostenator> I mean it's kind of cool being able to connect to your own node from mobile without revealing the destination IP by going through a bunch a lightning hops, but Tor seems easier.
< echeveria> provoostenator: it's heavy, makes a crap tonne of noise about its usage, and has a large attack surface. it's exactly what you don't want.
< wumpus> ( I mean, blocks and transactions *themselves* are relayed, there's no need to relay/bounce at the TCP level)
< wumpus> there's alrady no problem if you can't accept incoming connections, there's always someone who does listen that you an connect to
< gmaxwell> I guess what luke would want would be like getting your own node to sign headers then have some way of getting those headers from a public host.
< provoostenator> I think it's more about having a super light weight wallet on your phone that's really just a remote control for your node.
< gmaxwell> Which could be done, but ... it just seems kinda pointless to me. Luke's usecase is 99% conjecture... spv clients don't care about security beyond yolo hope no one attacks grade security, which is why they are spv lcients.
< echeveria> (an eighth of the listening IPV4 nodes also have 8332 exposed, btw)
< wumpus> that's valid idea but it has nothing at all with bitoin P2P anymore
< provoostenator> wumpus: I tend to agree that remote control is a more generic internet problem that can be solved outside of Bitcoin.
< gmaxwell> provoostenator: not superlight weight but just an spv wallet which has security that is tracable to the security of your own node (which runs on some headless host of yours)
< wumpus> remote control over bitcoin P2P? what?
< wumpus> i need coffee
< echeveria> gmaxwell: just have some central neutrino repo that you can add your own signatures to. done.
< gmaxwell> echeveria: right.
< echeveria> gmaxwell: block_header, signatures[]
< wumpus> provoostenator: definitely
< provoostenator> wumpus: you'd rely on your node at home (or wherever) to validate blocks, your phone has some private keys and wants to know the balance, so it just asks your node.
< wumpus> r;
< provoostenator> That's on one extreme of light-weightness, you could do something more hybrid like gmaxwell says where the phone does more work.
< wumpus> provoostenator: so electrum, basically
< echeveria> what I don't want to deal with is the bitcoin project being responsible for a bunch of people running outdated, vulnerable tor software that's doing nothing but harm them. include some scripts for setting this up in contrib/ if people really want it. I certainly do not.
< gmaxwell> echeveria: stop.
< gmaxwell> echeveria: wumpus already said we're not doing that.
< gmaxwell> you don't need to repeat.
< gmaxwell> Case made.
< echeveria> fine.
< gmaxwell> :)
< echeveria> sorry, this was mentioned to me out of band today as well.
< echeveria> I'd definitely like to see things like more p2p transports, don't let me detract from that.
< provoostenator> Funny how #11902 is tagged as "good first issue"
< gribble> https://github.com/bitcoin/bitcoin/issues/11902 | NAT-PMP port forwarding support · Issue #11902 · bitcoin/bitcoin · GitHub
< gmaxwell> take a step back. I believe the history of this discussion is that luke told someone somewhere don't use wallet X (some untrusted server SPV thing) because its insecure. Then someone said something like "well obviously I'm not going to run bitcoin core on my phone so how should I solve this"?
< gmaxwell> provoostenator: it is because you don't need to know essentially anything about bitcoin.
< gmaxwell> (something about _networking_ sure. :) )
< gmaxwell> good first issue doesn't mean good for newbies to development or something like that. :)
< provoostenator> I know, but I don't think this is easy for anyone unfamiliar with the codebase either, but I could be wrong. Don't want to discourage someone from trying :-)
< gmaxwell> (continuing) and luke's response was run your favorite SPV phone wallet against your own node. Which is good enough advice except (1) existing lite wallets (mostly) don't let you do that, (2) it's really hard to actually connect into your own node for most users. (3) you won't know for sure that you're connecting to your own node due to lack of auth.
< gmaxwell> Tor happens to solve (2) and (3).
< gmaxwell> though it's a blunt instrument.
< provoostenator> BIP 151 solves (3), but not (2).
< gmaxwell> technically BIP150 (or whatever replaces it... sipa and I really need to finish that)
< gmaxwell> (2) though can be substantially addressed through PMP/UPNP/etc. the users least likely to know how to punch a hole themselves are most likely to have a nat box that does pmp/upnp.
< gmaxwell> though maybe I'm being cynical but I don't see (1) changing. esp since we don't (and won't be) providing the kinds of crazy indexes that these wallets are written to need, they still rely on some somewhat trusted server regardless.
< provoostenator> Agreed, the combination of PMP/UPNP and ~BIP150 seems a more precise tool for this job.
< gmaxwell> Also if people want to improve things with tor there is a lot of stuff to do unrelated to bundling it.
< provoostenator> gmaxwell: now that we have descriptor support for importmulti, (1) is more doable, but it requires RPC access.
< gmaxwell> E.g. wumpus' work on a new addr message.
< gmaxwell> Or getting txn relay to preferentally relay through tor peers, to further frustrate tx forwarding spying.
< provoostenator> (but with RPC access you're right back to the remote control scenario)
< provoostenator> I think that as soon as you have trusted connection to your own node, you might as well just use it for everything.
< provoostenator> Except maybe holding private keys.
< gmaxwell> right.
< gmaxwell> using bitcoin-cli over ssh is pretty dandy. I assume thats what most people in here do, it's just not super gui-enduser friendly. :)
< provoostenator> Which is where #10102 comes in
< gribble> https://github.com/bitcoin/bitcoin/issues/10102 | [experimental] Multiprocess bitcoin by ryanofsky · Pull Request #10102 · bitcoin/bitcoin · GitHub
< provoostenator> With "some" additional modifications, as if that PR isn't big enough, you could just run QT on your phone and have it connect to a node+wallet instance over SSH anywhere.
< provoostenator> Or you run qt+wallet on the phone and connect to node process over SSH, but that would generate an insane amount of traffic the way the wallet currently scans for relevant transactions.
< provoostenator> IIUC currently the wallet tells the node "give me everything, I'll figure out what's relevant", rather than "here's what I care about, tell me what's relevant"
< bitcoin-git> [bitcoin] laanwj pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/b72c787dc8f7...f5a623eb66c8
< bitcoin-git> bitcoin/master a083f75 Gregory Maxwell: Update assumevalid, minimumchainwork, and getchaintxstats to height 563378...
< bitcoin-git> bitcoin/master f5a623e Wladimir J. van der Laan: Merge #15429: Update assumevalid, minimumchainwork, and getchaintxstats to...
< bitcoin-git> [bitcoin] laanwj merged pull request #15429: Update assumevalid, minimumchainwork, and getchaintxstats to height 563378 (master...201902-assumevalid) https://github.com/bitcoin/bitcoin/pull/15429
< gmaxwell> Danke.
< gmaxwell> fun fact, the new chainwork is 2x the old chainwork...
< bitcoin-git> [bitcoin] laanwj pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/f5a623eb66c8...29e82e460e19
< bitcoin-git> bitcoin/master 1435fab Pieter Wuille: Use RdSeed when available, and reduce RdRand load
< bitcoin-git> bitcoin/master 29e82e4 Wladimir J. van der Laan: Merge #15250: Use RdSeed when available, and reduce RdRand load
< gmaxwell> \O/
< bitcoin-git> [bitcoin] laanwj merged pull request #15250: Use RdSeed when available, and reduce RdRand load (master...201901_rdseed) https://github.com/bitcoin/bitcoin/pull/15250
< irc_viewer_test> gmaxwell: can you explain what it means for the chainwork to be 2x the old chainwork in simple terms?
< irc_viewer_test> gmaxwell: 2x more work is being done since when?
< gmaxwell> since ever
< gmaxwell> Probably this is not the best channel for stuff that is interesting once converted to simple terms. Some fun facts are only fun to geeks.
< gmaxwell> :)
< irc_viewer_test> haha
< irc_viewer_test> :)
< irc_viewer_test> sorry
< wumpus> doubled old value would be 0x51045fde384612c6a6b521a ... yep, close enough
< bitcoin-git> [bitcoin] domob1812 opened pull request #15435: Add missing #include (master...fix-include) https://github.com/bitcoin/bitcoin/pull/15435
< Ethan_> ‪New Crypto Security Concept:‬ Frozen Storage Much like Cold Storage except coins are sent to a special address with a special prefix on them. If private key is obtained by attacker who tries to spend them miners refuse to mine transaction from addresses with this prefix for specified time period. (i.e: 3 days give or take). If a spend is detected by owner of coins they will be able to reverse the spend before the tim
< Ethan_> In a situation where a standoff occurs a hard fork update can be legally applied for and issued to the miners for a possible small fee to incentivise them to agree to the update.
< Ethan_> In a situation where a standoff occurs a hard fork update can be legally applied for and issued to the miners for a possible small fee to incentivise them to agree to the update.
< Ethan_> These hard forks could occur yearly or whenever is suitable to minimise disruptions.
< Ethan_> The standoff situation would be a rare occurrence as coins located at these addresses are unlikely to be targeted.
< meshcollider> Ethan_: not the place
< meshcollider> Try ##altcoin-dev
< wumpus> what an horrible idea
< cjd> :D
< crised> Where can I learn the basics of bitcoin from developer point of view?
< instagibbs> crised, #bitcoin-dev or #bitcoin for general development questions
< bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/29e82e460e19...6ba3f1fdfd88
< bitcoin-git> bitcoin/master 6aaa0ab Gregory Sanders: Remove manual byte editing in wallet_tx_clone func test
< bitcoin-git> bitcoin/master 6ba3f1f MarcoFalke: Merge #15397: Remove manual byte editing in wallet_tx_clone func test
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #15397: Remove manual byte editing in wallet_tx_clone func test (master...wallet_clone_magic) https://github.com/bitcoin/bitcoin/pull/15397
< instagibbs> MarcoFalke, how is Travis letting through Python 3.5 if the .yml requests 3.4?
< instagibbs> (too late now, just asking)
< luke-jr> wumpus: BIP150/151 solve authentication when they're finally done, but I don't see any better solution for dynamic IPs and NAT traversal (when UPnP/NAT-PMP are unavailable).. at the end of the day, I'm not sure it makes sense to reinvent what already exists
< luke-jr> I suppose it could be simpler to add an even more-centralised dynamic DNS/tunnel service, but that seems likely even more controversial :/
< bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/6ba3f1fdfd88...f78cd3dd5115
< bitcoin-git> bitcoin/master 5b76c31 Carl Dong: doc: Add separate productivity notes document
< bitcoin-git> bitcoin/master f78cd3d MarcoFalke: Merge #15348: doc: Add separate productivity notes document
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #15348: doc: Add separate productivity notes document (master...2019-02-productivity-md) https://github.com/bitcoin/bitcoin/pull/15348
< MarcoFalke> instagibbs: travis runs the functional tests on ubuntu xenial and bionic, they come with python3.5+
< promag> should loadwallet() eventually use gArgs?
< promag> either that it should use a different args or some flags should be reset after init?
< promag> luke-jr: is #15428 wip or you'd like reviews?
< gribble> https://github.com/bitcoin/bitcoin/issues/15428 | GUI: Add Pairing tab with Tor onion address as copyable text and QR code by luke-jr · Pull Request #15428 · bitcoin/bitcoin · GitHub
< luke-jr> promag: I guess I need to address the no-wallet issues before code review, but concept review is welcome
< achow101> promag: are you working on a create wallet thing for the gui?
< bitcoinEnthusias> hey folks.... i
< promag> luke-jr: ok, will look into that
< promag> achow101: I was going to
< bitcoinEnthusias> hey folks.... i've got some noob questions concerning btc core development`, just out of interest. is this the right place to ask?
< jarthur> Probably, unless it's usage help.
< promag> a dialog, probably a wizard or with a advanced option, idk yet
< bitcoinEnthusias> is schnorr to be expected to be included in a main release 2019, what is the expected timeframe?
< promag> achow101: a drawing would be cool
< promag> achow101: for 0.19 right?
< achow101> promag: if you haven't started yet, I'll take a stab at it. i need it for the getting rid of default wallet with I'm doing
< promag> achow101: ah right, start without wallet?
< achow101> yeah, start without the wallet, but gui users need to be able to make one
< promag> achow101: ok, btw, what would happen if you always start with -nowallet ?
< promag> always prompt for "start by creating a wallet?"?
< luke-jr> bitcoinEnthusias: the next possible opportunity for code to be added is October, and typically softforks are only enabled on minor releases, so I'd give it at least a month after that. it's not impossible, but if you mean activation, I would be very surprised if it was during 2019. I have no particular info on Schnorr specifically, just going from normal processes.
< achow101> promag: i think so
< achow101> i don't think nowallet is an alias of disablewallet
< promag> achow101: -nowallet results in not loading wallets
< promag> not the same as --disablewallet
< bitcoinEnthusias> ok. thx. another question, as fees are rising again (i know vulnerable topic): is a blocksize increase completely out of roadmap, or is sth. like this still concidered for later releases?
< achow101> promag: right. so nowallet will become the default behavior
< bitcoinEnthusias> i think fees will rise again heavily if btc gains traction again
< jarthur> bitcoinEnthusias: on the protocol side, sipa has been organizing a BIP for Schnorr signatures. It hasn't officially been proposed yet, and typically an implementation would follow a proposal. https://github.com/sipa/bips/blob/bip-schnorr/bip-schnorr.mediawiki if you want to see the current state. bitcoin-dev mailing list a fine place to discuss the proposal
< achow101> I'm thinking there's going to be something that saves the last loaded wallet and loads that on following starts
< bitcoinEnthusias> @jarthur ah thx. i wonder why implementation/development is done in pyton and go while core is c++ :)
< jarthur> bitcoinEnthusias: the BIPs are designed to be readable and reviewable, and Python tends to work well for that.
< bitcoinEnthusias> i see
< bitcoinEnthusias> thx for the explanation
< luke-jr> bitcoinEnthusias: the only thing up for discussion at this time is a reduction in block sizes; take this topic to #bitcoin if you want to continue it
< bitcoinEnthusias> erm.... reduction?
< bitcoinEnthusias> ok
< bitcoinEnthusias> thx for the answers!
< gmaxwell> luke-jr: I am beside myself at what appears to be straight up malicious misinformation from you.
< gmaxwell> luke-jr: The only ongoing active thread on the mailing list is the discussion around segwit v1 schnorr signatures. Your absurd blocksize reduction proposal has never been even posted there.
< gmaxwell> Meanwhile there is a finished and widely reviewed bip on the schnorr signature scheme.
< luke-jr> gmaxwell: It's not clear to me what you're disagreeing with.
< gmaxwell> It's difficult for me to imagine how your answer to bitcoinEnthusias could have been _more_ misleading.
< gmaxwell> luke-jr: he asked about the status of some work, instead of telling him the status, you said the "the only thing up for discussion at this time is a reduction in block sizes"
< luke-jr> gmaxwell: he asked if it could be expected during 2019, so I gave an answer; then he asked about increasing block sizes, so I pointed out that was the opposite of what we needed
< luke-jr> if you don't think my answers went into enough detail, feel free to elaborate
< luke-jr> if you positively disagree with them, please be more specific
< gmaxwell> I don't agree that any reduction of block sizes is "up for discussion" in bitcoin core development. Quite the opposite: You have already been privately censured for creating drama in the press for annoucing some absurd reduction fait accompli when you hadn't even brought the subject up with other developers.
< luke-jr> nonsense. nothing has been announced, and it has been brought up, even though it's probably still too premature to make any serious action on. unlike increasing block sizes which is completely out of the question, there is an actual possibility of block size reduction (even if you think that possibility is small).
< luke-jr> ironically, you're now saying it's not up for discussion, and at the same time condemning that it supposed hasn't been brought up for discussion!
< luke-jr> kinda self-contradicting there
< gmaxwell> I'm saying your response is absurdly deceptive. You say that the signature stuff isn't happening in 2019, yet there is one completed BIP and a lot of discussion but then say that a reduction is the only thing "up for discussion" when it hasn't in fact been discussed (other than people privately WTFing you when we heard about it for the first time via false claims that this project was working
< gmaxwell> on it in the press).
< luke-jr> gmaxwell: do you expect Schnorr in 2019? what about my statement on that was unreasonable?
< luke-jr> the topic of block size is a separate question, I don't know how you're mixing the two questions together
< luke-jr> and I'm sure if I dig through logs I can find at least two discussions as counter-examples
< gmaxwell> luke-jr: no idea, but it's radically ahead of any reduction proposal which as far as I can tell is agressively not supported by major contributors to this project.
< luke-jr> gmaxwell: great, I don't see how my answers would suggest otherwise.
< sipa> i think both of these things are offtopic here; proposals should be complete and discussed in the wider development space before they can be considered for inclusion in core
< BlueMatt> (and very clearly neither has that level of support, so, seriously, stop spreading misinformation luke-jr)
< luke-jr> never claimed any level of support; no misinformation has been cited
< gmaxwell> You are also responsible for the reasonable and expected beliefs your statements create, not just the fine rules lawyer reading of the words.
< luke-jr> if you want to criticise what I actually said, fine, but making up strawmen is a waste of time
< gmaxwell> If you don't want to be yelled at for misinforming people, then when someone asks about active proposal X don't respond that it'll be a long time then suggest your hardly discussed hobby horse in a way that makes it sound more present and realistic.
< gmaxwell> I'm critizing the result, which you are responsible for.
< gmaxwell> You can't make carefully worded statements that mislead people and expect to not recieve complaint.
< bitcoin-git> [bitcoin] MarcoFalke opened pull request #15437: p2p: Remove remote debugging code (master...Mf1809-noBip61) https://github.com/bitcoin/bitcoin/pull/15437
< bitcoin-git> [bitcoin] MeshCollider pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/f78cd3dd5115...904308dca3ff
< bitcoin-git> bitcoin/master 0bedcba Jonas Schnelli: Use a single wallet batch for UpgradeKeyMetadata
< bitcoin-git> bitcoin/master 904308d MeshCollider: Merge #15433: Use a single wallet batch for UpgradeKeyMetadata
< bitcoin-git> [bitcoin] MeshCollider merged pull request #15433: Use a single wallet batch for UpgradeKeyMetadata (master...2019/02/wallet_key_upgrade_batch) https://github.com/bitcoin/bitcoin/pull/15433
< luke-jr> gmaxwell: I'm sorry that I was unclear, but I assure you it was not malice or carefully worded (I guess the lack of carefully wording is the real problem); what I meant was that *with regard to block size*, the only thing worth considering was a reduction - it wasn't intended to berate or address the earlier question about schnorr at all (and I would have expected from the context that was clear, but I guess not)
< palfun> hey folks
< palfun> what's the deal with the rpc wallet interfaces? I don't want to use node-side wallets, but it seems all nice affordances (ie "get unspent outputs") are only available on a per-wallet basis, rather than per-arbitrary-address
< achow101> palfun: doing things per arbitrary address is expensive and most people don't care about that
< palfun> isn't that how you implement clients/wallet software though?
< sipa> no
< palfun> or am I thinking about that the wrong way?
< sipa> i mean, you can
< sipa> but it requires a fully indexed blockchain on the server
< sipa> (and then trusting that server to do it right)
< achow101> palfun: no. wallet software typically just scan the blockchain for things that pertain to that wallet. to have everything per arbitrary address means that you are maintaining a lot of extra data for addresses that you don't care about
< palfun> ah right, that makes sense
< palfun> wait, but, then how do bip32 wallet clients work? they need to scan large amounts of addresses for outputs/transaction history right?
< sipa> they go through history
< sipa> once, for all addresses combimed
< sipa> and only the part of the chain after the address was created
< palfun> "go through history" here meaning "look at *all* blocks, see if any transactions include one of our addresses", right?
< sipa> yes
< palfun> and I guess they then just keep an eye out for new blocks and write them down whenever "one of our addresses" is involved, like some sort of specialized light client
< palfun> is that really the best way to do wallet software, assuming you want to just hook up with rpc to an arbitrary node? that's... somewhat painful, you have to "catch up" constantly
< sipa> bip37 allows client-side filtering (it has severe privacy concerns, and is not advised), or client-side filtering (bip157, which is still new)
< luke-jr> you mean server-side for BIP37, right?
< sipa> oops, yes
< sipa> bip37 is server side filtering
< sipa> palfun: it's only way to be sure you're not being lied to
< sipa> it's certainly not efficien
< sipa> *t
< palfun> ah cool, filters! but that's on the node protocol level, not the rpc one, right?
< sipa> right
< palfun> I had imagined it'd be possible to "just like implement simple wallet software" by calling out over rpc, but that's starting to seem... impractical
< sipa> yes
< luke-jr> palfun: you could use something like Electrum Personal Server (I think? never used it myself) to build the indexes you want, but there are scalability concerns to relying on this
< sipa> and privacy
< luke-jr> palfun: you can implement simple wallet software still, check out importaddress
< luke-jr> sipa: well, if you use your own Electrum server, it should be fine for privacy, no?
< palfun> luke-jr: I'd rather not put private keys onto nodes, I know people feel bad about doing that. I'm really looking to just support the simplest case of "I have a node, now let me hook a wallet up"
< luke-jr> palfun: importaddress doesn't require the private key
< luke-jr> it just tells the node to track the address in question
< luke-jr> (and the key backing said address)
< palfun> ohh I see, so it just does the indexing for you, giving you the read-benefits of a fully in-node wallet
< luke-jr> yes
< palfun> so for the bip32 case, you'd just feed it your first 20 addresses, see what turns up, and then proceed as appropriate
< palfun> cool, I'll have to play around with this! I bet I'll be back for more questions once I get to actually wrangling inputs/outputs into transactions (^:
< palfun> thanks luke-jr , sipa !
< belcher> read the source code of Electrum Personal Server because it does all that with rpc, so could be useful
< luke-jr> palfun: well, you ideally want to track from the creation of the wallet; rescanning is slow and doesn't work with pruned nodes
< luke-jr> belcher: oh, EPS doesn't actually build a full index?
< palfun> belcher: I thought Electrum used their own nodes for indexing etc? but that still goes over rpc? interesting, will take a look
< belcher> luke-jr nope
< luke-jr> belcher: interesting, good to know
< belcher> its a wrapper around bitcoin core's wallet that speaks the electrum server protocol, so it has no extra indexes and is compatible with pruning etc
< luke-jr> belcher: does that mean it doesn't get your wallet's history?
< belcher> palfun there are many electrum server implementations, the one which builds all the indexes is called ElectrumX and its much more complicated
< belcher> luke-jr it does, bitcoin core's wallet has the history
< luke-jr> maybe we should take this to #bitcoin
< palfun> luke-jr: right, so importing "used" bip32 wallets will be slow to detect all previous usage. does that still get done automatically, do I kick that off, or do it manually?
< * palfun> should probably not care about the "import wallet" case
< luke-jr> palfun: there's various RPC options
< palfun> I've pulled up the docs now, I see it can do rescans yeah
< palfun> cool, this seems fairly doable again. thanks! (:
< palfun> is there a practical difference between importaddress and importpubkey? docs recommend using the latter if you can, but I'm not sure I understand why
< luke-jr> palfun: IIRC (but my memory is fuzzy on this), there are some features missing with just importaddress, but I forget what
< palfun> looks like importmulti is probably preferable in most real-world cases anyway?
< palfun> is it just me or is the documentation around these things a bit shallow? perhaps idk where to look
< meshcollider> palfun: youre probably right, the importing RPCs (esp importmulti) have had quite a few changes recently so not a great deal of documentation exists at the moment I guess