< midnightmagic> is someone still mirroring the PRs, etc?
< gmaxwell> morcos: I am boggle at +class CompareTxMemPoolEntryByScore
< gmaxwell> oh no I'm not.
< gmaxwell> turns out that b and a look a lot like each other on the screen.
< wumpus> does anyone know of a profiling/monitoring tool that can how how much % of the time a lock is held and by what threads?
< wumpus> (for linux)
< wumpus> thanks for all the work on the tests MarcoFalke
< Lightsword> wumpus, I think valgrind may have something for that
< wumpus> thanks
< sipa> if contention is really significant i expect it to show up in profilimg
< wumpus> yes the problem I usually have with profiling is that it gives way too much information, where I just need to know a specific thing :)
< sipa> haha
< wumpus> linux has about a gazillion tools to monitor the system, or a process, but I can hardly ever find the one I need when I need it
< wumpus> something that shows a timeline *wtf is this process doing now*, per thread, in a kind of GANTT graph would be awesome, we had something like that for Sun at ASML but I have no clue whether it exists for linux
< Lightsword> dtrace?
< wumpus> yes it was a proprietary tool based on dtrace
< wumpus> yes saw that one, there's also systemtap, and oprofile, which have similar aims but use slightly different mechanisms
< wumpus> oh and sysdig!
< * wumpus> mumbles something about fragmentation
< Lightsword> because who doesn’t like to have 10 different profiling tools that work slightly differently :P
< wumpus> :-)
< wumpus> well I suppose people who specialize in this appreciate the flexibility, but if you just want to investigate something quickly it's easy to get stuck in the "what tool to choose... oh wtf I'll just roll something myself" stage
< * Lightsword> has noticed ethereum has decided to do the same thing by apparently trying to write their clients in as many languages as possible
< wumpus> oh yes don't get me started on programming languages :) we have to rewrite every single thing and library in every programming language, sometimes it's good but sometimes it's also a waste, what could we accomplish if we just cooporated on a single version of *thing* that works awesomely
< wumpus> "we" is general, especially the open source community
< wumpus> for c++ we even want multiple version of *thing* based on what the favourite combination of dependencies of the day is
< Lightsword> heh, seems that’s why c is so common with linux stuff, since those libraries are pretty much compatible with any language
< jonasschnelli> Any final feelings about RBF for createrawtx (https://github.com/bitcoin/bitcoin/pull/7159/files)?
< jonasschnelli> Should I remove the general opt-in boolean?
< jonasschnelli> And add a seq-nr per input?
< jonasschnelli> I like both approaches.
< jonasschnelli> The per-tx-global opt-in bool is a higher level function. The seq-nr per input is something we should have anyways.
< jonasschnelli> Maybe we can have both?
< jonasschnelli> Wait. The sequence option was added already.
< gmaxwell> needs nlock too.. globals could be used to autopopulate sequence/nlock if not overridden.
< sipa> hmm, i'm unsure
< sipa> the question is what createrawtransaction's goal os
< sipa> just only using explicit values would be more in line with its name
< gmaxwell> it bothers me that rawtxn are trivially distinguishable from txn created by bitcoin core itself for no good reason... and it's weird that there is _less_ functionality via rawtxn now.
< sipa> but given that it can now be used in conjunction in fundrawtransaction to just choose outputs, maybe there is a need fir slightly higher level behaviour
< gmaxwell> why make it gratitiously less useful?
< jonasschnelli> I'm also not sure if passing a CREATE_TX_RBF_OPT_IN - flag to FundTransaction is the way to go: https://github.com/bitcoin/bitcoin/pull/7159/files#diff-12635a58447c65585f51d32b7e04075bR747
< GitHub77> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/e662a7628801...a9149688f87c
< GitHub77> bitcoin/master 8efed3b Jonas Schnelli: [Qt] Support for abandoned/abandoning transactions
< GitHub77> bitcoin/master a914968 Jonas Schnelli: Merge #7707: [RPC][QT] UI support for abandoned transactions...
< GitHub144> [bitcoin] jonasschnelli closed pull request #7707: [RPC][QT] UI support for abandoned transactions (master...2016/03/abandon_ui) https://github.com/bitcoin/bitcoin/pull/7707
< sipa> gmaxwell: i dislike feature creep
< sipa> making createrawtransaction do things automagically may be confusing "i was making a raw transaction... why did it put things in the transaction i didn't tell it to?"
< gmaxwell> I dislike having functionality which doesn't get used but for small shortcoming that take away its utility.
< sipa> i just don't know how to reconcile it being "raw" with the suggested functionality
< gmaxwell> raw is the fact that it returns a rawtransaction to you.
< sipa> my preference would be just allowing setting nlocktime and nsequence
< sipa> it also doesn't create change outputs automatocally
< gmaxwell> Createrawtransaction is the only way to do manual coin selection from the command-line, it works great for that, and I use it that way several times per week. It has a nice workflow that allows offline signing and allows review of the completed transaction before signing.
< sipa> but have another RPC for higher level operations like this
< gmaxwell> it would just be an exact copy of createrawtransaction, with extra flags and or slightly different defaults.
< sipa> so you would suggest having the ability to set nlocktime and nsequence specifically, and also have options to change defaults based on rbf, transaction sniping, relative locktime, ...?
< gmaxwell> I wouldn't object to just having another RPC like that... but it seems a waste.
< gmaxwell> sipa: yes.
< gmaxwell> the problem with 'global defaults' however, is that I don't see a clean way to make it work with fundrawtransaction unless side information is added to the raw transaction.
< sipa> so what if you want the rbf/sniping/rellocktime/... logic, but not createrawtransaction?
< gmaxwell> sounds like arguments for sendmany.
< sipa> so we'd add those to both sendmany and createrawtransaction?
< sipa> just trying to think this through
< gmaxwell> See any reason why we wouldn't? sendmany's workflow requires that the transaction it creates be valid now, that isn't so for the raw workflow.
< gmaxwell> e.g. totally reasonable to use the rawworkflow to make a nlocked transaction for the future.
< sipa> it's just so ugly to have both high level logic and the ability to override
< sipa> say we didn't add csv support now, just optinrbf or not
< sipa> someone starts using csv by setting the sequence number explicitly, but also passes optinrbf=no
< sipa> and expects a transaction that is not replacable
< gmaxwell> I get what you're saying, but a sequenced transaction implies at least sequence succession relplacability. But yes, ... that would not by far be the biggest footguns around that interface.
< sipa> if createrawtransaction was instead just something that took an existing raw tx (empty be default?) and allowed adding inputs and outputs
< jonasschnelli> sipa: the question is, do we think that rawtx users need to know the MAXINT-2 RBF opt-in seqnr? I think we should abstract these magic numbers from the rpc-users.
< jonasschnelli> IMO it's on a different level then the rawtx abstraction
< sipa> jonasschnelli: createrawtransaction users also need to know that creating less outouts than inputs will send their life savings to miners
< jonasschnelli> sipa: not if they use fundraw. :)
< sipa> i totally agree that there should be a way for users to need to learn all the ugly details ofnthe semantics of all fields
< sipa> but so far, createrawtransaction has been the way to actually do control the exact values, and not the high level behaviour
< sipa> and mixing them may be seen as a recommendation for people to use createrawtransaction when it actually cannot guarantee safety without learning the details
< jonasschnelli> sipa: I agree. But knowing also the magic numbers we use for nSequence is another "level":
< jonasschnelli> At least we should offer some nSequence=BIP125 abstraction.
< sipa> yes, agree
< jonasschnelli> The per-tx-general opt-in-to-Bip125 is probably to high. I agree.
< gmaxwell> the magic nsquence is really MAX-1 which is not replacable when it logically should be (after all, the transaction is locktimed and non-final)
< sipa> but you should not be using createrawtransaction if you don't know what the semantics of the raw transaction fields are, or you'll shoot yourself in the foot
< jonasschnelli> Yes. Agree.
< jonasschnelli> But the BIP125 MAXINT-2 is a policy we should offer directly.
< jonasschnelli> {"txid": <txid>, "vout" : 0, "sequence": "BIP12"} <--- feels ugly
< sipa> what will you do to add csv supoort to createrawtransaction?
< * jonasschnelli> is thinking...
< sipa> i really dislike mixing magic smart behaviour with raw overriding
< gmaxwell> I don't really like mixing types in a single argument name.
< sipa> that feels like the rpc is trying to solve things at different levels
< jonasschnelli> flags per input? FLAG_BIP125_RBF, SEQUENCE_LOCKTIME_TYPE_FLAG?
< gmaxwell> e.g. sequence: 12345 / and seqtype: "BIP12" and have their use be mutually exclusive would seem preferable to me.
< jonasschnelli> no. flags per input is bad.
< sipa> i'd almost argue for a computensequencevalue RPC
< sipa> which just gives you the number to use, which you can put in crt
< sipa> but that's cumbersome for human users...
< jonasschnelli> RPC is not made for "human" users.
< jonasschnelli> I think its fine...
< jonasschnelli> Chains of commands is something that increase the modularity
< gmaxwell> the RPC is our commandline interface as well, and its usefulness aids in discoverability which makes it much easier to use as an RPC too.
< gmaxwell> like having an REPL makes python accessible to people.
< jonasschnelli> A "setoptinrbf <rawtx>" is probably a bad design and leads to overriding ther nsequence magics
< sipa> jonasschnelli: agree
< sipa> optinrbf should be something you choose per tx
< jonasschnelli> gmaxwell: imo RPC != commandline, .. bitcoin-cli == commandline
< sipa> some receivers may not want such transactions
< jonasschnelli> bitcoin-cli could combine stuff
< jonasschnelli> sipa: "setoptinrbf" would be per rawtx.
< jonasschnelli> It would just set the nSequence number of all inputs to MAXINT-2
< gmaxwell> jonasschnelli: that would harm discoverability, I think it is unfortunate to create wildly different interfaces.
< jonasschnelli> But I don't like the override character.
< gmaxwell> jonasschnelli: I hope it wouldn't do that, I hope it would move anything from MAXINT/-1 to -2 and leave the rest alone.
< jonasschnelli> gmaxwell: Yes. It would add another level.. which I agree its bad.
< jonasschnelli> gmaxwell: Yes. That could be nice (move MAXINT/-1 to -2)
< jonasschnelli> but would "optinrbf <rawtx>" not be to prominent for a policy flag?
< gmaxwell> (and unix has worked for decades where the programmatic interface to unix commands is the same as the human one)
< sipa> our 'default' of using positional arguments does not help
< gmaxwell> jonasschnelli: I don't think that it's "policy" makes it any less useful to set it. From a api fanout perspective it might be better to have a tweakrawtxn that is multimodal.
< sipa> multimodal?
< gmaxwell> positional arguments are sadness. but the json style of arguments is sadness squared. Thats a place where I think bitcoin-cli translating unix style arguments to json might be OK, as long as it was done consistently so someone could just learn the cli to rpc mapping.
< gmaxwell> sipa: as it tweakrawtxn "hex" optinrbf
< sipa> gmaxwell: ah, reimplementing bitcoin-tx as an rpc? ;)
< gmaxwell> Ha. Indeed.
< jonasschnelli> hehe..
< jonasschnelli> what speaks against having these tweak-flags in crt?
< gmaxwell> right now the api has excessive fanout. seperate top level functions for varioations on the same thing, side effect of positional arguments and legacy. It means when looking up a command you have to go through long lists.
< gmaxwell> crt?
< jonasschnelli> createrawtransaction
< jonasschnelli> createrawtransaction <inputs> <outputs> <flag|flag>
< gmaxwell> the existance of fundraw for one.
< jonasschnelli> fundrawtx interacts with the wallet
< gmaxwell> the point is that if you set flags in createraw, they won't be visible in fundraw and it might violate them.
< jonasschnelli> <flag|flag> could be {"bip125rbf":true, etc.}
< jonasschnelli> gmaxwell: good point.
< gmaxwell> where as createraw | fundraw | tweak | signraw | sendraw is a fine pipeline.
< jonasschnelli> hmm.. not bad. Fundraw could then remove the opt-in-rbf flag (because it can add inputs).
< jonasschnelli> alternative names for "tweak"?
< sipa> mutate
< jonasschnelli> alter?
< sipa> alter
< sipa> modify
< jonasschnelli> modify is probably most understandable... but that blitcoin-cli in RPC!
< sipa> in the bitcoin-tx source code they are called mutate
< gmaxwell> spindletx
< sipa> transmogrifyrawtx
< gmaxwell> Presotchangotx
< * jonasschnelli> lol
< gmaxwell> alterrawtx is probably fine.
< jonasschnelli> +1 for alterrawttransaction
< jonasschnelli> We could start with RBF...
< jonasschnelli> add CSV..
< gmaxwell> I guess thats more consistent. (the fact that transaction is spelled out is one of the sadder parts of the interface)
< jonasschnelli> and someone could also implement adding ins and outs.
< jonasschnelli> But once we said keep the utilities outside of the RPC level
< gmaxwell> We sometimes made made choices.
< jonasschnelli> I agree. RPC is very handy for utitilites...
< jonasschnelli> gmaxwell, sipa: If you interested to review the enc/auth BIP again, a new version is here https://github.com/bitcoin/bips/pull/362/files
< gmaxwell> 64-bit MAC, really? This leaks message lengths.
< gmaxwell> probably should remove the performance claims from the document? they'll just invite debate over the document text. AES-128-GCM is quite a bit faster than chacha when AES-NI is used. (I'm not arguing it here, just pointing out the argument)
< gmaxwell> Is there any need to limit the rekeying that strictly? if it's just a hash operation, someone can just send us transactions to make us hash over and over again.
< wumpus> when AES-NI is used <- but our odroid C2 doesn't have AES-NI :o
< jonasschnelli> gmaxwell: why 64bit mac? Because of the SHA512?
< gmaxwell> hah I know, I was advocating before that we only use chacha here... (the alternative is to support both and negoiate when both ends have aes-ni).
< wumpus> agree, let's just settle on that. I don't think the performance considerations in the cipher are the problem here so I also agree with not making it primary in the document
< jonasschnelli> gmaxwell: the rekeying is local policy, but should not be under 600 seconds to avoid dos (https://github.com/bitcoin/bips/pull/362/files#diff-0bd7a3b80179294f4bcb38cae13c8534R142)
< gmaxwell> jonasschnelli: it's using a 4byte length, which is very wasteful. (on average 2.5 bytes will be wasted) but then only an 8 byte mac which is quite weak.
< sipa> jonasschnelli: poly1305 does not have 256-bit security, and certainly not when you truncate the tag to 64 bits!
< gmaxwell> jonasschnelli: if the timeout is 600 seconds, then a sender cannot rekey for some multiple of that for fear that the far end has a different idea of the arrival time.
< jonasschnelli> sipa: Chacha20 offers 256bit security. I though the poly MAC does not affect the security itself,... only the authentication? Not?
< sipa> jonasschnelli: authentication is part of the security
< sipa> it protects against different types of tlattacks
< sipa> *attacks
< jonasschnelli> Okay... I see.
< gmaxwell> jonasschnelli: the cipher and authentication security go hand in hand, if you can compromise the authentication you can usually make the endpoints leak information about the content.
< jonasschnelli> But 256bit security does not reflect the overall robustness....
< jonasschnelli> But right. Let me better remove this line.
< jonasschnelli> gmaxwell: So you think truncate the TAG to 8 byte is still to long?
< gmaxwell> no it's too short.
< gmaxwell> I would suggest the length be made variable length, self-delimiting, encrypted like in the ssh spec... and that tag length be increased.
< jonasschnelli> Okay. So using the non-truncated full 128bit poly1305 tag?
< gmaxwell> much of the cost of a longer tag could be paid by making the length shorter. There are some things using 12 byte tags, but I'm not sure what I think about that.
< jonasschnelli> gmaxwell: length variable length -> do you mean varint serialization?
< sipa> gmaxwell: i think varint serialization is overkill
< jonasschnelli> The current message structure also has a int32 for the length. Maybe we keep the varint outside of the message header.
< gmaxwell> sipa: better than truncating the MAC to 8 bytes.
< wumpus> varint is slightly inconvenient, it's nice to be able to read network headers at once
< wumpus> how does ssh handle this?
< gmaxwell> uses a fixed width interger, it's encrypted however.
< gmaxwell> The varint suggestion was trying to recover the overhead from the longer mac tag, I'm not wed to the idea.
< wumpus> I don't think we should couple the MAC size discussion to the packet length field discussion in any case, at most you'd save ~2 bytes for the typical packet
< gmaxwell> Many of the messages in bitcoin are quite small, so the extra lengths do make a meaningful impact.
< wumpus> yes, but reading one byte of a time from a TCP stream :-(
< gmaxwell> average message size in bitcoin p2p is someting like 100 bytes right now.
< wumpus> let's increase the MAC size and leave the length at four bytes for now
< gmaxwell> yup
< jonasschnelli> Okay. Will do.
< wumpus> gmaxwell: well you had the idea to collate P2P packets into encryption packets ...
< wumpus> (I know, futur work.)
< sipa> wumpus: which is also in the bip, btw
< jonasschnelli> gmaxwell: re-keying minimum of 600 seconds is to large?
< wumpus> oh!
< gmaxwell> yes, though there is 4 byte lengths at each level. At least that helps with mac overhead.
< wumpus> well in the inner packet I'm not opposed to varints
< wumpus> those packets are in memory already
< gmaxwell> jonasschnelli: I'd make it 10 seconds or something very small.
< sipa> rekey every 10 seconds?
< jonasschnelli> sipa: min. 10 sec
< jonasschnelli> flexible local policy.
< jonasschnelli> A node could detect repeating re-keys and ban
< jonasschnelli> I just want a minimum timespan between re-key in the BIP.
< gmaxwell> I don't think a minimum timespan is needed, but if one is there it shouldn't be 600 seconds.
< jonasschnelli> gmaxwell: Yes. We could also leave that open to the implementation. But the most obvious attack vectors maybe should be covered by the BIP?
< sipa> if we care about overhead, the first thing to consider would be making those 12-byte message names varlen :)
< jonasschnelli> Indeed.
< wumpus> yes
< jonasschnelli> We could do this for the encrypted message stucture.
< sipa> jonasschnelli: rekeying every second will still be much lower overhead than normal transaction relay...
< wumpus> +1 for making those varlen, that padding is ugly and people have accidentally leaked memory contents into them in the past :)
< jonasschnelli> sipa: Hmm... right. Its just a 2xSHA256.
< sipa> jonasschnelli: no, it's an ecdh
< sipa> which is similar to a signature validation in time
< jonasschnelli> sipa: No. The ECDH is done already,... its only hash(old_key)?
< sipa> jonasschnelli: rekey means doing ecdh again
< gmaxwell> it shouldn't.
< gmaxwell> it doesn't in the spec.
< sipa> ah
< gmaxwell> sipa: it's logically just cranking the initial KDF csprng to the next state.
< jonasschnelli> okay. 1.) command varlen 2.) AEAD-tag 128bits, 3.) remove 256bit security mentioning.
< gmaxwell> on auth, this protocol looks like it would result in a bitcoin daemon announcing a persistant identity to everyone that connects to it?
< jonasschnelli> According to https://www.ietf.org/proceedings/88/slides/slides-88-tls-1.pdf AES-128-GCM is slower then ChaCha20+Poly1305
< gmaxwell> Is there a reason not to use the scheme I suggested where the client sends an auth challenge which is H(session-id||server-pubkey) and if the server reconizes one of his keys, he responds with a signature?
< sipa> jonasschnelli: aes-gcm is slowish without hardware support, very fast with
< gmaxwell> jonasschnelli: only if you don not have AES-NI. If you do AES-128-GCM is maybe 7 times faster.
< sipa> jonasschnelli: aes-gcm can get to 1 cycle/byte on modern hardware with asm code
< sipa> jonasschnelli: chacha20-poly1305 is hard to get under 4ish, i think
< jonasschnelli> a... i see. AES-NI: 892 MB/s, ChaCha20+Poly1305, -march=native = 560 MB/s
< jonasschnelli> gmaxwell: the identity would only be revealed by the requesting peer.
< jonasschnelli> So the requesting peer would know who he wants to give its identity price.
< jonasschnelli> The responding peers only reveals its identity if the requesting peers identity "was accepted".
< sipa> jonasschnelli: gmaxwell's protocol never reveals identity, and only works if both sides know the pubkey beforehand
< jonasschnelli> Hmm... yes: H(session-id||server-pubkey) makes sense.
< sipa> jonasschnelli: those numbers are certainly not the best possible ones
< jonasschnelli> okay.
< gmaxwell> then it hast to be mutual, but what if you just have a trusted node and many things that connect to it. You don't want to do per-peer setup on each side (if you do-- you could setup symmetric keys). The downside of my protocol is that if you have many different identities you'd have to try all of them, but hashing is cheap, and I don't see a reason to have a huge number of alternative identitie
< gmaxwell> s.
< jonasschnelli> hmm.. so we assume the requesting peer can link a IP to a pubkey
< sipa> i'd prefer focussing on encryption first :)
< gmaxwell> I'd also made a suggestion that auth trigger a rekeying. with the pubkey in the rekeying. Because this has a cute property of being forever forward secure even if ecc is broken, if the public keys are kept private.
< gmaxwell> jonasschnelli: well he hopefully knows who he thinks he's connecting to-- more like the other way around, he has something he wants to connect to (pubkey), and has an IP he thinks belongs to it.
< gmaxwell> but yea, maybe hammer out encryption first.
< jonasschnelli> Could the responding peer answer with a signature of the received hash(received-hash || session-id||server-pubkey)?
< jonasschnelli> I think auth and enc has some overlaps.
< jonasschnelli> (at least in thinking about a solution)
< * jonasschnelli> needs to go afk for 1h
< gmaxwell> All he should need to do is sign the session-id.
< Anduck> who answers from contact@bitcoincore.org?
< Anduck> i see lots of "contact us" etc. but no names
< Anduck> like who actually run it
< GitHub148> [bitcoin] gmaxwell opened pull request #7805: Eliminate TX trickle bypass, sort TX invs for privacy and priority. (master...sorted_invs) https://github.com/bitcoin/bitcoin/pull/7805
< jonasschnelli> sipa: I agree that encryption should be made first. But I just want to avoid that people start using it, and think: "okay, now everything is encrypted, let me use minimum BF FPR", but MITM is still trivial. Auth would allow to reduce the MITM scenario.
< GitHub1> [bitcoin] laanwj closed pull request #7543: [0.12] Backport BIP9, BIP68 and BIP112 with softfork (0.12...dot12_backport_bip68) https://github.com/bitcoin/bitcoin/pull/7543
< GitHub144> [bitcoin] laanwj pushed 24 new commits to 0.12: https://github.com/bitcoin/bitcoin/compare/c5f94f6584cb...834aaef7bd37
< GitHub144> bitcoin/0.12 15ba08c Alex Morcos: Implement SequenceLocks functions...
< GitHub144> bitcoin/0.12 0d09af7 Suhas Daftuar: Add RPC test exercising BIP68 (mempool only)
< GitHub144> bitcoin/0.12 0a79c04 Alex Morcos: Bug fix to RPC test
< sipa> jonasschnelli: bf for?
< sipa> bf fpr?
< jonasschnelli> bloomfilter false positive rate
< jonasschnelli> (in case you want to connect a SPV wallet to a node over enc. channels)
< sipa> jonasschnelli: i don't mean that we should encryption running first in production
< sipa> but just have it designed and perhaps implemented first
< sipa> as it interacts heavily with network code
< jonasschnelli> gmaxwell: H(session-id||server-pubkey) would have once downside: extending to an concept where you can connect to a peer where you don't have pre-shared identity-keys
< jonasschnelli> sipa: okay. Yes. That make sense.
< sipa> jonasschnelli: if you don't have pre-shared keys, what are you trying to accomplish?
< jonasschnelli> sipa: I agree. It's a different security level. But it would allow to make sure that further connections are going to the same node.
< sipa> jonasschnelli: that's by definition fingerprinting the node
< sipa> i'm unconvinced that's something we should aim for
< sipa> at least, it's a very different thing from what we've been doing recently, namely trying to avoid fingerprinting
< sipa> maybe there is some mechanism possible where the key is based on the ip, and you never leak identity information for other incoming network addresses
< jonasschnelli> sipa: Yes. Probably. Well, .. i though you could connect to a node, reveal you identity, get the pubkey if the remote node wants to show its identity, .. further MITM would be detectable. But agree. If MITM would be there in first place you have lost anyways.
< sipa> jonasschnelli: yes, that's called tofu (trust on first use)
< jonasschnelli> Ah.. that's the tofu. Thanks.
< sipa> but that may not something you want in general... for example if a node is available over tor and over a normal iov4, you don't want it to have the same identity on both
< jonasschnelli> sipa: hmm.. this would mean one identity per net-type?
< sipa> maybe something you only want for servers on static ips, and only on the listening side
< sipa> so i think it's an orthogonal use case
< sipa> not just per net type
< sipa> if you receive a connection on localhost, what key do you use? that may depend on whether it's a trusted local connection, or it's coming via a proxied tor
< jonasschnelli> If the authack signature would also contain the encryption-session-id (for the opposite com. direction) and the identity-pubkey from the responding peer, that would probably avoid an auth/authack initiated from the responding peer.
< jonasschnelli> auth: H(enc-session-id-A || indenity-pubkey)-A
< jonasschnelli> auth: H(enc-session-id-A || indenity-pubkey-A), authack: sig(H(enc-session-id-B || indenity-pubkey-B))
< jonasschnelli> no wait...
< jonasschnelli> auth: H(enc-session-id-A || indenity-pubkey-A), authack: sig(H(auth-request-hash || enc-session-id-B || indenity-pubkey-B))
< sipa> where does the pubkey of the other side come from?
< jonasschnelli> sipa: the identity pubkey from the other side.
< jonasschnelli> (should be node by the requesting peer because we assume pre-shared keys)
< jonasschnelli> *pre-shared pubkey*
< sipa> let's go over the use cases one by one
< sipa> i know 3
< sipa> 1) light client connecting to a trusted full node; in this case the light client needs to have the pubkey of the trusted node
< sipa> 2) full node only provides (part of) its services to known peers, for example bloom filtering, or block download, or whitelisting, ...; in this case the full node needs to have the pubkey of the client
< sipa> 3) tofu security between any nodes on the network
< sipa> i think 1 and 2 are fundamentally different from 3
< jonasschnelli> I think we should focus for 1/2.
< sipa> because 1 and 2 never need to reveal identities, only provide proof when requested
< jonasschnelli> Do you think 1 without 2 is something we should aim for?
< sipa> i think 1 and 2 are orthogonal
< jonasschnelli> Yes. I agree.
< sipa> and they can use the exact same code, just in the other direction
< jonasschnelli> Then we probably need: auth: H(enc-session-id-A || indenity-pubkey-A), authack: sig(requesting-hash)
< jonasschnelli> For both sides.
< sipa> i don't think that adds anything over signing just the session id
< sipa> Schnorr signatures internally compute a message hash, which is based on the message and the signing pubkey
< sipa> so they already do that internally
< jonasschnelli> if we assume ECDSA, would you then recover the pubkey from the sig to lookup the identity?
< sipa> ? you asked for it, you know it already
< sipa> "hey, are you X?" - "yes, here is proof"
< sipa> we could use bitcoin script for the signatures, so you can do multisig auth :p
< jonasschnelli> heh.
< sipa> "hey, are you X, Y, or Z?" - "yes, here is a scriptSig"
< sipa> no good use case for that though, so let's not exaggerate
< * jonasschnelli> thinking...
< jonasschnelli> For your usecase 2) when or how would the responding peer ask for "hey, are you X?" - "yes, here is proof"?
< jonasschnelli> A new message-type from the requesting peer?
< jonasschnelli> Or as soon as the requesting peer accesses a restrictied service?
< sipa> let's say there are two new messages "areyou"(H(peer-pubkey || receive-session-id)) and "yesiam"(H(my-pubkey || send-session-id), sig(key=my-privkey, msg=send-session-id))
< sipa> if you're making/accepting a connection to/from someone and want them to authenticate, you send the areyou as first message inside the encrypted channel, before version
< sipa> there should probably be an explicit "i don't need you to authenticate" ?
< sipa> actually, no
< sipa> you either send an 'areyou' (in which case you wait for a yesiam first), or you send a version yourself
< jonasschnelli> sipa: Hmm.. a requesting peer that seeks access to "filtering" would first send a "areyou"-message to hope the responding peer will also request auth over a "areyou" message?
< jonasschnelli> Or could the requesting peer not just send a "yesiam"(H(my-pubkey || send-session-id)
< sipa> the protocol doesn't continue if they don't respond
< jonasschnelli> sipa: for your usecase 2) the requesting peer first needs to authenticate the responding peer... i think thats fine.
< jonasschnelli> But I don't see a way to do 2) without 1)
< jonasschnelli> but however, let me think a bit about it. My brain usually needs some minutes for processing the input. :)
< sipa> jonasschnelli: 1 and 2 are exactly the same!
< sipa> except initiated by the one listening instead of the one connecting
< sipa> 1 is "are you the server I know? if not, i won't tell you anything about myself!"
< sipa> 2 is "are you the client I know? if not, i won't tell you anything about myself!"
< sipa> but the p2p protocol has no distinction between the one connecting and the one listening
< jonasschnelli> sipa: yes. But for 2) you might not want to ask every peer for authentication.
< sipa> ah, i see
< jonasschnelli> I think in practice, 2 makes only sense with 1
< jonasschnelli> Though, it could be policy (ask every peer, ask only peer where you have authacked).
< sipa> let's get encryption flushed out first :)
< sipa> that's a part that can be shared across all use cases
< jonasschnelli> okay... lets do that.
< jonasschnelli> I'll update the auth bip with all the stuff we have disusses but focus on the enc BIP
< GitHub28> [bitcoin] laanwj pushed 2 new commits to 0.12: https://github.com/bitcoin/bitcoin/compare/834aaef7bd37...e3341aa94e1f
< GitHub28> bitcoin/0.12 e10c044 BtcDrak: [0.12] Update release notes
< GitHub158> [bitcoin] laanwj closed pull request #7800: [0.12] Update release notes (0.12...docs) https://github.com/bitcoin/bitcoin/pull/7800
< GitHub28> bitcoin/0.12 e3341aa Wladimir J. van der Laan: Merge #7800: [0.12] Update release notes...
< wumpus> oh wow re: tracing and profiling https://github.com/brendangregg/FlameGraph
< GitHub163> [bitcoin] instagibbs closed pull request #7784: miner_tests number clarification (master...patch-4) https://github.com/bitcoin/bitcoin/pull/7784
< GitHub6> [bitcoin] instagibbs opened pull request #7807: Fixed miner test values, gave constants for less error-prone values. (master...mintest) https://github.com/bitcoin/bitcoin/pull/7807
< Luke-Jr> FWIW, it sounds like bc.i is making bogus problems for 0.12 txns
< Luke-Jr> the fee sniping chang
< Luke-Jr> change*
< gmaxwell> making bogus problems?
< btcdrak> ?
< gmaxwell> sipa: the client could send [h(session_id||serverkey), h(session_id||clientkey)] and the server could respond with a signature, and then "ah, you claim to be h(sessionid||clientkey||2), prove it with a signature."
< gmaxwell> sipa: and if the client doesn't wish to identify itself/not configured for mutual auth it just sends a random number in the clientkey field. Server can't meet that challenge, so server doesn't learn anything about client identity.
< gmaxwell> sipa: so this lets you do mutual auth without leaking client identities to people who don't already know them.
< GitHub124> [bitcoin] theuni opened pull request #7809: depends: some base fixes/changes (master...depends-updates) https://github.com/bitcoin/bitcoin/pull/7809