< GitHub109> [bitcoin] rebroad opened pull request #8583: Show XTHIN in GUI (master...ShowXTHINinGUI) https://github.com/bitcoin/bitcoin/pull/8583
< achow101> There appears to be a problem with verifying the email that wladimir sent for announcing the release key. See https://bitcointalk.org/index.php?topic=1596294.msg16030908#msg16030908
< phantomcircuit> indeed the signature is bad when you copy/paste from the website thing
< phantomcircuit> the actual email is correct though
< GitHub124> [bitcoin] pstratem opened pull request #8585: Remove last caller of IncOrderPosNext (master...2016-08-24-cwallet-incorderposnext) https://github.com/bitcoin/bitcoin/pull/8585
< phantomcircuit> can someone cancel all those travis jobs
< phantomcircuit> sipa: ^
< jl2012> wumpus: please do not include any email address within a PGP signed message. The signature is invalid because the "@" is replaced by "at": https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2015-June/009045.html
< jl2012> achow101, phantomcircuit: s/ at /@/ and you will have a good signature
< luke-jr> sigh, can't we just flip a switch so the ML sw doesn't edit it? :/
< kanzure> perhaps that's a "feature" enforced by linuxfoundation (which also doesn't make sense-- are they not signing email?)
< wumpus> I don't think I'm gonig to send asignedmessage with release announcement again, too many snags
< wumpus> the annouunce mailing list mutillated initial spaces converted to non-breaking spaces, \# converted to #
< wumpus> the -dev mailing list malformed the message in digest mode (which can't be disabled)
< wumpus> and now @'s are verboten too?
< wumpus> meh :-)
< wumpus> as if using GPG itsels isn't enough of a monster to get right (what, your key is only 2048 bits!)
< wumpus> sending a link to a .asc on a server may work
< * wumpus> first creates a gpg release mail signing key of 65537 bits
< wumpus> or maybe a bunch of sed scripts with transformations before validation
< paveljanik> I think we should abandon GPG and Bitcoin sign everything...
< wumpus> if only it was all GPG's fault
< paveljanik> and mailing lists ;-) Of course :-)
< wumpus> hehe
< paveljanik> sending announcements via OP_RETURN 8)
< paveljanik> think a bit of it...
< * luke-jr> glares.
< Lightsword> we could just upload the release itself using OP_RETURN’s :P
< * Lightsword> hides
< wumpus> in any case the release announcement doesn't need to be signed, people should verify the SHA256SUMS.asc tha tcomes *wth* the rekease
< wumpus> I tend to sign important mails to the mailing list, but this just created a diversion
< wumpus> verifying the release email itself does nothing, it provides no security, the binaries *at* the link may still be tampered with
< luke-jr> hm, that's a good point. this doesn't just screw up release mail, it screws up even when we want to sign discussion messages
< wumpus> would be nice if the archive had a 'RAW' button like github
< wumpus> that gives you the original text of the message, to paste into gpg
< wumpus> then again, the anti-@ 'feature' mentioned by jl2012 is probably against spam, so I doubht they'll disable that transformation. They may disable all others though.
< jl2012> or you may just use an attachment
< wumpus> put the text in an attachment? fullsigning the mail and putting the signature in the attachment would work worse because any footers added will invalidate it too
< gmaxwell> wumpus: just send the whole thing ascii armored.
< wumpus> gmaxwell: that'd work!
< wumpus> can't find any problems with that, it's what ASCII armoring is supposed to protect against. I guess it will generate no @, no # and doesn't depend on spaces
< wumpus> people without GPG can't read it anymore, but who cares, they don't take security seriously so shouldn't be using bitcoin in the first place right :)
< * wumpus> still remembers uuencode
< wumpus> GPG base64 characters are A-Za-z0-9+/=
< wumpus> does anyone happen to have the digest mail from bitcoin-dev or core-dev containing the 0.13.0 announcement?
< wumpus> I'd like to see how the text is mangled there
< gmaxwell> "okay, so, if we write our release notes in pig latin and make sure any numbers we use are prime..."
< wumpus> yes it reminds me of the often crazy requirements for passwords "needs at least one $, but no #, must be longer than 666 characters but shorter than 7" etc
< gmaxwell> jonasschnelli: re, service bit comment, that was my thinking too but I didn't make that argument because I didn't want to encourage another bitcoin classic like sybil attack. :)
< wumpus> cfields_: sorry for needing another non-trivial rebase for #8085, after the next one you should harry me until it is merged
< sipa> "Your password needs to contain at least an uppercase character, a digit, punctuation, a hieroglyph, and an extinct mammal"
< wumpus> yes, that
< gmaxwell> sipa: is there a configuration for gentropy for that?
< sipa> gmaxwell: not yet :)
< sipa> yes, the network refactors should be priority now
< gmaxwell> where is the gentropy webpage again?
< sipa> can we also merge feeler connections?
< gmaxwell> feeler++
< gmaxwell> feelers should be considered for backport
< gmaxwell> sipa: I mean the enscriptem version
< gmaxwell> (the backport comment, because of the problems we see in testnet with it taking a while to find many witness peers.)
< wumpus> jl2012: huh, there isn't even a @ in the release notes
< sipa> feeler is #8282
< gmaxwell> his shale kales fail thus nails fail yet ailments bewail but derailing bales assail
< GitHub188> [bitcoin] laanwj pushed 1 new commit to 0.13: https://github.com/bitcoin/bitcoin/commit/f2306fbe01426ce11c4df6a5c1b837106bc2c702
< GitHub188> bitcoin/0.13 f2306fb Wladimir J. van der Laan: doc: Clean out release notes after 0.13.0 release
< paveljanik> and Wshadow changes please...
< GitHub120> [bitcoin] rebroad opened pull request #8587: Provide bloom services to whitelisted nodes. (master...WhitelistedBloom) https://github.com/bitcoin/bitcoin/pull/8587
< wumpus> yes looking at #8282 now
< gmaxwell> :-/ more overloading of whitelisted.
< sipa> soon we'll need a separate confog file describing various peer's services
< wumpus> I think I've joked about this before, but it's starting to seriously look like whitelisting should be split up into a bitfield
< sipa> with its own turing-complete configuration language, of course
< wumpus> subnet 1.2.3.x { flags allow-bloom allow-hyperspace-travel ... }
< wumpus> yes, exactly
< gmaxwell> virtually no one uses it. and its changes in behavior (e.g. the forced relaying) have made it less useful (though at least 0.13 lets you turn that off)
< wumpus> yeah..
< wumpus> in any case if this is going to be more complicated it's going to need a proper design, instead of bolting on things
< wumpus> and proper documentation too
< gmaxwell> well, also the authentication bip thing is perhaps a better way to hand out access to varrious things.
< wumpus> I suppose there's use cases for allowing based on IP ranges as well as authentication
< gmaxwell> there are, though it just gets messy when we're peppering the code with permit this and permit that, and then putting them all under one banner.
< wumpus> yes, completely agreed, that was my point
< wumpus> rebroad is always like 'I need this, so I'll push this change, I don't care about others'
< wumpus> ego-commits
< gmaxwell> (Also, AFAIK there isn't even a reason to turn off bloom filter support generally...)
< sipa> i don't think he needs whitefiltering or bloom filters at all
< wumpus> disabling bloom filters reduces CPU and I/O load of a running node
< gmaxwell> I just mean at the moment there are no active attacks going on.
< gmaxwell> at least none that I'm seeing.
< wumpus> sure, but its true even without attacks, though in a lesser amount ofc
< gmaxwell> Yes. It's true.
< gmaxwell> in any case, for the use case I can think of for that: only provide bloomfiltering to your own mobile wallet, the authentication is exactly what you want: your wallet will move around, and you want the privacy of the encrypted link.
< sipa> the original idea behind whitelisting was to use it on an edge router, which your internal network nkdes connect to
< wumpus> yes, it makes sense to allow bloom filtering support selectively, no argument there
< wumpus> just overloading whitelisting for everything, bleh
< sipa> and it needed special configuration so that rebroadcasts would propagate
< wumpus> but it'd make sense to document what whitelisting is actually for
< wumpus> to prevent people extending it for what they think it is for
< wumpus> yes, that was the original idea
< sipa> agree
< sipa> it is unclear what the goal os at this point
< sipa> maybe peer authentication + mempool reconciliation are a good replacement in the future
< wumpus> yes
< gmaxwell> well partly it was a result of someone having a specific need (armory wanting to be able to trigger a rebroadcast, when the initial broadcast happened before the nodes connections were up, and similar)
< gmaxwell> and it got addressed with a more general tool.
< gmaxwell> but the armory usage was kind of at odds with other usage, for example elewhere I use whitelist to bypass my bandwidth limits to let local nodes use as much as they want, and to prevent my p2pool daemon from being disconnected.
< wumpus> yes, overloading the same option for a ton of different things isn't a good idea, it makes people get into each other's way, and causes unexpected behavior changes between functions that may be exploitable
< wumpus> it should be more granular
< wumpus> I wouldn't like rebroad to implement that though...
< gmaxwell> sort of a challenge, we want to solve specific problems with general tools... but we don't want to overload multiple tools into one feature. It can be hard to tell these things apart.
< wumpus> er between versions, not between functions
< wumpus> yes, it's always a challenge, it's clear where we don't want to go, not where we do want to go
< wumpus> trying to make general tools is a useful goal but only works with a clear view of what the use cases are
< wumpus> and that should be the beginning of the design not follow from it
< wumpus> the edge-router case is a clear one, for example
< wumpus> so is the 'allow BLOOM for LAN or localhost or authenticated peers only' for people using SPV wallets locally to connect to a node, but don't want to expose it to the whole world
< gmaxwell> yes, though that means different things for different uses. e.g. the armory one wanted it to bypass all standardness rules, for example. not what you normally want when the purpose of your edge router is to conceal your screwups from the network. :)
< wumpus> but these are completely different things and shouldn't be moved under one option
< btcdrak> wumpus: the announce list converted the message to HTML that's where the issues came from which we have solved. As for the LF list, the @ conversion is to protect against spam harvesters. For the mailing list does sending attachments work? I think it is important to have the SHA256SUMS sent to several locations. Certainly GPG signing simple messages is not
< btcdrak> hazardous. Maybe it it too much to GPG sign the entire release announcement. The part that needs to be signed are the torrent link and the hashes.
< wumpus> the thing is, 'whitebind'/'whitelist' *look* like something that would cover both cases. You allow internal connections with more privileges
< gmaxwell> The idea of some fancy acl thing (your set subnet 1234 flags space-modulator, example) makes me sad because 0.0001% of users would use it, and half of them would set it wrong. But something like that would be the only way to reasonable accomidate some things.
< wumpus> well at least a fancy ACL would be a general tool, that can be used for many different things, that doesm ake me happy
< wumpus> (e.g., instead of specific options with slightly different syntax)
< wumpus> btcdrak: yes
< wumpus> the hashes are already in SHA256SUMS.asc, I think adding them into the release announcement may have confused people to check the signature on that message instead of SHA256SUMS.asc
< wumpus> btcdrak: I usually just sign my mails to the development mailing list, doesn't haev much to do with it being a release announcement or not
< wumpus> gmaxwell: but I can still remember arguing against having any subnet/ACL matching in bitcoind because of similar reasons, so I understand your point, it's just that now we've crossed this threshold anyway we should try to do it in a consistent way
< wumpus> 'bitcoind is not a firewall tool!'
< gmaxwell> I think the authentication will make it more justifyable in fact... just because there will be more cases to use it.
< wumpus> maybe we could allow specifying a BPF filter *ducks*
< gmaxwell> wumpus: bitcoin script
< wumpus> right, with authentication you virtually *need* a system like that
< GitHub189> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/62a5a8a01866...026c6edac947
< GitHub189> bitcoin/master dbb1f64 Ethan Heilman: Added feeler connections increasing good addrs in the tried table....
< GitHub189> bitcoin/master 026c6ed Wladimir J. van der Laan: Merge #8282: net: Feeler connections to increase online addrs in the tried table....
< GitHub71> [bitcoin] laanwj closed pull request #8282: net: Feeler connections to increase online addrs in the tried table. (master...feelers3) https://github.com/bitcoin/bitcoin/pull/8282
< GitHub197> [bitcoin] laanwj closed pull request #8459: [0.13] release-notes: Do not encourage changing blockmaxsize to blockmaxweight (0.13...0.13_relnotes_remove_bad_advice) https://github.com/bitcoin/bitcoin/pull/8459
< wumpus> I'm not sure what to do with "leveldb: generate lib independent of locale sort" https://github.com/bitcoin/bitcoin/pull/8575 it's a change to leveldb,and we no longer need it since 0.13.0, and we likely won't do a leveldb subtree update anymore for 0.12.x.
< sipa> right
< GitHub185> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/026c6edac947...95a983d56dbd
< GitHub185> bitcoin/master fa1cf9e MarcoFalke: [test] Remove unused code
< GitHub185> bitcoin/master 95a983d MarcoFalke: Merge #8578: [test] Remove unused code...
< GitHub76> [bitcoin] MarcoFalke closed pull request #8578: [test] Remove unused code (master...Mf1608-qaUnused) https://github.com/bitcoin/bitcoin/pull/8578
< luke-jr> https://github.com/bitcoin/bitcoin/issues/8586 should probably be reopened.
< wumpus> luke-jr: are you going to troubleshoot/fix it?
< wumpus> if so, I don't mind reopening it
< wumpus> I tend to agree with MarcoFalke "However, if things are know to be broken and no one maintains/tests them, it would be better to disable qt4 right now.", but if we have someone that tests/fixes/maintains qt4 support we could keep doing that
< wumpus> I wonder how long this has been broken and gone unnoticed, not being able to switch tabs is quite serious
< btcdrak> is there any point in continuing with Qt4?
< * luke-jr> takes a look at the problem before deciding.
< sipa> arguabl, #8586 is still an issue, as we advertize support for Qt4.
< luke-jr> btcdrak: it's currently the only way to get native look/feel on KDE 4
< sipa> the solution may be discontinuing qt4, but that's still a cjangr
< sipa> *changr
< wumpus> I personally don't want to spend one minute of my time hndling qt4 issues, at least
< sipa> *change
< jonasschnelli> Yes. Neither do I.
< wumpus> #8263 was *assuming* things were working on qt4
< MarcoFalke> What is the advantage of having a native look/feel if you know that the application is untested and potentially buggy?
< wumpus> if they aren't, and people didn't even notice, well that's clear enough
< luke-jr> wumpus: well, it's only day 2(?) and people are reporting bugs
< jonasschnelli> I think we should not keep Qt4 compatibility only because of KDE 4
< * luke-jr> is happy there's been no problems with C++11 though
< wumpus> MarcoFalke: tend to agree
< MarcoFalke> luke-jr: The person reporting the bug apparently compiled against qt4 by accident
< wumpus> luke-jr: any update on when KDE is going to fix this situation?
< luke-jr> wumpus: KDE has moved on. KDE 4 is no longer supported. but KDE 5 is not usable yet. if the past tells anything, it'll be years before it is :/
< da2ce7> sipa: creative/good work on #8580, I don't feel qualified to ACK, however enjoyed reading the PR.
< wumpus> MarcoFalke: I don't understand how that can happen
< * luke-jr> checks KDE release dates
< MarcoFalke> wumpus: Forget to install qt5?
< wumpus> we choose qt5 by default now
< wumpus> oh, maybe that
< wumpus> luke-jr: well okay that is as close to an admission that this will never happen
< sipa> regarding c++11, i'd like to share a misconception i've had for a long time: i first read that rvalue references where "values where the receiver was responsible for cleaning up", but that's quite confusing - the destructor is still invoked by the caller. What they really are, is references to values that the receiver is allowed (but not required) to do anything with, including reusing its storage
< luke-jr> looks like it took 2 years for KDE 4 to stabilise
< sipa> s/receiver/callee/
< wumpus> but again, if you are willing to actively support Qt4, I'm fine with keeping support for it. Otherwise the clear decision of all other devs seems to be to remove it.
< luke-jr> if it's trivial, I'll just fix it; otherwise, fine with removing it
< sipa> luke-jr: well, one question is why haven't we noticed yet?
< jonasschnelli> If we support Qt4, someone needs to test and fix at least the RCs.
< sipa> seems you are using Qt4, but didn't notice this issue either?
< luke-jr> sipa: no devs use Qt4? :P
< luke-jr> I've been building against Qt5 for a while
< wumpus> jonasschnelli: yes, we need someone to step up to support it then, if no one is willing, then advertising qt4 support is wrong
< jonasschnelli> Agree
< luke-jr> I can probably switch back, it's no big difference to me
< jonasschnelli> luke-jr: your saying that you will continue to actively support Qt4? Which means testing release candidates and fixing stuff? Than I'll stop the PR for disabling Qt4 support.
< wumpus> sipa: interesting
< wumpus> sipa: yes I haven't really formed a conception around them either yet, more than 'maagic!'
< jonasschnelli> Qt4 != Qt4
< luke-jr> I can't reproduce the problem. :/
< da2ce7> wumpus: on the gpg signing the release announcement: I would recommend including signed gpg armoured copy of the announcement at the bottom of the email; Anyone who wishes to verify the message can do a diff between the two to see if anything important was changed.
< sipa> wumpus: it's quite clever... the only "magic" is that passed temporaries are automatically given the rvalue-reference class (which can be automatically converted into a lvalue reference, if the callee doesn't have an overloaded version that takes rvalue references)
< luke-jr> everything seems to work with Qt4 for me
< wumpus> da2ce7: yes that could work (although the mailing list was already complaining about me sending a 40kb mail...)
< sipa> wumpus: something else that was unintuitive to me is that when you have a function with a (type&& x) parameter, x is just an lvalue reference - the rvalue status is just used to determine which overloaded version to use
< wumpus> luke-jr: I had this problem with qt5 too, with the out-of-tree changes, btw https://github.com/bitcoin/bitcoin/pull/7911#issuecomment-212413442 But that was fixed. Bu tmay be a similar issue
< sipa> and if you want to pass x along to something else without copying, you need to use std::move
< sipa> wumpus: rvalue references are huge complication to the language, but they really feel like addressing a very deep problem
< luke-jr> wumpus: I wonder if a non-clean build dir (old bitcoin-config.h?) might also cause this
< wumpus> sipa: it does add a lot of complication to an already extremely complicated language, but yes I think overall it's worth it
< wumpus> sipa: it does allow for doing some things much more efficient in a cleaner way
< sipa> yes, it always felt non-C++-ish to me that there were many cases in which you couldn't cleanly avoiding copying
< sipa> C++-ish here meaning that you should always have the option to avoid unnecessary code execution
< wumpus> yes you had to use some really ugly hacks to avoid copying, if possible at all
< wumpus> (like adding a dummy element and swapping, etc)
< luke-jr> wumpus: I bet he had to `make clean` for Qt5, but didn't the first time around..
< wumpus> right, un-c++ish, it's intended to be an efficient language, if not, there are plenty of other languages to use that are more developer friendly
< btcdrak> It's pretty clear supporting Qt4 and doing QA for releases is going to be like pulling teeth.
< jonasschnelli> A Qt4-semi-disabling-step would be to disable the auto-detection
< wumpus> "and if you want to pass x along to something else without copying, you need to use std::move" I do wonder how std::move is implemented, or is it some compiler-internal magic?
< luke-jr> btcdrak: it seems to just work for now
< jonasschnelli> If someone really wants to compile against qt4, we could keep --with-gui=qt4 but disable the autodetect
< wumpus> jonasschnelli: good idea; you can still force using it if you know what you're doing, but it wil never choose it by default
< sipa> wumpus: it simply casts a reference to an rvalue reference
< sipa> nope, you can implement it yourself
< wumpus> sipa: oh, that's all
< luke-jr> jonasschnelli: that sounds reasonable; maybe a message to make noises if they want Qt4 support to stay?
< sipa> std::forward is more magic, but still does not require any internals
< jonasschnelli> luke-jr: Yes. Okay... lets try that.
< sipa> wumpus: you can create a function that takes a template parameter <typename T> (T&& x), in which case x is a universal reference (either an lvalue or an rvalue), and std::forward turns it back into the way it was called
< sipa> so you can create a function that takes either an lvalue or an rvalue, and passes it down the stack several times without losing its rvalue status
< wumpus> ah the 'perfect forwarding' thing
< GitHub187> [bitcoin] jonasschnelli pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/95a983d56dbd...d26234a9e2fa
< GitHub187> bitcoin/master 15df3c1 Andrew Chow: Persist the datadir after option reset...
< GitHub187> bitcoin/master 57acb82 Andrew Chow: Load choose datadir dialog after options reset
< GitHub187> bitcoin/master d26234a Jonas Schnelli: Merge #8487: Persist the datadir after option reset...
< GitHub68> [bitcoin] jonasschnelli closed pull request #8487: Persist the datadir after option reset (master...persist-datadir) https://github.com/bitcoin/bitcoin/pull/8487
< wumpus> it's kind of clever
< sipa> and the reason that std::forward doesn't happen automatically is that of course you're allowed to use x multiple times in the function body, as it is a normal variable
< wumpus> yes, makes sense
< sipa> so in that case you wouldn't want to auto destruct it on first use
< sipa> combining with substructural typing would have been nice :)
< btcdrak> that gitian script by achow101 makes me happy
< sipa> ffs can we send rebroad to a programming class
< luke-jr> Matt's doing one in NY, right? :p
< paveljanik> rebroad looking for a ban 8)
< GitHub152> [bitcoin] sipa opened pull request #8589: Inline CTxInWitness inside CTxIn (on top of #8580) (master...segwitinlinepain) https://github.com/bitcoin/bitcoin/pull/8589
< btcdrak> sipa: does #8589 replace both #8452 and #8580?
< sipa> yes
< btcdrak> may be better to close those two?
< sipa> well i don't know whether #8451 or #8580 will be accepted
< sipa> i guess i could wait with #8589 and #8452 until that choice is made
< GitHub70> [bitcoin] sipa closed pull request #8452: Code simplification: inline CTxInWitness inside CTxIn (master...segwitinline) https://github.com/bitcoin/bitcoin/pull/8452
< GitHub60> [bitcoin] sipa closed pull request #7984: Optional parameter for rescans to start at a specified height (master...rescan-fromheight) https://github.com/bitcoin/bitcoin/pull/7984
< GitHub54> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/d26234a9e2fa...9d0f43b7ca72
< GitHub54> bitcoin/master be1d451 Will Binns: contributing.md: Fix formatting...
< GitHub54> bitcoin/master 9d0f43b Pieter Wuille: Merge #8226: contributing.md: Fix formatting (line lengths and smart quotes)...
< GitHub120> [bitcoin] sipa closed pull request #8226: contributing.md: Fix formatting (line lengths and smart quotes) (master...binns-contributing-formatting) https://github.com/bitcoin/bitcoin/pull/8226
< GitHub179> [bitcoin] jonasschnelli pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/9d0f43b7ca72...0606f95b1ea8
< GitHub179> bitcoin/master 2f32c82 Jonas Schnelli: [Qt] show network/chain errors in the GUI
< GitHub179> bitcoin/master 0606f95 Jonas Schnelli: Merge #7579: [Qt] show network/chain errors in the GUI...
< GitHub28> [bitcoin] jonasschnelli closed pull request #7579: [Qt] show network/chain errors in the GUI (master...2016/02/gui_alert) https://github.com/bitcoin/bitcoin/pull/7579
< GitHub83> [bitcoin] MarcoFalke opened pull request #8590: Remove unused variables (master...Mf1608-unusedCode) https://github.com/bitcoin/bitcoin/pull/8590
< jonasschnelli> Travis failed on master: https://travis-ci.org/bitcoin/bitcoin/jobs/155043866#L2347
< jonasschnelli> compactblock test
< GitHub69> [bitcoin] sipa pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/0606f95b1ea8...53f8f226bd1d
< GitHub69> bitcoin/master f13c1ba Michael Rotarius: Move AdvertiseLocal debug output to net category
< GitHub69> bitcoin/master 53f8f22 Pieter Wuille: Merge #8462: Move AdvertiseLocal debug output to net category...
< GitHub172> [bitcoin] sipa closed pull request #8462: Move AdvertiseLocal debug output to net category (master...advertiselocal) https://github.com/bitcoin/bitcoin/pull/8462
< sipa> jonasschnelli: ugh
< jonasschnelli> probably a random error...
< jonasschnelli> (one of these)
< sipa> i'm restarting
< jonasschnelli> ok
< GitHub181> [bitcoin] rebroad opened pull request #8591: Do not relay or mine excessive sighash transactions (master...NoExcessiveSighashTxs) https://github.com/bitcoin/bitcoin/pull/8591
< GitHub149> [bitcoin] MarcoFalke closed pull request #8579: Performance: Prefer prefix operator for non-primitive types (master...Mf1608-perfIter) https://github.com/bitcoin/bitcoin/pull/8579
< GitHub76> [bitcoin] sipa closed pull request #8591: Do not relay or mine excessive sighash transactions (master...NoExcessiveSighashTxs) https://github.com/bitcoin/bitcoin/pull/8591
< GitHub80> [bitcoin] hejunbok opened pull request #8592: 0.13 (master...0.13) https://github.com/bitcoin/bitcoin/pull/8592
< GitHub7> [bitcoin] hejunbok closed pull request #8592: 0.13 (master...0.13) https://github.com/bitcoin/bitcoin/pull/8592
< GitHub39> [bitcoin] jl2012 opened pull request #8593: Verify all incoming txs unless the witness stripped size is >100kB (master...verifyalltx) https://github.com/bitcoin/bitcoin/pull/8593
< luke-jr> MarcoFalke: https://github.com/bitcoin/bitcoin/pull/6996 my rebase already has the tests taken care of too..
< MarcoFalke> Nice, then ask sipa to force push :)
< luke-jr> implicitly did 16 days ago :p
< sipa> luke-jr: how does it deal with invalidateblock?
< sipa> you could return later to something with a lower chainwork
< luke-jr> sipa: dunno, I would assume like any other invalid block declared precious? O.o
< sipa> i guess it's such an edge case it doesn't matter
< luke-jr> doesn't preciousblock just give a small bias toward the block in question, in best-chainwork selection?
< sipa> yes
< sipa> right, i guess it doesn't matter even
< sipa> thanks, i'll update the branch
< Lauda> Is the credit list @release of a version updated frequently/automatically/manually?
< luke-jr> Lauda: you mean the list of contributors at the end of the release notes? typically comes from a git log, sometimes with manual adjustments
< Lauda> Yes, that list. Thanks for the information.
< sipa> well it's not automatically updated at all... it is compiled by wumpus at release time
< Lauda> Based on what? Seems some people are on the list due to just having a pull request or two that eventually got closed.
< sipa> commits
< Lauda> One has to have a merged commit to be on that list?
< luke-jr> Lauda: git shortlog v0.12.1..v0.13.0 | perl -nle 'm/^(\S.*)\s\(.*$/ && print $1' | sort
< MarcoFalke> Is the script wumpus uses public?
< sipa> Lauda: yes
< sipa> that's the necessary and sufficient condition
< MarcoFalke> In some corner cases it does misattribution.
< MarcoFalke> E.g. I open a pull but I am not author of the commits
< Lauda> I think it did, sec.
< MarcoFalke> I reported this once so it might be fixed by now
< wumpus> for the credits list I prefer erring on the safe side, e.g. including more than necessary instead of less
< wumpus> MarcoFalke: any specific ones?
< MarcoFalke> This was in 0.12.x
< MarcoFalke> already fixed
< * luke-jr> ponders what wumpus's LC_COLLATE is
< wumpus> that script simply looks who submitted the pull, so if you submitted someone elses' pulls then that needs to be manually corrected
< wumpus> it's pretty dumb and I end up doing a lot of work myself
< MarcoFalke> I wonder if the script could be included in the repo.
< MarcoFalke> Maybe people will help you maintain it
< wumpus> you should look over the list and correct things that aren't correct
< wumpus> oh no way, it's crap
< MarcoFalke> ha, I guessed it
< MarcoFalke> :)
< MarcoFalke> I wouldn't want to share my scripts ...
< luke-jr> Lauda: git shortlog --use-mailmap v0.12.1..v0.13.0 | perl -nle 'm/^(\S.*)\s\(.*$/ && print $1' | sort | uniq # just needs LC_COLLATE and a mailmap file :P
< Lauda> Just.. :p
< wumpus> maybe at some point I get to cleaning it up for release, but I only tend to touch it for releases
< luke-jr> Lauda: you should see my scripts for merging translation files.. :P
< Lauda> It must be 'nice' looking :)
< wumpus> luke-jr: I don't even know what LC_COLLATE does
< wumpus> let alone having changed it manually :)
< wumpus> it's not even defined in my env
< wumpus> postgresql?
< sipa> sort order
< Lauda> It has the flag explanation
< Lauda> Collation order (LC_COLLATE) to use in the new database. This affects the sort order applied to strings, e.g. in queries with ORDER BY, as well as the order used in indexes on text columns. The default is to use the collation order of the template database. See below for additional restrictions.
< luke-jr> wumpus: your sort program does a different order than mine :p
< wumpus> my sort program probably sucks...
< luke-jr> wumpus: run `locale` and check the output
< wumpus> LC_COLLATE="en_US.UTF-8"
< luke-jr> hmm
< wumpus> maybe I should set it to Chinese
< wumpus> or Russian
< sipa> wumpus: do you use sleepsort?
< luke-jr> wumpus: German was a closer approximation in my trial-and-error experience
< wumpus> sipa: yes, the parallelism, it's awesome!
< luke-jr> de_DE.utf8 got me the same order for the linguas files.. until 0.13.0
< wumpus> I don't know what I should feel about people trying to reverse my environment variable settings :)
< luke-jr> lol
< Lauda> haha
< luke-jr> it's what happens when doc/translation_process.md results in a resorting ;p
< wumpus> python has locale independent sorting AFAIK, but yes, the list-authors script is a shell script
< wumpus> and itindeed calls the sort utility
< wumpus> so, so far you're right
< wumpus> it's curious how all those utilities leak information
< wumpus> luke-jr: good idea on using --use-mailmap, I was doing a manual processing step for that
< afk11> bitcoincore.org is behind CloudFlare - following the gitian instructions lead to a 403 if CloudFlare decides it doesn't like you. https://github.com/bitcoin/bitcoin/blob/master/doc/release-process.md#fetch-and-create-inputs-first-time-or-when-dependency-versions-change
< btcdrak> afk11: which step?
< afk11> wget anything..
< Lightsword> is it hitting a captcha page?
< afk11> osslsigncode-Backports-to-1.7.1.patch loaded from cfields directory on bitcoincore.org in this case.
< afk11> Lightsword, yep
< wumpus> I don't think that's correct
< luke-jr> hm, I thought there was a different subdomain for downloads
< Lightsword> yeah…that’s their ddos detection getting triggered, btcdrak maybe you can lower the sensitivity
< wumpus> that's where it redirects, and that one's not behind cloudflare
< Lightsword> wumpus, dev.bitcoincore.org is showing behind cloudflare for me
< afk11> I'll PR that link instead
< afk11> oh
< wumpus> ok, i didn't use to be, I give up
< wumpus> maybe someone else can host the file somewhere else and give a fallback url...
< Lightsword> there should be a way to whitelist urls in cloudflare to turn off the ddos protection
< Lightsword> although not sure if that’s available in the free plan
< wumpus> dev. shouldn't be behind cloudflare in the first place
< wumpus> it's really eating the internet, isn't it cloudflare
< wumpus> thanks luke-jr
< gmaxwell> sipa: in the feeler stuff that was just merged, what is the purpose of FEELER_SLEEP_WINDOW, when the timing of the feelers is accomplished via the possion next-send?
< wumpus> btcdrak: any idea what is happening there?
< sipa> gmaxwell: ethan explained that in a comment
< gmaxwell> ah, I see it. that should have ended up as a comment in the code.
< gmaxwell> As it stands, if I later reworked that code and didn't know they both went in at once, I might have removed the latter delay.
< afk11> thanks luke-jr!
< btcdrak> wumpus: this is why I wanted to change fallback URL remember.
< btcdrak> afk11: are you running behind Tor?
< gmaxwell> sipa: I wonder if their test before evict shouldn't be accomplished by having a seperate table of recently evicted entries which are prioritized for feeler probes.
< gmaxwell> rather than having the eviction directly trigger a connection.
< btcdrak> afk11: it tends to happen on tor exits. settings are on minimal which basically flags Tor exit nodes.
< sipa> gmaxwell: i don't think i ever reviewed test before evict
< wumpus> btcdrak: but how did dev.bitcoincore.org end up behind cloudflare?
< gmaxwell> sipa: ha, I was asking you because you had a bunch of outdated comments on the PR.
< wumpus> I'm surprised that doesn't interacts wth the OSX SDK download which checks based on source IP
< sipa> gmaxwell: test-before-evict or feeler?
< gmaxwell> oh oh sorry. I was commenting on the comments in feeler about test before evict.
< afk11> btcdrak, yeah, it's using tor. might be better to host it elsewhere, people do this on servers after all
< wumpus> I'd hope that patch gets merged in upstream at some point so we don't need it anymore at all
< btcdrak> wumpus: it's in our PMs iirc with lots of details. firstly the fallback URL points to bitcoincore.org so there is a redirect to dev.bitcoincore.org. so the initial request hits CF, game over. the firewall settings are on min, but you cant disable it on the free plan and it flags some IPs very very occasionally if they are "abusive" tor exits usually. You
< btcdrak> didnt want to change the fallback URL in gitian setup, so I had to use a redirect. secondly there isnt a valid SSL cert on dev so it uses the CF one, plus there was the issue of transient hosting (and wanting the least maintenance options).
< btcdrak> afk11: you're the first person who has complained, I'm not really sure why a server would be running exclusively behind tor unless it's a dnm :-p
< wumpus> btcdrak: I know bitcoincore.org is on cloudflare and that we've set up the redirects on bitcoincore.org
< wumpus> btcdrak: what I don't understand is why dev.bitcoincore.org is behind cloudflare
< btcdrak> it has to be to get the SSL.
< wumpus> dev.bitconcore.org has no SSL of itself? okay
< wumpus> yes, I probably forgot
< btcdrak> unless you want to buy a certificate. but it doesnt matter, the gitian thing would fail for afk11 anyway because of the redirect
< wumpus> well we could easily change that link
< Lightsword> btcdrak, just use letsencrypt for SSL on dev
< afk11> btcdrak, in this case, it's a qube over tor!
< luke-jr> …
< luke-jr> so you're saying people connect to CloudFlare over SSL, and then it connects to the real server unencrypted?
< luke-jr> wtf is the point
< btcdrak> letencrypt only issues certs for 3 months.
< Lightsword> btcdrak, letsencrypt has autorenew...
< btcdrak> luke-jr: no, the server is SSL
< afk11> you can set a crontab (certauto is a tool for it I think) to auto-renew
< btcdrak> it's just expired
< luke-jr> aha, ok
< luke-jr> how about StartCom?
< Lightsword> btcdrak, expiring for letsencrypt certificates shouldn’t ben an issue
< wumpus> hosting files is one of our bigger problems
< btcdrak> if we were to change the gitian fallback URL we could just point it to github tbf
< wumpus> but that's what you get with an almost zero project budget :)
< Lightsword> since you normally run an autoupdater on the server that will periodically renew the letsencrypt cert before it expires
< btcdrak> Lightsword: no, all those moving parts fail and it's a lot of maintenance in practice.
< Lightsword> hmm, I thought they had made it fairly reliable by now
< btcdrak> the gitian fallback should be hosted on github releases page and that would solve the issue.
< wumpus> is there some cheap https-supporting hosting that is not behind cloudflare?
< luke-jr> what happened to bitcoin.org's maintainers (not bitcoin.org itself) offering to host bitcoincore.org also?
< btcdrak> luke-jr: their hosting is due to run out soon.
< luke-jr> ah
< wumpus> they have the same proble
< sipa> who is bitcoin.org's maintainers, and who is bitcoin.org itself?
< btcdrak> actually, Pindar is getting us a budget for about 5 years infrastructure hosting.
< luke-jr> sipa: saivann/harding vs theymos/Cobra/someoneelse
< sipa> ah, ok
< btcdrak> should have the funding by October/November.
< gmaxwell> letsencrypt even sends email if you're going to expire.
< btcdrak> when wumpus and I spoke, he was quite clear about having a long term solution. also infrastructure hosting is a pita
< gmaxwell> Please never again setup pretextual security like that, it's an embarassment.
< Lightsword> for nginx should work
< btcdrak> gmaxwell: pretextural what?
< Lightsword> or certbot
< btcdrak> sorry, I mean what is pretextural
< wumpus> huh gmaxwell?
< gmaxwell> proxying through a third party to 'add SSL' when the connection just leaves cloudflare unencrypted and unauthenticated.
< gmaxwell> It appears secure to the user, but it's secure to a third party, and not even secure to the backend.
< afk11> certbot works quite well
< btcdrak> gmaxwell it's _not_
< * luke-jr> assumed btcdrak meant CloudFlare pinned the expired cert or something
< btcdrak> it's SSL to SSL
< wumpus> that's not even the case
< btcdrak> btw, this is the PR https://github.com/bitcoin/bitcoin/pull/7351
< afk11> btcdrak, it's not. it can be (gets you over CN mismatches), but they will proxy to a HTTP server
< btcdrak> afk11: no they wont, because it's set to Full SSL
< wumpus> gmaxwell: also dev.bitcoincore.org is *not* used for anything security critical, there are just some fallback files and gitian checks the hash
< wumpus> the only reason ssl was necessary there is that browsers frown on redirecting https to http
< sipa> i'm confused
< btcdrak> afk11: you can also configure the webserver to only accept connections from a certificate signed by CF
< gmaxwell> okay I was responding to "why is ... behind cloudflare" "it has to be to get the ssl".
< Lightsword> wumpus, bitcoincore.org is HSTS preloaded so http through browser won’t work at all for chrome and firefox
< wumpus> no, it's because of a cert issue with dev.bitcoincore.org, which is hopefully temporary
< gmaxwell> Okay.
< wumpus> Lightsword: right, that too
< gmaxwell> I thought this was another instance of "something hosted on github pages, then people complained about it not having ssl because security, so someone layerd cloudflare on top". My apologies.
< Lightsword> doesn’t github force https?
< btcdrak> remember also, bitcoincore.org has HSTS preloading so we cannot use http:// on that domain.
< luke-jr> HSTS affects subdomains? :x
< wumpus> I'm not sure why we're even having an argument about this, is there an actual problem?
< wumpus> luke-jr: you can configure it to
< Lightsword> luke-jr, yeah it’s required to be on for HSTS preloading
< btcdrak> ^
< wumpus> please keep this channel restricted to bitcoin core development
< sipa> we need #bitcoin-core-org-dev :)
< btcdrak> LOL
< luke-jr> almost time for meeting btw
< wumpus> all the http and gpg etc talk is getting heavily off topic, yes I know I have been participating too
< sipa> wumpus: it looks like you've spent an awful lot of time on administrative stuff with 0.13
< gmaxwell> It's helpful to collaborate on things, even if they're offtopic. But not polite to people trying to read the logs. :)
< wumpus> sipa: yes :(
< btcdrak> more channels!
< * btcdrak> dashes for a tea break before the meeting
< gmaxwell> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier
< instagibbs> here
< CodeShark> morning
< cfields_> hi, here
< kanzure> greetz
< murch> evening
< jonasschnelli> Meeting?
< sipa> pŗėșểñť
< * luke-jr> gives sipa some frogs as a present.
< gmaxwell> yum.
< paveljanik> topics?
< MarcoFalke> no chair?
< paveljanik> I'd like to ask for more ACKs on Wshadow PRs
< gmaxwell> too bad, we haven't started so you can't ask for that yet.
< gmaxwell> :P
< kanzure> hehe
< wumpus> #startmeeting
< lightningbot> Meeting started Thu Aug 25 19:02:37 2016 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
< lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
< btcdrak> here
< paveljanik> I'd like to ask for more ACKs on Wshadow PRs
< paveljanik> ;-)
< cfields_> seconded. Will ACK/re-ACK.
< gmaxwell> paveljanik: as you wish.
< instagibbs> paveljanik, pr #?
< cfields_> (i caught a bug of my own with -Wshadow yesterday)
< instagibbs> for those not initiated
< MarcoFalke> paveljanik: I ACKed all. Anything I missed?
< gmaxwell> Give the PP@.
< gmaxwell> gr. PR#
< paveljanik> 8449, 8472, 8191, 8163, 8109
< paveljanik> but yes, Marco's link is even better
< wumpus> anything that really needs to be discussed at the meetng?
< CodeShark> no, we've already accomplished everything - there's nothing more to do ever
< gmaxwell> woot.
< sipa> i'd like to bring up the various proposals for segwit DoS protection
< gmaxwell> ^
< instagibbs> ack
< petertodd> ack
< btcdrak> ack
< wumpus> well I'm very happy we finally managed to release 0.13.0, congrats everyone!
< paveljanik> Do we have any numbers of 0.13.0 nodes?
< cfields_> topic suggestion: codesigning update
< paveljanik> congrats to wumpus!
< * jonasschnelli> is checking the 0.13 nodes on his seeder
< instagibbs> paveljanik, few hundred ones bitnodes has reached
< btcdrak> beer for wumpus
< sipa> wumpus: congrats to all, and thanks a lot wumpus
< wumpus> #topic proposals for segwit DoS protection
< gmaxwell> Some of this is more general than segwit. There as some existing minor vulnerablities that have been ignored, which were pointed out in segwit form. Good to resolve those too.
< murch> paveljanik: 322 (according to bitnodes)
< gmaxwell> Congrats on 0.13. It looks like a great release.
< jonasschnelli> cat dnsseed.dump | grep "0.13.0" | grep " 1 " | wc -l ---> 62
< sipa> so the main issue is #8279
< jonasschnelli> 213 seeds in non-good state (probably just set up)
< jonasschnelli> s/seeds/nodes
< gmaxwell> there are a lot more parties running it than accepting inbound, as typical.
< sipa> and there have been several proposed solutions, including: verify all inputs (even if the transaction is too big or below feerate), not entering failed witness txn into the reject table, making feefilter mandatory, ...
< gmaxwell> sipa: I think all of these changes are beneficial.
< kanzure> this is separate from the malleability confusion someone posted about the other day, ya?
< luke-jr> sipa: making feefilter mandatory, or merely always using it?
< instagibbs> kanzure, the linked github issue explains it
< sipa> luke-jr: you need to be able to rely on it, and be able to ban peers who disregard it
< gmaxwell> luke-jr: me means giving it an ack message and punting peers that violate it.
< sipa> luke-jr: as you're moving the responsibility for filtering things you won't accept to the sender
< luke-jr> ok, so <0.12 nodes wouldn't be kicked
< gmaxwell> right.
< petertodd> segwit is a good opportunity to make feefilter mandatory, given we're completely changing what nodes we connect too
< kanzure> instagibbs: for example; someone was unaware that adding witness data to a non-segwit script was consensus-invalid. but er, your link seems more productive anyway, so nevermind.
< btcdrak> petertodd: i think that is the plan
< sipa> yes, but it seems hasty to do that along with 0.13.1
< gmaxwell> in any case, "verify all inputs" was a proposal from before segwit was even imagined.
< petertodd> sipa: sure, but 0.13.1 also "hastily" has to stop fetching stuff from non-segwit nodes
< sipa> petertodd: by hasty i mean we haven't implemented/tested/... a mandatory feefilter yet
< sipa> we have tested the relay behaviour of segwit vs non-segwit peers
< gmaxwell> I believe the primary reason we didn't just make the verify all inputs change is that it was proposed before feefilter, during spam attacks, and there was a lot of rejected stuff coming from even cooperative peers. That should be resolving now.
< petertodd> sipa: but thats the thing, you can just make it mandatory if the peer is a segwit peer, and leave the current behavior the same otherwise - non-segwit peers can't send us segwit txs at all (other than ones stripped of their witnesses which is easy to detect)
< sipa> petertodd: we still need a new network message etc
< sipa> otherwise peers can just ignore your feefilter
< petertodd> sipa: maybe I'm missing something, but why do we need a new network message?
< luke-jr> sipa: I think petertodd is saying "segwit peers MUST NOT ignore feefilter"
< gmaxwell> petertodd: because the protocol is async, we don't know if the far end has recieved our feefilter request yet.
< sipa> petertodd: you don't know what the last feefilter message you sent is that your peer received
< petertodd> sipa: ah right, duh...
< gmaxwell> so you send a feefilter, remote sends an inv.. you get the data.. oops failed filter.. Was the far end bad or just too fast?
< gmaxwell> so the suggestion there is to add a feefilterack, and punt peers that should send it but don't send it 'fast enough'
< petertodd> sipa: I had it in my head that feefilter worked the other way around, and already had a new inv with fee info...
< luke-jr> are there any possible nodes where implementing feefilter would be a burden? I think we already need fee info to relay transactions anyway, right?
< gmaxwell> luke-jr: yes, thats part of the consideration.
< sipa> i expect that most of the problems are solved with just having feefilter
< sipa> and we can just do #8525 now
< luke-jr> hm, what if we change feerate's definition and want to get rid of the current algo some day?
< sipa> (don't store witness txn in the reject cache)
< gmaxwell> I think we should be able to always validate now that feefilter is in place. The only argument I recall against always validating is that that a significant fraction of all recieved txn failed the fee check. That should be much less true now.
< petertodd> sipa: that could be a problem in the future if we have peers with different non-fee-related acceptance criteria than us
< CodeShark> I kinda like the idea of an inv with fee data - slightly larger messages but no need for state
< gmaxwell> reject cache was 90% implemented for lack of a fee filter.
< sipa> gmaxwell: that would open up many attacks... i don't think those are impossible to fix, but it needs a lot of analysis
< gmaxwell> I disagree.
< sipa> i like the idea of always validating (as #8593 does), but it feels risky
< gmaxwell> An uncached UTXO lookup eclipses the validation costs on most systems by orders of magnitude.
< petertodd> gmaxwell: yeah, I don't see how an attacker who can DoS attack by sending specially crafted txs is ever stopped by the other validation rules, and yes, UTXO looking is expensive too
< CodeShark> can't we consider extreme cases?
< CodeShark> what's the maximum possible validation cost for a single transaction?
< sipa> huge... remember we can't have any feerate or size based rejection criteria beforehand
< gmaxwell> well you can have a stripped size limit.
< sipa> yes, #8593 does that
< sipa> but you can't use fee
< sipa> or at least not actual feerate, only stripped-size-feerate
< CodeShark> I'm not sure if this is what petertodd was referring to above - but I was imagining an inv that sends not just transaction hash - it also sends fee amount and tx size
< petertodd> sipa: so, an attacker who was trying to DoS attack you would just pay enough fee, with a tx that triggered O(n^2) complexity
< petertodd> sipa: so I don't see the value of feerate in preventing DoS attack there anyway
< sipa> an attacker can now craft a very expensive valid transaction that has a feerate too low to be acceptable to you, but stripped-size-feerate high enough
< sipa> and you'll validate it, but not relay
< gmaxwell> CodeShark: improved invs are a fine thing, longer term (though we should be spending our time on reconcillation, IMO) -- but I think the focus of the discussion is on shorter term since we likely need some improvements for 0.13.
< petertodd> right, but relaying is worse! we're then attacking others, and there's still no guarantee the tx will get mined, allowing the attacker to doublespend and repeat
< sipa> petertodd: well in no case will any of this relay
< gmaxwell> An alternative jl2012 suggested to validating all was to randomly validate some percentage. This will allow you to punt a peer sending you garbage, but you only take a fraction of the cost.
< CodeShark> gmaxwell: it seems simpler to just create a new static inv structure than to have to manage session states and do feefilteracks and stuff like that
< petertodd> sipa: sure, so then why not kick peers that send us DoS attack txs? IsStandard() has been around long enough that we could get away with that
< sipa> CodeShark: i think mandatory feefilter is a better solution than adding 10 kB of data to each inv
< gmaxwell> CodeShark: not so, right now inv bandwidth is already _greater_ than transaction bandwidth. You can't really escape the need to have people not send you things you're totally disinterested in.
< CodeShark> 10kB of data?
< CodeShark> the fee is uint64 and the tx size is varint
< sipa> CodeShark: txid, wtxid, fee, feerate, stripped feerate, size, sigops, bytes hashed, ... anything else that may affect your policy
< CodeShark> ok, fair enough
< sipa> inv bandwidth is larger than tx bandwidth by a factor of 5 on my node
< luke-jr> sipa: let's make it configurable! /s
< sipa> and that's between feefilter nodes
< sipa> so... do we think jl2012's approach of validating everything (or a fraction thereof) is the way to go for 0.13.1
< sipa> ?
< CodeShark> I haven't done the math
< sipa> the code is simple at least
< sipa> CodeShark: getpeerinfo
< sipa> shows bytes per message
< gmaxwell> This is why I think we need reconciliation for any inv improvement long term.
< sipa> yes
< sipa> but the meeting time is half, let's not stray too far
< sipa> i'd like to know what we're going to do for 0.13.1
< instagibbs> sipa, your guess is as ~~good as~~ better than mine
< CodeShark> sipa: I mean, I haven't done the math for validation cost edge cases
< gmaxwell> sipa: I jl2012's sampling suggestion is a pretty good idea, coupled with not reject filtering them.
< sipa> gmaxwell: right... either fully validate, or don't store in reject
< sipa> i like
< sipa> ok, other topics?
< petertodd> I suspect we'd be better off kicking peers who send us >100KB txs
< petertodd> (rather than any kind of probabalistic thing)
< afk11> hardware signing BIP?
< sipa> there were a few other topic ideas before
< wumpus> #topic code signing
< petertodd> afk11: that's off-topic to Bitcoin Core development right now
< gmaxwell> petertodd: yes, we can do that-- and I think we should--, but I think it's not sufficient.
< wumpus> @cfields_
< wumpus> petertodd: it's on topic for the future though
< petertodd> gmaxwell: well, I mean in combination with fully validating
< petertodd> wumpus: sure, why I said "right now" :)
< gmaxwell> petertodd: okay. lets talk after meeting.
< cfields_> so, i just wanted to bring this up before i finish the work, in case anyone has any suggestions for improvements
< cfields_> I have a working codesigner for osx from linux, so an osx machine is no longer needed for the release process
< gmaxwell> \O/
< sipa> great
< cfields_> but i'm curious to see if anyone has any suggstions for more complicated/distributed signing schemes, before just implementing it as it was before
< CodeShark> cfields_: nice
< btcdrak> cfields_ except for extracting the MacOS SDK...
< jonasschnelli> cfields_: very nice. Lots of devs are looking for something!
< jonasschnelli> like that
< wumpus> cfields_: nice work!
< sipa> cfields_: what cryptography does it use?
< cfields_> (as of now it produces valid signed binaries given very specific constraints, it just needs to be tidied up and plugged into gitian)
< luke-jr> cfields_: plugged into gitian?
< petertodd> cfields_: nice license!
< jonasschnelli> GNU AFFERO GENERAL PUBLIC LICENSE,... hell!
< gmaxwell> :)
< * luke-jr> agrees it's a good choice of license. ☺
< cfields_> sipa: there's no choice in that, i haven't even looked into the signature type
< gmaxwell> If it is RSA or schnorr we can secret share the key.
< cfields_> sipa: it basically collects all of the files to be embedded, creates a custom wonky structure out of them, signs, and embeds the final hash in the binary
< petertodd> jonasschnelli: I like AGPL because I'm not stinkin' commie
< cfields_> along with a mostly redundant detached xml with the same hashes
< gmaxwell> cfields_: how are you signing if you don't know how it's signing? :)
< cfields_> sorry, not my license choice. 99% of the code is borrowed from an ios tool
< jonasschnelli> I guess you need to update "sudo xcode-select --switch /Applications/Xcode-4.6.3.app"
< cfields_> gmaxwell: PKCS7_sign(), openssl does the hard work :)
< gmaxwell> nothing wrong with AGPL for a tool like this, licensing is a tool.
< jonasschnelli> Hard to download older version of Xcode
< gmaxwell> cfields_: how big is the signature? :P
< cfields_> gmaxwell: i can dump you some samples after the meeting
< gmaxwell> great, in any case, I'd like to talk about integrating multiparty signing into it. it might be easy to accomplish.
< cfields_> but back to the point: is anyone interested in coming up with a signing scheme that deviates from what we have now?
< cfields_> oh, i suppose that's why you want to know about sig type :)
< gmaxwell> Yes.
< sipa> yes.
< jonasschnelli> gmaxwell: my OSX code signing certs are RSA 2048 Bit
< michagogo> cfields_: your signer is missing a README
< petertodd> cfields_: I'm already working on codesigning as my first application for dex
< sipa> oh, let's just switch bitcoin to use prime factoring as PoW
< cfields_> same here, what i'm not sure about is how much you're allowed to change around the sig format
< cfields_> (for osx, the signing cert must be signed by apple)
< wumpus> yes, IIRC PKCS7 is simply RSA wrapped in ASN.1 mess
< cfields_> yes
< cfields_> and apple adds a weird little scripting language around 'designated requirements'
< wumpus> congrats on cracking that :)
< sipa> cfields_: well there could be several signers, and we get a cert for the combined key?
< jonasschnelli> The question is, does code-signing really increases the security of bitcoin-core...
< cfields_> so you can add other restrictions, but again, i'm not sure how much apple lets you deviate from the default
< luke-jr> jonasschnelli: imagine where common gitian builders all sign themselves and combine the sig
< gmaxwell> the straightforward way of doing threshold RSA needs a trusted dealer. but at least its multiparty after setup.
< petertodd> jonasschnelli: I suspect the code-signature-verification is what increases the security of bitcoin-core :)
< jonasschnelli> petertodd: i rather say verifing of gitian signatures...
< luke-jr> jonasschnelli: essentially we'd get the OS to verify the gitian builds in this case
< cfields_> jonasschnelli: yes and no. the code-signing itself, not really. but code-signing does allow you to force-on the osx sandbox, which is interesting for security
< jonasschnelli> OSX code signatures are from "Bitcoin Foundation"... anyone could hold the key
< sipa> i'm sure we can get a new cert
< petertodd> jonasschnelli: the gitian signatures aren't ever going to be much more secure than the code signatures
< jonasschnelli> cfields_: Yes. But thats just a restrriction from apple...
< cfields_> jonasschnelli: (we haven't messed with the sandboxing yet, afaik)
< jonasschnelli> and sandboxing is impossible for bitcoin core right now
< petertodd> jonasschnelli: also, lots of people don't use the gitian builds (I've never personally run one)
< gmaxwell> We can get a new cert and we should improve the maintaince of it so no one person needs to hold the key-- it's a good practice.
< kanzure> someone should make sure to watch petertodd do a gitian build in milan :)
< cfields_> jonasschnelli: ah ok, i've been meaning to look into it. if you already have, i'm very curious to hear your thoughts after the meeting
< jonasschnelli> cfields_: sure. But nice work! Linux based OSX/iOS code signing is highly appriciated by lots of devs.
< cfields_> just to reiterate though: osx-codesign is 99% ripped from a tool from Saurek. I just ported to osx and updated.
< kanzure> saurik
< cfields_> *Saurik. Thanks.
< sipa> saurik?
< cfields_> jonasschnelli: right, mostly I got tired of having to have my macbook for signing.
< kanzure> /whois saurik
< cfields_> and it limits the amount of people who can do the signing
< jonasschnelli> Indeed
< jonasschnelli> though you can run a OSX VM on a Linux machine. :)
< cfields_> heh
< jonasschnelli> (and violate apples TOC)
< * luke-jr> peers at his common channels with saurik
< gmaxwell> cfields_: so I think good job, continue on, and I (and sipa?) will look into threshold signing for 2048 bit RSA and see if we can't get it so that no single party holds that key.
< kanzure> sipa: he is known for various ios reverse engineering things
< CodeShark> is 4096 overkill?
< sipa> ah, cool
< cfields_> gmaxwell: perfect, thanks.
< gmaxwell> (as an aside, this could also be done with wumpus' release key)
< kanzure> sipa: (pm)
< jeremyrubin> 4096 not overkill
< sipa> CodeShark: 3000 bits RSA is approximately equivalent with 256 bit EC
< jonasschnelli> CodeShark: 2048 gives use 118bit of security.. ECDSA 128 I guess... enought?
< gmaxwell> CodeShark: I believe we don't get to choose the parameters of this key.
< jonasschnelli> Yes. Apples does it.
< CodeShark> oh, right
< gmaxwell> Otherwise it would be 4096 bit. because duh.
< cfields_> gmaxwell: i'll try to find out if other signature types are possible.
< * jonasschnelli> is trying a 4096 key
< gmaxwell> (the size and performance are basically irrelevant)
< jeremyrubin> unrelated question while everyone is in meeting: having a lot of trouble getting my code to pass travis, but can pass locally on a few diff machines. If anyone has any ideas, ping me after meeting.
< jonasschnelli> To shortly come back to afk11 topic: there has been a proposal for detached signing: https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2016-August/013008.html
< kanzure> jeremyrubin: got the core dumps yet?
< jonasschnelli> Which would allow decoupling the keys from the wallet(system)
< jeremyrubin> kanzure: nope; i tried changing the travis script to dump them but couldn't see anything.
< kanzure> you can probably call gdb bt from inside the after_failure segment
< jonasschnelli> to end the wild-west APIs/interfaces that are currently been implemented by wallets and hardware-wallet vendors
< wumpus> I very much like the idea of standardizing detached signing
< wumpus> yes, exactly
< jonasschnelli> One of the main problems users have and will have with wallets is to keep private keys private.
< wumpus> it makes no sense to try to support detached signing in bitcoin core until there is some kind of standard, if every hw vendor is going to hook in their own code in their own way it isgoing to be a mess
< btcdrak> jonasschnelli: well you'd hope hardware signing devices will become more commonplace
< CodeShark> they will
< instagibbs> 7 minutes, any more topics?
< CodeShark> it's not something we should hope for - it's something we should make happen
< wumpus> no doubt about that, though it's two-sided, software also needs to support them
< jonasschnelli> Yes. I proposed the URLScheme standard... even, I don't like it in the first place.. but its probably the only choice if we'd like to address all systems and platforms.
< wumpus> CodeShark: jonasschnelli is helping make it happen
< sipa> the topic is still codesigning :)
< petertodd> btcdrak: also note that in systems like Qubes, detached signing in another VM is a significant security improvement even w/o special hardware
< btcdrak> yes. detached signing is more for the bitcoin-dev ML
< petertodd> btcdrak: Qubes does this for GPG already
< gmaxwell> jonasschnelli: I think at this point the discussion should stop and some prototypes should be tried. You've seen there is some support and some opposition, and thats fine.
< wumpus> do we still have anything to say about codesigning?
< btcdrak> petertodd: interesting
< jonasschnelli> gmaxwell: Yes. I'm working on a PoC with static data between Core and iOS.
< sipa> wumpus: i believe not, i was only casually complaining about the random switch of topic :)
< jonasschnelli> my fault. sry.
< gmaxwell> jonasschnelli: okay, even simpler-- I think we should change our standard operation to have walletpassphrase and decrypt and sign done by a seperate mlocked process.
< wumpus> petertodd: yes, it should be general enough to support that scenario as well, but I think that's handled if it works for hw signing, just expose a 'virtual' hw device
< wumpus> sipa: agreed, it's a bit of a sudden switch
< wumpus> time to end the meeting maybe
< wumpus> #endmeeting
< lightningbot> Meeting ended Thu Aug 25 19:57:10 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< murch> sipa: average UTXO set size did turn out to be a very valuable evaluation criteria by the way.
< jonasschnelli> gmaxwell: I think it should be split into two applications
< sipa> murch: cool
< gmaxwell> jonasschnelli: How so?
< jonasschnelli> wallet does only hold pubkeys,... it can retrieve new keys or signatures over the sign:// URL scheme
< jonasschnelli> sign://getkeys?amount=10
< jonasschnelli> sign://sign?type=bitcointx&somedata=xyz
< CodeShark> wallet stuff really needs to be a separate project - bitcoin-core-dev should focus on consensus and p2p stuff :p
< gmaxwell> CodeShark: I disagree.
< jonasschnelli> If we scale and all uses that jump on the nicely scaled bitcoin bus and use webbased or heavy centarlized insecure walltes... we can stop focus on consensus and p2p
< jonasschnelli> *users
< gmaxwell> jonasschnelli: now thats batching things into having 'multiple wallet' data stores.
< jonasschnelli> No really. Just separating private-key from the wallet
< jonasschnelli> *keys
< gmaxwell> you mean seperating the wallet from the wallet.
< CodeShark> the thing that needs to be separated is the signer
< jonasschnelli> No.. the wallet is much more then the private-keys...
< CodeShark> along with all private key data
< jonasschnelli> private-keys do only sign data... 90% is wallet logic.
< CodeShark> and the signing request protocol can be defined abstractly
< gmaxwell> jonasschnelli: Yes it is, but the thing 99% of the care goes into is the keying material.
< gmaxwell> jonasschnelli: in any case, all I am suggesting there is that the wallet be able to store a blob on behalf of the signer, and be able to return that blob to the signer when asking it to sign.
< CodeShark> the signing device doesn't need to store history and perhaps only very little state
< sipa> CodeShark: i think jonasschnelli is well aware of that
< CodeShark> I was replying to gmaxwell's "separate the wallet from the wallet" comment
< jonasschnelli> CodeShark: yes. It just needs the data that needed to verify the data-to-sign
< gmaxwell> jonasschnelli: and then my example of having a seperate interface to collect the passphrase and sign, works fine-- the signer would ask the wallet to store its encrypted key data. Then the signer needs no filesystem access.
< jonasschnelli> If you sign a PDF, you need the PDF along with the sha256(PDF)
< gmaxwell> (for that case)
< jonasschnelli> gmaxwell: what would be in the blob stored on behalf of the signer?
< jonasschnelli> Some king of authentication?
< sipa> jonasschnelli: encrypted keys in gmaxwell's examplr
< jonasschnelli> kind
< sipa> jonasschnelli: but you shouldn't care about what it is
< gmaxwell> jonasschnelli: in the case of the detatched signer which I think we should use in the GUI by default, it would be the encrypted master key. But bitcoin-core wouldn't really know or care what it is.
< michagogo> If anyone here has an iOS device and hasn't heard yet: update to iOS 9.3.5 ASAP.
< da2ce7> gmaxwell: I agree making the Hardware wallet storage-less other than securely holding a decryption key is a very attractive route.
< michagogo> A trio of critical security holes was just patched. And in the meantime stay away from untrusted links.
< CodeShark> the requestor sends a blob + metadata, signer displays metadata to user and asks for authorization, user authorizes, signer returns signed blob
< sipa> da2ce7: i think you misunderstand
< sipa> da2ce7: an actual hardware wallet would have its own state, of course
< gmaxwell> or maybe it wouldn't that would be up to its design.
< CodeShark> the interface could even provide for abstract sighash type
< jonasschnelli> sipa, gmaxwell: thanks... need to think about it. need to go afk.
< sipa> da2ce7: gmaxwell is just suggesting that if thr protocol allows the signer to ask the wallet to store data, it could replace our current wallet encryption scheme as well, by moving the signing part to a separare process
< kanzure> is the hardware wallet BIP thread the right place to pimp "libconsensus in HSM" stuff?
< gmaxwell> NO
< gmaxwell> actually I think that thread should be dropped for now.
< gmaxwell> I think too many speculative things are going for "BIP" now, this isn't really what the bip process is intended for. BIPs shouldn't be for speculative ideas. They should be so that implementations are documented and can interoperate.
< gmaxwell> There are many discussions that are being phrased as "lets have a BIP about X" when they should be "I think it would be useful for me to do X"
< gmaxwell> only after establishing that X is something that people might want to actually do, should it move onto "now we should have a standard for doing X"
< gmaxwell> talking about it in the context of standards first seems to be having bad effects where people oppose ideas seemingly because they don't want to implement more things.
< MarcoFalke> jeremyrubin: You need to solve the merge conflict for travis to pick it up
< MarcoFalke> I will try to reproduce locally
< jeremyrubin> MarcoFalke: ah; will do. It's picked up on my fork
< MarcoFalke> ok
< jeremyrubin> I just added something to print out the test-suite.log after_failure
< jeremyrubin> so that's the newer build
< jonasschnelli> gmaxwell: the hardware wallet signing thing is not really a BIP now. It's just something to think about I have sent to the ML...
< jonasschnelli> But I think a BIP would be the right think for a hardware wallet interface standard..
< jonasschnelli> *Thing
< sipa> sure, eventually
< * jonasschnelli> sometimes thinks that a wallet with privet-keys is like a home door with the key under the doormat
< instagibbs> jonasschnelli, you can start a SIP ;)
< * instagibbs> ducks
< gmaxwell> jonasschnelli: that is going a bit far, ultimately if your computere is totally compromised you probably can't use bitcoin correctly no matter having a hardware wallet or offline signing or what not.. at most you can slow the bleeding.
< gmaxwell> because your compromised computer will substitute any addresses you attempt to pay to.
< gmaxwell> and it will tell you that you've been paid, when you haven't been.
< jeremyrubin> MarcoFalke: I rebased onto master fyi so you'll want to -D your local if you pulled it
< jonasschnelli> Yeah. Agree. But...
< jonasschnelli> A hardware wallet or let say, a detached signer (assume a smartphone) can verify the data(tx) independently.
< jonasschnelli> Even if you computer is totally compromised, your signer would reveal that.
< sipa> jonasschnelli: how does the hw device verify it's sending to the right address
< sipa> it can show you, sure
< jonasschnelli> It would show it on a display?
< sipa> but how do you know the right address if your computer is hacked?
< jonasschnelli> Fees are a bit tricky
< jonasschnelli> That right... Your browser window where the address listed could be already compromised.
< jonasschnelli> In the world of "addresses", we have to assume it has been transmitted untempered
< gmaxwell> jonasschnelli: thats what I mean, if your computer is compromised it doesn't really matter what it displays, at least right now.
< jonasschnelli> At least the signer can identify if a receiving address is owned by the signer and also if a change address is
< jonasschnelli> But you might scan in a QRcode address...
< jonasschnelli> The receiving part can be made relatively secure. But I agree, the pay-to part - in case the pay-to addresses origin for you is your computer - cannot be made much more more secure through a signing device.
< gmaxwell> and similarly, there are two ways you can be robbed, by sending where you don't intend, or by thinking you were paid when you weren't. Existing hardware wallets do nothing about the latter.
< jonasschnelli> Malware targeting only you wallet software would be identified. A clever forged multi-app attack not.
< MarcoFalke> jeremyrubin: This also fails locally on my 14.04vm
< MarcoFalke> Might be easier to debug
< jonasschnelli> gmaxwell: how can the later be exploited? You mean by displaying an attackers address instead of the owner ones?
< jeremyrubin> MarcoFalke: Awesome; maybe it's a virtualization thing
< jeremyrubin> Are you just using a vanilla 14.04?
< gmaxwell> the latter is that the attacker 'pays you' but never authors a transaction and your compromised host displays the transaction as confirmed.
< MarcoFalke> jup, I don't recall any specifics I changed
< jonasschnelli> Ah. I see. Yes. Hard to detect on the host even with a signing device... You would need access to a trusted offline in-sync air gapped node :-)
< gmaxwell> well it's not really that wild, you'd just need your wallet to hand all the payments to the hardware wallet with spv proofs and then it could display your balance and lists of transactions.
< gmaxwell> so at least that could be done.
< jeremyrubin> cool -- do you see anything interesting in the dump? (spinning one up now)
< gmaxwell> but the address substituion attacks don't have as easy a fix.. you need a PKI and send signed payment requests to the wallet. ;(
< luke-jr> instagibbs: SIPs are for altcoin stuffs ;)
< btcdrak> cfields_: I have successfully extracted the MacOS SDK from Xcode DMG directly linux :)
< luke-jr> btcdrak: how?
< btcdrak> black magic :)
< GreenIsMyPepper> ooo that's awesome ^_^
< btcdrak> GreenIsMyPepper: I think I have been cheated! the archive size doesnt match sigh
< luke-jr> ☺
< luke-jr> btcdrak: the files are all empty
< btcdrak> =) oh well
< cfields_> btcdrak: woohoo!
< cfields_> oh, heh
< btcdrak> cfields: :/
< cfields_> yes, hfsmount hasn't been touched in ages afaik
< cfields_> iirc there are files missing too
< btcdrak> I assume the extraction of the partition by 7z is fine, just the hfsmount is incompatible?
< gmaxwell> so I have a suggestion. We have the data, right? uncompress the download, and the take as side information a list of offsets to extract bytes from.
< gmaxwell> We can have side information as part of the process.
< GitHub34> [bitcoin] gmaxwell opened pull request #8594: Do not add random inbound peers to addrman. (master...no_inbound_addr) https://github.com/bitcoin/bitcoin/pull/8594