< Luke-Jr>
is there a way to get [skip ci] without polluting the commit message?
< sipa>
ask a maintainer to kill the travis job :p
< Luke-Jr>
sipa: are you okay with me asking people to remove it from commit messages and ask maintainers instead? :P
< Luke-Jr>
seems like a waste of time
< sipa>
yeah :)
< sipa>
i just wouldn't bother
< CodeShark>
hmm, SoftForkMajorityDesc needs to be moved into the soft forks unit as well...
< CodeShark>
would be nice to define the validation rules for each soft fork in its own module as well...so then the entire soft fork can be specified as a unit that just needs to be added and linked to Bitcoin Core
< CodeShark>
no more need for PRs for that :)
< CodeShark>
other than the trivial link
< CodeShark>
totally doable, too
< CodeShark>
I know how to do it...but it would require moving a few things around in main.cpp
< CodeShark>
hopefully can be done without being too extremely invasive
< CodeShark>
the only real question I have is whether this is desirable...should we make it *that* easy to define and deploy a soft fork?
< CodeShark>
would require some changes to script/interpreter.cpp as well
< CodeShark>
but I think that it could be done in a minimally invasive way...keeping what's already there more or less in tact
< CodeShark>
*intact
< GitHub142>
[bitcoin] dgenr8 opened pull request #6785: Backport to v0.11: In (strCommand == "tx"), return if AlreadyHave() (0.11...already_have_0.11) https://github.com/bitcoin/bitcoin/pull/6785
< GitHub108>
[bitcoin] Diapolo closed pull request #6288: [Qt] fix debug console window handling when e.g. minimized (master...show-rpconsole) https://github.com/bitcoin/bitcoin/pull/6288
< wumpus>
Luke-Jr: [skip ci] only works in the commit message, not the pull title?
< Luke-Jr>
wumpus: ?
< wumpus>
Luke-Jr: I mean - what does travis look at, the pull request title or the commit message?
< Luke-Jr>
no idea, I just don't like it in the commit message :p
< Luke-Jr>
aj: just the usual softfork expectations given SPV limitations
< Luke-Jr>
1-block confirmation is never secure for SPV anyway
< aj>
Luke-Jr: afaics for softforks that upgrade OP_NOP's SPV clients won't have that problem
< aj>
Luke-Jr: (or won't have it any worse than normal anyway)
< Luke-Jr>
aj: I don't follow then.
< aj>
Luke-Jr: (1) afaics, apart from elegius, 99.9% of existing hashpower won't mine OP_NOP txns but will mine nSequence transactions
< Luke-Jr>
oh, I see what you mean
< aj>
Luke-Jr: (2) post softfork, 5% of miners haven't upgraded. but eligius has, so no one will mine OP_NOP stuff, but 5% will mine bad nSeq/relative-timelock txns
< aj>
Luke-Jr: so SPV clients go from .1% (made up number) of hashpower being a problem to 5% of hashpower being a problem
< Luke-Jr>
aj: I don't think there's ever been a softfork without this "problem"
< Luke-Jr>
well, except BIP66 I guess
< Luke-Jr>
not even that actually
< Luke-Jr>
because older block versions were disallowed
< Luke-Jr>
anyway, 5% of hashpower won't get more than a block or two deep
< aj>
Luke-Jr: OP_CLTV won't have that problem, afaics
< Luke-Jr>
perhaps not. but it will be the first not to.
< aj>
Luke-Jr: (wow)
< Luke-Jr>
and SPV 1-block confirmation is not secure regardless
< Luke-Jr>
aj: the versionbits stuff does improve on this
< Luke-Jr>
after the softfork goes active, there is a difficulty-adjustment period (2 weeks) before it is enforced
< Luke-Jr>
so other miners can upgrade
< aj>
Luke-Jr: versionbits maybe makes it a little harder, in that after activation the bit's not set anymore, so you can't even compare the nVersion of the latest block to see that something odd's happening?
< Luke-Jr>
aj: SPV nodes need the full header-history anyway, so they can implement versionbits
< aj>
Luke-Jr: i think, without a soft-fork upgrade, to exploit a SPV client, you need to use your own hashpower to mine a will-be-orphaned block to trick SPV clients; but immediately after soft-fork activation, you have 5% of other people's (ie *free*!) hashpower that will mine will-be-orphaned blocks for you?
< Luke-Jr>
aj: only if 5% got left behind
< Luke-Jr>
aj: the expectation is not that those 5% continue mining invalid blocks
< aj>
Luke-Jr: hmm, is there any way to tell how much hashpower dropped off in previous upgrades?
< Luke-Jr>
dunno, probably just Deepbit
< aj>
Deepbit
< aj>
?
< aj>
oh that was a mining pool, not a stats site? :)
< Luke-Jr>
aj: a mining pool that once had over 50% ;)
< CodeShark>
two things: 1) clients that do not recognize the version can (and *should*) warn the user that something might be up. 2) 1-confirmation reorgs are typical in the daily course of business...around soft fork activations, thin clients (that don't validate blocks) should be even more careful
< Luke-Jr>
CodeShark: will versionbits impl kill mining when it detects it is being softforked out?
< Luke-Jr>
ie, getblocktemplate should probably fail if some activated softfork is unsupported
< Luke-Jr>
CodeShark: btw, I was thinking of the softfork plugin thing a bit ago also, so sounds good to me :P
< CodeShark>
:)
< Luke-Jr>
maybe harder than it seems though in practice
< CodeShark>
I have some ideas for how to do it...but if we're going to be doing a bunch of refactors after 0.12 is released, I figure this is a good area on which to focus :)
< CodeShark>
we don't want to have to backport each individual soft fork perpetually...and it might actually be easier to backport the plugin thing
< aj>
be easier to backport pluggable soft forks once the plugin thing exists too, presumably
< CodeShark>
yes, of course
< sipa>
what do you mean by plugin system?
< CodeShark>
I posted something on the mailing list - but I'm guessing you unsubscribed :)
< sipa>
yes
< CodeShark>
I can forward it to you if you want
< CodeShark>
or you can look for the link on linuxfoundation.org :)
< sipa>
shouldn't need any changes in primitives... that's only for data types, their construction, and serialization... softforks can't change those
< CodeShark>
CBlock needs a new default version
< CodeShark>
or could be implemented in getblocktemplate, I suppose
< CodeShark>
in principle I agree
< CodeShark>
CBlock should not be where the default version gets set
< sipa>
yes, indeed
< sipa>
those default versions there don't belong
< sipa>
you say "easily merged units"... that's pretty vague
< CodeShark>
right - primitives should just handle data access and serialization
< sipa>
what do you mean specifically?
< btcdrak>
sipa: will you consider joining again after we get the new list and push offtopic stuff to the new list?
< sipa>
btcdrak: maybe
< CodeShark>
sipa: by adding two lines to the makefile and one line to an init function
< CodeShark>
and two files to the repo :)
< CodeShark>
at least for starters
< CodeShark>
long-term it would even be possible to deploy at runtime
< sipa>
CodeShark: so you pretty much mean... changing the code location organizatiin so that soft forks are the top level?
< CodeShark>
yeah, I suppose you could say that
< sipa>
what if soft forks require wallet changes? or block creation code changes?
< CodeShark>
we start with the consensus hooks...then worry about UI :p
< sipa>
ok
< sipa>
my concern is readability... hooks result in less readable execution flow, and much harder to reason about things like "whatever this hook does, the result is always soft fotk safe"
< sipa>
but... perhaps
< CodeShark>
well, if you take a look at how BIP65 and BIP66 are deployed, there aren't too many places really
< CodeShark>
BIP65 is a tad bit more complex in the sense that it also defines an op code
< CodeShark>
so we need a hook in the script interpreter
< sipa>
yeah, i don't like that
< sipa>
not for consensus
< CodeShark>
we can keep that part separate for now, I suppose - the soft fork deployment would also modify script/interpreter.cpp if we don't add a hook
< wumpus>
consensus code should be as simple and straightforward and self-contained as possible, IMO, no hooks and plugins
< CodeShark>
I think this is about as simple as it gets, though
< CodeShark>
it makes any change to consensus very easy to review
< CodeShark>
the diff becomes trivial
< wumpus>
does it? interaction/crosstalk issues tend to be problematic for plugin-like systems, what sipa says
< wumpus>
needs to be easy to follow the execution flow
< CodeShark>
a soft fork has well-defined abstract behavior
< wumpus>
and *in which sequence* things are done
< CodeShark>
the implementor just needs to specify what those behaviors actually are
< CodeShark>
the execution flow is even easier to follow with this kind of architecture...because in the stable consensus code itself the specifics of the rule are encapsulated...and in the rule definition itself there's nothing else BUT the rule definition
< Luke-Jr>
CodeShark: so… let's say a softfork with segregated witness..?
< CodeShark>
can we do that cleanly with a soft fork?
< Luke-Jr>
in theory, I don't see why not
< sipa>
I don't see how.
< Luke-Jr>
probably entails p2p protocol changes though
< sipa>
Only in a very non-useful way could you do it, with external data blobs that blocks commit to (like extension blocks).
< Luke-Jr>
well, we can't segregate the existing scripts - but P2SH-like..
< sipa>
no
< sipa>
segregated witness changes how transactions refer to each other
< sipa>
i guess you can do it by making transactions not contain scriptSigs
< Luke-Jr>
exactly
< sipa>
and have separate witness structures that are relayed separately, and committed to in a block's coinbase
< sipa>
ok
< sipa>
yeah, that wouldn't easily fit into a hook structure, i think
< CodeShark>
I'm just thinking about the consensus structures themselves - not the relay mechanisms. I see the p2p stuff as a separate layer
< sipa>
yes, but even the consensus changes wouldn't easily fit in
< * Luke-Jr>
, after breaking softfork plugins, runs off to bed. :P
< CodeShark>
sipa: I just think of the witness structures as additional validation context
< CodeShark>
they can be anything, really - any additional state required at validation time
< sipa>
ok, how does that fit in?
< CodeShark>
we'd probably need to rearchitect the script interpreter a bit to be more abstract
< wumpus>
no...
< CodeShark>
or we could just accept that not everything is solved by the plugin and some kinds of soft forks still would require additional modifications to the consensus code (but they would be far fewer than before, and the routine ones would all be covered)
< CodeShark>
routine ones being stuff like nVersion changes and the kind of stuff we currently do in main.cpp
< Luke-Jr>
when/if at some point we dynamically call libbitcoinconsensus, we could implement softforks as a forked lib, and have the node call libbitcoinconsensus_softfork's CheckBlock etc and also libbitcoinconsensus_original CheckBlock etc to enforce softfork behaviour; this makes the changes *simpler* and less risky
< CodeShark>
yes, agreed, Luke-Jr. That's sorta what I meant by abstracting the script interpreter a bit more
< Luke-Jr>
CodeShark: no, in this case we'd have entire duplication of the consensus lib ☺
< Luke-Jr>
(so it can be reused for hardforks as well)
< CodeShark>
no...the soft fork plugins could use the consensus lib once it's ready
< Luke-Jr>
CodeShark: well, that's not what I just described
< Luke-Jr>
and we'll want to do these dynamic calls anyway if we support sidechains in a single node ever
< CodeShark>
I'm saying at the core level, there are a few things that are fundamental and invariant - amongst these are the block header tree, PoW, and version rules
< CodeShark>
then on top of this you have core structures (blocks and transactions) which generally can only change their fields with a hard fork
< CodeShark>
so let's say they are invariant as well
< CodeShark>
then on top of this you have context (UTXO, timestamp)
< CodeShark>
if these checks work, then you run the script
< sipa>
we don't even get to do simple refactors
< CodeShark>
sipa: understand this is not a one or two month plan :p
< CodeShark>
this is probably out at least a year
< CodeShark>
point is unless we start thinking about this kind of stuff well in advance it will probably never happen
< sipa>
i'm not convinced it's very useful
< CodeShark>
ok, then I have some homework to do :)
< sipa>
the trivial softforks it is targetting are already simple to review
< CodeShark>
for you, sipa
< CodeShark>
I'd rather you spend your time doing other stuff :p
< CodeShark>
for most people it's still very hard
< sipa>
the non-trivial ones will become harder, because they'll need to change the plugin api...
< CodeShark>
not if we abstract things well enough
< CodeShark>
there's a sequence of validation steps that can be easily abstracted regardless of the specific rules
< sipa>
i don't see how you could create an abstraction that keeps working with seggregated witness, for example
< CodeShark>
isn't segregated witness effectively additional state required for validation?
< sipa>
seggregated witness, from the point of existing code, removes script and sigScript entirely
< CodeShark>
let
< sipa>
and then jntroduces a new primitive data structure, to which consensus rules are applied
< sipa>
i think you're biased by having seen a few simple soft forks
< CodeShark>
removes script and sigScript?
< sipa>
yes, transactions would end up having an empty scriptSig
< CodeShark>
I get that part - what do you mean by "script"
< CodeShark>
?
< CodeShark>
scriptPubKey?
< sipa>
by script i mean the script subdir in the code
< CodeShark>
oh...so interpreter.cpp
< CodeShark>
and that stuff :)
< sipa>
of course, it would get reused in a new piece of code
< CodeShark>
yes, as I pointed out the script interpreter could be far better abstracted to cover vastly more potential use cases...and with the benefit of hindsight
< sipa>
but from the point of existing code - and that's what your abstraction framework can deal with - it just completely deletes the concept of script from bitcoin
< sipa>
how do you deal with cases where more data needs to be exposed to the script interpreter?
< CodeShark>
I imagine a function Validate(context, transaction)
< CodeShark>
where context could be extended
< CodeShark>
but for now it covers the current block tree
< CodeShark>
and utxo set
< sipa>
you can't just expose the entirety of consensus state to transaction validation, or it could break caches that assume certain data doesn't influence validation outcome
< sipa>
no, it's not that easy
< sipa>
script execution can purposefully not depend on certain data
< sipa>
like height
< sipa>
to avoid transactions going from valid to invalid
< CodeShark>
who says the context structure needs to maintain everything? it could be pruned...and if Validate requires missing context when called it can throw an error
< CodeShark>
I agree it isn't trivial
< CodeShark>
it's a challenge - but I think it's doable
< sipa>
i think you're underestimating the costs to others
< CodeShark>
the trick here is figuring out what sorts of context can be excluded from what types of checks
< CodeShark>
and what sorts of context are desirable to exclude
< CodeShark>
again, this is not a one- or two-month plan
< sipa>
ok: bottom line, i am very skeptical about the benefits
< sipa>
now let's talk about things we can actually do
< CodeShark>
but we CAN do this...it's just going to take a much longer time than one release cycle
< CodeShark>
the argument that it doesn't have that many benefits is an acceptable one - but that it isn't possible I don't accept :)
< CodeShark>
you might be able to convince me of the former
< sipa>
it's certainly possible with high costs
< CodeShark>
given our current process, I agree
< sipa>
but i don't think it's the right direction to.go in
< CodeShark>
ok - that's more fruitful discussion
< sipa>
i would aim to have less abstractions, not more
< sipa>
because they complicate reasoning and proving what code can do
< CodeShark>
that depends on how the logic is chunked, though
< sipa>
maybe
< sipa>
i'm not saying there are no benefits
< CodeShark>
the mind is capable of handling craploads of complexity as long as it's well-encapsulated with a simple interface
< sipa>
but it will come at a high cost, and not just reviee cost
< sipa>
CodeShark: unless the API is perfect, and the different modiles are guaeanteed to be isolated from each other, i don't think the abstraction you can introduce is sufficient for consensus purposes
< sipa>
you will need to know what all plugins are doing to see whether the result is consensus safe
< CodeShark>
if the plugins have interdependencies it does potentially add a significant amount of complexity, especially circular dependencies
< sipa>
that's not what i mean
< CodeShark>
with this architecture you could simulate whatever rules you wanted and review specific units that focus exactly on the rules in question
< sipa>
in theory
< CodeShark>
sipa: I just don't think bitcoin can perpetually keep on adding new features at the protocol level if each proposed change requires a handful of people such as yourself to review the same lines of code over and over again :)
< CodeShark>
this is process scalability
< sipa>
CodeShark: i will still review them if they are in plugins
< sipa>
i don't see how this changes anything
< wumpus>
at least some people are going to know those lines of code *very* well then
< CodeShark>
yes, but the author can be tasked with proving them out considerably more before you even touch them
< wumpus>
... which is kind of the goal, people need to understand the consensus rules and how everything fits together
< CodeShark>
we can have a review process where stuff gets better screening before it gets to you
< sipa>
i don't believe that
< sipa>
i think more abstraction will only result in more ways a plugin could screw with consensus
< sipa>
unless it is very minimal, and thus nearly useless
< GitHub133>
[bitcoin] laanwj opened pull request #6789: Update miniupnpc to 1.9.20151008 (master...2015_10_mitigate_upnp_buffer_overflow) https://github.com/bitcoin/bitcoin/pull/6789
< wumpus>
this is going to force our hand re: new 0.10.x and 0.11.x releases
< morcos>
would you still consider trying to bundle the other stuff in?
< wumpus>
I think everything that is currently in the branches should be included (including the LowS enforcement)
< wumpus>
so it can be announced as dual fix: upnp security issue and mallability nuisance
< wumpus>
I don't think the CLTV softfork should be included
< sipa>
i wonder whether we should disconnect softforks entirely from releases
< sipa>
eg have 0.11.1 and 0.10.3 now, and whenever ready, we do a 0.11.1-cltv and 0.10.3-cltv release
< wumpus>
internal versions 0.11.1.1 and 0.10.3.1?
< sipa>
i guess we could coopt the lowest digit for that
< wumpus>
we never use the fourth number
< sipa>
we have used it occasionally for platform-specific releases
< sipa>
like windows-only bugfixes
< sipa>
but that's not a big deal
< wumpus>
but I'm ok with the idea...
< sipa>
i mean in terms of branding... that would make it much more transparent which softforks are implemented, and that they are independent from the client code largely
< wumpus>
yes
< sipa>
in backports you could keep the naming after activatiin, while for new major releases you could have a "0.12 includes softforks bip16, 34, 66, and 68 in all releases"
< sipa>
to avoid needing huge names for historical softforks
< wumpus>
though it may be hard to get people to upgrade to a 0.11.1-cltv if they already have 0.11.1
< btcdrak>
I dont like having a -cltv release.
< btcdrak>
I think we should patch with miniupnp on top of the current 0.11.1 and 0.10.3 branches with all their fixes and release that.
< btcdrak>
let's not get overly pedantic
< btcdrak>
let's release what we have.
< wumpus>
that's already my plan btcdrak
< wumpus>
the softfork releease would be what is planned for the end of the month
< btcdrak>
wumpus: +1
< wumpus>
by the normal numbering that would be 0.11.2 now, but there's something to be said for (but also against) making it a "special" release because it only enables a softfork
< btcdrak>
that's overly complex because then if you need to release security/maintenance, you have to have maintenance version of the softfork release. Better the status quo and then it will get a lot easier with versionbits (which is near first draft implementation I hear from CodeShark)
< btcdrak>
OT: how serious is this miniupnp issue?
< wumpus>
I don't have a strong opinion about version numbers, I prefer just counting up, but if many people want to do a specially-named release I'm not opposed
< wumpus>
as I understand it, any local device could feed invalid data to the UPNP discovery and overflow a buffer
< sipa>
I understand the downside... it's just an idea.
< sipa>
I think it makes sense to help understand people that the network rules are more or less independent from the software
< wumpus>
btcdrak: the TALOS description is quite detailed; I don't know more, haven't tried to exploit it
< wumpus>
sipa: I absolutely agree - just don't know if version *numbers* are the right way to communciate that :-)
< sipa>
fair enough
< wumpus>
may be better to write something in the release notes, if someone is good at writing those things
< wumpus>
anyhow that's a problem for end of the month
< gavinandresen>
remote exploit is serious enough to warrant an alert, in my humble opinion
< wumpus>
yeah, probably
< wumpus>
I've thought about sending one to make people disable upnp
< wumpus>
but reconsidered quickly
< wumpus>
so let's get the new versions out first
< sipa>
yeah
< CodeShark>
one day, in about a century or two, we'll be able to move to IPv6 and this will no longer be an issue :p
< wumpus>
CodeShark: don't be crazy, this is not #bitcoin-sciencefiction :p
< CodeShark>
heh
< sipa>
ipv6 has a 2^96 times larger address space, so it will take 2^96 times as long to deploy, obviously
< morcos>
i understand why we want to logically separate soft forks, but practically speaking it doesn't really work that well, especially with ISM kind, either everyone adopts them or not
< morcos>
its not like its an option you can really choose in addition to getting your upgrades
< btcdrak>
I agree with you morcos. But I think it all goes away with versionbits because there will be a way for people to disable the softfork "vote" on their node.
< morcos>
how about this, let release two versions of 0.11.1 simultaneously, one with soft fork and one without, so that way people don't really have to update twice
< morcos>
but no one can argue we were witholding critical security updates if they didn't go along wiht the softfork
< CodeShark>
lol - the one without is called a command line option...the one with is called the default settings
< morcos>
i just think the downside of now possibly 4 releases in the time span of 4-5 months outweighs the confusing the soft fork aspect
< btcdrak>
morcos: releasing often is always a good thing
< CodeShark>
I am very close to having a functioning versionbits implementation, fwiw
< morcos>
not when people NEED each of the upgrades
< morcos>
ok so 5
< morcos>
:)
< sipa>
fair enough
< morcos>
0.12 although a major version upgrade will be essentially required b/c of mempool limiting
< CodeShark>
in principle, we should probably have version numbers for the protocol itself
< CodeShark>
separate from the software
< btcdrak>
morcos: I think users would be happier to know CVEs get patched asap, and not have to wait for a patch Tuesday. If we need an extra in between release or two for security reasons then that's just how it has to be. miniupnpc is forcing our hand here
< sipa>
CodeShark: we have those...
< CodeShark>
the block header version is sorta that...but not exactly
< sipa>
block version numbers for consensus
< sipa>
protocol version numbers for protocol
< CodeShark>
with versionbits, we're moving away entirely from sequential numbering
< morcos>
btcdrak: yes possibly if we could more concretely agree on the roadmap, then more releases would be ok. mini-upnp patched now.. softfork to follow for CLTV in 3 weeks, 0.12 with mempool limiting a couple months later (likely before CLTV activates) , CSV a month or so later
< morcos>
then people can decide which to skip and not just blindly upgrade? but is that asking too much
< gavinandresen>
Anybody object to me tweeting: “If you're running bitcoind NOT behind a firewall, restart with -upnp=0 -- there is a critical miniupnpc library vulnerability”
< sipa>
where can i read about this vulnerability?
< CodeShark>
but people behind a firewall might just shut down their node
< sipa>
presumably, such situations are more dangerous in corporate networks, where upnp doesn't worn anyway
< gavinandresen>
“doesn’t work” != “isn’t vulnerable” though
< sipa>
sure, just saying that the place where this risk is largest, is one where upnp has no use anywah
< gavinandresen>
yup
< sipa>
so i would say it's the opposite... if you're running behind a firewall it's more reason to disable upnp
< CodeShark>
depends on whether it's a private LAN or a larger shared network, though
< gavinandresen>
How about: “If you are running bitcoind on a public LAN, restart ...."
< sipa>
sounds good
< sipa>
actually
< sipa>
no need to even restart
< sipa>
miniupnp is only called at startup
< sipa>
so putting it in your config would be enough, to prevent upnp from running next time
< CodeShark>
unless someone already injected a delayed payload...in which case it's already too late
< gavinandresen>
sipa: ACK, I added a comment to the pull request. Unfortunately, I have to turn into a pumpkin now (committed to attend an event here all day) so I can’t help gitian build
< wumpus>
yeah@gavinandresen in the GUI too <wumpus> to disable upnp in Bitcoin Core: run with -noupnp, or disable checkbox in GUI under Options → Network → Map port using UPNP
< wumpus>
but they'll probably forget to enable it again after the new release
< wumpus>
I think it will be pretty hard to exploit this - it's a buffer overflow on the stack, and we have address space randomization, -fstack-protector- non-executable stack, enabled for the builds
< GitHub102>
bitcoin/0.11 17ea542 Wladimir J. van der Laan: doc: #6077 was reverted, don't mention in release notes...
< morcos>
wumpus: did you raise minrelaytxfee?
< morcos>
it looks like no, i know there was some controversy about it, and maybe 10x is too big a jump, but its going to be a long time until 0.12 is released, and these mempool attacks are a big problem
< morcos>
i think raising the default to at least double is worthwhile, maybe we could get some ACKS on that and get it merged too?
< michagogo>
hm, `make -C ../bitcoin/depends download SOURCES_PATH=`pwd`/cache/common` is erroring
< michagogo>
or, well
< michagogo>
it's giving me this message: `/bin/sh: 1: test: qtbase-opensource-src-5.5.0.tar.gz: unexpected operator`
< michagogo>
and now it seems to be redownloading qt
< wumpus>
morcos: that was not done
< wumpus>
maybe for the release end of this month
< cfields>
michagogo: that's harmless, you can ignore it
< michagogo>
Hm, did qt get bumped since 0.11.0?
< wumpus>
#6471 92401c2 Depends: bump to qt 5.5 yes - I remember this was for the TLS1.0+ requirement
< michagogo>
Oh, I just realized my script for a complete gitian build probably won't work for 0.10
< michagogo>
I guess I'll do that one manually tomorrow night
< michagogo>
cfields: btw, looks like the mirror on bitcoincore.org isn't up to date
< cfields>
ugh, i keep meaning to setup a cronjob for that
< cfields>
what's missing?
< michagogo>
qt
< michagogo>
Also, argh
< cfields>
ok, pulling it in
< michagogo>
apparently if one component errors out it doesn't keep everything else that was fetched
< michagogo>
That's really bad
< cfields>
yea, the multi-sources are a corner-case and kinda handled hammer-style
< michagogo>
Hm
< michagogo>
I think this might mean I won't be able to kick the build off before I go offline
< michagogo>
a 60MB download, took a long time
< michagogo>
and is now going to have to rerun because a <3 MB file failed to download
< michagogo>
Is dependency download from within gitian working atm?
< cfields>
depends on the config. I think it's only an issue with lxc
< michagogo>
Im using lxc :-(
< cfields>
ok, links should be good now
< cfields>
setting up a cronjob
< Cocodude>
Hello. My bitcoind is taking up > 4 GB RAM right now. Is there a recommended way to bring that down? Is it somewhat due to the UTXOs?
< paveljanik>
Cocodude, #bitcoin please.
< michagogo>
Okay, I *think* I got the build kicked off now.
< michagogo>
If you see a PR from me with the sigs for 0.11.1rc1 in a few hours, it worked. If not, I'll figure out why an fix it tomorrow night. Now, I g2g.
< GitHub52>
bitcoin/0.11 717152c Wladimir J. van der Laan: Bump version to 0.11.1
< cfields>
wumpus: unsure
< GitHub44>
[bitcoin] gmaxwell opened pull request #6792: Defaults UPNP to off when discovery is disabled. (master...upnp_off_on_ip) https://github.com/bitcoin/bitcoin/pull/6792
< gmaxwell>
I continue to be unhappy with the miniupnp codebase. At least shutting it off more often will help. :)
< wumpus>
I'd love to get rid of miniupnpc, if we all agree it's a bigger risk than the gain it is to the network we can still remove it for rc2
< wumpus>
(too bad we don't really have a way to measure)
< gmaxwell>
We could go even further in turning it off, e.g. defaulting it to off if the user appears to have a routable IP.
< wumpus>
there are other upnp libraries but they aren't any better with regard to codebase, UPNP *is* an ugly protocol with XML and al, so that's not an option
< petertodd>
wumpus: removing miniupnpc while adding automatic tor hidden service support would be a decent compromise...
< wumpus>
petertodd: it's the better way of hole-punching :)
< petertodd>
wumpus: yup!
< gmaxwell>
petertodd: not quite a replacement unless we were getting more nodes to come along with a tor client though.
< wumpus>
it should be able to interface with the TBB that the user has installed
< petertodd>
gmaxwell: yeah, easiest to do in the context of things like ubuntu packages which can depend on tor
< gmaxwell>
We had upnp off for bitcoind for a long time. Perhaps we should just default it off for non-QT?
< wumpus>
alternatively we could switch to an UDP-based protocol and use UDP hole punching on the router *ducks*
< wumpus>
I really don't like that solution gmaxwell, having anything depend on UI-or-not
< gmaxwell>
wumpus: well, I'm in damage mitigation mode there. The more places we can turn it off with almost no effect, the better.
< wumpus>
on average, UI users are generally the most vulnerable to all kinds of attacks
< wumpus>
so giving them the libupnp burden is wrong
< gmaxwell>
I agree. But assuming (bad assumption) we don't want to generally turn it off, how much can we turn it off with no effect?
< wumpus>
(also UI users are most likely to have the wallet enabled)
< gmaxwell>
and I think the PR I just opened is pretty fine, as I think would going back to the configuration where bitcoind has it off by default.... but I agree these get less of the benefit.
< gmaxwell>
wumpus: at the same time UI users are less likely to be subject to heroic attack efforts.
< wumpus>
I'd just like to make the decision to turn it off
< gmaxwell>
Also UI users are more likely to have worse vulnerabilities on their system.
< wumpus>
yes, I want to remove miniupnpc completely
< gmaxwell>
How about we default it to off in the update?
< wumpus>
fine with me too
< petertodd>
gmaxwell: AC
< petertodd>
gmaxwell: ACK
< gmaxwell>
I believe this will result in a substantial reduction in reachable nodes. I don't know how substantial. If it does, we can deal with it. They likely weren't the most usable nodes.
< wumpus>
"<gmaxwell> Also UI users are more likely to have worse vulnerabilities on their system." probably , but what if they're running the UI on the only computer they had that was safe :p
< gmaxwell>
Absolutely, I was just speaking in terms of averages.
< wumpus>
well I like the torrent solution... pressure people to add a port forward by showing an ugly icon :-)
< petertodd>
gmaxwell: what % of reachable nodes correspond to residential ip addrs?
< wumpus>
though we already do, the 'network connectivity' icon is never full without incoming connections, but it could be more insistent in warning 'it appears that the port is unreachable, see XXX for instructions on forwarding'
< Luke-Jr>
wumpus: what is the purpose in removing the --[enable|disable]-upnp-default
< wumpus>
Luke-Jr: we don't have --XXX-default options for other settings either
< Luke-Jr>
wumpus: is there any similar option? particularly, that enables a compile-time feature disabled by default
< wumpus>
I don't think so
< Luke-Jr>
it only makes sense to remove this one, if the default is always ON
< wumpus>
"set this at runtime instead"
< wumpus>
come on, do you really care about this?
< Luke-Jr>
wumpus: yes, because I will now have to support a patch to do it
< wumpus>
why not just set it in bitcoin.conf instead?
< Luke-Jr>
that involves manipulating bitcoin.conf
< wumpus>
yeh it does...
< Luke-Jr>
people who install bitcoin-qt[upnp] shouldn't need to take additional steps to enable UPnP
< Luke-Jr>
people who don't want UPnP would disable it at compile-time
< wumpus>
with qt it's even simpler, you can enable it in the options
< Luke-Jr>
that's not simpler
< wumpus>
--with-upnp *adds the feature* it doesn't enable it
< wumpus>
it never did
< Luke-Jr>
exactly why the separate option is needed
< wumpus>
zmq is exactly the same
< wumpus>
--with-zmq builds with libzmq, it doesn't *enable* it
< sipa>
Luke-Jr: but as things are right now, we at least temporarily seem to not want it enabled by default
< wumpus>
if you want it, enable it in your config
< Luke-Jr>
nobody uses ZMQ though; whereas UPnP *should* be used
< wumpus>
no, UPnP should not be used, it's a grave risk
< wumpus>
this isn't the first vulnerability in it
< Luke-Jr>
it's a grave risk to disable it too
< wumpus>
well at least not a remote code execution...
< Luke-Jr>
sipa: so we should distribute binaries with it disabled by default, I agree with that
< btcdrak>
so we just need to point users to instructions on how to port forward.
< btcdrak>
there must be plenty tutorials already out there
< wumpus>
btcdrak: agreed, same as torrent clients have done for years, upnp is unreliable at best
< paveljanik>
btcdrak, +1
< Luke-Jr>
but if someone is explicitly compiling a build with UPnP, they shouldn't need extra steps
< sipa>
Luke-Jr: but someone who compiles themselves should get it on by default? those users are exactly the ones who can set a config option too
< btcdrak>
wumpus: we should just pull upnp out. I mean, this is people's money we're talking about potentially
< Luke-Jr>
sipa: for example, installing on Gentoo, I have no sane way to set a config option for the user
< Eliel>
sipa: I think luke's point is that to the user it feels like setting _two_ config options that way.
< wumpus>
Luke-Jr: as said, it's the same for the other optional features, apart from the wallet
< wumpus>
I'm done arguing this, agree with btcdrak, I'd prefer to remove upnp support completely, this is already a compromise
< Luke-Jr>
…
< btcdrak>
Let me go find some port forwarding tutorials
< wumpus>
why do you want to expose users to the risk of upnp on gentoo by default?
< Luke-Jr>
btcdrak: you're assuming the user even has access to the router
< * Eliel>
wonders how much work it'd be to implement just the subset of upnp that bitcoin needs.
< Luke-Jr>
wumpus: it's not by default
< Luke-Jr>
wumpus: by default, it won't even be built with UPnP support
< sipa>
Luke-Jr: then why do you need a compile-time option for it?
< gmaxwell>
Luke-Jr: we went years without having upnp on by default for bitcoind. We saw no benefit from making it more on by default in our binaries.
< wumpus>
if they don't have access to the router, upnp shouldn't be enabled either, it breaches the security policy too
< Luke-Jr>
sipa: so when users enable it, they actually get it
< sipa>
Luke-Jr: they can enable it by enabling it!
< Luke-Jr>
gmaxwell: huh? the binaries have always had it on by default, no?
< gmaxwell>
Luke-Jr: they do get it, by enabling it in the config! making it compile time is kinda psycho!
< Luke-Jr>
sipa: they enable it system-wide by setting USE=upnp and recompiling
< gmaxwell>
Luke-Jr: no bitcoin-qt did but bitcoind didn't until the switch off of makefiles, AFAIK.
< Luke-Jr>
wumpus: UPnP just fixes a bug in NAT
< Luke-Jr>
gmaxwell: but Bitcoin-Qt is what end users use
< Luke-Jr>
obviously servers (the target for bitcoind) do not need UPnP
< Luke-Jr>
anyway, --[enable|disable]-upnp-default has real use and zero cost. removing it for no reason is just annoying and wastes time.
< wumpus>
the trinary option USE_UPNP confuses me every time, but apart from that, you're probably right...
< wumpus>
but I don't want to wake up to people adding similar options to set other defaults at compile time, if you have --upnp-default why not --minfee-default, --zmq-default, --maxreceiveoption-default and so on...
< sipa>
--reindex-default would be useful for systems with crappy disks
< wumpus>
heh
< Luke-Jr>
meh
< Luke-Jr>
IF it is going to be removed in master, can we at least keep it around in backports?
< wumpus>
I think the point of this is to backport it
< wumpus>
but ok, I'll change my pull to only change the descriptors, this is not worth wasting so much time over
< skyraider>
what's the value of the master public key and master chain code in the Bip32 Test Vector 1? this doesn't seem to be specified.
< wumpus>
someone that cares enough could still remove the --upnp-default setting in master
< skyraider>
there is a value for "master" - a 16-byte value of some kind. however, "master" has no semantic meaning; unclear what this is.
< wumpus>
skyraider: questions about BIPs are really off topic here
< skyraider>
ok sorry
< sipa>
skyraider: read BIP32, it clearly specifies what master means
< skyraider>
correct channel?
< Luke-Jr>
#bitcoin-dev
< wumpus>
#bitcoin or #bitcoin-dev
< sipa>
oh, it calls it seed
< gmaxwell>
wumpus: at least I understand luke's complaint now. Basically the model is that gentoo users who want to use upnp ubiquitiously on their system would set a useflag.
< gmaxwell>
which doesn't seem totally nutty. But the fact that supporting this craps up our code base, is .. unfortunate.
< wumpus>
I suppose the useflag could do a sed -i s/UPNP_DEFAULT = 0/UPNP_DEFAULT = 1/ src/net.h as well .. ok that moves the hackyness to gentoo :)
< gmaxwell>
I certantly would prefer to make it so the entire action of the default is via changing that global const.
< wumpus>
though I still think that it would be just as acceptable to have a distinction between 'adding support for a feature' and 'enabling a feature'
< gmaxwell>
E.g. I'd also be fine with UPNP_DEFAULT = false being overridable with a UPNP_DEFAULT_ON ifdef.
< gmaxwell>
and then gentoo need not even SED.
< gmaxwell>
could just set a cflag.
< Luke-Jr>
CXXFLAGS="-DUPNP_DEFAULT_ON" would also be fairly simple
< Luke-Jr>
not sure it's better than sed though, if it was a single well-defined location
< wumpus>
setting a cflag wouldn't work, the default is a constant, not a macro
< gmaxwell>
Luke-Jr: the sed is more likely to get disrupted by code motion.
< gmaxwell>
wumpus: I mean having a macro that changes the constant.
< Luke-Jr>
gmaxwell: true
< wumpus>
ugh, if you're going with a macro anyway, then you may just as well keep the configure option
< wumpus>
as we have a macro for that now: USE_UPNP
< wumpus>
but ok, I'm done with this, not going to spend anymore time on it
< sipa>
i think this discussion has now wasted more time than the combined benefit of the 5 gentoo bitcoin users who set use=upnp
< sipa>
i don't care
< wumpus>
yeah...
< Luke-Jr>
>_<
< gmaxwell>
wumpus: I'd happily change it if it were my PR to be just like you suggest plus letting the constant get overridden with a define, with no build system explicit support for it.
< gmaxwell>
It's a few more bytes and not an unreasonable ask.
< gmaxwell>
And it's nice to get rid of the crazy overload of the define value.
< gmaxwell>
and make this simpler.
< wumpus>
please just keep it as it is now
< wumpus>
I'm sure you have something more serious to do
< wumpus>
can we make a decision about mempool limiting? :p
< wumpus>
well, BIPs are on topic when it relates to implementing them in Bitcoin Core, but not on their own
< paveljanik>
wumpus, 0.10 doesn't contain 649f5d9c1100bb3cbace724900f035939df5ea11. It was backported to 0.11 only. OK?
< morcos>
wumpus: it's a bit tricky to think of what the right value should be for minrelaytxfee. i agree we should only be thinking of what we want it set to in 0.11/0.10
< morcos>
but if we set it too high, and then magically in the future things just work nicely with lower tx fees b/c of limited mempool or something
< morcos>
any pre-0.12 nodes will just miss out on txs unnecessarily
< morcos>
and if we left the fee low on those, they'd probably be ok, bc it'd be hard to carry out the attack when most of the network wouldn't relay
< morcos>
so maybe the right solution here is to raise it up higher (to temporarily protect against mempool attacks) and then once 0.12 is release, the next 0.11 release can reduce it again?
< morcos>
i'm commenting here instead of pull, b/c i'm just brainstorming what makes sense
< GitHub123>
[bitcoin] TheBlueMatt opened pull request #6796: Update debian/changelog and slight tweak to debian/control (master...debian) https://github.com/bitcoin/bitcoin/pull/6796