< gmaxwell>
jonasschnelli: sipa: both of you are rehashing old arguments that have already been made to voskuil on the list. He is either flagrantly incompetent or actively working against the public interest. He is not worth your time or attention, and he gets _no say_ in this. Supporting authenticated links is something each indivigual node can decide to do for itself.
< gmaxwell>
sipa: your addnode argument was directly made my me previously in a private email, which he has dishonestly described as a threat (because I said that I thought I would write him to him privately before mentally writing him off, and he responded that this was 'a threat'-- absurd.)
< gmaxwell>
Abusive asshole like that are a big part of the reason I unsubscribed. You just enable them when you argue with them where you don't have to.
< bitcoin-git>
[bitcoin] sdaftuar opened pull request #9959: Mining: Prevent slowdown in CreateNewBlock on large mempools (master...2017-03-cnb-optimizations) https://github.com/bitcoin/bitcoin/pull/9959
< gmaxwell>
jeremyrubin: I can't make sense of your comment on #9955
< isle2983>
it is a set of functionality previously submitted to contrib/devtool, but closed because it was decided that was the wrong spot
< isle2983>
the pointer was to bitcoin-maintainer-tools, so I ran with that
< isle2983>
the scripts use a common shared framework
< isle2983>
hopefully it will help accelerate further automation in the future
< bitcoin-git>
[bitcoin] NicolasDorier opened pull request #9960: Trivial: Add const modifier to GetHDChain and IsHDEnabled (master...consthd) https://github.com/bitcoin/bitcoin/pull/9960
< jeremyrubin>
gmaxwell: if a miner doesn't want to support segwit, and won't mine them, maybe they should not keep such transactions in their memory pool
< jeremyrubin>
gmaxwell: e.g., to keep room for the most-fee nonsegwit txns
< luke-jr>
jeremyrubin: what if a miner does want to support segwit, but is locked-into mining software that doesn't support it, on some machines?
< jeremyrubin>
this still applies
< jeremyrubin>
they will make more money evicting segwit txs
< luke-jr>
jeremyrubin: they won't be supporting segwit with that logic, and they might not make more money either so long as their mempool is a decent size
< jeremyrubin>
luke-jr: false. they can still validate segwit blocks.
< jeremyrubin>
luke-jr: and check the witnesses (full validation)
< luke-jr>
jeremyrubin: all miners need to validate segwit blocks once it activates; that's not support, it's just mining
< jeremyrubin>
that's not completely true.
< jeremyrubin>
A miner can mine a non segwit block, it will just get orphaned by the segwit-mining majority
< jeremyrubin>
(if any of the txs have a witness)
< jeremyrubin>
and even in that case, such a miner might not want segwit transactions in their mempool if they can't mine them?
< warren>
I suspect your discussion there may be confused by absolutism on one hand, and a common misconception that I also made on the other.
< jeremyrubin>
warren: that sentence doesn't say anything
< jeremyrubin>
In general, if I am a miner, and I have a 10 MB mempool (let's say), and there is a backlog of 100MB of transactions, and the top 10MB are all segwit transactions I absolutely want to exclude them from my mempool if I can't mine them
< warren>
jeremyrubin: pre-segwit nodes see segwit transactions as non-standard and it won't enter the mempool to begin with
< warren>
nevermind, I don't care enough about this to continue
< jeremyrubin>
warren: context is a miner wanting to be a fully validating segwit node running on the latest, but with segwit incompatible mining hardware #9955
< jeremyrubin>
The question is, if you're never going to mine a transaction why would you want it in your mempool? (other than, perhaps, compact blocks)
< luke-jr>
jeremyrubin: it would be fairly irresponsible for a miner to have a mere 10 MB mempool in the first place..
< luke-jr>
the default of 300 MB is for non-mining nodes
< jeremyrubin>
luke-jr: was only for convenience picking a number.
< luke-jr>
point is they should have more than enough
< jeremyrubin>
no
< jeremyrubin>
there is actually an attack that you can do with a more well resourced miner (larger mempool) against a less well resourced node that can't mine segwit txns
< jeremyrubin>
e.g., let's say I have a consistent segwit fee around 10 sat/byte, filling up at all times the top 10MB of the mempool
< jeremyrubin>
all txns will be selected from that range always
< jeremyrubin>
so then, I issue 290 MB of txns that are at 9 sat/byte, with segwit, such that they won't be mined
< jeremyrubin>
let's say I also have a bunch on non-segwit txns at 9 sat/byte
< jeremyrubin>
I can push them out with my segwit txns
< jeremyrubin>
so that your less well resourced miner now has a mempool full of txns you can't mine
< gmaxwell>
jeremyrubin: we don't know in advance of someone calling GBT what they're going to call it with, and they may call it with a mix of arguments.
< gmaxwell>
jeremyrubin: their normal operation as a node shouldn't be impared ... e.g. logical conclusion of what you suggest is that someone who doesn't mine shouldn't have a mempool at all-- which is clearly not true, as its integral to other functions of a node (fast block propagation, fee estimation, txn relay)
< gmaxwell>
luke-jr: you say a lot of confusing things, there is nothing wrong with having a 300 mb mempool as a miner and CNB is effectively the same speed for all mempool sizes. Having a small mempool may also adversely impact your block reception speed.
< gmaxwell>
tiny mempools also prevent users from being able to create low priority transactions that get mined in off hours... which is bad because without a supply of lower priority transactions the spare capacity will instead by used for less useful/efficient uses that continually rebroadcast.
< luke-jr>
gmaxwell: I'm saying a small mempool is bad..
< gmaxwell>
oh okay!
< gmaxwell>
I had read your comment backwards.
< gmaxwell>
jeremyrubin: okay thanks for at least clarifying what you meant. (I don't think it's a good idea at all-- but at least it makes logical sense to me now)
< jeremyrubin>
gmaxwell: that's fair; what I'm suggesting might be better off as a startup time argument
< jeremyrubin>
gmaxwell: I also think that you can relay a txn w/o having it in your mempool (e.g., to your immediate peers)
< jeremyrubin>
gmaxwell: w.r.t. fee estimation, you could collect the fee estimate information for such transactions without storing them
< sipa>
jeremyrubin: BIP152 relay speed depends on having transactions in your mempool
< sipa>
(or at least, stored somewhere)
< gmaxwell>
jeremyrubin: not as soon as there is a dependency, and we also use the mempool as a dynamic rate control.
< jeremyrubin>
yes, as I said, compact blocks is the place where it does make sense to keep them
< gmaxwell>
jeremyrubin: you have to remember information about them, it could be just their IDs and feerates, rather than the whole things. but you would get less accurate results since you'd overestimate the time it took for things that were evicted.
< gmaxwell>
In any case what you are suggesting is 99% orthorgonal, since you do not know at the rest of operation what your future GBT calls will be.
< jeremyrubin>
yeah; this would only make sense where you know that none of your calls would need it
< bitcoin-git>
bitcoin/master 53a2ba3 practicalswift: [util] Remove redundant call to get() on smart pointer (thread_specific_ptr)
< bitcoin-git>
bitcoin/master e3e7db8 Wladimir J. van der Laan: Merge #9538: [util] Remove redundant call to get() on smart pointer (thread_specific_ptr)...
< bitcoin-git>
[bitcoin] laanwj closed pull request #9538: [util] Remove redundant call to get() on smart pointer (thread_specific_ptr) (master...remove-redundant-get-on-smartptr) https://github.com/bitcoin/bitcoin/pull/9538
< bitcoin-git>
[bitcoin] laanwj opened pull request #9963: util: Properly handle errors during log message formatting (master...2017_03_handle_exception_tinyformat) https://github.com/bitcoin/bitcoin/pull/9963
< jonasschnelli>
I still try to understand why importing a P2PKH address into the wallet leads to ISMINE_WATCH_UNSOLVABLE (and not ISMINE_WATCH_SOLVABLE).
< jonasschnelli>
The comment for ISMINE_WATCH_SOLVABLE says: "Indicates that we know how to create a scriptSig that would solve this if we were given the appropriate private keys"
< jonasschnelli>
Isn't this true for P2PKH (== importaddress could identify P2PKH)?
< luke-jr>
I suspect the requirement of the public key might play a part
< luke-jr>
you could calculate it from the private key, but that may not count
< jonasschnelli>
luke-jr: Yes. When I import a pubkey it will marke the P2PKH watch-only as solveble.
< jonasschnelli>
But I don't see a clear reason for that. I understand that for P2SH but not for P2PKH...
< wumpus>
I just had the wallet RPC test pass... on a leveldb-based wallet on top of #9951. It's a bit of a hack right now but hopefully enough to be able to run all the RPC tests against the cloudabi bitcoind at some point.
< bitcoin-git>
[bitcoin] practicalswift opened pull request #9964: Add const to methods that do not modify the object for which it is called (master...const) https://github.com/bitcoin/bitcoin/pull/9964
< wumpus>
seems that a dump may be incomplete if it could not find all keys
< wumpus>
(not that I've ever seen an instance of that problem, but I just wonder that seeing the code)
< bitcoin-git>
[bitcoin] jonasschnelli opened pull request #9965: Allow to use unsolveable P2PKH watch-only in fundrawtransaction (master...2017/03/watch_solve) https://github.com/bitcoin/bitcoin/pull/9965
< sipa>
ploink
< wumpus>
#startmeeting
< lightningbot>
Meeting started Thu Mar 9 19:00:43 2017 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
< gmaxwell>
All the 0.14 nodes sending final alerts constantly is somewhat protective, but unforuntately not absoltely protective.
< gmaxwell>
sipa: already looked for that, there were no major ones, but some obscure & dead ones that have been nagged.
< gmaxwell>
There are also funds paid to the alert key in the network, I believe 0.01 BTC or so. :P
< wumpus>
heheh
< cfields>
heh
< gmaxwell>
in any case, one possibility would be to not worry too much about it, announce again that version <0.12.1 are vulnerable (there is a lot else they're vulnerable too, I think).
< wumpus>
surprised no one ever swiped that
< btcdrak>
sipa: most altcoins are on <0.12 codebase, and many never changed the alter key
< gmaxwell>
btcdrak: not so.
< wumpus>
from what I've seen, altcoins generally do change the alert key, at least from the bitcoin one
< gmaxwell>
btcdrak: (didn't change the _litecoin_ alert key, yes, but I searched extensively for this already, months ago.)
< wumpus>
many have the litecoin or feathercoin key
< wumpus>
yes :)
< sipa>
ah
< btcdrak>
ah yes, most coins forked from litecoin (and didnt change the network magic either)
< morcos>
remind me again what the advantage is of disclosing the key?
< achow101>
morcos: making the alert system entirely unreliable (as if it weren't already)
< gmaxwell>
to not be responsible for someone else abusing it, among other things.
< gmaxwell>
we don't believe the key is actually secure right now in any case.
< sipa>
there is no hurry in disclosing it either
< sipa>
just a nice to have to close that chapter
< wumpus>
to force people to not use it anymore
< btcdrak>
they key is useless now anyway that final alert is out.
< wumpus>
and also for sake of history
< paveljanik>
We can make people upgrade because of the planned alert key discolsure...
< luke-jr>
not sure how litecoin alerts are affects if they use a different key
< gmaxwell>
we can't make anyone upgrade, and even if we could we wouldn't. :)
< morcos>
yes but if there is even nuisance harm that can be done with it.. i think we should announce that its not considred secure but that we're not going to aid people doing nuisance with it by publicly disclosing it until a) we see nuisance or b) some predetermine future point
< wumpus>
litecoin is not affected luke-jr - what they do is up to them
< btcdrak>
wumpus: they also removed the alert system, but there's a lot on 0.10.x nodes in litecoin
< gmaxwell>
morcos: well we're at the predetermined future point now, we announced.. 6 months ago we'd do this after 0.14. The issue there is on careful review of the old code I found several DOS vulnerablities.
< wumpus>
yes, this was announced on a timeline
< morcos>
right, so change the predetermined point to be further out... the risk is only that someone blames us for nuisance right?
< achow101>
maybe just push back the date even further until <0.12.1 nodes are even fewer?
< BlueMatt>
yea, I would vote push another release given old 0.12.0 isnt otherwise insecure?
< wumpus>
right, if anyone abuses the key now, they may blame us. If it is public knowledge not so much.
< gmaxwell>
It's fine with me, we should emphasize that those older versions are insecure. If someone dosses them later it's not the end of the world.
< morcos>
Except of course we will be blamed if we make it public and someone creates nuisance with it!
< gmaxwell>
BlueMatt: I believe it is otherwise insecure at a greater magnitude than the alert DOS attacks.
< paveljanik>
gmaxwell, yes, and by emphasizing it, maybe people update...
< gmaxwell>
Well anything that has alerts is displaying the hardcoded final alert now...
< achow101>
what version did BU remove the alert system? If it is in their 0.12.1, people could dos them, and I don't think they would take kindly to that (drama and FUD and all that)
< btcdrak>
so let's announce the vulnerability first that should motivate more people to upgrade old systems or disable the alert system.
< gmaxwell>
which says something like (allcaps) warning alert key compromised! upgrade required!.
< jtimon>
ACK "<sipa> there is no hurry in disclosing it either"
< luke-jr>
gmaxwell: can you get a CVE assigned for one or more of the old DoS?
< gmaxwell>
I think some of you are underestimating the cost of this thing... but sure, no rush required.
< wumpus>
anyhow, everyone with the alert key could disclose it, even anonymously
< wumpus>
don't even know who has it exactly so...
< luke-jr>
yes, nobody can be blamed individually if nobody knows who disclosed it
< wumpus>
anyhow, any other topics?
< luke-jr>
FWIW, ignoring alert keys, I see 539 vulnerable nodes (0.94%)
< gmaxwell>
Can we get a clear plan of action here? We'll CVE old versions and remind people these old things are insecure? should we review other fixes and determine if we want to set a higher bar for holy-shit-dont-run-that than 0.12.1?
< wumpus>
you can't be entirely sure that all of them are vulnerable, they may have patched the code to remove the alert system
< luke-jr>
sure, but that's unlikely
< gmaxwell>
I think 5% dropping offline would be inconsequential, so avoiding the dos is just a politeness to the users to avoid disrupting other things they may have going on.
< wumpus>
many are running old versions to be able to run their own (outdated) patches on top
< achow101>
gmaxwell: how about CVE the dos vulns you found, post messages an all forums about vulnerable nodes, and then release the key a few weeks later?
< luke-jr>
if we're really worried, we could release a 0.12.2 with just those exploits fixed
< morcos>
if it was me, i wouldn't do any of it... lets put our effort where it matters.. 0.15
< luke-jr>
oh, duh
< luke-jr>
so why wouldn't people held back by patches just rebase?
< luke-jr>
to 0.12.1
< wumpus>
0.12.0 can easily upgrad to 0.12.1, 0.11.x maybe less so
< btcdrak>
considering there's an alert message broadcasting to all those vulnerable nodes right now saying they should upgrade, I think we've probably done enough.
< wumpus>
not going to do a new 0.11.x release though
< luke-jr>
get and publish a CVE, then 2 weeks later publish the key
< luke-jr>
?
< wumpus>
I guess we could push a patch to the 0.11 branch that disables the alert system
< gmaxwell>
it's a trivial patch.
< wumpus>
yes
< achow101>
luke-jr: +1
< gmaxwell>
nothing before 0.8 still works at all, and the 0.9x nodes I have looked at are all fake.
< gmaxwell>
0.10 / 0.11 are probably still running in practice.
< gmaxwell>
luke-jr: okay fine with CVE and we can patch the old branch. thats enough plan for me for now.
< wumpus>
it's a one-line patch, could even do a tag, just not going to build executables for them anymore
< gmaxwell>
Thanks.
< sipa>
sgtm
< achow101>
I suppose the bitcoin.org alert page should be updated with that info then
< jtimon>
I'm happy to review any backports for an alert patch, even if they're old, won't go below 0.8, sorry
< wumpus>
there's always issues in major releases that only get found when the final is tagged, it's a fact of life
< gmaxwell>
achow101: any kind of bug that requires special configuration to find is much less likely to get caught, this is why you hear groans when people propose solving problems with more config options. :)
< wumpus>
no number of rcs can solve that
< luke-jr>
9955 is more of a new feature IMO, but I don't care if people want to backport it
< morcos>
agreed, we should tag those for 0.14 or backport or whatever we say, but not cause for expedited minor release
< wumpus>
ok tagging them
< cfields>
wumpus: 0.15 schedule sgtm.
< luke-jr>
note that backporting 9955 is likely a hazard since priority mining doesn't pay attention to fIncludeWitness I think
< wumpus>
cfields: thanks, good to know
< luke-jr>
should be a trivial thing to fix, but might not conflict
< achow101>
are we going to do the release notes wiki thing again?
< wumpus>
yes, at the end of the 0.15 release cycle
< achow101>
ok
< sdaftuar>
priority mining did pay attention to fIncludeWitness, so i think it would be fine, but i haven't tested
< wumpus>
achow101: I think it worked quite well; the only problem that came up is merging it with that is in the tree. So better to leave it until the end when people start caring about writing release notes, and until then leave it up to pulls to add something there
< luke-jr>
oh, it's tested in a different place, okay
< achow101>
anything else?
< sipa>
seems not
< wumpus>
I don't think so
< wumpus>
#endmeeting
< lightningbot>
Meeting ended Thu Mar 9 19:44:45 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< bitcoin-git>
bitcoin/0.11 0bace83 Wladimir J. van der Laan: net: Disable P2P alert system
< sdaftuar>
ah, i will re-read more carefully
< jl2012>
It just stirs the header
< jl2012>
Stores
< achow101>
wumpus: coulda just set DEFAULT_ALERTS to false
< wumpus>
achow101:did that exist in 0.10 and 0.11?
< achow101>
yes
< achow101>
0.11 had it false too
< achow101>
(but not tagged)
< wumpus>
achow101: anyhow, this is complete and clear
< achow101>
indeed
< jl2012>
sdaftuar: it does not ban for sending header for childs of an invalid block
< wumpus>
going to tag v0.11.3 and v0.10.5
< btcdrak>
wumpus: ~1
< btcdrak>
+1
< sdaftuar>
jl2012: ah, ok. right now, banning peers who are on an incompatible chain is used to avoid being partitioned from the peers who share our consensus rules.
< wumpus>
no RCs necessary for this, I'd say
< sdaftuar>
jl2012: there's some discussion of this in #9530
< sdaftuar>
jl2012: i think we can replace the banning with something else, like rotating outbound connections periodically and try dropping any outbound peers on an incompatible chain in favor of a new outbound peer, but i think we'd need to write that code first before we change the DoS banning behavior we have now
< wumpus>
btcdrak: wouldn't that normally be the 0.14 release date?
< kanzure>
oh, i assumed the meeting was over. 'scuse me.
< wumpus>
yes, the meeting is closed
< wumpus>
kanzure: will take a look
< kanzure>
i don't think it's directly usable in bitcoin.git because the rpc tests need to actually use real rpc :) (and also we're not using petertodd/python-bitcoinlib in bitcoin rpc tests anyway)
< jl2012>
sdaftuar: maybe giving a score lower than 100 for sending child of invalid block?
< jl2012>
Before we have a new DoS system
< sipa>
please, no more dos scoring
< jl2012>
So the plan is completely removing that?
< sipa>
i think so. things are either invalid and expensive, in which case we ban, or not, and we don't ban
< sipa>
and independently there is a mechanism for keeping good peers around (which can use relay speed, first to relay blocks/txn, resource comsumpion, seemingly on the same chain, ...) as a low-frequency kicking (but not banning)
< morcos>
sipa: that wasn't super clear
< morcos>
there is expensive to produce and expensive to consume
< morcos>
both matter
< sipa>
right, ratio between cost to the attacker and cost to us is what matters
< btcdrak>
wumpus: yes maintenance end will be when 0.14 is released, but what about EOL.
< wumpus>
btcdrak: what did we use for other releases?
< * luke-jr>
peers at race conditions not fixed by adding atomic<>
< gmaxwell>
sipa: we can have another mechenism unrelated to DOS to make sure peers on incompatible chains aren't wasiting our slots and contributing to partioning us. Bans are a dumb way to accomplish that.
< wumpus>
just kicking is usually enough
< sipa>
gmaxwell: that's exactly what i said
< sipa>
12:09:44 < sipa> and independently there is a mechanism for keeping good peers around (which can use relay speed, first to relay blocks/txn, resource comsumpion,
< sipa>
seemingly on the same chain, ...) as a low-frequency kicking (but not banning)
< wumpus>
only spy nodes are persistent enough to keep reconnecting to the same nodes
< gmaxwell>
ah I missed the 'seemingly on the same chain'.
< gmaxwell>
e.g. we can easily tell when a peer is on a chain we consider recently invalid. (if they advertise a block to us, we'll fetch back its headers, and eventually find it as a child of a block we rejected as invalid)
< gmaxwell>
at that point he should be ranked as first to get dropped.
< sipa>
right
< gmaxwell>
can just be a flag, that gets set when we see he's on an invalid chain, unset if we get a valid block from him. and connection rotation could strictly prefer to punt ... all obvious.
< luke-jr>
tfw you rebase something just to realise you forgot to pull master
< wumpus>
'well, that was easy!' 'oh no...'
< luke-jr>
I think #8694 needs fresh re-reviews :x
< bitcoin-git>
[bitcoin] jnewbery opened pull request #9966: Control mempool persistence using a command line parameter (master...mempoolpersistenceoption) https://github.com/bitcoin/bitcoin/pull/9966
< gmaxwell>
apparently the latest electrum nags you to opt into RBF if you tell it to pay a low fee.
< dcousens_>
gmaxwell: suprising, I'd of thought CPFP would be the easier route
< dcousens_>
aka, UX escape hatch
< gmaxwell>
dcousens_: CPFP is not that useful.
< gmaxwell>
Requires that you have change, and involves 2x the transaction data.
< gmaxwell>
I mean, it's useful, but RBF is more useful.
< dcousens_>
gmaxwell: true, just remembered as you wrote that it would require the change
< gmaxwell>
Think: you were trying to pay low fees and now you have to pay twice a high fee. .. and you don't even always have the option.
< BlueMatt>
gmaxwell: that seems like a reasonable route
< gmaxwell>
15:34 < cbits_> gmaxwell: yep, if you slide the fee slider all the way to the left, the rbf box becomes visible and automatically checked. :-)
< dcousens_>
gmaxwell: so RBF doesn't show unless its low fee?
< luke-jr>
CPFP is more useful for the recipient than the sender, really.
< luke-jr>
prompting to enable RBF for fallback fee seems like a good idea for Core
< gmaxwell>
dcousens_: no, you could set it to default on in the prior version, but few users did until it was too late.
< cbits_>
Electrum uses dynamic fees by default now. The last two slider options for fee preference are within 10 blocks, or within 25 blocks. The within 25 blocks option makes the rbf box visible and sets it on.
< luke-jr>
cbits_: why isn't it always visible?
< cbits_>
Not sure, probably to be neutral. Less boxes for users to check or figure out. You can make it always visible in settings.
< luke-jr>
i c
< achow101>
luke-jr: it can be set to always be visible