< wumpus>
OH I was misreading that, looking at the github page it seemed like jtimon was still adding commits to it, but it says "added a commit to jtimon/bitcoin that referenced this pull request", I think that's new?
< sipa>
i don't expect wumpus to sleep at 2:30 pm
< sipa>
:)
< wumpus>
I have a strange sleep/wake rhythm sometimes, but no, I don't expect so either :-)
< NicolasDorier>
yeah, I just saw that too sorry, last time I checked was only the const change :p
< btcdrak>
sipa: I dont expect him to sleep at all. I will buy some match sticks to keep his eyes open 24/7
< sipa>
btcdrak: quality of review may suffer slightly...
< wumpus>
it *is* only the const change, github is just making it confusing
< GitHub37>
bitcoin/master 6f3d616 Jorge Timón: Trivial: Make CBlockIndex param const in ContextualCheckBlockHeader and ContextualCheckBlock
< GitHub37>
bitcoin/master 381917f Wladimir J. van der Laan: Merge #8347: Trivial: Make CBlockIndex param const in ContextualCheckBlockHeader and ContextualCheckBlock...
< GitHub87>
[bitcoin] laanwj closed pull request #8347: Trivial: Make CBlockIndex param const in ContextualCheckBlockHeader and ContextualCheckBlock (master...0.12.99-consensus-const-lost) https://github.com/bitcoin/bitcoin/pull/8347
< NicolasDorier>
thanks
< jtimon>
awesome, thanks guys
< jtimon>
NicolasDorier: regarding some of our diffenrces in getflags parameters, they will be resolved by removing issupermajority instead of moving it
< NicolasDorier>
ooooh that's super cool if we can remove it
< NicolasDorier>
how? hard coding the activations ?
< NicolasDorier>
gmaxwell: do you plan to make a PR soon about removing ISM completely ? I can work on redoing it if needed, it simplify the code I'm writing for verifyBlock in consensuslib
< jtimon>
NicolasDorier: yeah, hardcoding the activations in Consensus::Params like CodeShark did with bip34, the other day gmaxwell said he was working on suh a branch
< jtimon>
Ideally we would use an array, I wish I had seen the PR for BIP34 when it was open...
< jtimon>
I'm happy to write that too, the hardest part for me would be too look at the heights and block hashes
< jtimon>
but yeah, whoever writes it, it simplifies things for libconsensus encapsulation
< jtimon>
sipa: I've been thinking more about the consensus vs script flags, I guess it has the advantage of having a much shorter list of consensus flags, basically only one per consensus deployment (well, we can keep bip68 and bip112 as separated flags), while the script flags have many more thant in principle don't need to be exposed in libconsensus
< sipa>
jtimon: right; if the script code is at some point duplicated to form a consensus-only form, we could combine the flags there
< sipa>
but as long as the script code is more generically useful, it will have flags that are not necessarily consensus, and it feels cleaner to have the script code independent
< jtimon>
but I still believe that the refactor is much simpler if we put temporarily the locktime flags and the new ones (bip30 and bip34) in the script flags and then we separate them
< jtimon>
those non consensus script flags are hidden behind the flags in script/bitcoinconsensus.h for libconsensus anyway, that's why I wasn't seeing the point at first
< jtimon>
probably that's where the consensus flags should be instead of consensus/falgs.h as previously suggested, but then main would need to include script/bitcoinconsensus.h
< jtimon>
s/falgs/flags
< jtimon>
mhmm, no converter is necessary if the flags in script/bitcoinconsensus.h and script/interpreter.h just keep matching in their bit numbers...
< sipa>
that's a terrible idea
< sipa>
i've argued several times that they should be made independent
< sipa>
internal constants should not leak into a public API
< jtimon>
I also prefer the converter function, but that's what we're doing right now
< jtimon>
makes sense
< jtimon>
ok, this helps me understand your whole reasoning much better, thanks
< sipa>
but yes, that is what we're currently doing - but i wouldn't extend that practice further
< jtimon>
I'll write a converter function and see how disruptive it looks compared with temporarily putting non-script flags in script/interpreter.h
< jtimon>
erik from libbitcoin also complained about the flags, I believe that was one of the reasons they don't use our API directly and copy the code instead, but IIRC the main one is having to serialize/decerialize the tx to be signed
< jtimon>
should talk to him again to remind me his thoughts
< jtimon>
in fact, I should have taken notes
< jtimon>
I have a very good reason for trusting my memory instead of taking notes most of the time, but I forgot what it was ;)
< luke-jr>
Lightsword: my hot wallet has too much in it to risk upgrading yet; I should probably deal with that
< Lightsword>
luke-jr, can’t you just backup wallet.dat before doing anything with latest version?
< luke-jr>
Lightsword: sure, but bdb issues are the least probable kind of loss
< luke-jr>
I have no particular concerns, just don't like to use new code with lots of funds
< Lightsword>
what are the other probable kind?
< luke-jr>
Lightsword: sending the wrong amount somewhere, or to fee
< * Lightsword>
wonders why that would be less likely to happen with an older wallet
< luke-jr>
no changes to the old code
< luke-jr>
the usual reason stable software is preferred over bleeding edge
< luke-jr>
anyhow, looks like it was a build system issue, and got it to build
< Lightsword>
yeah…but 0.10…is two releases behind the latest stable
< luke-jr>
not even a year old
< luke-jr>
0.10.4 was released 2015 Nov
< Lightsword>
I thought 0.11.2 or whatever was earlier since 0.10.4 was a backport
< luke-jr>
v0.11.0 was 2015 Jul
< luke-jr>
only a year
< GitHub177>
[bitcoin] luke-jr opened pull request #8386: mining: Optimise for typical mining use with blockmaxsize (master...blockmaxsize_opti) https://github.com/bitcoin/bitcoin/pull/8386
< GitHub184>
[bitcoin] luke-jr opened pull request #8387: [0.13] mining: Optimise for typical mining use with blockmaxsize (master...blockmaxsize_opti-0.13) https://github.com/bitcoin/bitcoin/pull/8387
< GitHub33>
[bitcoin] luke-jr closed pull request #8387: [0.13] mining: Optimise for typical mining use with blockmaxsize (master...blockmaxsize_opti-0.13) https://github.com/bitcoin/bitcoin/pull/8387
< GitHub51>
[bitcoin] luke-jr opened pull request #8388: [0.13] mining: Optimise for typical mining use with blockmaxsize (0.13...blockmaxsize_opti-0.13) https://github.com/bitcoin/bitcoin/pull/8388
< gmaxwell>
are there NO web block explorers that show transaction version numbers?!
< gmaxwell>
hmph, when connecting blocks at start RPC can return "Loading banlist" for the duration...
< wumpus>
it's supposed to release the lock every block, but sometimes it appears it doesn't
< roasbeef>
gmaxwell: smartbit and http://srv1.yogh.io/ do, a few other do as well but can't recall off the top atm
< achow101>
gmaxwell: blockcypher does under "advanced details"
< jonasschnelli>
<phantomcircuit:#bitcoin-core-dev> jonasschnelli, can you check 8152 again
< jonasschnelli>
Yes. Will do.
< sipa>
wumpus: i think we should try adding a yield and see if there is a performance degradation
< sipa>
wumpus: in ActivateBestChain
< gmaxwell>
roasbeef: achow101: Thanks, I tried like 5 of them before giving up.
< wumpus>
sipa: Iworth a try I guess
< sipa>
wumpus: as far as i know there is no guarantee that the scheduler gives a fair distribution of cpu time in case there are multiple waiting threads
< sipa>
in ActivateBestChain we release cs_main but pretty much instantly grab it agaim
< wumpus>
there is no guarantee, no, but with multiple cores I think usually a waiting thread should get the lock
< wumpus>
but indeed if you grab it immediately again, that may be a special case
< gmaxwell>
we could just add explicit sleeps when connecting more than a few blocks.
< wumpus>
it may be re-locked before anyone else even sees
< wumpus>
adding sleeps sounds really ugly
< wumpus>
there must be a better way
< sipa>
yes, getting rid of locks that are held for long periods of time :)
< wumpus>
I'm aware a yield is effectively a sleep for one timeshare, but at least it's as short as possible
< sipa>
maybe we can release cs_main during signature verification (but after fetching inputs from the cache), and grabbing it back afterwards and compare if the tip is still the same, and apply the changes
< wumpus>
maybe yield-every-N-ms-or-more instead of, by definition, every block? this avoids the yield slowing things down in the beginning when blocks validate really fast
< wumpus>
then again maybe it's not a performance issue at all
< sipa>
well there are two questions: 1) is yield sufficient to fix this problem at all?
< sipa>
2) what performance overhead does yield have if there are no orher waiters
< wumpus>
the issue is only noticable when cs_main is held for, say, more than a second
< wumpus>
(2) is up to 20ms, depending on the scheduler
< wumpus>
it just gives away the rest of the timeslot
< sipa>
it gives away the rest of the timeslot if there is another candidate for taking the timeslot
< wumpus>
yes, which is very likely on a modern multitasking OS
< wumpus>
but indeed it depends on factors not under bitcoind's control
< sipa>
we should benchmark :)
< jeremyrubin>
I'm using this_thread::yield() in some code right now while I wait on something, seems to work well enough
< jtimon>
sorry, late
< jonasschnelli>
no
< jonasschnelli>
meeting is in 23 mins. :)
< jtimon>
jonasschnelli: oh, yeah of course, how could I need to be reminded again? I even got it right the last weeks, I was of course just pinging other people...besides there's no start meeting in the backscroll...
< sipa>
jtimon: calendars help :)
< sipa>
jtimon: bitcoin core irc meeting is 7pm iceland time
< btcdrak>
We should hold the next coredev.tech in Iceland @jonasschnelli
< * sipa>
in favor!
< jonasschnelli>
btcdrak: Yes. Sure! Always wanted to go to see how the vikings live today. :)
< jonasschnelli>
Reikjavik must be awesome
< btcdrak>
Yes, I already has a look they have nice meeting venues
< sipa>
nature just outside of reykjavik is much nicer even
< jtimon>
yep, I should finally put this meeting in my google calendar, but I'm usually at home in front of the pc at 7 pm iceland, which will only change from CET twice, I should be able to memorize this, I just saw the 1X:34 in the clock and panicked
< jtimon>
I'll just fo it I guess, f$%#ng goole...
< jonasschnelli>
there is a PR to export the seed in dumpwallet
< jonasschnelli>
we could consider that for 0.13
< jonasschnelli>
its easy to review.
< jonasschnelli>
Low impacts
< wumpus>
jonasschnelli: if it's low-impact, why not
< luke-jr>
wumpus: the default policy not performing as well as inadvisable policies is apparently an issue, which I just PR'd a fix for an hour or so ago
< sipa>
1) just a height after which the previous softforks activate
< btcdrak>
wumpus: yes
< jtimon>
well, my preference would be to introduce a getflags function first
< gmaxwell>
when are we branching? I've been holding off on patches until after the branch.
< sipa>
gmaxwell: branch was a few days ago
< wumpus>
gmaxwell: we've already branched a few days ago
< btcdrak>
gmaxwell: we branched already
< luke-jr>
how many exceptional blocks violate softfork-added rules?
< sdaftuar>
i'm curious to know what you're thinking about doing for regtest
< btcdrak>
for some reason github didnt ping the channel on new branch creation
< gmaxwell>
oh I missed that.
< instagibbs>
sdaftuar, any issues you can think of?
< sipa>
sdaftuar: ugh... a command line option to enable the rules individually (from the start, over the entire chain)?
< jtimon>
but knowing that ISM is going to be reomved, just don't touch any ISM call and leave it all prepared for ISM calls to be simply removed from main and replaced with the new code inside versionbits::getflags()\
< wumpus>
luke-jr: 8388 is a pretty large change, isn't that a lot to do last-minute? also you don't have a description on the pull at all
< gmaxwell>
jtimon: ISM things are not versionbits.
< gmaxwell>
We went over this before.
< btcdrak>
yes we did
< jtimon>
I was previously of the opinion that a height was enough but I changed my mind, yes, block hash too
< gmaxwell>
sipa: see consensus.BIP34Height = 227931;
< gmaxwell>
sipa: in chainparams.cpp
< btcdrak>
jtimon: gmaxwell said he is working on a ISM removal PR
< gmaxwell>
"like that"
< luke-jr>
wumpus: well, I didn't expect counting bytes to be used as an excuse to have the release notes pressure miners to use bad policy configs
< jtimon>
gmaxwell: agreed, I only want a getconsensusflags function, if it's defined in versionbits or consensus/header_verify.cpp or somewhere else I don't care so much
< gmaxwell>
sdaftuar: what I'd done is just set regtest to 0, but hadn't checked to see what tests doing that would break.
< jtimon>
but I strongly oppose to define getflags in main.cpp
< gmaxwell>
jtimon: the things which are ISM should not be flags, becuase they're not optional anymore.
< wumpus>
luke-jr: ok, fair enough, I do think it requires more discussion
< btcdrak>
jtimon: but that doesnt mean you can stuff them into an unrelated unit
< sdaftuar>
gmaxwell: i guess we can just change the tests that test activation of ISM things
< sdaftuar>
gmaxwell: so maybe that will work fine?
< morcos>
gmaxwell: they still need flags right? for when they are active and when they aren't?
< gmaxwell>
sdaftuar: yea, thats what I was thinking. the obvious alternative is to set it to something not 0, like 5.. and let the tests handle it.
< sdaftuar>
i wonder if there are cases where we might want to test how a chain behaves when some blocks don't satisfy a rule (like bip34) and then later ones do.
< gmaxwell>
morcos: some of the things are not accomplished via flags like mechenisms, e.g. the version number checks. They're not a script feature.
< jtimon>
btcdrak: yep, I just want to coordinate with him, in fact, I believe NicolasDorier and I will benifit more than gmaxwell from this coordination, gmaxwell doesn't really need us and can do it all in main
< sipa>
sdaftuar: versionbits does not need that anymore, and i don't care strongly about testing that for purely historical features
< sipa>
jtimon: please
< sdaftuar>
sipa: yeah i think you're basically right, we can just enumerate each of the ISM soft forks and make a decision, there's only a handful
< sipa>
sdaftuar: exactly
< wumpus>
jtimon: where to put it and what solution to use are orthogonal issues
< * luke-jr>
observes that P2SH was actually more like BIP 9 than ISM was
< gmaxwell>
Some future softforks that get virtified might even have no violations anywhere in the chain, in which case, I'd propose just removing all conditional logic for them entirely.
< jtimon>
wumpus: agreed, but two people trying to complete verifyblock would benefit in sharing the most common refactors as possible
< gmaxwell>
I believe CSV might be an example of that.
< jtimon>
gmaxwell: I agree, but I was talking much shorter term here
< jtimon>
as in, whithin the refactor window
< gmaxwell>
adding a bunch of additional 'flags' for things like height checks would be undesirable code smell.
< gmaxwell>
as would moving ISM logic into a file called versionbits.cpp.
< sipa>
agree
< luke-jr>
gmaxwell: GBT will likely pose a problem for that, but probably not insurmountable (worst case we can neuter GBT by removing the clients-can-modify-the-template aspect, since nobody much uses it afaik)
< CodeShark>
I had proposed a softforks unit to solve this a while back ;)
< gmaxwell>
To be honest, I am really frustrated right now with some of the pointless nitpicky behavior being driven by 'refactoring' all of a sudden. It's making me want to not be involved in the project.
< NicolasDorier>
btw, I'm almost done with the PR removing ISM
< gmaxwell>
I don't have the time to endlessly debate minutia with people who are being very tunnel vision about varrious things.
< CodeShark>
?
< GitHub128>
[bitcoin] jonasschnelli opened pull request #8389: [0.13] Create a new HD seed after encrypting the wallet (0.13...2016/07/hd_enc) https://github.com/bitcoin/bitcoin/pull/8389
< jtimon>
gmaxwell: the movement of ISM to versionbits was only in preparation to more cleanly remove it later and not having to include main.h from consensus files, but yeah no need to move it, just delete it beforehand
< wumpus>
well code that is going away doesn't need to be moved
< jtimon>
of course, agreed
< wumpus>
but refactoring main.cpp is also important - we've held it off for so long now
< CodeShark>
Making sure the code doesn't get cluttered in the long run and establishing good habits for this early on are not tunnel vision, imho
< sipa>
agree
< wumpus>
after all that time we still have that huge c++ file with so many different responsibilities
< jtimon>
all I want to agree is that is "going away" to getflags, and that getflags has some place to exist (it may not be versionbits.o)
< jtimon>
it won't go away
< sipa>
jtimon: is getflags the "one set of flags for everything"? if so, nack
< kanzure>
while we're busy refactoring everything i would like libconsensus and a pony
< jtimon>
it will be replaced and we can leave it all preapare for where it will be replaced with something like bip34 and that will simplify things, I believe
< sipa>
ok, can we move on?
< wumpus>
in any case, this doesn't seem to be a constructive topic in the meeting anymore
< CodeShark>
yes, let's move ob
< sipa>
or are there still things about ISM?
< CodeShark>
on
< wumpus>
#topic checkqueue.h optimization
< btcdrak>
ping jeremyrubin
< jtimon>
sipa: thank you for being so direct. After our discussions on the subject, I agree the script flags should be separated, but for now it should be libconsensus flags and script flags, yes
< jeremyrubin>
Hi
< jeremyrubin>
So all I wanted to say is that I am doing some refactoring of checkqueue, have some nice improvements preliminarily.
< jeremyrubin>
Will push something to my fork if you're particularly interested in it
< jtimon>
sipa: I know you won't like non-script flags in script/interpreter even temporarily, I'm working on changing that, but that's just hwat I have right now
< wumpus>
jeremyrubin: looking forward to the pull request :)
< jonasschnelli>
jeremyrubin: nice!
< sipa>
jeremyrubin: looking forward to it, obviously
< sipa>
(as long as it doesn't rely on MIPS assembly)
< jtimon>
wumpus: well, it is constructive for me, I'm getting useful feedback
< jeremyrubin>
just wanted to note it as I've heard some other people looking at it, so don't want to duplicte work
< jeremyrubin>
that's all!
< cfields>
jeremyrubin: along those lines, I've been working on optimizing the sigcache, after morcos pointed out some cool speedups. maybe we should work together to come up with a good representative bench for testing improvements?
< wumpus>
jtimon: ok that's good; it didn't seem to be going the right way. Seems that gmaxwell wants to get rid of ISM as soon as possible, so refactoring the ISM part is just a waste of work
< cfields>
morcos: or are you still hacking on that? ^^
< NicolasDorier>
I will propose a PR for removing ISM in one or two hours normally
< gmaxwell>
I wanted to remove it in 0.13 but didn't want to introduct a conflict with the SW merge so I held off.
< morcos>
cfields: yep, we're working on it together!
< gmaxwell>
It saves like two whole milliseconds from block connecting times. :P
< wumpus>
NicolasDorier: great
< jeremyrubin>
cfields: sounds good -- will msg out of meeting
< CodeShark>
wumpus: focusing too much on ISM given its impending death is probably not the most urgent thing - my concern is more ovet general habits
< CodeShark>
*over
< cfields>
morcos: ah, ok.
< wumpus>
CodeShark: well the topic was removing ISM
< sipa>
gmaxwell: that's 12 minites of sync time :o
< morcos>
gmaxwell: i think with the lock free stuff jeremy is working on, i can get validation times down from 200ms to sub 50ms on my machine
< sipa>
morcos: impressive
< sdaftuar>
he has a lot of cores :P
< jonasschnelli>
heh
< BlueMatt>
sdaftuar: but across 2 cpus, so numa :/
< wumpus>
oh no numa, is that still a thing
< jtimon>
wumpus: I also want ISM to go away as soon as possible and I'll rebase on top of the PR as soon as it appears, I was just asking for an agreed trivial plan on how to "just don't touch ISM, the replacement will go here" that we could work on in the meantime
< gmaxwell>
wumpus: any multisocket system is numa.
< gmaxwell>
(these days)
< jtimon>
wumpus: in my experience "will go here" can take a lot of time
< wumpus>
gmaxwell: I didn't know, haven't seen much multisocket systems in recent times, I was hoping it was a crippled thing from the past :)
< jtimon>
so we should totally not touch ISM in any consensus refactor PR, but where the replacement should go is something we can discuss after the meeting
< wumpus>
in any case optimizing bitcoind for numa is very much out of scope :p
< btcdrak>
we seem to have drifted off topic
< wumpus>
anything else to discuss?
< NicolasDorier>
#topic Handling Dust on the wallet
< wumpus>
hey, I'm supposed to set the topic
< NicolasDorier>
oh sorry :D
< NicolasDorier>
noob here
< wumpus>
(but no problem with this one)
< gmaxwell>
Has anyone picked up that simulator work and improved it?
< sipa>
it also does not work (for logs) if someone else than the chair does it
< wumpus>
#topic Handling Dust on the wallet
< NicolasDorier>
so the problem is boring: wallet code is preventing to create output below dust using ::txminRelayFee
< jtimon>
I think the ISM removal topic is mostly finished, TODO adapt any refactor for ISM to be removed before-hand, TODO decide where the replacement need to go during the refactor
< NicolasDorier>
problem is that when we bumped it long time ago, some transaction could not be relayed anymore
< NicolasDorier>
causing some stress to users
< NicolasDorier>
several way to fix the problem
< jonasschnelli>
You can if you add new/fresh larger coins? right?
< jonasschnelli>
(econimical painful)
< morcos>
NicolasDorier: I dont' have a good sense for how to make this work properly and be something the at doesnt involve fiddlign with a bunch of variables
< sipa>
NicolasDorier: if all wallets at all times would not create outputs that were uneconomical to spend, there would be no issue, i think
< luke-jr>
I thought the wallet avoided change below some number much larger than dust?
< MarcoFalke>
yes
< morcos>
But I think its clear we should have the MinimumOutput be higher than the dust level, so that when we raise the dust level, we know prev releases are still not generating dust
< NicolasDorier>
luke-jr: problem is that it is not "much larger"
< NicolasDorier>
it is "exactly"
< luke-jr>
NicolasDorier: 0.01 BTC last I checked
< MarcoFalke>
but this does not affect when you pay someone dust
< gmaxwell>
This was intentional; not the stress of course, but not relaying transactions with produce outputs which are a loss to consume. But it sounds like you're not talking about the wallet, but relay behavior.
< sipa>
gmaxwell: no, this is about wallet
< luke-jr>
MarcoFalke: sure, but that's user error
< gmaxwell>
The wallet tries to produce change >0.01 btc as luke mentions. (which is its own stupidity, see my earlier simulator question)
< NicolasDorier>
for calculating the smallest output
< MarcoFalke>
gmaxwell: I asked murch to present his findings of his masters thesis in one of the meetings
< jonasschnelli>
I think it should try much higher min change outputs if possible.
< jtimon>
btw, GetDustThreshold and IsDust are still in primitives/transaction.h (libconsensus), unrelated, but maybe cheap to do both at the same time
< morcos>
NicolasDorier: i also forgot about that 0.01 btc thing
< jonasschnelli>
We don't know the fees in – lets say – 2 years.
< luke-jr>
jonasschnelli: how much higher? this could be a privacy problem
< gmaxwell>
NicolasDorier: if select coins does not make an exact match, it attempts again with amount + 0.01 btc as the target.
< luke-jr>
jonasschnelli: we have fee learning logic..
< jonasschnelli>
luke-jr:If possible 1:1 like the spend itself
< jonasschnelli>
:-)
< luke-jr>
jonasschnelli: that harms privacy
< jonasschnelli>
(and would also not be possible)
< gmaxwell>
luke-jr: it can be done in ways which don't. Please see my comments on the remove extranious inputs PR.
< NicolasDorier>
gmaxwell: this is coin selection, but it does not prevent the creation of an output below 0.01 right ?
< MarcoFalke>
no
< jtimon>
luke-jr: please tell me "fee learning logic" is in policy/fees.o and not consensus/deeplearning.o
< MarcoFalke>
Those are different objectives to optimize
< gmaxwell>
NicolasDorier: it may not be possible to prevent that except by turning small amounts into fees.
< luke-jr>
jtimon: afaik yes
< luke-jr>
gmaxwell: if both outputs have the same or near-same value, any observer can see the approximate tx value, no?
< jtimon>
luke-jr: cool :)
< gmaxwell>
luke-jr: I'll explain more outside of the meeting.
< luke-jr>
ok
< NicolasDorier>
ok I'll need to study more about this 0.01 thing and the implication, will catch up with morcos
< gmaxwell>
the whole logic with that stuff is braindamaged imo.
< NicolasDorier>
I heard someone is doing some work in that area (Xekyo)
< gmaxwell>
manages to fail under every kind of metric I can come up with, except for consistency with the existing code. (which, unsurprisingly, is the property the tests test for... :) )
< murch>
marcofalke: My thesis is still in the making, sorry.
< MarcoFalke>
NicolasDorier: it's murch in this channel
< NicolasDorier>
oh ok
< gmaxwell>
okay, is there anything else?
< wumpus>
no proposed topics at lesat
< kanzure>
well, i am assembling a list of things to talk about in person
< kanzure>
so i will be heckling some or most of you regarding proposed subjects
< jtimon>
well, you could talk more about that to close the meeting, no?
< gmaxwell>
wumpus: I have a topic.
< gmaxwell>
wumpus: sigops max size, and the sigops per byte limit.
< kanzure>
jtimon: what would you like to hear?
< wumpus>
#topic sigops max size, and the sigops per byte limit
< luke-jr>
(gmaxwell: btw, I have no strong opinions on the coin selection stuff, if it's one of those minutia you'd rather not spend time on.)
< jtimon>
kanzure: nothing specifically just things to talk about in person, I didn't folloed your link yet
< gmaxwell>
We have ongoing complaints that the bytespersigops limit has made some bare multsig outputs difficult to spend (normal software behavior won't work)
< jonasschnelli>
Shouldn't this be included in 0.13 then?
< luke-jr>
jonasschnelli: probably
< gmaxwell>
It's unambigious why the limit was introduced. There is was a consensus sigops exhaustion attack resulting in small blocks.
< sipa>
i think 8365 should be in 0.13; i think 8364 is needless complication
< sdaftuar>
sipa: i agree
< gmaxwell>
8365 corrects it in the way I originally proposed (so I like it)
< sipa>
(but apart from that i'm not strongly against 8364)
< luke-jr>
gmaxwell: that isn't why the limit was introduced, though.. it was to filter spam
< wumpus>
for 0.13 we should prefer a simple solution
< sipa>
luke-jr: in your world view perhaps
< jonasschnelli>
Im in favor of #8365 (at least for 0.13)
< luke-jr>
sipa: considering I wrote it..
< gmaxwell>
Luke proposes 8364 in addition, driven by a concern that allowing high sigops transactions but with high fees is sending an implicit endorsement that it's okay for random transactions to burn up lots of actual signature operations needlessly if they pay for it.
< sdaftuar>
the PR that introduced the limit lacks explanation
< sipa>
luke-jr: sure it may have been your intention; that was certainly noy clear to me (or many, i think)
< jtimon>
is this about #8362 ?
< sipa>
luke-jr: i disagree that it helps at all with preventing spam, and only encourages further bloating
< wumpus>
sdaftuar: because it was a DoS fix
< luke-jr>
jtimon: no, 8364 and 8365
< gmaxwell>
It was extensively discussed in here. It's revisionist to suggest that it was merged for any reason other than consensus sigops exhaustion.
< jtimon>
luke-jr: thanks, scrolling back
< gmaxwell>
I wouldn't have supported it otherwise.
< gmaxwell>
In any case, 8365 corrects the issue though sdaftuar expressed some concern that it also causes small miscosting of a small amount of transactions.
< gmaxwell>
(since most of the time no sigops flooding attack is happening)
< sdaftuar>
gmaxwell: in the current form, #8365 sets the conversion at 20, not 50
< sdaftuar>
by default
< luke-jr>
I don't think we can solve that without a much more complicated change..?
< CodeShark>
luke-jr: I still adhere to the view that "spam" lacks a technical definition here
< sdaftuar>
so i think it's fine as is. if another sigops attack were to hit the network, miners could bump it up to 50 and avoid being attacked
< luke-jr>
CodeShark: in this context, it is non-pubkey data fed to sigops as a key
< sdaftuar>
in its current form, users affected by the old policy have a better alternative to bloating up their transactions
< wumpus>
CodeShark: storing unnecessary data in the utxo set counts as spam to me
< sdaftuar>
we can think about better policies in the long run later, imo -- this is good enough for now
< wumpus>
sdaftuar: for 0.13 that's the most important
< luke-jr>
8365 as-is, is a regression of used behaviour
< sdaftuar>
wumpus: yep agreed
< CodeShark>
wumpus: if someone is willing to pay for it it's income to someone else
< sipa>
luke-jr: as-is, i think it will have as sole effect that some people who are now bloating their transactions to bypass the limit, will stop doing so
< gmaxwell>
luke-jr: I don't agree that it is. since the change manages to except the counterparty data storage transactions, I'm not aware of anything that could be classified as spam that exists now that it usefully blocks.
< wumpus>
CodeShark: everyone with a full node suffers from it, without being paid for it
< CodeShark>
wumpus: that's a problem with incentives, not spam
< wumpus>
CodeShark: that's just like e-mail spam - everyone with an email account has to handle the extra messages
< sipa>
can we keep philosophical discussions for elsewhere?
< luke-jr>
gmaxwell: can you rephrase?
< wumpus>
uhm, okay
< gmaxwell>
luke-jr: point me to a transaction that 8364 would block that should be blocked? the only thing the sigopsperbyte limit was blocking was the sigops exhaust transactions (blocked by 8365) and counterparty data storage transactions, which 8364 jumps though enormous hoops to not block.
< GitHub86>
[bitcoin] jonasschnelli opened pull request #8390: [Wallet] Correct hdmasterkeyid/hdmasterkey name confusion (master...2016/07/hd_masterkeyrename) https://github.com/bitcoin/bitcoin/pull/8390
< sipa>
wumpus, CodeShark: sorry for interrupting, but i don't think you guys disagree at all - only about what certain words meam
< luke-jr>
gmaxwell: both of those should be blocked, and AFAIK 8365 doesn't block anything at all?
< luke-jr>
gmaxwell: (to be clear, Counterparty should be using OP_RETURN only for their data)
< gmaxwell>
luke-jr: 8365 closes the sigops bloat attack vector.
< wumpus>
only roughly one minute to go
< gmaxwell>
I think we should close the meeting.
< wumpus>
#closemeeting
< wumpus>
#endmeeting
< lightningbot>
Meeting ended Thu Jul 21 19:59:17 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< luke-jr>
gmaxwell: it just costs more to perform the attack?
< gmaxwell>
luke-jr: no, jesus chrsit luke. It _eliminates_ the attack.
< sipa>
luke-jr: ideally, they are disincntivized to be produced at all. but the current code (which blocks them) has as only result that they bloat their transactions to bypass the restriction
< gmaxwell>
luke-jr: The attack is that someone make a few small transactions that use up all the block capacity, while paying less than all the other users who were left waiting.
< gmaxwell>
if someone actually wants to outbid the other users and use up the block capacity, they can do that-- and nothign can stop them (except for eventually running out of funds to blow), which is how open markets work. :)
< gmaxwell>
8365 fixes things so that users making weirdly shaped transactions can't use up capacity in excess of what they've paid for.
< gmaxwell>
Thus no reason to create a weirdly shaped transaction.
< sipa>
gmaxwell: that would only be the case if the factor were 50, not 20... but it is the best we can do today without potentially impacting transaction relay
< sipa>
ok, i'm going to catch some pokemon
< sipa>
i mean
< BlueMatt>
.........
< sipa>
i'm going for a walk
< luke-jr>
lol
< dstadulis>
lol
< gmaxwell>
did sdaftuar measure the impact? or did we just gimp the fix on conjecture?
< gmaxwell>
I guess at 20 it's no more gimped than the code currently in place.
< jonasschnelli>
sipa: haha..I tried that once on my speedbike. Almost crashed in a car...
< luke-jr>
gmaxwell: ok, so how does 8365 close the data storage abuse vector, that bytespersigop used to at least make clear was abuse?
< gmaxwell>
luke-jr: none of these things have any impact on data storage abuse.
< sipa>
luke-jr: clearly people dom't care about whether we consider it abuse or not
< gmaxwell>
non the bytespersigops, not 8364, not 8365.
< gmaxwell>
bytespersigops had a accdential and coincidental negative effect on counterparty data storage transactions, that 8364 introduces a bunch of code to avoid (by counting signature operations instead of consensus sigops)
< luke-jr>
sipa: the OP_RETURN fiasco suggests otherwise, no?
< jtimon>
yeah I heard that pockemon thing is awesome and they're going to make a mario kart version soon in collaboration with google and tesla, but I can't find it on gihub, I'm searching on bitbucket next...
< jtimon>
sorry, #bitcoin
< sipa>
luke-jr: but OP_RETURN was a deliberately created alternatove for data storage
< sipa>
luke-jr: not blocking something intended to be a dos fix, that people perceive as accidentally stopping some transactions
< luke-jr>
no, it was created for commitments to external data, not storage :/
< sipa>
in the long term, we can't rely on any of these techniques
< sipa>
we can't expect people to behave nicely
< sipa>
but we can design systems that make some behaviour expensive, regardless of intent
< sipa>
and in the long term, that should be enough
< gmaxwell>
80 bytes wasn't needed for 'commitments to external data'...
< jtimon>
luke-jr: IMO "the right way" ie pay2contractOrHash should replace the rpc interface for op_return
< gmaxwell>
there is no rpc interface to op_return.
< sipa>
furthermore, OP_RETURN probably actually did help us get rid of stuff that would otherwise permanently have been added to the utxo set (though i thonk at least the increase to 80 bytes was a mistake)
< luke-jr>
sipa: the problem isn't OP_RETURN itself, but people portraying it as us endorsing data storage on-chain
< gmaxwell>
jtimon: you were right, I had no clue that was there. wtf.
< gmaxwell>
jtimon: result of PR #6346
< gmaxwell>
I give up.
< jtimon>
gmaxwell: oh, awesome, thanks, traceability++, shall you comment in 6346 ?
< midnightmagic>
he left.
< midnightmagic>
whoah
< midnightmagic>
how did commitid d7078533ebd32bb5071f4dba8e3d9c5a3b1f0d4c get merged
< sipa>
luke-jr: i'm aware
< jtimon>
I only saw that when doing a one old version of elements thing for assets, I assumed this was known by everyone since I was unfamiliar with rpc in general
< jtimon>
I was mostly only familliar to the rawtx prc which I was modifying
< jtimon>
s/prc/rpc
< jtimon>
I just looked weird at the op_return and moved on
< sipa>
heh, i don't remember seeing that PR
< midnightmagic>
me neither. I can't tell people not to use OP_RETURN anymore.
< btcdrak>
Counterparty are about to stuff Ethereum contracts into them :-p
< sipa>
counterparty isn't using our rpc interface
< midnightmagic>
you reviewed it and ACK'd it
< * btcdrak>
sniggers at the back of the room
< jtimon>
let's put effort in the fix before tha blame, please, even if git blame is the source of all fixes
< instagibbs>
I'm guessing the amount of op_return data via that interface is really tiny...
< instagibbs>
I only found out about it last week
< sipa>
yes, seems nobody even knew about it...
< jtimon>
I guess so too, that's how I thought it was allowed, but if there's nobody agains, we should really get this out
< jtimon>
I mean, it's in the "standard" policy, but I consensus first, policy later
< sipa>
wumpus: good news: yield() doesn't seem to affect performance noticable (i didn't do an extensive benchmark, but early in the chain but:
< sipa>
with 2173 blocks per second, i doubt we're losing much cpu time
< sipa>
wumpus: the bad news: it doesn't fix the 'Loading block index' message during activation
< sipa>
oh, wait, that's expected maybe
< sipa>
instead of showing something about banlist
< sipa>
bad news is that i can't reproduce the bug, i guess
< NicolasDorier>
I am trying to remove ISM right now... however, I'm hitting problems with tests testing the ISM logic for old soft fork. Especially, normal ISM have two phase, activation (at 75% new rules are enforced for block version X) and version enforcement (at 95% if nVersion < X, reject). However, the difference between both threshold is only relevant during the
< NicolasDorier>
transition between the two thresholds and a few block afterward. We can safely replace the activations checks by enforcement checks.... Except that if I do that, then tests rightly break...
< NicolasDorier>
I'm tempted to just remove the old ISM sf tests
< NicolasDorier>
and make regnet with all the ISM SF activated from block 0
< NicolasDorier>
I mean block 1
< NicolasDorier>
thought ?
< sipa>
for regtest we probably need command line switches to set what is activated (and at what height?)
< NicolasDorier>
mh it seems rather bothersome for code going away
< NicolasDorier>
I'd like ideally to remove old ISM SF tests, and default regnet to have everything activated from block 1. Adding those command line switches just so the old tests continue to work seems like a waste
< NicolasDorier>
or maybe hard coding some number for regtest
< NicolasDorier>
and testing that
< NicolasDorier>
hard coding some heights
< luke-jr>
sipa: I knew about it, but didn't strongly oppose it on the basis that createrawtransaction is a low-level thing that users shouldn't be exposed to anyway. And I was hoping for the author to add more useful tools (contracthash-like), but that didn't get added. ☹
< sipa>
luke-jr: it would have been nice if it hashed the data first
< sipa>
luke-jr: but i guess it's hard to change now
< sipa>
NicolasDorier: i agree... old ISM tests can probably go away
< sipa>
NicolasDorier: though only those testing activation... not those testing the new behaviour
< NicolasDorier>
yes
< luke-jr>
sipa: yes, but then there'd be no marker possible for example
< luke-jr>
sipa: as soon as libsecp256k1 has contract-sign primitives though, I hope to put it to use to hopefully kill off proof-of-existence spam ;p
< btcdrak>
luke-jr: how would that work?
< sipa>
btcdrak: we can make a signature commit to data
< sipa>
like a hash
< sipa>
except it independently also remains a valid signature
< sipa>
and you can't even tell it commits to something
< btcdrak>
oh
< btcdrak>
this sounds like black magic...
< sipa>
the commitment is obvious to anyone who knows the data being committed to
< luke-jr>
add a merkle tree, and you can do unlimited proof-of-existence in the background when you send txs
< luke-jr>
even fingerprint your wallet so you can prove its state at any given point in history, if you want to
< sipa>
luke-jr: unfortunately, very few people seem interested in timestamping, and many seem to be interested in using the blockchain as a broadcast medium
< luke-jr>
well, maybe we should add a p2p mechanism that relays data along with tx so long as the tx fee pays for the data size too?
< sipa>
or find a way to actually make the payment protocol take off, so all that stuff can just remain between sender and receiver?
< luke-jr>
I'm not sure that stuff has a sender/receiver >_<
< sipa>
if it doesn't, then why does it belong in bitcoin?
< sipa>
either it's data needef for the world to verify the transaction, or it is private information between sender and receiver of a payment
< luke-jr>
it doesn't. :<
< * luke-jr>
wonders what it would take to get p2sh^2 deployed and in use
< sipa>
luke-jr: people would move to storing data in inputs
< luke-jr>
sipa: where in policy-accepted inputs can data be stored?
< luke-jr>
wait, forgot we allow any script now
< luke-jr>
we could potentially lock that down again?