< meshcollider>
is there any style preference for atomic_bool vs atomic<bool> and similar for other atomic types?
< meshcollider>
Its not mentioned in the style guidelines
< BlueMatt>
meshcollider: not atm, I dont believe
< meshcollider>
ok sweet as
< cfields>
i have a slight preference for std::atomic<foo>, because that allows everything to be the same. atomic_bool is fine, but obviously doesn't work for atomic_myfoo
< cfields>
(i'm pretty sure i haven't even been consistent with my own preference though)
< meshcollider>
might as well change the couple of atomic_ to atomic<> and add it to the developer notes then aye? Or not worth it
< cfields>
i don't really mind either way
< achow101>
happy segwit activation
< BlueMatt>
indeed it is
< fanquake>
Has anyone had trouble gitian building the signed windows build? I'm getting a message digest mismatch. Never seen that before.
< gmaxwell>
w
< wumpus>
fanquake: for rc2? no, haven't heard issues reported
< gmaxwell>
wumpus: someone on reddit was saying it was segfaulting for them, I prodded them to open an issue.
< fanquake>
wumpus yes rc2. Had no issues with rc1.
< fanquake>
Details here if anyone is interested. https://pastebin.com/WCPjd1gE Will submit built sigs and play around with it later.
< wumpus>
meshcollider: both are used, I don't think the project needs an opinion of which one to use. Usually a CPU supports a limited set of atomic types, which is a subet of the typedef-ed ones.
< meshcollider>
ah in that case should i drop the commit to change them from my PR?
< wumpus>
it's allowed by the standard to define your own atomic<mystruct> but I don't know what it will do, fail or emulate with mutexes somehow
< wumpus>
yes, no need to make a rule about it, it doesn't affect safety
< meshcollider>
nah it was just for the sake of consistency since most cases used atomic<bool>, only like 10 in the whole project used atomic_bool
< meshcollider>
removed
< wumpus>
we should avoid creating too many little rules, that people have to slog through before contributing, my original idea with the guidelines in developer-notes was to add useful review criteria with rationale how they make the code safer
< wumpus>
then some things like consistent variable naming which are reasonably important to be able to distinguish various scopes
< wumpus>
btw has anyone tried cross-compiling bitcoin core for windows from ubuntu zesty/17.04 or pre-.10
< wumpus>
I'd like to know if they solved some of the issues that made 16.04 so terrible e.g. the lack of sane support for std::threads by default, I'll try but I wonder if someone else did already
< wumpus>
fanquake: whoa, I never saw that one before. Maybe cfields can comment what it means. Did your unsigned assert match?
< sipa>
wumpus: agree, for many things it really doesn't matter what approach is used
< fanquake>
wumpus my win-unsigned sigs match yours. Will PR them in a sec.
< wumpus>
fanquake: rah! I expected an error like this if the thing-to-be-signed mismatches, so the signature, which is added without heed to what it is added to, mismatches. But if the unsigned matches I can't think of any reason for that error.
< luke-jr>
[05:29:49] <gmaxwell> wumpus: someone on reddit was saying it was segfaulting for them, I prodded them to open an issue. <-- this "someone" I have labelled as a troll FYI, maybe a bogus report
< wumpus>
gmaxwell: luke-jr: well if he has steps to reproduce it, and can open an issue, I don't care if it's a troll :)
< luke-jr>
wumpus: sure, just saying, if he doesn't answer, it's probably made-up
< wumpus>
agree, making up segfaults just to sabotage the rc process would be a new low, but yeah you never know here
< bitcoin-git>
bitcoin/master 79191f5 Joe Harvell: Add option -stdinrpcpass to allow RPC password to be read from standard input
< bitcoin-git>
bitcoin/master affe927 Wladimir J. van der Laan: Merge #10997: RPC: Add option -stdinrpcpass to bitcoin-cli to allow RPC password to be read from standard input...
< bitcoin-git>
[bitcoin] laanwj closed pull request #10997: RPC: Add option -stdinrpcpass to bitcoin-cli to allow RPC password to be read from standard input (master...stdinrpcpass) https://github.com/bitcoin/bitcoin/pull/10997
< instagibbs>
GAit, looks like a "normal" failure: assert wait_until(lambda: len(self.nodes[1].getrawmempool()) == 5)
< GAit>
What do you mean normal? it shouldn't fail and indeed it doesn't fail on the other runners nor locally.
< GAit>
the test does this: have a node with 5 tx in mempool, dump that to file, copy file to second node, start second node, make sure it has those 5 tx - without synchronizing manually.
< fanquake1>
GAit I restarted the tests
< jnewbery>
fanquake1 - I was just looking at the Travis output and it disappeared under my feet :(
< jnewbery>
in any case, if it's a new test and it failed first time on Travis it's probably reproducible
< instagibbs>
GAit, it means that it appears to be related to your changes, or maybe you got unlucky
< fanquake1>
jnewbery sorry. Looks like the same two have failed anyways.
< GAit>
jnewbery: I merged mempool_dump into mempool_persist. instagibbs: def related to my changes, what i don't get is which is causing it to be racy. Maybe os.rename but i'd be very surprised.
< GAit>
jnewbery: seems we get the same failure, but only on some builders
< jnewbery>
ok, I'll try to reproduce locally
< jnewbery>
oh, it might be a merge conflict with master. You no longer need to assert on the return value of `wait_until()` (after #11068)
< jnewbery>
Try rebasing on master and changing the `assert wait_until()`s to `wait_until()`s
< GAit>
thanks, but i didn't rebase so i don't see how master can hurt it? - but very happy to rebase. Do you think I can also squash things while at it?
< jnewbery>
There's no merge conflict, but the semantics of that function have changed in a different file, which I believe is what caused your build failure
< jnewbery>
if you merge or rebase locally and then test I expect it'll fail in the same way
< jnewbery>
And yes, probably a good idea to squash at this point
< GAit>
thanks! i thought i was crazy :) indeed fails locally now, will fix and squash
< jnewbery>
GAit: no problem. I'll re-review once you've pushed
< GAit>
jnewbery: done but i'd wait for travis before reviewing
< promag>
jnewbery: do you think travis should merge with master and then run the tests?
< GAit>
promag: that is what it does now and what made it fail, wait_until changed in master after i branched so I had to rebase
< GAit>
i guess is better to fail earlier rather merge day but it was confusing for me
< promag>
right, the problem is when master forwards but you don't push to your branch, and at that moment your branch can have problems.
< GAit>
yes and if the code is still mergable it may suffer only too late. But rebasing often (which I like) breaks reviews doesn't it?
< promag>
usually unless you point the diff between the old and new commit.
< instagibbs>
wish github had gitlab feature of "diff since last push"
< GAit>
jnewbery: i like what you are doing in #11121
< jnewbery>
GAit thanks. Just working on some nits from kallewoof and MarcoFalke. I'll push an updated branch soon. Reviews always appreciated :)
< GAit>
jnewbery: :) - do you know why there are sleeps in the tests all over the place? if they are flacky without perhaps we need a better way to wait for the node to be ready?
< GAit>
mempool_persist seems to work fine locally without the sleeps (and 3 whole seconds faster!)
< jnewbery>
GAit - yes there are probably sleeps that can be removed
< MarcoFalke>
Indeed, we should remove all sleeps other than for explicit polling
< adrh>
hey guys. i need some help if you can help me. I've been busting my head off...i want to build a script that uses sendtoaddress and i really can't get it to work
< luke-jr>
lock order issue, so probably wouldn't affect prod bins
< gmaxwell>
what did the lockorder debugging print
< luke-jr>
it's in the pastebin
< luke-jr>
I don't understand how the walletdb.cpp lock is still in the lock order here
< ryanofsky>
probably the easiest way to fix this is to just change LoadWallet LOCK(wallet) to a LOCK2(main, wallet)
< BlueMatt>
ryanofsky: that feels strange...maybe cs_main in CreateWalletFromFile in wallet.cpp instead?
< luke-jr>
ryanofsky: do you understand the problem? why is walletdb's lock relevant here?
< BlueMatt>
taking cs_main inside of walletdb is...yuck
< BlueMatt>
luke-jr: cause its a lock inversion
< BlueMatt>
walletdb takes the wallet lock, so...lock inversion?
< luke-jr>
BlueMatt: but walletdb's lock is released before we get to the wallet lock?
< BlueMatt>
no it isnt
< BlueMatt>
ReadKeyValue calls back into wallet
< luke-jr>
BlueMatt: to ReacceptWalletTransactions?
< luke-jr>
how?
< MarcoFalke>
wumpus: Mind to create a gitian.docs under bitcoin-core?
< BlueMatt>
luke-jr: yes
< BlueMatt>
LoadToWallet
< MarcoFalke>
You wanted me to write a fedora guide for it
< luke-jr>
BlueMatt: that doesn't call Reaccept
< bitcoin-git>
[bitcoin] promag opened pull request #11125: Add bitcoin-cli -stdinrpcpass functional test (master...2017-08-stdinrpcpass-functional-test) https://github.com/bitcoin/bitcoin/pull/11125
< promag>
jnewbery: ^ as soon as you can? ty!
< wumpus>
MarcoFalke: you want a repo specifically for gitian docs?
< BlueMatt>
luke-jr: the conflict is MarkConflicted, not Reaccept
< * luke-jr>
peers at his editor
< luke-jr>
oh, the backtrace was in Reaccept
< MarcoFalke>
jup, they don't really belong in the main repo
< MarcoFalke>
Also, a GitHub wiki is not suitable
< MarcoFalke>
It is either open to be edited by anyone (with a GitHub account) or effectively none
< wumpus>
but I can't see a gitian.docs repo ever containing more than one or two files - what about a more general 'docs' repo, which could potentially include other things as well?
< luke-jr>
wumpus: +1
< * luke-jr>
wonders how to make a test that would pick up on this wallet lock issue
< luke-jr>
if (prevtx.nIndex == -1 && !prevtx.hashUnset()) { <-- this line needs a comment :/
< MarcoFalke>
wumpus: Fine with me
< wumpus>
MarcoFalke: ok
< luke-jr>
otoh, gitian docs may be unversioned, whereas other docs might be versioned?
< wumpus>
nah, all documentation tends to be about the newest versions
< wumpus>
could add sections for building older versions
< luke-jr>
wumpus: I just mean, it makes sense to tag docs per release, but not gitian docs necessarily
< wumpus>
I'd prefer not to do any tagging/brancing in the docs repo
< wumpus>
document change control tends to be very diffrent from source code
< luke-jr>
wumpus: so we keep release-specific docs with the main repo?
< wumpus>
yes, there are no plans to move anythingbesides the gitian building doc at this point
< sipa>
short topic: thanks and congrats everyone with SegWit... it seems it activated on the network without a glitch
< wumpus>
yes, congratulations everyone!
< jonasschnelli>
wohoo!
< achow101>
yay!
< cfields>
hi
< wumpus>
it took lots of time, and lots of sweat, but we did it
< kanzure>
whole bitcoin community did it
< gmaxwell>
sipa: well except the glitch of miners universally having maxblocksize=1000000 in their configs, resulting in overly small blocks even though there have been several oppturnityies to make bigger ones.
< sipa>
gmaxwell: everything on its time
< luke-jr>
s/glitch/good thing/
< sipa>
misconceptions are hard to fix
< gmaxwell>
You may well have jinxed us, since things blowing up might only happen with a >1MB block. :)
< BlueMatt>
yes, some miner appear to be confused by maxblocksize/maxblockweight options, we need #11100
< wumpus>
we should probably publicize some segwit miner's faq thing
< wumpus>
on bitcoincore.org
< achow101>
wumpus: isn't there already one?
< luke-jr>
wumpus: not if they don't encourage big blocks. :/
< luke-jr>
err, not if they do*
< kanzure>
luke-jr: it would have been better if the expectations around that would have been communicated, then i would agree with you about good thing. expectation mismatch is not ideal.
< wumpus>
achow101: there is a segwit faq but it doesn't include this
< gmaxwell>
achow101: says nothing about needing to remove the maxblocksize setting and setting maxblockweight=4000000
< luke-jr>
there is no such need, nor is it good for them to do so
< kanzure>
luke-jr: it's about being honest about expectations. if miners expected the software to do something different, then we need to better document the configuration.
< wumpus>
kanzure: +1
< harding>
Is there anything else that should go into a current segwit faq for miners?
< wumpus>
there is apparently a lack of documentation for these optons and what they do, that's clear, no matter what your opinion on what their value should be is
< kanzure>
sounds like there's some requests for a post-activation faq
< wumpus>
#topic segwit faq for miners
< luke-jr>
hm, I thought we had documented it before, but I can't find it
< BlueMatt>
blockmaxsize really just needs to be removed, it has caused significant confusion with several mining pool operators...if you want to limit the block's size-in-bytes you can do that yourself as a gbt client (isnt that what gbt is for?)
< gmaxwell>
wumpus: it's super confusing for people. esp the fact that the maxblocksize setting (which doesn't make any sense anymore) says it defaults to 750k so then I tell people to remove it and set maxblockweight=4000000 and they don't believe it'll make a >750k block.
< wumpus>
not sure if there's any other ones, have there been any other frequent questions about segwit by miners?
< luke-jr>
BlueMatt: size is often the bottleneck today, so it makes sense to limit by size
< gmaxwell>
so then I say, "it will but fine, just set blockmaxsize=4000000" and then they think if they set if over 1M they'll make invalid blocks. :(
< BlueMatt>
luke-jr: ok, then gbt clients can do so
< gmaxwell>
luke-jr: size is a bottleneck in _what_?
< BlueMatt>
wumpus: I've literally heard confusion to the tone of what gmaxwell described from every mining pool operator I spoke to
< luke-jr>
gmaxwell: IBD mainly at this point I guess
< BlueMatt>
that option needs to die
< sipa>
luke-jr: with BIP152, the size hardly matters, except if you're ridiculously bandwidth constraint to the point you'd be unable to keep a node synced in the first place
< sipa>
*constrained
< achow101>
luke-jr: IBD is too late to save already
< luke-jr>
sipa: blocksonly?
< gmaxwell>
luke-jr: not for anyone with an internet connection beyond a few megabits in speed.
< gmaxwell>
luke-jr: IBD time is limited by chainstate database operations for parties that aren't totally network starved.
< luke-jr>
achow101: it's nothing like as bad as it could be
< cfields>
i've seen a ton of questions about segwit and "the addresses that start with a 3"...
< kanzure>
ya i have seen mostly non-miner questions
< BlueMatt>
anyway, so aside from luke is there anyone who objects generally to removing -maxblocksize? (ala #11100)
< wumpus>
BlueMatt: IMO the solution to lack of documentation should be documentation, changing around things more so there's differences between versions to explain too
< cfields>
a quick update and faq about segwit addresses, how they look, and how they'll look soon would be helpful imo. even if it's just re-hashed info
< luke-jr>
not long ago, the blockchain was 100 GB. now it's 150 GB. and every month 10 GB more with 2 MB blocks :/
< kanzure>
we can also copy-paste old segwit faq details anyway
< kanzure>
(or link to them)
< sipa>
luke-jr: prune
< luke-jr>
sipa: that doesn't eliminate IBD
< wumpus>
'take the option away and the problem goes away' is a bit too simplistic at this point
< BlueMatt>
wumpus: 11100 now removes it but keeps it somewhat compaible in spirit with what most miners appear to still have in their config files...
< BlueMatt>
ehh, deprecates it
< BlueMatt>
and clearly states blockmaxsize is deprecated
< BlueMatt>
which roughly aligns with the expectation of everyone i spoke to
< luke-jr>
11100 is stupid.
< BlueMatt>
there was confusion as to why blockmaxsize still existed in the context of segwit
< meshcollider>
deprecating makes sense to me
< morcos>
even i have trouble keeping up with what the interaction of these different options is
< kanzure>
needs an interaction matrix visualization
< kanzure>
where the cells indicate interaction :-)
< morcos>
we need to move towards getting rid of blockmaxsize, its confusion for no benefit....
< achow101>
is blockmaxsize taken into consideration with segwit or just ignored?
< sipa>
achow101: it is!
< wumpus>
deprecation is likely a good idea on the longer run, but not a fix to the misunderstanding created
< instagibbs>
taken into account
< gmaxwell>
achow101: it does something stupid, it truncates block construction when you reach that size.
< achow101>
bleh :/
< luke-jr>
it does what it's supposed to do
< gmaxwell>
it originally went away and luke threw a fit. With segwit not active it made sense as a legacy support thing but it's active now.
< morcos>
wait a second
< BlueMatt>
wumpus: I disagree here, I was explicitly asked why it was not deprecated, because the consensus limit is around weight, so thats what people expect to be setting
< luke-jr>
gmaxwell: you're rewriting history now
< BlueMatt>
deprecation is, itself, documentation
< wumpus>
BlueMatt: but we're already in the screwed state that those options exist
< morcos>
i might be wrong (b/c the interaction is complicated) but isn't it just flat out wrong to say the default is 750K
< morcos>
isn't the default infinity?
< sdaftuar>
morcos: no, i don't think so.
< sipa>
morcos: i honestly don't know anymore
< achow101>
morcos: how so?
< morcos>
if you don't set blockmaxsize it doesn't even calculate it
< gmaxwell>
morcos: the default is 1/4 the configured weight which defaults to 3m.
< sdaftuar>
if you set nothing, you get a 750k default, i believe. and a 3M weight max.
< wumpus>
I'm fine with marking it as deprecated anyhow
< morcos>
ok well we'll get to the bottom of it in a second, but look how ridiculous it is that we don't know
< sdaftuar>
if you set just blockmaxweight, then blockmaxsize is effectively infinity, i think.
< sdaftuar>
as we indicated we may do in the future in the 0.13 release notes, iirc
< morcos>
Announcement that you should set only blockmaxweight and you should set it to 4,000,000. Plus change code to ignore blockmaxsize unless specifically set, and deprecate it.
< gmaxwell>
Well we should remove the silly option. The maximum size is not really correlated with anything that is interesting to be set anymore.
< luke-jr>
miners producing blocks larger than 750k (or 1 MB at most) is malicious toward the network, and advising they do so or deprecating the blockmaxsize option needed to control this, it what is absurd.
< luke-jr>
morcos: absolutely not
< jtimon>
BlueMatt: ack on removing blockmaxsize and leaving only maxblockweight
< gmaxwell>
Luke is faulty.
< wumpus>
luke-jr: so they can still choose to do that with that option removed, right?
< wumpus>
luke-jr: e.g. by setting the blockmaxweight lower?
< sdaftuar>
or by using the gbt results and truncating themselves
< BlueMatt>
anyway, so it sounds like #11100
< wumpus>
that doesn't need two options
< luke-jr>
wumpus: not effectively
< sdaftuar>
luke-jr: why not?
< morcos>
In general it is good that we have a culture of arguing things to death here, sometimes useful things result. But in this case, we've been over this, we've heard Luke's arguments, and last time we gave in to end the argument
< sipa>
luke-jr: the assumption behind segwit is that size doesn't matter nearly as much as a new metric, weight
< wumpus>
the complication here is having two options that interact in a complicated way
< BlueMatt>
luke-jr: you're the one who always advocates for policy at the other end of gbt, or isnt that the design of gbt?
< wumpus>
please don't confuse it with what the value should be
< praxeology1>
I don't think you should depricate the maxblocksize option. If anything, if you want the software to enforce a legacy block size of 1MB, then hard code the limit... and remove the maxblocksize setting from the configuration.
< sipa>
luke-jr: i'm sorry to hear you disagree about that point, but that ship has sailed
< luke-jr>
the only way to limit size to 750k with blockmaxweight is =750000, which will make 250k blocks unless 100% of tx are segwit
< morcos>
Unless someone else agrees with Luke, I htink its just time to say unfortunately we're proceeding against his objections on this issue
< jnewbery>
Concept ACK 11100
< gmaxwell>
praxeology1: oh jesus christ you're also confused
< wumpus>
yes, that sounds confused
< luke-jr>
sipa: that ship has not sailed. that's not an argument.
< gmaxwell>
praxeology1: THERE IS NO "LEGACY BLOCK SIZE" LIMIT ANYMORE! Compatibility with old nodes achieved implicitly through how weight is constructed.
< BlueMatt>
praxeology1: these options only effect the type of blocks miners mine, and are currently a maze of confusing
< sipa>
luke-jr: segwit is active, regardless of your preferences, you should assume that rational actors will produce what the rules allow them to
< BlueMatt>
praxeology1: whats worse, the current options default to pissing away a significant amount of profit (so are always being overridden in practice, which has resulted in much confusion)
< jnewbery>
what, dead again?!
< sipa>
luke-jr: then Bitcoin is evolving to actually be tested
< gmaxwell>
luke-jr: you've failed to address any of the points which I raised against your last rant on this on github.
< praxeology1>
OK... so if there is no legacy limit anymore... then why would there be a configuration option for it? :o Are you saying post segwit activation, it has no effect now?
< kanzure>
does 11100 completely encompass the proposed changes
< BlueMatt>
kanzure: I believe so
< sipa>
luke-jr: i'm only interested in a Bitcoin that survives under the assumption that miners act rationally
< achow101>
praxeology1: it has an effect, kind of. it really shouldn't though
< sipa>
luke-jr: and you should too
< luke-jr>
gmaxwell: your "points" depend on assertions that have no evidence at this time, which you are trying to CAUSE to be true
< gmaxwell>
luke-jr: in particular expecting people to throw money on the floor to protect interests like keeping IBD times slightly lower is a failed expectation. (which we knew would fail, which is also why many of us don't support those reckless blocksize uncapping proposals)
< wumpus>
if you think a defaults change kills bitcoin, you're severly underestimating it
< luke-jr>
sipa: incentives are too broken for that to be possible right now
< sipa>
luke-jr: how so?
< luke-jr>
sipa: unless "rationally" includes altruism
< instagibbs>
I motion to move onto a new discussion unless there's some new information to be shared
< sdaftuar>
seconded
< wumpus>
people are already overriding it on large scale, no one is using that stupid default
< sipa>
seconded
< BlueMatt>
ok, other topics?
< wumpus>
#topic 0.15.0 release
< luke-jr>
wumpus: overriding it to 1 MB, not 4 MB
< BlueMatt>
second round of congrats on segwit?
< morcos>
i'm against moving on unless we have agreed to get rid of the option and make an announcement
< kanzure>
i could see luke-jr maybe working on the calculation there and showing numbers at some point. but probably not this moment. in the mean time i think 11100 review status should be checked?
< BlueMatt>
morcos: I think we did
< morcos>
we can't constantly do the wrong thing because luke is willing to argue longer than the rest of us
< morcos>
at some point we must overrrule
< morcos>
with 3 r's
< luke-jr>
morcos: you're the one who wants to do the wrong thing
< BlueMatt>
morcos: go ack 11100, I dont think there was any concept objection
< sdaftuar>
luke-jr: he is not "the one" who wants to do what he is suggesting
< sdaftuar>
he is one of many
< gmaxwell>
morcos: a point of order, I prefer we don't force 'final' decisions to be made in the meetings due to attendance complications. I think what you suggest it clearly the general thrust of where things are going, however.
< BlueMatt>
okkkkkk, so 0.15.0
< instagibbs>
reviewing the PR is the action item
< gmaxwell>
s/it/is/
< wumpus>
this is getting really unruly
< BlueMatt>
dooglus finally got some backtraces on #9683, which I think needs to be investigated for 0.15
< wumpus>
I'm going to end this meeting if this continues
< BlueMatt>
also #11126
< wumpus>
any reports of new regressions with rc2?
< achow101>
probably not
< wumpus>
to be honest dooglus has been having issues no one else is having, for a long time, we should assist him but I'm not sure we should hold up a release for it
< BlueMatt>
wumpus: well are we gonna release today? I'll take a look at his issues this afternoon :)
< BlueMatt>
i mean I'm not saying hold it up
< meshcollider>
only issue I've heard with rc2 has been fanquakes gitian issue :)
< BlueMatt>
just make sure its not an obvious "oh, thats broken" kind of thing
< wumpus>
11126 is an issue that only affects DEBUG_LOCKORDER in ithe initialization; nothing else is running at that moment
< wumpus>
so it doesn't cause any real issue
< BlueMatt>
ah, ok, i hadnt investigated it significantly
< cfields>
i've had a few crashes lately that i mentioned here i think. I've finally been able to track them down to local changes. So, no concern for 0.15.
< BlueMatt>
hmmmm, looks like dooglus' qt crash is the same one I couldnt debug
< BlueMatt>
i wonder if he's also running qt over X forwarding
< BlueMatt>
that appears to make it more likely
< jonasschnelli>
Seems like a free/alloc race in the table weak loading
< BlueMatt>
it doesnt appear to be harmful, at least, some race in setting the table sort order or something like that
< jonasschnelli>
Could also be a Qt bug
< BlueMatt>
indeed, could be
< BlueMatt>
mine was 9883
< jonasschnelli>
They are related
< jonasschnelli>
setSortingEnabled()
< jonasschnelli>
I'll investigate
< BlueMatt>
thanks
< jtimon>
the deault of one is a function of what you set in another? that interaction should definitely go away asap, ideally to me by removing the size one, I don't care the weight default is 2000000 or whatever
< jonasschnelli>
If its racy,.. could be due to a large wallet
< jonasschnelli>
(many txns)
< wumpus>
#9883 crashed in the leveldb iter?!
< BlueMatt>
the wallet I saw that on is not so large...maybe 1k txn?
< wumpus>
oh wait, wrong thread, that was just going on at the time
< jonasschnelli>
wumpus: not that I know... I think it crashes via setSortingEnabled
< BlueMatt>
dooglus also has some crashes where quick-exit races openssl mutex unlocking....I assume that is an openssl bug or so
< BlueMatt>
its the old mutex-locked-during-destruction crash, though, so not harmful
< wumpus>
jonasschnelli: yes, I see now, that's a more recognizable trace
< BlueMatt>
just a scary warning
< wumpus>
is something outside the GUI thread calling Qt functions perhaps?
< jonasschnelli>
BlueMatt: your 9883, self compiled Qt? What version?
< BlueMatt>
yes...uhhh, sec
< BlueMatt>
wumpus: that was my assumption
< wumpus>
if so, that can explain the crashes with remote X, as well as this
< BlueMatt>
jonasa: qt5 something
< wumpus>
*only* the GUI thread may call Qt GUI functions, the rest should queue signals on the GUI thread
< wumpus>
I haven't seen any violation of that, though
< jonasschnelli>
wumpus: Yeah. But that would very likely show another thread calling a relevant Qt function during the crash
< jonasschnelli>
(which it doesn't)
< wumpus>
PSA regarding 0.15.0 final - I will not be able to tag releases while in the US, and I'm leaving wednesday night 30th, and return to NL the 12th
< BlueMatt>
anyway, it appears to be an only-at-startup thing and afaict, at least for me is a very reliable either-crashes-or-works-fine thing, so I'm still not too worried
< wumpus>
jonasschnelli: you'd say so, but not necessarily, it could just mess up some internal table state, or even internal X state
< BlueMatt>
(hence the closure of the original bug, may be a qt bug)
< achow101>
wumpus: so, final this week?
< jonasschnelli>
wumpus: Yes. Possible.
< wumpus>
achow101: yes, or second half of september
< gmaxwell>
I would rather have 0.15 out sooner rather than later. so far I haven't seen any clear blockers.
< wumpus>
and I guess we should work on 0.15.1 during coredev
< BlueMatt>
gmaxwell: agreed
< jonasschnelli>
wumpus: Yes. Ideally.
< wumpus>
segwit wallet support etc
< jonasschnelli>
Yes
< luke-jr>
wumpus: maybe tag locally, and delay pushing?
< wumpus>
luke-jr: well it's more complicated, I could possibly push the tag, but I can't upload executables
< luke-jr>
wumpus: there are others who can, right?
< jonasschnelli>
I think wumpus should do it.
< wumpus>
yes, but they'll have to us their own release signing key
< jonasschnelli>
It's just 12 days (max +/-)
< BlueMatt>
luke-jr: no (I mean theoretically we can *give* others access, but I see no reason to do so at this time)
< achow101>
I think we should sign with a differnt release key and see if anyone notices :)
< meshcollider>
yeah wumpus should do it, its not that time critical
< jonasschnelli>
achow101: they will...
< * luke-jr>
shrugs
< wumpus>
oh people notice, my mail was full after stupping using my personal key for that
< wumpus>
even though it was announced on the mailing lists etc
< wumpus>
anyhow... let's release 0.15.0 soon if possible
< wumpus>
if there's still something you'd like to debug over the weekend I can wait until after that
< BlueMatt>
anyway, so I have no objections to tagging 0.15 this weekend/tomorrow if no other issues come up in that time
< wumpus>
but hearing that cfields's concerns were with local code puts me at ease a bit about 0.15.0 at least :)
< luke-jr>
the lock issue can probably be ignored, I guess?
< wumpus>
luke-jr: not ignored, just fixed on master and next 0.15 branch release
< luke-jr>
sure, that's what I mean. ignored for 0.15.0
< luke-jr>
would be nice to get a test that reproduces it, but I'm not sure how
< wumpus>
well as I understand it there are no threads in flight yet at that point
< gmaxwell>
we understand it, it shouldn't block the release. (it's in init)
< cfields>
wumpus: if that puts you at ease, i should invent some local problems and solve them more often :)
< meshcollider>
😂
< BlueMatt>
lol
< wumpus>
cfields: haha!
< wumpus>
can cfields create an issue even cfields cannot debug
< cfields>
haha
< wumpus>
ok, any other topics?
< BlueMatt>
Segwit is active!
< sipa>
what? how?!
< wumpus>
yeah! we can remove the point 'activate segwit' from the weekly agenda
< BlueMatt>
heh
< kanzure>
i have a topic. it can wait.
< luke-jr>
what is it waiting for?
< wumpus>
ther's only 15 minutes left so don't wait too long
< kanzure>
i am collecting topic ideas for coredev.tech meetup. either pm me with things you want to see or in here.
< kanzure>
and i will publish small document somewhere
< wumpus>
segwit wallet support
< kanzure>
it's not a schedule. just a braindump sorta.
< kanzure>
oh.
< kanzure>
okay added
< wumpus>
(which is very general, probably should be divided up...)
< jonasschnelli>
kanzure: thanks for collecting stuff for the CoreDev.tech in SF!
< kanzure>
sure
< jnewbery>
wumpus: I'd like #11053 to be reopened / reconsidered
< wumpus>
GUI support, bech32 yes/no for this release, etc
< jnewbery>
refactor: Make all #includes relative to project root
< cfields>
jnewbery: +1. I think my concerns there were mistaken for a NACK. I pretty much regret bringing them up.
< luke-jr>
wumpus: eh, at least sending bech32 seems a necessity
< wumpus>
cfields: no, not really, it just made me reconsider whether it's really a good idea
< wumpus>
cfields: ideally we'd clear out src and move everything to subdirs
< cfields>
wumpus: agree with that.
< wumpus>
cfields: on the other hand this doesn't make things worse at least
< wumpus>
cfields: #include "" is severly misused in our source code
< wumpus>
if used it should be used to include relative to the directory of the source file, but it's used as a kind of wildcard include
< wumpus>
*find it somewhere!*
< bitcoin-git>
[bitcoin] laanwj reopened pull request #11053: refactor: Make all #includes relative to project root (master...2017_08_includes_absolute) https://github.com/bitcoin/bitcoin/pull/11053
< gmaxwell>
12:44:01 < wumpus> yeah! we can remove the point 'activate segwit' from the weekly agenda
< wumpus>
anyhow reopening the pull, it wasn't my intent to stop discussion on it, just to make it clear there's no urgency in mergin a certain solution
< gmaxwell>
we should also go dig up the wallet improvements we couldn't make without segwit... and start working on them. :)
< instagibbs>
wumpus, oh thought you just implemented those changes just now, haha
< jnewbery>
thanks wumpus
< gmaxwell>
(perhaps see the log of the meeting where we added that item)
< cfields>
wumpus: agreed, i just don't see a ton of benefit in changing something that works
< cfields>
however, if benches show that it's noticibly faster, i'm all for it :)
< wumpus>
cfields: well the current state prevents having files with the same name in multiple places in the tree sanely
< sipa>
cfields: eh, what are you talking about?
< gmaxwell>
how are include paths going to change the speed of anything?
< wumpus>
cfields: that's what kind of triggered this, we can't have wallet/init.h and init.h right now because #include "" as we use it now gets confused
< wumpus>
gmaxwell: it can reduce the amount of probing the compiler has to do
< meshcollider>
(compile speed not runtime speed)
< cfields>
wumpus made that point ^^ in the PR, i believe
< gmaxwell>
okay, I'll look at the PR then.
< wumpus>
gmaxwell: e.g. including "primitives/primitives.h" in "primitvies/transaction.h" makes it look in primitives/primitives/transaction.h
< wumpus>
gmaxwell: which can affect compile speed in some setups
< wumpus>
gmaxwell: it's not the primary reason for making the change, though
< BlueMatt>
ok, other topics?
< cfields>
anyway, i'm happy to see it reopened
< luke-jr>
how about meeting plans?
< luke-jr>
or should we just discuss that in the dedicated channel later?
< sipa>
yes
< wumpus>
I thin kthe best reason for it is that it makes it immediately clear where to look for a file to developers
< luke-jr>
(specifically, are people sticking around Friday, or leaving as soon as it's over?)
< BlueMatt>
luke-jr: dedicated channel, i think
< wumpus>
instead of eh, it's included with "", is it in the current dir? no? oh maybe it's under src/ directly
< meshcollider>
which channel is that?
< kanzure>
#bitcoin-core-dev
< jonasschnelli>
##
< kanzure>
i didn't say what you thought i said
< kanzure>
luke-jr: anyway there's enough people local to the area that you'll find friendlies even if the visitors leave.
< meshcollider>
I guess there's some sort of filtering happening yae
< wumpus>
and that concludes the meeting, thanks everyone
< wumpus>
#endmeeting
< lightningbot>
Meeting ended Thu Aug 24 20:00:11 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< wumpus>
we can back to shouting at each other about default values now :)
< morcos>
well re: the shouting, i think there is time to argue on github about the changes for next release
< morcos>
what i'm more interested in is letting miners know what they need to do
< * BlueMatt>
thinks we should just merge all currently open prs for 0.15 and see what happens
< kanzure>
chaos branch?
< gmaxwell>
luke-jr: I'm concerned that you seem to be fixated on size rather than on the greater goal of keeping bitcoin decenteralized, secure, usable. Size is increasingly not an interesting magic metric for that, moreover, single miners reducing their size does nothing to help here-- if it did there would be much less reason for a blocksize limit.
< cfields>
morcos: let's forego some of the arguments and s/what they need to do/what their options are/ :)
< morcos>
well i think the only option that has a legitimate chance of getting approved is something that says
< kanzure>
gmaxwell: luke was arguing that his reasoning is directly related to IBD. perhaps IBD growth should be capped some other way in a more explicit way.
< morcos>
If you want to just produce the blocks with the most fee that you can without limiting anyting about them more than required by consenus then you should do X
< cfields>
morcos: yes, exactly.
< wumpus>
BlueMatt: luckily that's never going to work, most PRs conflict with a few other PRs :)
< luke-jr>
gmaxwell: there's only so much improvement that is possible. supposedly that's 17% per year, which gives us only room for 300-450k blocks. I don't think we have a magic leap to fix that..
< BlueMatt>
wumpus: merge at random until the only remaining ones conflict, then resolve conflicts at random until it compiles :p
< kanzure>
(would an IBD limiting value get to safely ignore signatures for the same reason that a signature validation skip mechanism was implemented for firstpass?)
< gmaxwell>
kanzure: the only thing that can save IBD is something like a utxo sync; any plausable blocksize results in a growth rate well in excess of computers/links getting faster, so its only a question of when things fail not if, absent. it.
< wumpus>
BlueMatt: it would be kind of nice if github had a way of showing what PRs don't conflict, or *shudders* what the biggest subset of PRs would be that could be merged without conflicting
< kanzure>
well that's an interesting argument.
< gmaxwell>
luke-jr: we do, stop syncing the whole history, and only sync the last year worth or whatever.
< morcos>
luke-jr: do you have an opinion on Core making such a statement via tweet or blog post or both? where X is set maxblockweight=4000000 and don't set maxblocksize
< luke-jr>
gmaxwell: UTXO sync is a change to the security model, and you were just going after praxeology1 for wanting to work on it earlier unless I totally misunderstood that?
< kanzure>
gmaxwell: ok then what about argument about limiting IBD until utxo commitment sync stuff
< gmaxwell>
luke-jr: and as I mentioned in the meeting for many (perhaps most) IO costs in maintaining the chainstate dominate sync time, so witness size increases don't matterh much.
< luke-jr>
morcos: yes, I think we shouldn't recommend things that harm Bitcoin.
< kanzure>
well i think IBD concerns weren't about validation cost, mostly bandwidth over the interwebs
< morcos>
i don't think my statement recommended anything
< kanzure>
sorry, i mean, those specific IBD concerns
< gmaxwell>
luke-jr: what is the security model of a bitcoin with no nodes? But with the assumevalid approach I don't know that it's really a meaningful change in the security model.
< luke-jr>
gmaxwell: we aren't only getting witness size increases with Segwit..
< morcos>
i think it was just informing miners of how to do something that most of us (probably including you) guess that a lot of them want to do, without commenting on its "goodness"
< sipa>
luke-jr: the worst case I/O effect of a block has not changed at all with SegWit
< cfields>
luke-jr: if bitcoin is as fickle as a default setting, the value of that setting isn't the issue...
< luke-jr>
morcos: that statement sounded like a recommendation. maybe a neutral blog post that explains how to get different policies configured would be fine
< sipa>
luke-jr: the average just got closer to the worst case, which is a good thing
< gmaxwell>
luke-jr: moreover the thing you're insisting on (maxsize) doesn't correlate with those IO costs well.
< luke-jr>
cfields: all the more reason we should make the default something advisable (or nothing at all)
< kanzure>
surely it was bandwidth not IO concern?
< jtimon>
luke-jr: if we delete the maxsize and set de default weight to 2MB, do you think many miners won't change it to 4MB ?
< gmaxwell>
luke-jr: if you want to argue that maxsize=4m maxweight=3m .. at least that makes more sense (though I would disagree there too)
< morcos>
luke-jr: at some point we have to consider what our customers (miners in this case) want. i suppose we could just get jeff to tweet it, since they claim to be running his software anyway.
< gmaxwell>
kanzure: read faster. For a lot of systems they are _not_ bandwidth limited in IBD.
< kanzure>
but is that luke's concern?
< gmaxwell>
kanzure: they're chainstate IO limited, etc.
< luke-jr>
morcos: they can configure the settings for what they want; it doesn't mean we need to suggest what those should be, if it's against what others might want
< gmaxwell>
It's a fact that setting in any way other than 4m vs 4m will be leaving money on the floor, potentially quite a lot.
< luke-jr>
morcos: I think documenting how to use the settings is good. I don't think suggesting settings harmful to the network, over more benevolent settings, is good.
< luke-jr>
gmaxwell: only if other miners set that
< luke-jr>
no miners right now AFAIK set 4/4
< gmaxwell>
luke-jr: no, you'll still miss out on what you could have earned even if no one else sets it.
< morcos>
ok in the meantime i'm just going to tweet it myself in the event that some miners don't know, i think many would like to know what to do
< gmaxwell>
which is why expecting people to set it some other way is unrealistic and has simply failed in the past.
< luke-jr>
gmaxwell: you have a probability to get it next block of yours
< luke-jr>
gmaxwell: do you disagree with #3229 ?
< luke-jr>
(it was originally shot down by Hearn and Garzik)
< gmaxwell>
I think it's phenominally dumb but better than the current behavior. (not really much of an endorsement)
< luke-jr>
sigh
< luke-jr>
either miners are setting the config or not. if they are, then defaults don't matter, and we should either set what would be ideal, or require configuration. if they aren't, then the same conclusions make sense
< gmaxwell>
I don't think these things should be configurable at all.
< gmaxwell>
it's harmful to the network to have discrepencies.
< gmaxwell>
Do we have a setting to set a maximum number of 1-bits in a seralized block. No.
< gmaxwell>
Would we support such a setting if someone asked for one, ... almost certantly not.
< gmaxwell>
discrepancies*
< sipa>
if you want to less insane example; we also don't have a setting to control how many sigops in a block (which correlates much more strongly with actual cost to the network)
< gmaxwell>
(I mean that for both weight and size, but especially size because it doesn't very map well to anything that matters and causes a lot of confusion)
< wumpus>
we wouldn't add it, but if we had such an option since the beginning, then removing it would still be difficult
< luke-jr>
size does matter though
< sipa>
luke-jr: yes, a bit
< luke-jr>
at the very least, there is the clear case of the blocksonly node
< sipa>
so do many other things
< gmaxwell>
it's easy to demonstrate that the size option is seriously confusing people; they think it's controls the "legacy block size" and that if they set it over 1mb their blocks will be invalid. :( so much disinformation spread by people like garzik means that people are primed to have a screwed up understanding, and it's not like it's especially clear even without being setup to fail.
< gmaxwell>
luke-jr: what about the blocksonly node?
< luke-jr>
gmaxwell: its bandwidth requirement is primarily affected by the block sizes
< gmaxwell>
luke-jr: it matters a small bit yes, but other things matter a lot more.
< sipa>
luke-jr: let's work on compressed blocks then
< gmaxwell>
luke-jr: you can reduce its bandwidth requirement by almost 30%, in a way that isn't economically unstable and won't fail in a couple months, by helping finish the compacted seralization. But do you work on that? No. You throw rocks here and emotionally exhaust people who are working on things like that.
< luke-jr>
compressed blocks are going to make 100% year-over-year improvements? :/
< praxeology1>
True or false... 0.14.2 nodes by default currently limit the weight to 4M?
< sipa>
praxeology1: to 3M
< sipa>
praxeology1: for mining, that is
< sipa>
the consensus rules are obviously not configurable
< gmaxwell>
luke-jr: expecting to trick miners with a default that works against their interests isn't going to meaningfully keep the size down, all it's going to do is amplify segwit 2x drama.
< luke-jr>
gmaxwell: it's not a trick
< wumpus>
tip: you can see the default for every option with bitcoind --help
< praxeology1>
sipa: yea I am talking about consensus rules, not what miners create by default
< sipa>
praxeology1: that's entirely offtopic
< sipa>
and irrelevant here
< wumpus>
praxeology1: huh, then you shouldn't ask 'by default'
< luke-jr>
praxeology1: the consensus rule is max weight of 4M.
< gmaxwell>
wumpus: elsewhere he's suffering the "but if miners set things higher they'll produce invalid blocks" confusion.
< wumpus>
(unless you mean this in the extremely twisted sense of 'without source code changes'? )
< sipa>
praxeology1: the weight, which is defined as 3*witsize + basesize must be less than 4M
< sipa>
praxeology1: that implies that basesize must be less than 1M
< praxeology1>
I'm trying to find out what you guys are talking about... it wasn't clear to me that "maxweight" configuration doesn't effect the consensus rule
< sipa>
praxeology1: we're only talking about the mining code
< gmaxwell>
sipa: don't use witsize as your variable. sounds like the size of the witnesses.
< sipa>
-blockmaxsize and -blockmaxweight control the size of blocks the mining code produce
< sipa>
but the mining code will never ever produce a consensus-invalid block, regardless of settings
< gmaxwell>
praxeology1: consensus rules are not configurable because that would be extremely negligant, incompetent, and pointless.
< gmaxwell>
negligent*
< * luke-jr>
notes miners could set blockmaxsize=8000000 safely
< praxeology1>
sipa: gmaxwell: ok, its clear to me know what you guys are talking about... thanks. Sorry for injecting myself into the conversation, now I see I don't even care about it (has nothing to do with consensus rules)
< praxeology1>
gmaxwell: given one was using rolling utxo hashes to verify the state of a utxo snapshot... what kind of data structure would be used to store old snapshots for new nodes to synch from?
< praxeology1>
maintaining such a data structure w/ no block height delay on the hash... sounds hard
< praxeology1>
Make the heights one can synch from periodic instead of each block. Store the spends in a new DB for that specific snapshot height. remember the block height for new utxos in the current chainstate
< praxeology1>
So then to reproduce the utxos at a given height... its the set of spent txs in that snapshot's DB, plus any utxo in the current chainstate from a block <= given height.
< sipa>
so make a snapshot, then process that snaoshot in the background into a compact utxo representation, and continue with normal procoessing
< sipa>
and yes, periodically
< praxeology1>
sounds good to me
< praxeology1>
then we just need some kind of standardization on how the "compact utxo representation" is hashed so that people can download from multiple sources piecewise
< sipa>
yes
< praxeology1>
Hopefully also some automated process for people to generate these utxo hashes, and communicate them over insecure networks
< praxeology1>
Or is there talk bout making it a commitment?
< praxeology1>
*trying to get a feel for the core dev's current position on this... thinking it might be a touchy subject
< sipa>
i think it's gettingna bit offtopic here
< praxeology1>
Well, whether the rolling utxo hash was committed on, or was communicated outside of blocks... would have a big impact on... how it was communicated
< bitcoin-git>
[bitcoin] MarcoFalke opened pull request #11129: [qa] util: Poll cookie file size before reading (master...Mf1708-qaPollCookie) https://github.com/bitcoin/bitcoin/pull/11129
< luke-jr>
it's too bad the UTXO set doesn't store the absolute output index in the blockchain
< sipa>
luke-jr: it's easy enough to add if that's needed
< luke-jr>
sipa: wouldn't it require a re-sync for all pruned nodes?
< luke-jr>
otoh, maybe there's some way to do a gradual upgrade..?
< meshcollider>
sipa: doesn't BIP 152 cover at least the design step? Or is it still being thought out
< sipa>
meshcollider: bip152 is completely unrelated
< sipa>
meshcollider: we're just talking about an efficient encoding for transactions
< sipa>
luke-jr: if you really mean absolute index, yes
< sipa>
hmm, even for relative index in the block
< luke-jr>
relative index isn't quite as useful
< luke-jr>
I was thinking have all the inputs just address by index
< meshcollider>
Oh am I confusing compressed blocks with compact blocks
< luke-jr>
I suppose relative index in the sense of "<this tx abs index> - <relative index> = <input's TXO abs index>" might be good
< luke-jr>
might make varint more optimal
< gmaxwell>
luke-jr: I can't figure out what you're talking about.
< luke-jr>
I suppose a per-block index might also be combinable with UTXO height, but I don't see a clear advantage to that
< luke-jr>
gmaxwell: replace the input txid+index with an absolute tx index in the blockchain
< gmaxwell>
how does that have anything to do with the utxo set?
< luke-jr>
you'd need to store/index by the absolute tx index to do the lookup
< luke-jr>
saying blockchain tx #1934 isn't useful unless the node can easily figure out what tx that is
< luke-jr>
utxo*
< gmaxwell>
I think what you're trying to suggest is this: make a transfered block smaller by rekeying its inputs with indexes.
< sipa>
luke-jr: i think needing a UTXO set in order to just decompress a block is already a no-go
< gmaxwell>
I think all you need is an index to txid table, seperate from the utxo set. Which you can build on the fly as you sync.
< sipa>
luke-jr: as that means an attacker can make you do arbitrarily much I/O before you can figure out PoW
< gmaxwell>
but as sipa is pointing out it would be a layering mess.
< sipa>
s/figure out/validate/
< luke-jr>
hm
< gmaxwell>
luke-jr: the compacted seralization I proposed (which sipa later refined) reduces tx sizes by around 28% without doing anything like that, working just one tx at a time... so it's useful for loose transction relay as well as blocks.
< gmaxwell>
yes, there is a lot more savings if you could replace txin hashes with indexes, but ... layering mess.
< luke-jr>
what if we add a commitment to the index-based format?
< gmaxwell>
yes, we could do something like have a compressed block, format where there was a tree-root in the software, assume valid style, no commitment needed.
< gmaxwell>
but: I think we'd better off spending effort on from utxo sync (needed to escape eventual doom); or on compact seralization (works also for loose transaction relay)
< meshcollider>
gmaxwell: is your serialisation format posted anywhere? Sorry I'm just catching up
< gmaxwell>
meshcollider: yes, but sipa's revised version is better and I don't think thats posted anywhere.