< BlueMatt>
lol, did a chainsync with infinite dbcache and, on shutdown, bitcoind did a 1.5G malloc
< luke-jr>
>_<
< BlueMatt>
wait...wtf
< BlueMatt>
and on startup, with a tiny dbcache...leveldb did a 2.1G alloc
< BlueMatt>
that cant be right
< BlueMatt>
yea, it is
< BlueMatt>
heh, the first time you restart after that it does insane things
< GitHub82>
[bitcoin] catilac opened pull request #8004: signal handling: fReopenDebugLog and fRequestShutdown should be type sig_atomic_t (master...fix_signal_handler) https://github.com/bitcoin/bitcoin/pull/8004
< cjcj>
What is the git process for including a specific PR (#6853) into the v0.11.2 branch?
< Guest7895>
cjcj: it gets backporter if it's considered critical
< Guest7895>
v0.11.2 is a release, not a branch, by the way
< Guest7895>
6853 is not a bugfix, and certainly not a critical one
< btcdrak>
cjcj: all changes should be made to master, and then they can be backported as required. You backport to the specific branch 0.12, and 0.11
< btcdrak>
cjcj: normally you just mark it as "requires backport", and the maintainers will cherry-pick backport it after merge, but if it has lots of merge conflicts with the backport branch then you may need to open a PR for it (but wait until master merge first).
< cjcj>
btcdrak: I never intended to PR this. It would be for my own use only. The 0.12 branch doesn't work for some reason for my case, and #6853 is the only PR from 0.12 I really need. Was just wondering of a painless way to include it, but I think I will just build from CodeSharks fNoRetarget branch.
< Guest7895>
cjcj: well you can git cherry-pick
< Guest7895>
though i'm interested why 0.12 does not work for you... perhaps that's an indication of a bigger problem
< btcdrak>
what does "does not work for me" mean specifically?
< cjcj>
Guest7895: Will check out that command. Am not very familiar with git yet. I don't know yet why 0.12, but I bet the problem is on my end. For now 0.11.2 just works so I will continue from there for now.
< cjcj>
btcdrak: The program crashes when I make an RPC request in 0.12 after running for a while, but it works fine in 0.11.
< sipa>
perhaps you should open an issue for that?
< sipa>
or is this with heavy local modifications?
< cjcj>
sipa: I think the problem is on my end, but I will debug the issue further when I got time and open an issue if it doesn't resolve.
< cjcj>
I'm using python-bitcoinlib as well, so maybe the issue could lie there.
< cjcj>
Are there heavy differences in the RPC code between 0.11 and 0.12?
< sipa>
you'll need to give a bit more information about what you're doing
< sipa>
some things changed a lot, others not at all
< wumpus>
yes, you need to be more specific about what is not working, 'it doesn't work' is not a good bug report
< wumpus>
what are you doing, what error do you get back
< wumpus>
does bitcoind crash ,if so can you provide a traceback
< wumpus>
etc
< wumpus>
if you just want to cherry-pick one commit, use the git cherry-pick command, it may be easy, it may also be very hard if the surrounding code changed a lot between 0.11 and 0.12 (large chance)
< cjcj>
bitcoind doesn't crash, only the python script I'm running. Will provide a traceback once I have removed some personal info from it.
< sipa>
ah, ok :)
< GitHub26>
[bitcoin] avar closed pull request #8003: Get rid of a compiler warning due to #if 0'd test (master...fix-unused-function-compiler-warning) https://github.com/bitcoin/bitcoin/pull/8003
< GitHub154>
[bitcoin] avar opened pull request #8005: Add a comment indicating that the btc devs don't want a warning fixed (master...note-that-unused-function-compiler-warning-should-not-be-fixed) https://github.com/bitcoin/bitcoin/pull/8005
< gmaxwell>
wumpus: whats the goal in the rounds of merging right now? just cleaning out backlog?
< wumpus>
just moving forward
< gmaxwell>
Good.
< gmaxwell>
#7934 seems good to me I've had it running since my utACK without issue.
< fanquake>
wumpus issues that could be reviewed/closed by inactivity 6835 6355
< wumpus>
yes #6835 won't be merged anyway - it at least used to be kept up to date by people that cared about it, but it can just as well be a separate branch on someone's repository without being a pull request
< fanquake>
Also #7149
< wumpus>
not sure about #6355, seems it just received very little review and testing
< wumpus>
generally I don't close pulls for inactivty, only issues, if the OP doesn't respond to requests for more data. In this case the author can't help that his PR received so little attention
< wumpus>
bah #7149 has a lot of changes for a 'bugfix'
< fanquake>
Seems that #7814 fails on osx when you run the extended test suite
< MarcoFalk_>
fanquake, which test / exception?
< fanquake>
MarcoFalk_ will post to GH
< fanquake>
Looks like you've just missed signmessages.py
< MarcoFalk_>
When was this merged?
< MarcoFalk_>
Is probably a merge conflict
< MarcoFalk_>
You are running merge(master, pull) ?
< fanquake>
When did I merge it? 10 minutes ago
< fanquake>
Looking at the PR, you haven't touched signmessages.py at all
< MarcoFalk_>
Oh, actually you need to compile if you also want to run the signmessage.py test
< GitHub118>
[bitcoin] Tyler-Hardin opened pull request #8006: Qt: Add option to disable the system tray icon (master...disable-tray) https://github.com/bitcoin/bitcoin/pull/8006
< jonasschnelli>
Whoha! PR/Issue# >8000!
< wumpus>
it will take some getting used to 5-digit PRs/issues
< wumpus>
ACTION: (gmaxwell) try to extract some feedback e.g. from roasbeef to reimplemented, who might be aware of other limitations in the spec
< phantomcircuit>
im here
< sdaftuar>
hi
< cfields>
here
< gmaxwell>
wumpus: I've failed to do that so far, sorry.
< wumpus>
no rush I suppose
< wumpus>
any other topics?
< anchow101>
segwit versionbit
< nickler>
I've had a look at the btcd segwit PR, it includes around 5 tests
< wumpus>
#topic segwit versionbit
< anchow101>
The bip still says tbd for bit and date.
< * sipa>
randomly proposes bit (1 << 4)
< * instagibbs>
tries rng, gets 4
< wumpus>
if there's no special reason to pick a specific bit I'd suggest previous_bit+1
< btcdrak>
8 is lucky in China
< sdaftuar>
previous_bit + 1 makes sense to me...
< btcdrak>
wumpus: ack
< sipa>
so (1 << 1), also fine
< anchow101>
+1
< BlueMatt>
I'm with btcdrak
< wumpus>
otherwise it leaves holes, not a big deal, but dealing out consecutively may reduce the chance of accidentally duplicate assignments
< btcdrak>
are we ready to think about dates? even for testnet?
< jl2012>
i think we should set the testnet date now?
< gmaxwell>
sipa: whatever number you're proposing please post it to the mailing list.
< jl2012>
start 1 Apr 2016, end 1 Jan 2018?
< wumpus>
probably we should have some living document that keeps track of current bit assignments, outside the bips
< NicolasDorier>
for testnet do we need a date ? we did not for csv
< anchow101>
NicolasDorier, the dat for csv on testnet was March 1st
< anchow101>
*date
< NicolasDorier>
ok my bad
< btcdrak>
wumpus: maybe we can add a file bip-0009/assignments.md in the bips repository
< anchow101>
If the release can be out before June, what about June 1st for a mainnet start date? And May 1st for testnet?
< gmaxwell>
Dates should not be set until the software is known ready for release, and we are not currently there.
< gmaxwell>
There is no need to be over-eager.
< sipa>
i think we need to have a deployment active on testnet before even beginning to consider a start time on mainnet
< wumpus>
btcdrak: sounds good to me
< gmaxwell>
I think june first would be fine, but it could be set the day before, for all the system cares.
< wumpus>
#action add a file bip-0009/assignments.md in the bips repository to keep track of an overview of current bit assignments separate from their bips
< btcdrak>
jl2012: no need to have such a long expiry date for testnet.
< wumpus>
okay
< wumpus>
so do people agree on june 1?
< morcos>
for testnet?
< sipa>
for testnet?
< morcos>
i don't see why not make it earlier
< wumpus>
that's what the discussion is about right?
< morcos>
it kind of doesn't matter, just make it may 1st and it happens when it happens
< sipa>
indeed
< wumpus>
for mainnet it'd be kind of crazy to decide on an activation date now IMO
< btcdrak>
morcos: ack
< sipa>
we're not testing the deployment logic and teansitions
< gmaxwell>
morcos +1 for testnet.
< sipa>
may 1st for testnet sounds finr
< wumpus>
may 1st? more time travel? I've seen enough deloreans this week
< jonasschnelli>
hah
< morcos>
i might be obnoxious and start now... :)
< instagibbs>
morcos, hostile softforks incoming
< * sipa>
is going to disappear
< gmaxwell>
This date is not something that needs to be set _in advance_, and it also shouldn't be set without coordiating with other implementers (at least in principle)
< morcos>
wumpus: its what we did with csv, it just means you can starg signaling immediately
< wumpus>
okay, no decision on a date then
< wumpus>
#action discuss testnet activation date on bitcoin-dev mailing list
< morcos>
gmaxwell: i kind of disagree, i think that the code is mature enough that we should activate on testnet now
< gmaxwell>
morcos: I'm not talking about testnet.
< morcos>
gmaxwell: teh rest of us are :)
< wumpus>
we AREE talking about testnet
< gmaxwell>
Testnet is fine. do whatever with testnet. If it causes turbulance there, oh well.
< wumpus>
please don't confuse things
< gmaxwell>
wumpus: _YOU_ are talking about testnet jl2012 and anchow101 were not.
< gmaxwell>
I already +1 morcos for testnet.
< wumpus>
huh *confused*
< jl2012>
no, I'm talking about testnet
< phantomcircuit>
haha
< morcos>
ok so to summarize, email to bitcoin ML stating we are setting the testnet activation start date as may 1st because we believe at this point the activation start date is likely the only consensus change remaining with segwit
< gmaxwell>
Because it's testnet and the delayed start logic doesn't apply there, we don't care about creating turbulance there if miners upgrade ahead of nodes.
< wumpus>
makes sense
< morcos>
this will allow anyone to test their various versions of segwit (different implementations and backports) against each other potentially even before merging
< anchow101>
morcos: ack
< morcos>
gmaxwell: yes there is no reason to delay, but there is reason to agree on the start date so that we all activate at the same time
< gmaxwell>
morcos: yes, may first is fine.
< btcdrak>
ok so (1<<1) with activation may 1st for testnet, and (1<<1) and date TDB for mainnet
< jonasschnelli>
ack
< achow101>
yes
< morcos>
btcdrak: ack
< paveljanik>
ack
< morcos>
but what does TDB stand for? :)
< * btcdrak>
palms face
< gmaxwell>
Totally delicious burger.
< jl2012>
ack 1 May testnet, how about expiry date?
< cfields>
ack, but we need to get the gbt changes in place quickly so that testnet is a valid representation of what miners will be running
< btcdrak>
j2012: 1 year.
< morcos>
ack 1 year
< BlueMatt>
sgtm
< btcdrak>
(1<<1) with activation may 1st and expiry 1 year for testnet, and (1<<1) and dates TBD for mainnet
< BlueMatt>
phantomcircuit: when will we see testnet fork?
< morcos>
cfields: can you summarize what GBT changes are needed still?
< morcos>
does #7935 have anything at all to do with segwit?
< cfields>
morcos: there's a proposal to bip9 that would require that miners set a flag signaling awareness of segwit
< morcos>
ok i haven't read through all that but i kind of thought it was orthogonal to segwit. we already have versionbits SF's in the process of being activated. is segwit somehow materially different. if not, lets not confuse the issues
< gmaxwell>
morcos: it's just that a non-sw aware miner can't use GBT w/ segwit and keep mining while they can use CSV.
< sdaftuar>
gmaxwell: i don't follow why that is, can you explain?
< cfields>
morcos: assuming that's adopted, some miners won't be creating blocks with commitments, so i'd like to make sure that we're testing on testnet. Otherwise it's not a great representation of mainnet mining.
< gmaxwell>
I could be speaking out of my rear, my understanding at a distance was that non-SW ready gbt clients won't insert the commitment.
< sdaftuar>
but the commitment is created by bitcoind
< gmaxwell>
sdaftuar: classical GBT does not include a coinbase transaction, the client generates it using information from the template.
< morcos>
if can't use GBT means can't change the txs selected by bitcoind then maybe you're right, but that seems a secondary problem
< cfields>
sdaftuar: if a miner is too old to understand how to insert the commitment, bitcoind can provide only non-witness txs, so that the miner continues to produce valid blocks
< morcos>
maybe we should take this up after the meeting.
< gmaxwell>
sounds fine.
< cfields>
ok. i only mentioned because i'd like to start upstreaming the mining/pool patches if we're going to deploy on testnet. And can't do that until the gbt stuff is finalized
< cfields>
but fine to discuss later, i don't think it'll be an issue
< wumpus>
nickler mentioned btcd segwit PR tests, but I'm not sure that was a topic suggestion
< morcos>
cfields: i'd just like to distinguish between necessary changes and changes that are only needed if miners are going to be modifying the tx selection created by bitcoind. the second category in my mind should not stand in the critical path
< NicolasDorier>
I had problems with customers when mintxrelayfee where bump because occasionally wallet would produce bellow mintxrelayfee dust for other nods.
< wumpus>
#topic uneconomical outputs in wallet based on current fees
< nickler>
wumpus: nope I was referring to the action item that mentioned roasbeefs implementation
< BlueMatt>
also compact block bip, if anyone has bothered to read that
< wumpus>
nickler: okay :)
< cfields>
morcos: this has nothing to do with miners modifying tx output. it's that miners need to opt-in to segwit in order for bitcoind to give it witness tx. And that opt-in signal hasn't been implemented yet.
< gmaxwell>
BlueMatt: was that a topic suggestion?
< wumpus>
any opinions on the wallet issue mentioned by NicolasDorier?
< BlueMatt>
gmaxwell: yes
< gmaxwell>
NicolasDorier: I'll take a look at the issue.
< NicolasDorier>
Breadwallet had issue also because of that when the mintxrelayfee was bumped
< NicolasDorier>
so I think we should fix the wallet to not use mintxrelayfee
< NicolasDorier>
but estimatedfee for determining the dust (only wallet part)
< NicolasDorier>
would prevent having reliability issue in case it need to be increase in the future
< wumpus>
it sounds sensible, wallet and relay policy are different things, although the mintxrelayfee should probably be the floor
< gmaxwell>
or the dust threshould should just be made an infrequently changed fixed constant.
< NicolasDorier>
gmaxwell: I am talking only about wallet, not relay policy
< NicolasDorier>
ah
< NicolasDorier>
I get your point. But well the problem would be the same with a constant. If we get a spam attack, we would increase it
< NicolasDorier>
and then some wallet will produce below dust rejected by updated nodes
< morcos>
yes, we should do both things
< gmaxwell>
lets discuss on the issue.
< NicolasDorier>
ok
< NicolasDorier>
there is another quick topic I want to talk about
< morcos>
we should separate wallet functionality to use some smarter higher value for "dust" and the floor for dust shoudl be a separate variable than the muliple of min relay that it is now
< morcos>
(floor for dust = policy relay limit for dust)
< NicolasDorier>
ok, seems good I'll start working on it. It made me some pain nin the past
< gmaxwell>
morcos: I agree.
< NicolasDorier>
My other quick topic is
< NicolasDorier>
long time ago I made a PR to remove unused flag and code
< NicolasDorier>
morcos a jtimon had a better idea
< NicolasDorier>
instead of removing the flag
< NicolasDorier>
transforming it into one flag for all consensus stuff
< NicolasDorier>
I'm thinking working on it but
< NicolasDorier>
if I understand it seems to be better to do such kind of work after the merge of segwit ?
< gmaxwell>
NicolasDorier: usually if that question arises the answer is yes.
< wumpus>
yes I think for such non-trivial consensus refactoring it's better to wait until after segwit
< NicolasDorier>
ok so I'll keep it for later
< wumpus>
ok
< wumpus>
#topic compact block bip
< gmaxwell>
Next subject?
< gmaxwell>
I read it!
< BlueMatt>
you're the only one :'(
< sdaftuar>
not true...
< gmaxwell>
BlueMatt: would you like other people to read it?
< BlueMatt>
I would :p
< BlueMatt>
next topic?
< cfields>
heh, ack
< cfields>
(ack to reading)
< btcdrak>
^
< wumpus>
so there's nothing about the contents to be discussed?
< BlueMatt>
wumpus: not really...just hoping for feedback
< wumpus>
that's what I mean, no feedback
< btcdrak>
it's pretty dense reading, might need another week...
< BlueMatt>
wumpus: I think all the outstanding decisions were concluded between gmaxwell and I
< BlueMatt>
true
< gmaxwell>
I gave a fair amount of feedback to Matt and he updated prior to putting it up.
< BlueMatt>
so action to our army of devoted full-time code-reviwers? :p
< wumpus>
(haven't read it yet)
< morcos>
too much happening. we need to clone ourselves. at least wumpus
< gmaxwell>
BlueMatt: when will the PR be up?
< BlueMatt>
morcos: yea, that
< NicolasDorier>
will read it. It takes me more time than most of you to understand it, can't say anything meaningful about it after reading it for 10min :p
< gmaxwell>
BlueMatt: are you just waiting on feedback?
< BlueMatt>
gmaxwell: I could do that this week...
< NicolasDorier>
jonasschnelli: not this week figure out
< BlueMatt>
hey, we're early!
< NicolasDorier>
this is golden week :p
< wumpus>
yes it's an inconvenient time for japan
< morcos>
cfields: sorry if you guys have been going over all this GBT stuff already.. i tried looking through the code and BIP PR's but seems like there is a bunch of detailed GBT stuff in there that has pretty much nothing to do with how most miners use it as far as i can tell
< morcos>
is the problem that miners will replace the coinbase entirely, because the commitment won't actually change if the miners aren't doing witness txs right, so as long as they kept the output, i dont' think they should care
< cfields>
morcos: np. It's fresh on my mind because i looked at it yesterday/today, otherwise I'd be clueless
< sipa>
back
< jonasschnelli>
landed?
< cfields>
morcos: no, either way, miners will be using what bitcoind provides. We're not talking about modifications here
< morcos>
sipa: segwit activated on testnet while you were gone. pretty awesome huh
< cfields>
(er, "will be using what bitcoind provides" for the sake of this discussion)
< sipa>
jonasschnelli: yes, sfo
< morcos>
cfields: yes i understand that we have narrowed the discussion to that case (although since the PR's are not narrowed to that case, they are harder to get through)
< sdaftuar>
so to clarify: bitcoind will be including a coinbase tx with a valid commitment in response to gbt, right?
< sipa>
sdaftuar: if there is at least one witness transaction in the block, and there is no commitment already
< sipa>
(i think)
< sdaftuar>
sipa: agreed
< morcos>
what i'm asking is why in that case is it necessary for the miner to be segwit aware? changing the coinbase doesn't change the commitment, but i'm guessing the problem is that miners override all the coinbase outputs and so that commitment will be lost, and so bitcoind would have to do more work to add it back in and recalc the merkle for the header
< gmaxwell>
morcos: norma gbt right now has no coinbase transaction in it.
< sdaftuar>
^ is morcos' guess here the issue?
< gmaxwell>
normal*
< sdaftuar>
gmaxwell: ah. i am just now seeing where that coinbase gets stripped out of the response
< cfields>
right, what gmaxwell said. miners insert the coinbase. but old miners don't know to insert the extra txout.
< sdaftuar>
ok understood now. so the idea would be to add a mode where CreateNewBlock just doesn't pick witness transactions, which old miners could use?
< cfields>
if you only give them non-witness tx's, they'll continue to function fine. if they opt-in for the new serialization, no need to filter
< cfields>
sdaftuar: right.
< gmaxwell>
sdaftuar: right, and that mode would be default unless a special flag was sent.
< sdaftuar>
got it
< gmaxwell>
meaning that there is a guarenteed way to deploy with no mining infra updates.
< sdaftuar>
yep
< cfields>
but also a false sense of security while testing. so that's why i'd like to have the opt-in ready for miners to turn on asap
< morcos>
yes this makes sense to me now too. but INCREDIBLY frustrating. we have to rewrite the API for minings. it's absurd to have all this consensus critical logic outside of bitcoind by default.
< morcos>
sdaftuar points out that extra nonce makes that hard to fix
< morcos>
cfields: so is there a PR that does what you're suggesting?
< cfields>
morcos: afaik there's no implementation of the BIP yet.
< cfields>
luke-jr: have you coded something up, or should I jump on it?
< cfields>
(sorry, proposed BIP changes)
< cfields>
morcos: to be more specific: the bip changes are PR'd, but the specific segwit case isn't implemented yet afaik
< CodeShark>
yeah, already read through the scrollback. thx :)
< gmaxwell>
morcos: if only someone had suggested that any hardfork would mask out the constant zero bits in PREV in the header so miners could use it as nonce....
< instagibbs>
BlueMatt, I assume the XOR-adding scheme wraps around mod 2^64
< BlueMatt>
instagibbs: yes
< BlueMatt>
wait, no?
< BlueMatt>
wait, whats your question?
< gmaxwell>
he's asking if the addition is uint64_t addition, it is.
< instagibbs>
wasn't too confusing, but it wasn't stated in the bip
< instagibbs>
(I'm also prejudiced from previous conversations so better to ask here)
< gmaxwell>
I have a constructive proof that this scheme is not optimal but I don't think anyone cares.
< BlueMatt>
yea, I'm not convinced its worth caring, and optimal versions are much more expensive
< BlueMatt>
considering we're running that on so many txn, I'd prefer not.....
< gmaxwell>
In an optimal scheme, for any to txids A and B, there should be a salt input C that makes them collide. If there is no such C, then someone trying to create collisions could avoid the pair A,B, and thus increase their success rate. For this scheme, if A and B share the same even/oddness in each 64 bit word, then no C can make them collide. QED.
< gmaxwell>
Yes, I don't think it matters but it's useful to know that this exists.
< instagibbs>
just mentioning uint8 in the definition would be best, ill continue reading post family business
< roasbeef>
nickler: Yeah, the test coverage on my PR leaves much to be desired. Most of the lingering TODO’s are related to increasing test coverage across the various packages (txscript+blockchain especially).
< roasbeef>
nickler: Once I get the tests in, I’ll be restructuring the commits as they’ve started to sprawl a bit as I’ve fixed bugs, tweaked API’s, etc. I’ve been busy with other non-bitcoin stuff (this whole graduating thing), but hope to get the PR to it’s final form (insert Frieza meme ;) ) in the next week or two.
< nickler>
roasbeef: sounds great! My comment was not meant to be judging; the idea was to exchange tests between multiple implementations if possible. Let me know when you've worked on the PR again. Good luck with your graduating thing.
< warren>
To get rid of this patch what should we upstream? Matt's guess was to add an optional parameter to an RPC
< roasbeef>
nickler: No worries, I took no offense :). Great, I think there's a lot of value in exchanging tests across implementations. I plan to port over core's new json script, sighash and transaction tests to btcd. I'll contribute any additional cases I add upstream to core.