< cfields>
gitian builders: sigs for rc3 have been pushed
< cfields>
a few extra +1s from Windows testers would be great. rc3 switches us to a new cert using sha2, in order to comply with Microsoft's new rules this year
< gijensen>
I upgraded from 0.12RC2 to 0.12RC3, now bitcoind crashes seemingly at random during a reindex. Here's the crash error and debug.log tail: https://pastee.org/th29d any ideas?
< wumpus>
despite it probably being a false positive, the lock usage in the network code is pretty gross in places, cfields how's the net rework going? :)
< cfields>
heh
< cfields>
wumpus: i've been distracted by segwit for a few weeks, but i got back to cleaning it up yesterday
< wumpus>
cfields: ok, good to hear
< wumpus>
I have trouble parsing *which* locks are conflicting from that log message though. It's supposed to detect A->B B->A lock conflicts, but it looks like, except cs_main the two sequences have nothing in common?
< PRab>
cfields: With the new windows cert, the email address is unknown.
< PRab>
The old one was hello@bitcoinfoundation.org
< PRab>
I don't think its a problem, just different.
< wumpus>
doesn't seem like an issue, unless it prevents it from validating
< PRab>
Also, if the point of going to a new cert was to use sha2, I'm not sure if it worked. When I bring up the cert, I still see "COMODO SHA-1 Time Stamping Signer" as the Countersignature.
< PRab>
For good news, rc3 just passed my super simple smoke test. (It installed and launched without issue)
< wumpus>
ouch! But seems that's the name of the signer, maybe COMODO didn't change that. Is there a direct way to check the algorithm?
< cfields>
PRab: the rules are... complicated.
< PRab>
checking...
< cfields>
our new cert is sha256. From what i gather, for compat, best bet is still using an actual sha1 digest for the payload, and sha1 for the timestamp
< PRab>
gotcha. I'm just poking around the various View Certificate pages and see a pretty good mix of sha2 and sha1.
< wumpus>
so it uses sha256 only for the value to sign. Doesn't seem much safer if the cert still contains sha1 hashes, but okay, if it makes microsoft happy
< cfields>
PRab: yes. for the most part, sha1 is still fine "until pre-image attacks are possible". It's the cert's algo that matters.
< PRab>
For example, on the cert the "Signature hash algorithm" is sha256, but "Thumbprint algorithm" is sha1.
< cfields>
wumpus: heh, you'd think. the actual signed value can still be md5 even
< wumpus>
(may be easier to generate a collision on the cert than on the payload or timestamp)
< wumpus>
right
< PRab>
Gotcha. Well from my point of view, it installed without any warning on Win 7 64bit.
< cfields>
wumpus: i assumed the idea was that if collisions became possible, you can always re-sign yourself. But if your cert is vulnerable, you're screwed.
< cfields>
*re-sign with a stronger hash
< wumpus>
but, seems to me, the idea of win7 forcing the hash algo change is to preempt abuse by stopping to use an algorithm that may be compromised in the near future. If you need to resign you're probably too late, someone could have published corrupted executables with your name on it
< wumpus>
anyhow, the canonical way of checking if bitcoin core executables are untampered with is by checking the GPG signature (which uses sha256), or checking the sha256 hash against the gitian signatures. The windows signing is pretty much because windows requires it.
< wumpus>
GPG signature on SHA256SUMS.asc, that is
< cfields>
wumpus: i think a ton of the complication comes from the fact that there's an ugly overlap in what some OS versions requre vs. what they permit
< cfields>
ie i think winxp and maybe even win7 will flag if you use sha256 across the board
< cfields>
eventually i just settled with what _seemed_ to be the most compatible based on what i can find, since as you said, we only do it because it's required
< cfields>
but by all means if there's a better combination we could be using, we should do so
< PRab>
Honestly, the level of build reproducibility and verification that bitcoin core has is so far ahead of pretty much anything else that I don't worry about it too much.
< wumpus>
it's not completely theoretical, e.g. the Flame malware used a collided MD5 signature to pose as certified windows kernel driver. But compatibility is most important here, I agree.
< PRab>
I'm more worried that somebody could create a signature hash collision on a windows update and mitm a windows machine.
< wumpus>
(or did it use that to hijack windows update? I think that happened too, though I don't remember precisely. Anyhow, SHA1 isn't by far as broken yet as MD5)
< cfields>
wumpus: either way, I'll pr a change that adds a script for signing, same as how OSX is done. That way the paramaters are in git and not something the signer can tweak arbitrarily
< wumpus>
makes sense
< Luke-Jr>
cfields: did you manage to get two? how difficult is the process?
< michagogo>
cfields: one of them does 64-bit and then 32, the other is the opposite. That's mildly annoying when you're trying to gauge progress from tailing the log.
< michagogo>
5:51:57 <cfields> ie i think winxp and maybe even win7 will flag if you use sha256 across the board <-- Um, do we actually still support XP?
< wumpus>
we should add the "system requirement for buildling" back in the build-unix.md, it's gone a few days and we have someone complaining you can't compile with 512MB of memory https://github.com/bitcoin/bitcoin/issues/7471
< wumpus>
the corrent wording suggest that tweaking gcc will make it possible to compile with any amount of memory
< wumpus>
michagogo: we've never officially dropped support for winxp, so I suppose so. I wouldn't recommend using it but as long as it is easy to support too, why not. 70% of PCs on the world probably still runs it :p
< wumpus>
(while those tweaks are already needed to compile *with 1GB* of memory)
< Luke-Jr>
wumpus: Microsoft doesn't support XP AFAIK?
< Luke-Jr>
in an ideal world, unmaintained systems would be banned from the internet :P
< wumpus>
the BOFH in me tends to agree, but realizing that'd probably mean banning all poor countries from the internet I'm not sure that's an ideal world :P
< wumpus>
and although Microsoft did offer a free upgrade to windows 10, that doesn't apply to XP
< GitHub8>
[bitcoin] laanwj opened pull request #7472: rpc: Add WWW-Authenticate header to 401 response (master...2016_02_www_authenticate) https://github.com/bitcoin/bitcoin/pull/7472
< wumpus>
... also barring hardware which wouldn't have drivers for W10 anyway so the upgrade would be bricking
< wumpus>
in an ideal world, they would just use some free OS
< GitHub16>
bitcoin/0.12 996c27d Wladimir J. van der Laan: doc: add PR authors to release notes...
< cfields>
michagogo: ah
< cfields>
michagogo: switching them around would be reasonable then, sure
< cfields>
Luke-Jr: no, sorry. Didn't apply for new certs this time around, to avoid complications. Just replaced our old one with a new one without updating any info, in order to avoid the verification process
< Luke-Jr>
cfields: oh, so it'll be BCF cert still? O.o
< cfields>
Luke-Jr: yes. Otherwise, I was afraid it would trigger a verification of "Bitcoin Core", which of course doesn't exist as a corporate entity. That could've made the process drag on for a while, so I took the easy route this time and just fixed the sha256 issue, leaving all else the same
< Luke-Jr>
i c
< morcos>
gmaxwell: can i pick your brain about a couple of things? i'm trying to think of some longer term projects to work on and had some questions
< morcos>
#1 It appears 300M for a mempool is much more ample than I was guessing. I think its quite unlikely we'd need a disk based mempool anytime soon. It appears to be quite rare that we have more than few M's backed up with any fee
< morcos>
And the rest of the mempool is just shoved full of 1 sat/kB txs
< morcos>
Along those lines, I wonder if there is any cost to all these 1 sat/B (sorry B, not kB) txs bouncing around
< morcos>
we had made the calculation that the slow decay of rolling min rate combined with the limit on free relay would be enough to keep these low
< morcos>
but i'm not sure we fully though through all the system effects, espeically the fact that some communication happens trying to inv and send these same txs over and over
< morcos>
Is there anybody working on measuring all of this bandwidth utilaztion to know whether its worth worrying about?
< gmaxwell>
INV traffic on a node is already most of the bandwidth.
< morcos>
What do we think about adding ban points of some kind for bad P2P behavior?
< gmaxwell>
(at least when it has a full compliment of peers)
< morcos>
IS there any reason we don't send the fee rate with the inv for txs
< morcos>
?
< gmaxwell>
we do have the reject filter so that we'll not waste time rerequesting things.
< morcos>
The reject filter gets reset after a block though right?
< gmaxwell>
Just not in the protocol though it would be a good idea... or even better: being able to tell out peers not to even bother INVing transactions below some threshold towards us (because we'll not fetch them)
< gmaxwell>
since INVs are already so much of the bandwidth it's more important to suppress them than to suppress sending the transaction.
< morcos>
Also I was thinking about this idea of extending fee estimation for longer time horizons, but its not clear to me if the idea that there is usage that doesn't care about immediate confirmation is theoretical or actually exists
< gmaxwell>
(the reason for this is that the inv is 38 bytes, but you send or recieve it from _every_ peer, while the transaction is transfered less)
< morcos>
Yes, I think it could be reasonable to send a min threshold which expires after some time, and you coudl update it more frequently than that timeout
< morcos>
You could even assume that each of your peers minimums decays
< gmaxwell>
There are absolute use cases for non-immediate confirmation; but the UI and human factors (people worrying that it'll never go through) probably diminish them a lot.
< morcos>
This of course all ties into free/priority txs. I.e. Are we ok getting rid of FREE txs regardless of making a decision about priority.
< gmaxwell>
that question is probably 80% of why a fee filter wasn't done two years ago. Of course one could also say "send me things with priority over X" ... but then thats more functionality to specify.
< gmaxwell>
and implement.
< gmaxwell>
BBIAB.
< * gmaxwell>
back
< morcos>
gmaxwell: i think the simple version of just sending a feefilter threshold every X minutes and having it expire after Y minutes would work pretty well
< morcos>
and wouldn't really change the free tx use case
< morcos>
that is once your mempool is no longer limited and you're accepting free txs you'd just send a 0
< morcos>
free rate limiting i suppose still might trigger excess invs...
< phantomcircuit>
morcos, or you set the fee filter to match the mempool minimum fee amount
< morcos>
phantomcircuit: yes thats what i meant
< phantomcircuit>
that would actually significantly reduce bandwidth
< gmaxwell>
morcos: is there any reason for it to expire?
< gmaxwell>
(if you want it gone you can just send a filter of 0)
< morcos>
gmaxwell: well the way mempool limiting works is your effective min decays
< morcos>
so for instance imagine the case where there is a temporary huge fee boost and everyones mempool b/c limited to some relatively significant fee
< morcos>
your own acceptance decays
< morcos>
accentance threshold that is
< morcos>
so you'd want your peers to decay your filter for you or just periodically update them
< gmaxwell>
right, so after a block that cleared things back, you'd end up sending a new value. ... but sending a filter update message once per block isn't a ton of overhead.
< gmaxwell>
The amount of decay is highly dependant on local state and policy.
< morcos>
yeah i was thinking like phantomcircuit said you just watch your own threshold, and when it has deviated signficantly then you send an update
< morcos>
but yeah maybe the expiry isn't needed
< morcos>
just seemed safer
< gmaxwell>
My intuition about the expiration is that it's another periodic counter to manage, and you have some racing around the expiration. (meaning that e.g. banning peers that send you things you don't want might be less sane)
< gmaxwell>
One question I have: feerate or ancestor feerate?
< morcos>
it has to be the same as what governs admittance to your mempool
< gmaxwell>
I guess without a way to relay a group as a group that question isn't so useful.
< morcos>
which is feerate
< morcos>
if that changes then yeah we'd need a way to change
< gmaxwell>
well perhaps the way we'd change that is with a package relay. And then it's just something that offers a package, and the filter just applies to the feerate of the package.
< morcos>
right, i think that would work
< Tasoshi>
Sorry to interupt, but I suppose I must ask if I may. Our community is so terribly divided gmaxwell, is there a way to return to the cheerful days?
< morcos>
sending packages gets dangerous though, b/c of the asymmetry between ancestor and descendant branching. but thats a problem for the future. need CPFP mining first.
< Tasoshi>
The unthinkable is happening with r/bitcoin being censored/banned, that affects all for it breaches all principles
< Tasoshi>
hard forks are being considered
< Tasoshi>
these are difficult times for bitcoin
< morcos>
Tasoshi: Thats a good coverstion, but sometimes we need to step away from it for our sanity. This channel is not the place. This is where we try to get work done.
< Tasoshi>
it shouldnt be so
< Tasoshi>
morcos gmaxwell is not accesible anywhere as far as I am aware
< Tasoshi>
I understand of course it is difficult for him too
< morcos>
Not on this channel please
< Tasoshi>
but I thought to ask, if I may
< Tasoshi>
are we to burn the world?
< Tasoshi>
how can bitcoin operate when r/bitcoin is banned and censored, on what basis would anyone find this community welcoming at all
< Tasoshi>
I know you want to focus on the code
< Tasoshi>
there you are...
< Luke-Jr>
morcos: CPFP mining has been live on the network since 2012 (or 2011)