< Luke-Jr>
I don't remember a time where the contributors page didn't bug that way
< Luke-Jr>
gmaxwell: it was, but with a different user
< Luke-Jr>
(disclaimer: I did not view that exact page before)
< gmaxwell>
perhaps we should change the merge script to refuse to merge commits that don't have dots in their email addresses.
< Luke-Jr>
do we have any recent like that?
< gmaxwell>
Luke-Jr: it was messed up like that from whatever moment saracen added sirius-m@1a98c847-1fd6-4fd8-948a-caf3550aa51b to their email list.
< gmaxwell>
Thus my latest example, I just went and reproduced using sirius-m@1a98c847-1fd6-4fd8-948a-caf3550aa51b
< Luke-Jr>
gmaxwell: sure, but that was a long time ago
< gmaxwell>
oops I ment s_nakamoto@1a98c847-1fd6-4fd8-948a-caf3550aa51b in the penultimate case.
< Luke-Jr>
I noticed it at least as early as when we were contacting devs to sign that joint letter
< gmaxwell>
in any case, I went and reserved all the other dotless names in the history. .. looks like it only lets a single github user claim them, first come first serve.
< Luke-Jr>
pretty sure I had seen it before then, though
< Luke-Jr>
gmaxwell: ☹ now GitHub shows bogus stats for you
< Luke-Jr>
at least ignoring saracen was easy
< gmaxwell>
Luke-Jr: nah, it shows up as seperate users.
< midnightmagic>
Luke-Jr: just trying to be complete. Is that a duplicate?
< Luke-Jr>
midnightmagic: no, just stale :p
< midnightmagic>
:-)
< midnightmagic>
lot of tags I've been remiss on..
< wumpus>
at this point I would not recommend building any 0.10 versions < 0.10.3 and 0.11 versions < 0.11.1
< wumpus>
__unless you somehow swap the miniupnpc version, but in that case it wouldn't match so what is the point
< wumpus>
may make sense to do a 0.9.6 with just miniupnpc upgraded, thoug most likely anyone still on 0.9.x will be running a self-built heavily patched version anyway, probably without upnp
< wumpus>
but first priority is getting 0.11.1 (and possibly 0.10.3) to people
< GitHub172>
[bitcoin] luke-jr opened pull request #6825: [WIP] Backport bugfixes to 0.11 (2015-10-14 / a1d623d) (0.11...backport-bugfixes-to-0.11-20151014) https://github.com/bitcoin/bitcoin/pull/6825
< midnightmagic>
wumpus: I build only for end-binary verification and triangulation. All resulting artifacts are discarded. This is for gitian signature completion only. I explicitly build older builds to help triangulate continuing reproducibility.
< wumpus>
github should be automatically reporting that, weird
< wumpus>
(at least, pushes to master on the detached sigs repository should be reported)
< cfields>
wumpus: 0.10.3 didn't use the repo, still have to manually fetch it
< wumpus>
oh? I am, somehow, using the automatic fetching with 0.10.3
< * wumpus>
must be doing something wrong, let me check
< * cfields>
checks the releases rc2 dmg
< wumpus>
... ugh
< cfields>
wumpus: yikes. mismatch on rc2 :\
< wumpus>
it's interesting that no one reported the invalid signature, the only conclusion from that is that no one tried the osx dmg for 0.10.3rc2 - which I guess makes sense, 0.10.x is bound to have very few users
< cfields>
yea
< cfields>
also points out that you were correct to suggest writing the checksum of the unsigned file into the sig payload a whlie back
< wumpus>
yes
< wumpus>
attaching the wrong signature should ideally fail :)
< cfields>
yea
< cfields>
for windows it does, that actually saved me on rc2
< cfields>
i'll make it simulate that for osx via checksums. adding now.
< kanzure>
"Presumably, the server closed the connection before. sending a valid response." (http BadStatusLine error) when calling getinfo. any hints on replicating this? i am only seeing this intermittently on some remote testing server (in the middle of a bunch of other testing), so not sure how to isolate this particular problem.
< wumpus>
kanzure: that tends to happen when the http connection times out server-side, python is pretty bad at handling that
< wumpus>
kanzure: see commit ddf98d1
< kanzure>
why would getinfo timeout so quickly?
< wumpus>
the problem as I understand is it not that the call that times out, but the http connection you're sending it on may have timed out *before* you're even able to send it
< wumpus>
but this only applies to master, which has http timeouts
< wumpus>
there may well be a different issue, but it may help troubleshooting
< kanzure>
wumpus: for that to be the source of my problem, the connection must have been opened long before i made the getinfo call?
< wumpus>
yes - at least "-rpcservertimeout" seconds, and you must be using master
< kanzure>
hmm maybe i'll go try -rpcservertimeout=0
< kanzure>
yes this is on master (ish) (a week old maybe)
< morcos>
cfields: any chance you want to think about nLastBlockFile for a sec?
< cfields>
morcos: sure. give me 5 min to wrap something up?
< morcos>
sure
< cfields>
morcos: what's up?
< cfields>
wow, that took 30. felt like 5. sorry bout that :\
< morcos>
cfields: ok so i've been trying to make sure there isn't an issue with pruning accidentally deleting the wrong block files
< morcos>
and there is a small issue in that during a reindex, the state of vinfoBlockFiles are still being updated, so if you submitblock you could munge a bunch of stuff or cause pruning and delete even worse stuff
< morcos>
so that needs to be fixed
< morcos>
but in the course of trying to trace through all this stuff, nLastBlockFile is kind of confusing
< morcos>
what is is really supposed to represent
< morcos>
where you're going to write the next block to?
< morcos>
why when you do a reindex does it follow along as you process blocks from your block files and possibly end up not in your latest block file?
< morcos>
wouldn't it make sense if it was really your last block file?
< cfields>
hmm, let me take a look. as i remembered, it meant "the file the next write should go to", but i don't remember the reorg specifics
< morcos>
ok, in particular i'm wondering whether it might make sense for line 2506 to read nLastBlockFile = max(nFile, nLastBlockFile)
< morcos>
i just dont understand why it ever needs to go backwards
< morcos>
it turns out the belt and suspenders searching for additional contiguous vInfoBlockFiles in LoadBlockIndexDB is actually required, because after a reindex, nLastBlockFile can end up somewhere earlier.
< morcos>
interestingly pruning can cause gaps in blockfiles, but because you can't reindex after pruning, and pruning only searches up to nLastBlockFile, you can't end up with gaps after the position of nLastBlockFile
< cfields>
hmm yes, without max() that looks very strange
< morcos>
i mean it works, and you might actually end up writing new blocks in the spare space inside earlier blockfiles if they are small blocks
< cfields>
still reading and re-remembering
< morcos>
but its just kind of confusing
< cfields>
right
< cfields>
ah yes, that was the logic as i remembered it. It points to the first place where there's space to write a block, but not necessarily the last one
< morcos>
well after a reindex, it points to the last block that was processed. irrelevant as to where there is space, but then when you go to write you move forward from there until you find space to write
< cfields>
still looking
< michagogo>
wumpus: I'd have caught the rc2 dmg thing if I were at my computer
< michagogo>
But that unfortunately hasn't happened much at all lately
< michagogo>
I've been doing something new these days (starting a couple months ago), 9-5, with a 2-3.5 hour commute each way :-/
< michagogo>
Doesn't leave me much leisure time.
< cfields>
morcos: ok, i believe i remember the logic now
< morcos>
cfields: sorry to make you page that all in, but i took so long tracing it out this morning, and i wanted to document or something for the future, so just making sure i have it right
< cfields>
morcos: fKnown is only (as far as i can see?) true in the case of a reindex. Imagine the typical reindex case. something has reported corruption, so you're scanning all blocks on disk blindly. There's a good chance that some are valid and some aren't. If you get through 100 blockfiles, and the last valid block was in file 50, it makes sense to resume writing at 50 rather than at 100.
< morcos>
cfields: hmm.. i see
< cfields>
so (as i read it) it can't actually go backwards, only start from the point where the reindex finished
< morcos>
but in the case you point out, there still coudl be valid blocks in say file 51
< morcos>
that have height less than the block in file 50
< morcos>
you'll still do the right thing, and i agree you can now reclaim all the space between 50-100 that doesn't contain valid blocks
< morcos>
so if we changed that line to max, wouldn't you still get the fast majority of that benefity
< morcos>
vast
< cfields>
morcos: in the case where there's a valid block in 51, i believe the intention is to move it to 50 (since presumably the end of 50 is corrupt so it's been marked for writing), then set nLastBlockFile to 50. At that point, the block in 51 is a dupe and can be overwritten
< morcos>
move?
< morcos>
what happens now is it stays in 51, and nLastBlockFile stays at 50
< morcos>
even if we shutdown and restart (without reindex) thats the case
< morcos>
luckily this extra searchign for vinfoBlockFiles in the database finds the information for 51, so you know about it
< cfields>
mm nm, that's not right
< wumpus>
michagogo: no problem - blame the (unavoidable) hurry around this release
< cfields>
morcos: i thought i remembered the index pos being updated, but that's not the case. looks like you're right..
< cfields>
morcos: so yes, after a reindex, it'll point to the file with the last valid block, even if there are completely invalid files inbetween
< cfields>
morcos: for the sake of clarity, i think it'd suffice to stick the "nLastBlockFile = nFile" in the if(fKnown) case?
< morcos>
did you mean !fKnown
< morcos>
that was my first inclination, but then it never gets updated if you do a reindex, hence my suggestion for max
< morcos>
instead of it pointing at "the file with the last processed valid block", i think it should point at "the last file with processed valid block"
< cfields>
no, if(fKnown). that's the reindex case.
< morcos>
but reality i don't care so much about changing it as making all the logic clearer, which is confusing in my opinion, not sure how to clarify
< cfields>
ugh, wait
< morcos>
but in the !fKnown case it defintiely needs to happen, because nFile might get incremented if we move to a new file
< cfields>
of course, brainfart
< cfields>
morcos: ok. yes, i agree with max()
< morcos>
ok, i'll put it out there in a pull, i think its easier to reason about that way. if people don't like it then i'll just try to comment the existing logic
< morcos>
btw, when you fixed the calculation of the size in vinfoBlockFiles, we had thought that was from garabage data, but it turns out that another way that can happen is if you have unconnected blocks
< morcos>
if you ever get a 2 block reorg announced to you that you don't accept, you'll have headers for both blocks in the fork and the block for only the tip. if you then reindex, that block will not be processed, b/c it has an unknown parent, so its the same thing as having garbage data in your block file
< cfields>
morcos: the part i'm not sure about is whether blocks are always processed 100% in order when reindexing. obviously it tries, but when it stores out-of-order children, i'm not sure that it plays them back in order
< morcos>
it doesn't update the state of the vinfoBlockfiles but its taking up room.
< cfields>
if the order is completely correct, then i think the current code should work ok without max()?
< morcos>
the current code does work, its just confusing
< cfields>
morcos: aha, that makes sense
< morcos>
i don't like the idea that nLastBlockFile can be smaller than your vinfoBlockFile.size()
< morcos>
seems fragile or something
< cfields>
morcos: maybe just delete the garbage files after reindex then?
< morcos>
but what i'm saying is they are not necessarily garbage
< morcos>
blockfile 51 has vaild blocks part of chainactive in it. but chainactive.tip is in block file 50. then nLastBlockFile = 50
< morcos>
after a reindex
< morcos>
i would prefer nLastBlockFile to = 51 in that case
< morcos>
cfields: lets not waste any more time on it i'd say. i don't feel strongly about making the change, i just want to comment this so future me and yous don't have to spend so long trying to figure out what the idea is
< cfields>
morcos: see above. reindexing attempts to process in-order, so tip should always be in 51. is that not the case?
< cfields>
morcos: sure. but now after discussing, i'm left scratching my head about how things look after a reindex
< morcos>
cfields: reindex reads the block files in order, but doesn't call ProcessNewBlock until it can conenct them. It's PNB through AcceptBlock which calls FindBlockPos and updates nLastBlockFile. so the last block to have ProcessNewBlock called on it will be the the one whose file nLastBlockFile points to
< morcos>
since blocks can be stored on disk out of order, that block can be back in block file 50.
< morcos>
i hacked up a test to demonstrate this, because i thought there was a bug in that you wouldn't know about the blocks that were in files 51 and higher
< morcos>
but turns out in LoadBlockIndexDB, you keep searching for vinfoBlockFiles beyond nLastBlockFile so you do know about them, but nLastBlockFile still points at 50
< cfields>
morcos: but because PNB isn't called until they can be connected, isn't the tip the last to be processed?
< cfields>
or maybe the out-of-order ones themselves aren't played back in-order ?
< morcos>
yes, the tip is the last to be processed, but this is reindex, and so if the tip is stored in block file 50, then your are passing in that block pos to FindBlockPos and we're in the fKnown case and its using the file where the tips is stored to set nLastBlockFile
< michagogo>
Great. My script should (I think) automatically pick up on that, and pr the sigs including the signed.
< michagogo>
And if I didn't mess up the && chain of commands, should even do 0.10.3 too
< michagogo>
(Should almost definitely get the win, Linux, and OS X unsigned -- the question is the signing)
< michagogo>
It would be nice to have this channel subscribed to the gitian.sigs repo the same way it is to bitcoin
< michagogo>
(Also, idk if I've ever seen a -n channel before)
< GitHub90>
[bitcoin] Michagogo opened pull request #6830: Add historical release notes for October 2015 bugfix releases (master...add-release-notes) https://github.com/bitcoin/bitcoin/pull/6830
< michagogo>
wumpus: sigs PR'd
< GitHub66>
[bitcoin] Michagogo opened pull request #6831: Bring historical release notes up to date (0.10...0.10-add-release-notes) https://github.com/bitcoin/bitcoin/pull/6831
< GitHub36>
[bitcoin] Michagogo opened pull request #6832: Bring historical release notes up to date (0.11...0.11-add-release-notes) https://github.com/bitcoin/bitcoin/pull/6832
< * michagogo>
preps the Reddit post
< * michagogo>
checks if there's a release PR for bitcoin.org