< luke-jr>
jonasschnelli: are you sure we can get a corrupt db from kill? that would be a pretty serious bug..
< luke-jr>
I mean, a non-trivial part of the *purpose* of a db engine is to prevent corruption
< luke-jr>
I know we have had some issues with literal power failures, but it seems absurd for kill to corrupt things
< gmaxwell>
we shouldn't be, it would be a serious bug. During IBD unclean poweroffs will pretty reliable corrupt things because we skip syncing, but otherwise it shouldn't.
< BlueMatt>
sipa: i think i got all your nits on #9014, though some of your comments (from 15 minutes ago) were on outdated code (how did you do that? I pushed many hours ago)
< sipa>
BlueMatt: i was reviewing commit by commit
< BlueMatt>
oh, you missed some SQUASHME commits :/
< sipa>
no, i just wasn't looking at them at the time
< BlueMatt>
sipa: when you get to a stopping point, i can just squash and force-push...jeremy was happy with where it was and i assume only you and jeremy have looked at it
< luke-jr>
inclined to ask him if he can provide remote access for debugging, but not sure there's a need if jonasschnelli has a way to reproduce something similar already
< gmaxwell>
interesting, slushpool just failed to take a pretty attractive CFPF transaction set that my node would have mined.
< jonasschnelli>
Luke-Jr: my tests showed my that sudden shutdowns (power off situation) will result in corrupt databases (=require for re-sync) during IBD
< luke-jr>
jonasschnelli: quite different from a kill
< jonasschnelli>
kill is a flexible term. :)
< jonasschnelli>
./kill is more specific
< jonasschnelli>
I ran into different corruptions on Linux/Debian 1) when running out of memory 2) random corruption on USB device, 3) on OSX when force shutdown a process (lldb), 4) Window 10 in VMWare sudden power off
< GitHub60>
bitcoin/0.13 a32d7c2 Cory Fields: release: bump required osx version to 10.8. Credit jonasschnelli....
< blur3d>
Hey Core. Just a thank you message for your all your work. Keep doing what you guys are doing. Ignore the outside pressures, and keep bitcoin from becoming centeralised. All of the fear mongering ignores the simple fact that the bitcoin network would be near impossible to replicate from scratch. Any transitional periods may have some discomfort, but compromising the network for short term bandaides is for fools.
< rabidus_>
i'll sign that message also.
< midnightmagic>
are there any plans to stuff some detached sigs into the repo for v0.13.1rc3..?
< btcdrak>
blur3d: thank you for saying.
< btcdrak>
midnightmagic: there are no plans for a binary for 0.13.1rc3
< btcdrak>
so there is no point building gitian sigs for rc3
< gmaxwell>
some people build sigs for rc3 to allow comparison, etc.
< gmaxwell>
I hope in the future we manage to make it so unmodified rc->final doesn't change the gitian sigs.
< jonasschnelli>
I guess midnightmagic is refering to code-signature detatched sigs (OSX/WIN) these would only be required if there would be binary releases (which we won't do for rc3 IMO)
< luke-jr>
not sure why rc3 was even tagged :p
< gmaxwell>
Causes more people to test.
< gmaxwell>
and update their tests.
< timothy>
gmaxwell: how? you still have to change the version
< gmaxwell>
timothy: the version in the software is already 0.13.1. There is only some build automation that inserts the git-tag.
< timothy>
oh ok
< timothy>
I tough you still have rc3 in version name
< gmaxwell>
timothy: but for tagged builds (e.g. releases) we could display the hash of the binary instead of setting the tag, or some hash of the source instead.
< gmaxwell>
and then the binary would really be identical.
< gmaxwell>
timothy: we try to change nothing. even really changing the tag implies some small risk.
< timothy>
I agree :)
< wumpus>
blur3d: thank you
< gmaxwell>
(as there are processor design flaws that are alignment sensitive; a different string size could change offsets in the build, and mean that a release could be produced which consistenty crashed on a small set of hardware that tested fine with the rc)
< gmaxwell>
"This adds some 4-byte NOPs to this IC stub on x86 if CPU family is 20 and model is 0-2. According to AMD engineers, limiting the number of branches per cache line might help, so I'm hopeful this will work."
< luke-jr>
gmaxwell: if we have such issues, I suspect we have bigger problems than changing the version number? :p
< wumpus>
gmaxwell: that sounds like GPU compiler design :)
< wumpus>
it doesn't work? add moar nops
< gmaxwell>
there are a bunch of these -- amd seems to be especially guilty. branch predictor bugs where under particular sets of instructions it'll just ignore a jmp.
< luke-jr>
:|
< gmaxwell>
and then mozilla puts out a new update that does nothing but change the PGO weights around a bit and then the crashes go away.
< wumpus>
oh yes I'm sure the abuse of branch prediction caches to subvert ASLR (locally) is just the beginning, if people are really going to look into those bugs deeply I'm sure some are exploitable
< gmaxwell>
then another update to change someting insignificant... and all those hosts are crashing again.
< wumpus>
(e.g. make it skip the jmp that rejects the authentication)
< gmaxwell>
hmmm
< luke-jr>
wumpus: would you entertain support for TQt btw? or is that something I'd need to maintain out of tree?
< gmaxwell>
Se, Greg ate d witness.
< gmaxwell>
I never knew I named that after myself.
< timothy>
gmaxwell: is arm better? :P
< wumpus>
luke-jr: TQT?
< luke-jr>
wumpus: Qt3 fork
< wumpus>
timothy: unfortunately, no, though the specific bugs are different
< wumpus>
(except for rowhammer, everyone loves rowhammer)
< luke-jr>
with stuff like thread support, and maintained by TDE
< wumpus>
luke-jr: why would you fork qt3? that seems ancient
< gmaxwell>
timothy: no. beyond actual silicon flaws it's hard to find ANY arm board that can handle being run full out without becoming unstable. None of the boards are built for actual usage.
< luke-jr>
wumpus: originally, because KDE 4 went downhill, and they wanted to maintain KDE 3
< gmaxwell>
Most reliable I've used has been the novena, though without a active fan on it, the libsecp256k1 tests will still make it hit a thermal emergency cutoff.
< timothy>
I tried to have a full node on a banana pi. it crashes often :P
< wumpus>
I've good experiences with cubox-i's, seems they got the cooling right
< wumpus>
imx6 is also supported in mainline linux + the graphics drivers were partially written by me :)
< wumpus>
32 bit, though
< gmaxwell>
imx family ends up in a lot of industrial control stuff too, probably less likely than armcores that are only used in smartphones to be buggy.
< wumpus>
yes, even in planes, you'd hope they take stabilty seriously :)
< luke-jr>
I'd kinda hope the planes don't rely on standard CPU stability
< wumpus>
not only, for any vehicle control system, there's always fallbacks
< tulip>
luke-jr: there's documentation about plane control out there, they seem to be nominally dual or triple redundant, even going so far as to have quorum between multiple devices. you can commonly get CPUs designed for doing medical control now which do lock stepped ARM cores and a comparator between them.
< luke-jr>
tulip: yeah, those would be the *non-*standard CPUs :p
< wumpus>
I'm more worried about cars in that regard
< gmaxwell>
luke-jr: lockstep cpus are a basically off the shelf part now, ti hercules, for example. It's inexpensive too, though not very fast.
< rabidus_>
e
< wumpus>
luke-jr: but re: qt3, I think it would be a shame to introduce that just now that everything is converging on qt5
< luke-jr>
Talos ships with TDE? :P
< luke-jr>
but yeah, I kindof agree
< * luke-jr>
kicks his IRC client that takes a full second to change channel tabs since compiling it against Qt5
< wumpus>
and ideally, focusing on a single version means that less effort has to go into compatiblity and more can to improving the experience for users
< wumpus>
why does Talos ship with TDE?
< luke-jr>
common lead guy
< wumpus>
right
< luke-jr>
☺
< GitHub119>
[bitcoin] laanwj opened pull request #9020: rpc: Remove invalid explanation from wallet fee message (master...2016_10_wallet_message) https://github.com/bitcoin/bitcoin/pull/9020
< wumpus>
instagibbs: re: #9016 are you sure that problem will always be logged to the debug log when a transaction coming through RPC is rejected? If not, pointing the user to debug.log could result in a wild goose chase
< wumpus>
we did make the transaction validation a lot less noisy in recent versions
< GitHub197>
[bitcoin] fanquake opened pull request #9022: Update release notes to mention dropping OS X 10.7 support (0.13...0-13-1-osx-notes) https://github.com/bitcoin/bitcoin/pull/9022
< gmaxwell>
so interesting, last block (by bitclub network) had 866 transactions, prior blocks (all same max size)-- 1942, 1422, 2215, 2145, 1905.. I wonder why bitclub's transactions were so much larger?
< gmaxwell>
they also collect a lot more fees.
< gmaxwell>
1.239 BTC vs, .513 .839 .723 .689
< gmaxwell>
I see it earlier too. antpool mined a block with .714 btc in fees in 2288 transactions, then bitclub with 1783 transactions but .950 btc in fees.
< wumpus>
interesting, a custom strategy to maximize fees?
< gmaxwell>
well I know that earlier both slush and antpool failed to mine a fairly attractive CPFP (where the parent had ample fees for relay).. the bitclub picked it up.
< gmaxwell>
so it ~might~ be 0.13 vs not.
< gmaxwell>
right now my GBT returns 1.03 btc in fees for a 1MB block.
< gmaxwell>
hm. if it weren't 4am I'd hack update tip to do a gbt and log the total fee amount from the result before processing a block.
< gmaxwell>
in any case if 435976 has less than 1.06 btc in fees, something is up.
< gmaxwell>
Wed Oct 26 11:10:29 UTC 2016 435975 109856937
< gmaxwell>
1477480260 435976 99407716
< gmaxwell>
so 435976 could hav collected 1.09 by my observation, collected 0.953 instead (not that much worse), and immediately after it the next block could connect 0.994 ... pretty good.
< gmaxwell>
Take that mining is unstable people. :P
< gmaxwell>
also bc.i is like seriously behind.
< gmaxwell>
oops I was looking at 435975 there.
< gmaxwell>
also wtf, getblock needs a "true" for its verbose argument, while getrawtransaction needs a "1".
< gmaxwell>
okay 435976 took 1.003 which is pretty close to what I saw right before.
< wumpus>
that's slightly curious, maybe getrawtransaction author expected an 'even more verbose' format at some point and pass '2'? it'd have made sense for `getblock` and https://github.com/bitcoin/bitcoin/pull/8704
< wumpus>
getblock \"hash\" ( verbose ) ( extraVerbose ) is a bit silly as APIs go
< wumpus>
but it'd better have been called verbosityLevel then
< luke-jr>
IIRC originally there was a bunch of flags on an Object controlling verbosity
< wumpus>
really? seems like a reversion then
< wumpus>
in any case the RPC could be trivially changed to accept 'true' for '1' too
< wumpus>
(and false for 0)
< gmaxwell>
yea, that might be reasonable. I've notied this true/1 thing before and thought I was nuts. :)
< gmaxwell>
1477480364 435976 101621240
< gmaxwell>
1477481079 435976 108481648
< gmaxwell>
435977 takes 1.01339 btc in fees.
< gmaxwell>
1477481089 435977 94256908
< gmaxwell>
so either miner at 435977 has 715 seconds of latency from gbt->mining, or their transaction selection is less optimal than 0.13.1 on my desktop with defaults + weight=4m.
< luke-jr>
gmaxwell: or they're being paid out of band as most big pools seem to now
< gmaxwell>
I looked, don't appear to have a wad of free transactions.
< luke-jr>
hm
< gmaxwell>
oh well I know why that miner's selection would be suboptimal.. 0.12.x code (they claim to run BU)
< luke-jr>
XD
< wumpus>
wasn't far of the mark with misconfigured/weird software hypothesis
< GitHub151>
[bitcoin] jnewbery opened pull request #9025: getrawtransaction should take a bool for verbose (master...getrawtransbool) https://github.com/bitcoin/bitcoin/pull/9025
< adiabat>
Hi, I have an admittedly nit-picky request for the rpc calls
< adiabat>
could we put "weight" in the getrawtransaction return?
< adiabat>
right now getrawtransaction returns "size" and "vsize", while getblock returns "strippedsize", "size", and "weight"
< adiabat>
the "true" vs "1" sillyness in rpc calls just reminded me. If you want the "weight" of a transaction you have to calculate it from size and vsize.
< adiabat>
also I may be missing something but I don't think verbosity does anything to the getblock command
< jonasschnelli>
adiabat: This would be easy to implement... maybe give it a try?!
< jonasschnelli>
Or open an issue on github
< jonasschnelli>
You need to add a call to GetTransactionWeight()
< jonasschnelli>
somewhere near entry.push_back(Pair("vsize", (int)::GetVirtualTransactionSize(tx)));
< adiabat>
ok, sure
< sipa>
adiabat: originally i wanted weight to be purely an internal thing, and have vsize be the exposed value
< adiabat>
sipa: Hmm ok, should we put vsize in the block info instead?
< sipa>
adiabat: nah, weight is better in any case
< sipa>
i'm just mentioning it to explain why weight isn't in gettransaction
< adiabat>
OK I'll look at putting weight in the getrawtransaction return data
< adiabat>
also the 'verbosity' on getblock, not sure what that does... if it does nothing, maybe can get rid of it
< sipa>
it is about returning 1) just txids 2) full tx info 3) full tx data
< btcdrak>
wumpus: I have found a bug in univalue JSON export, where do I submit the patch?
< jtimon>
quick question on the blocksigning stuff, what flags should I use for block signing (ie #define BLOCK_SIGN_SCRIPT_FLAGS (SCRIPT_VERIFY_P2SH|SCRIPT_VERIFY_WITNESS) is what I have for now)
< jtimon>
some obviously don't make sense like cltv and csv
< jtimon>
for others like SCRIPT_VERIFY_MINIMALIF or SCRIPT_VERIFY_NULLFAIL feels like why not?
< wumpus>
btcdrak: upstream to jgarzik/univalue, and we can merge it to bitcoin-core/univalue if that takes too long / is urgent
< jtimon>
what's the opposite operation of ParseHex() ?
< sipa>
HexStr
< Chris_Stewart_5>
HexStr?
< jtimon>
thanks!
< jtimon>
they were on the same file *hides*
< sipa>
hiding in plain sight, as they say
< btcdrak>
wumpus: well it isnt urgent. Problem is it will need to be merged into Core along at the same time as a tweak of some test cases in the test/data/*.json files.
< GitHub114>
[bitcoin] sdaftuar opened pull request #9026: Fix handling of invalid compact blocks (master...fix-invalidcb-handling) https://github.com/bitcoin/bitcoin/pull/9026
< morcos>
gmaxwell: i actually have a long running node that has been calculating that but only considering txs it actually saw in its mempool. got to run now.. but can get you the results form that later
< morcos>
don't think its that different, thank goodness
< gmaxwell>
btcdrak: can you merge in my data where it overlaps and show the difference?
< btcdrak>
gmaxwell: done
< phantomcircuit>
btcdrak: the average transaction sizes are really high
< gmaxwell>
wow, gbminers block 436404 failed to collect .246 btc in fees.
< gmaxwell>
bitclub is the only miner that beat my node in these observations
< gmaxwell>
and only slightly
< gmaxwell>
average amount of missed fees by 'block source':