< jl2012>
when I'm working on rpctests, sometimes the node will generate empty blocks despite the mempool is not empty. It usually happens when the transactions in mempool are big. Why would that happen and how could I force it to mine those txs?
< gmaxwell>
will it continue to mine empty blocks, not just a one time event?
< sipa>
jl2012: perhaps an interaction between max block size an cpfp mining?
< sipa>
if you build a chain of dependent transactions that does not actually fit in a block
< gmaxwell>
and then there is nothing else to include other than that chain.
< jl2012>
all parents are confirmed. And keep mining empty blocks
< jl2012>
oh, not really keeping mining empty blocks forever
< jl2012>
there is only 1 tx in mempool, but it'd cleanup the mempool only if I generate about 500 blocks
< luke-jr>
fee too low, so it's waiting for priority?
< sipa>
anything particular about this tx?
< jl2012>
about 80kB
< jl2012>
luke-jr: so it'd rather mining empty blocks, than mining a tx with insufficient fee?
< luke-jr>
jl2012: it shouldn't happen with the current code afaik, but that's what it sounds like if it's mining it after blocks
< luke-jr>
I assume there's no locktime?
< jl2012>
no locktime
< sipa>
so after 500 blocks, it does mine it?
< jl2012>
yes, about 400 blocks. I'm try to find the exact number
< gmaxwell>
"Interesting"
< sipa>
is the height it activates a nice multiple of 144 after the activation of segwit?
< jl2012>
it was it's just generate 1 block, print the mempool
< jl2012>
nothing strange
< jl2012>
from the output i assume it was mined at the 11th block, but it re-appeared after that
< sipa>
that's not possible...
< sipa>
or shouldn't be possible
< sipa>
its inputs are spent after mining it
< jl2012>
by the way the tx is not segwit
< jl2012>
maybe i should really see if it is really mined. But that's kind of random
< sipa>
you can call gettransaction to see its number of confirmations, if the transaction applies to the local wallet
< jl2012>
that's not. But I could read the content of blocks
< gmaxwell>
how do these tests mine?
< gmaxwell>
IIRC getblocktemplate can cache templates.
< jl2012>
gmaxwell, the mempool has only 1 tx, if the mempool is cleaned up after mining one block, the tx is not mined (i suppose)
< warren>
btcdrak: wow
< btcdrak>
warren: I assume you are referring to this "<cfields_> wumpus: i've added thin toolchain builders to depends that rid us of the ubuntu toolchain dependency. Only needed once for bootstrap, then we're self-hosted"
< warren>
cfields_: can the same binary result be reached from another distro's toolchain?
< phantomcircuit>
luke-jr: there seems to be a bug in getblocktemplate
< phantomcircuit>
running master
< phantomcircuit>
"coinbasevalue": 333038622,
< phantomcircuit>
2016-09-28 10:43:08 ERROR: ConnectBlock(): coinbase pays too much (actual=332777360 vs limit=312500000)
< phantomcircuit>
(this is on testnet)
< sipa>
phantomcircuit: ugh!
< phantomcircuit>
or possibly im just not including transactions?
< luke-jr>
phantomcircuit: in or outside the context of jl2012's stuff?
< phantomcircuit>
hmm
< phantomcircuit>
this seems like im not including transactions actually
< sipa>
phantomcircuit: that would explain
< phantomcircuit>
im just guessing based on the difference
< jl2012>
sorry, seems just a bug in my code
< luke-jr>
>_<
< jl2012>
false alarm
< sipa>
jl2012: goox
< sipa>
good
< jl2012>
I have 2 nodes. I generated blocks in one, and the other one is not synced
< jl2012>
so sometimes it will generate a block with the new tx, but the block is then get orphaned
< sipa>
i wonder if we shouldn't try to integrate a buildinfo-less mode into the source tree itself
< sipa>
./configure --disable-buildinfo
< wumpus>
maybe, but this is just an experiment for now
< wumpus>
I don't think that's an argument against the tool though?
< sipa>
sure, just making an unrelated comment
< sipa>
but in general of things are useful for the project jn general, i think they should be in repo
< wumpus>
having it outside the source tree has its advantages, it means easier experimentation, quicker iteration etc
< wumpus>
not everything having to go through series of ACK
< wumpus>
these are just my private tools anyhow
< wumpus>
I think I tend to disagree with that in practice, not everything has to be in the repo
< wumpus>
sure, if something is needed all the time like the github-merge script it makes sense to have it in the repo
< wumpus>
although even of that I have my own version, which I incidentally make improvements to, once there's enough I'll file a PR. But having to do that all the time gives a lot of overhead for everyone involved.
< wumpus>
I have some other scripts too, a few other code analysis tools, and to build release notes, author lists etc. All can be useful, but it's pretty much a meta-project to bitcoin itself.
< wumpus>
it isn't part of the bitcoin core release cycle either
< wumpus>
changes can happen any time, even between rcs, or between rc and final
< sipa>
fair enough
< sipa>
no objection to moving it to bitcoin-core bte
< sipa>
*btw
< wumpus>
thanks
< wumpus>
I also think we can just be more liberal with changes in a tools repo. If you think something is useful, just do it, no need for long review cycles etc.
< sipa>
right, agree
< wumpus>
(or maintaining a listof 130 PRs *despairs*)
< jonasschnelli>
BlueMatt: A nice... was looking through the code and couldn't find the Auth/MAC
< BlueMatt>
jonasschnelli: in UDPMessageHeader - chk1 and chk2
< BlueMatt>
those are just the hash of the message = the magic (yea, yea, not how you're supposed to do a MAC....whatever)
< BlueMatt>
s/=/+/
< * jonasschnelli>
is checking the code
< BlueMatt>
see FillChecksum
< jonasschnelli>
need to checkout first,... github not searchable on forks. :(
< BlueMatt>
its all in one file, you can just read that :p
< jonasschnelli>
std::get<3>(msg) is a key or something?
< sipa>
it gets the 3rd element of a tuple
< jonasschnelli>
Yes.
< jonasschnelli>
its declared as a uint64_t
< jonasschnelli>
Wonder how it gets popuplated
< jonasschnelli>
Or is FillChecksum(uint64_t magic, UDPMessage& msg, const unsigned int length) similar to our TCP p2p message packeges where magic is the network magic?
< jonasschnelli>
Okay. Got it...
< jonasschnelli>
UDPConnectionInfo.local_magic gets populated through addudpnode
< GitHub61>
bitcoin/master ac01ff2 Wladimir J. van der Laan: doc: Add privacy recommendation when running hidden service
< GitHub61>
bitcoin/master 7b05af6 Wladimir J. van der Laan: Merge #8823: doc: Add privacy recommendation when running hidden service...
< GitHub121>
[bitcoin] laanwj closed pull request #8823: doc: Add privacy recommendation when running hidden service (master...2016_09_tor_recommendation) https://github.com/bitcoin/bitcoin/pull/8823
< sipa>
jonasschnelli: yes, you can, but it's hard to create an API that always has expected results
< sipa>
we have so many different ways through which we can start treating a particular script as ours
< sipa>
maybe it's an imported address
< sipa>
maybe it is p2sh for a redeemscript that contains something we know
< sipa>
maybe we have the private key
< sipa>
maybe we have a pubkey
< sipa>
i think it's impossible to explain all those conditions
< sipa>
so what does 'importing a pubkey' mean
< jonasschnelli>
Yes. I see your point.
< sipa>
should it make sends to that pubkey spendable? or also p2pkh involving those keys? what about a 1-of-3 multisig that involves those keys?
< sipa>
by forcing the user to state exactly what they want, you avoid that
< sipa>
and you can check whether you can provide the functionality they're asking for
< jonasschnelli>
But explicit requiring the scriptPubKey could be a little bit annoying if you just want to import a P2PKH key?!
< sipa>
it's a compromise between convenience and clarity, yes
< jonasschnelli>
A wait... there is an "address" option for the scriptPubKey... okay. let me continue the testing..
< jonasschnelli>
Makes sense so far.
< sipa>
also, if you just want a p2pkh, you should import just that address, and set watchonly
< jonasschnelli>
Agree. A plain pubkey import is kind of unspecified.
< sipa>
the idea is that 1) you state what you want to see made spendable/solvable/watching, and then 2) all necessary information to accomplish that (which may mean giving redeemscripts, or pubkeys, or private keys)
< GitHub163>
[bitcoin] paveljanik opened pull request #8826: Do not include env_win.cc on non-Windows systems (master...20160928_leveldb_no_win) https://github.com/bitcoin/bitcoin/pull/8826
< GitHub21>
[bitcoin] jnewbery opened pull request #8830: [test] Add option to run bitcoin-util-test.py manually (master...test-bitcoin-util-manually) https://github.com/bitcoin/bitcoin/pull/8830
< pedrobranco>
Hi, anyone is experiencing high disk space usage when running bitcoin tests (env. is osx)?
< wumpus>
what is 'high disk space usage' these days?
< pedrobranco>
62gb in temporary test folder
< wumpus>
for the RPC tests I guess, not the unit tests?
< wumpus>
does it at least clean up everything again? the RPC tests have recently been changed to execute tests in parallel, this also means that more disk space is required for the instances running at the same time
< pedrobranco>
rpc tests only
< wumpus>
you can reduce the parallelism with some command line option if it's too much
< pedrobranco>
i have 80G free space and it uses it all before ending rpc tests.
< gmaxwell>
62GB sounds like a lot, do the rpc tests really do that? seems implausable to me.
< pedrobranco>
i will now run p2p-fullblocktest only
< wumpus>
final size is 2.1G /tmp/testyxoclyo1/14241 for p2p-fullblocktest.py -nocleanup
< wumpus>
blocks.db ends up at 1.1G, the rest goes to the node0 dir. So it must be something local there that causes some file to balloon, do you have any changes compared to master?
< wumpus>
(in qa/rpc-tests/test-framework/blockstore.py)
< wumpus>
jonasschnelli: you're on OSX right? did you ever notice p2p-fullblocktest using exorbitant disk space?
< jonasschnelli>
I can't remember when i last ran this test...
< jonasschnelli>
let me try
< jonasschnelli>
(currently compiling, will start in a min)
< wumpus>
well it's part of the default pulltester run
< jonasschnelli>
Yes. Never noticed big data dirs...
< jonasschnelli>
But I have a 1GB SSD..
< jonasschnelli>
TB
< jonasschnelli>
:-)
< * jonasschnelli>
started ./qa/rpc-tests/p2p-fullblocktest.py --nocleanup
< wumpus>
BTW bitcoin core on FreeBSD nearly works out of the box with ports-provided development packages, not nearly the struggle it is on OpenBSD. Though I don't think I've run the RPC tests yet :)
< jonasschnelli>
oh
< jonasschnelli>
Unexpected exception caught during testing: error('cannot add item to database',)
< wumpus>
hah there goes your 1TB SSD? :-)
< jonasschnelli>
Got the exception after Test 94: PASS [132]
< jonasschnelli>
du -sh /var/folders/hp/kb9p9q8x4k3_z_ccy588hxrc0000gn/T/testhuweo6jb/15776
< wumpus>
they're all key-value stores, which offer the same high-level interface but differ under the hood
< jonasschnelli>
Is the dumb module much slower? If no, why not use dump for the tests?
< jonasschnelli>
*why not use dumb
< wumpus>
I think dumb is much slower, that is what the manual says at least
< wumpus>
I haven't *tried*
< wumpus>
but it seems it's the safe choice
< jonasschnelli>
Theres also a gnu version...
< luke-jr>
gnu is GPL, but I'm not sure we care for Python tests
< wumpus>
gah, we're not going to include it in our source tree
< wumpus>
although BSDs generally don't haev that installed by default
< wumpus>
but dumb works fine as a fallback
< wumpus>
I think we should default to the current db for Linux, and on all BSD variants use dumb
< wumpus>
or use dumb everywhere if the perfrmance is acceptable for this use
< jonasschnelli>
Can you easy switch between dumb/db based on the platform in python? Probably yes.
< GitHub123>
[bitcoin] MarcoFalke opened pull request #8832: [rpc] throw JSONRPCError when utxo set can not be read (master...Mf1610-rpcUtxoFail) https://github.com/bitcoin/bitcoin/pull/8832
< * luke-jr>
wonders how badly things would blow up if we used a map :p
< wumpus>
jonasschnelli: yes, they are just modules that have the same interface