< bitcoin-git>
[bitcoin] practicalswift opened pull request #13815: build: Add linter to make sure the return value of Parse[...](...) is checked (master...check-return-values-from-parse) https://github.com/bitcoin/bitcoin/pull/13815
< kallewoof>
So confused. Why would the `allowhighfees` argument refuse to become an integer despite client.cpp listing it as 'to be converted'? self.nodes[2].sendrawtransaction(hexstring=rawTxSigned['hex'], allowhighfees=Decimal("0.00007000"))
< kallewoof>
Actually, wrong line, it was this one: assert_raises_rpc_error(-26, "absurdly-high-fee", self.nodes[2].sendrawtransaction, rawTxSigned['hex'], Decimal("0.00001000"))
< luke-jr>
kallewoof: client.cpp is entirely irrelevant for this
< kallewoof>
luke-jr: It determines if argument should be converted or kept as a string, right?
< luke-jr>
kallewoof: for bitcoin-cli, which isn't tested by assert_raises_rpc_error
< * kallewoof>
blinks
< kallewoof>
So how does it know whether to format arguments or keep as strings?
< sipa>
kallewoof: it doesn't
< sipa>
JSON-RPC on the wire is JSON
< sipa>
not strings
< sipa>
if you use Decinal in python, it is sent as a JSON number
< sipa>
bitcoin-cli needs to know what arguments to treat as string literals and which ones not
< kallewoof>
I'm confused. THis works: self.nodes[2].sendrawtransaction(hexstring=rawTxSigned['hex'], allowhighfees=0.00007000)
< kallewoof>
And this doesn't: self.nodes[2].sendrawtransaction(hexstring=rawTxSigned['hex'], allowhighfees=Decimal("0.00007000"))
< sipa>
oh, i must be wrong then
< kallewoof>
Unless Decimal() gives something that univalue does not consider to be isNum().. but if I write() it, it gives me [ "0.00007000" ] (with quotes)
< sipa>
this has nothing to do with univalue
< kallewoof>
Wait, it does, because the error is when ! isBool() and ! isNum()
< sipa>
JSON has 4 data types; numbers, strings, arrays, objects
< sipa>
the *python* code is converting what you pass to the sendrawtransaction function into those 4 types
< sipa>
isNum failing on the bitcoind side just means you're not sending a JSON number
< kallewoof>
My code goes if isBool() {..} else if isNum() {..} else throw jsonrpcerror...
< sipa>
so it's presumably sending a JSON string
< sipa>
oh, bools too; sorry, 5
< kallewoof>
Right -- I thought Decimal() gave a JSON number. Guess not.
< sipa>
i guess not :)
< luke-jr>
probably should be made to
< kallewoof>
I'm honestly pretty sure it is, and I'm screwing something up, but rounding errors are no biggie in this case so I'm going with the native variant.
< fanquake>
wumpus: heh, I saw that "loss" as well. Got emailed twice for some reason
< wumpus>
fanquake: well I didn't see my comment so I posted another one, then suddenly they both appeared
< fanquake>
wumpus GH playing tricks on you
< fanquake>
Also please don't merge 13809 yet, few things to be fixed up first, to avoid lots of followups
< wumpus>
OK, please comment that there so anyone else won't merge it either
< mryandao>
typically most linux distribution follow XDG specification for looking up config files, would this be preferable in Core? I know there was a previous closed issue which mentioned this, but we could keep the datadir cache separate in ~/.bitcoin as is
< mryandao>
the upside of this is that daemon configuration can be easier to backup for users who just scoop the entire XDG_BASE_DIR
< harding>
mryandao: do you think there's lots of people who backup just .config/ and not their whole home directories? The stuff in my .config represents perhaps a few hours of tweaking over years, but the stuff in the rest of my home directory represents many hours of work each day over the same number of years.
< mryandao>
harding: personally, i only backup my .config directory since my homedir would have accumulated a bunch of unwanted stuff in new installations.
< mryandao>
especially when it comes to hidden directories in home :/
< harding>
mryandao: ok. I guess I have a different backup philosophy than you do. I use cheap external HDs, and so I just grab everything indiscriminately. I think there probably is an advantage to fully separating stuff we really want users to backup (e..g bitcoin.conf and wallets) from stuff that's just nice to backup (e.g. old blocks), but I personally don't think that's enough of an advantage to make them have to go looking around
< harding>
various places in their home directory for different Bitcoin-related resources.
< fanquake>
wumpus I was looking at the wrong things before. Feel free to ignore my nit on 13809 and merge it.
< bitcoin-git>
[bitcoin] scravy opened pull request #13816: travis: build and run tests on os: osx (master...run-functional-tests-on-macos) https://github.com/bitcoin/bitcoin/pull/13816
< bitcoin-git>
[bitcoin] Sjors opened pull request #13818: More intuitive GUI settings behavior when -proxy is set (master...2018/07/gui-proxy) https://github.com/bitcoin/bitcoin/pull/13818
< bitcoin-git>
[bitcoin] MarcoFalke opened pull request #13821: qa: Re-enable test for unknown arg in conf file (master...Mf1808-qaConfWarn) https://github.com/bitcoin/bitcoin/pull/13821
< bitcoin-git>
[bitcoin] MarcoFalke closed pull request #13821: qa: Re-enable test for unknown arg in conf file (master...Mf1808-qaConfWarn) https://github.com/bitcoin/bitcoin/pull/13821
< bitcoin-git>
[bitcoin] achow101 opened pull request #13822: bench: Make CoinSelection output groups pass eligibility filter (master...fix-out-groups-bench) https://github.com/bitcoin/bitcoin/pull/13822
< bitcoin-git>
bitcoin/master 2fb0066 practicalswift: net: Add missing verification of IPv6 address in CNetAddr::GetIn6Addr(...)
< bitcoin-git>
bitcoin/master 7d36237 Wladimir J. van der Laan: Merge #13776: net: Add missing verification of IPv6 address in CNetAddr::GetIn6Addr(...)...
< bitcoin-git>
[bitcoin] laanwj closed pull request #13776: net: Add missing verification of IPv6 address in CNetAddr::GetIn6Addr(...) (master...call-IsIPv6-in-GetIn6Addr) https://github.com/bitcoin/bitcoin/pull/13776
< ossifrage>
This is a curious error: " dropped: non-selectable socket"
< ossifrage>
They seem to happen in runs over several minutes
< bitcoin-git>
[bitcoin] MarcoFalke opened pull request #13823: qa: quote path in authproxy for external multiwallets (master...Mf1808-qaAuthProxyQuotePath) https://github.com/bitcoin/bitcoin/pull/13823
< bitcoin-git>
bitcoin/master 494634a Andrew Chow: bench: Make CoinSelection output groups pass eligibility filter...
< bitcoin-git>
bitcoin/master 0fb9c87 MarcoFalke: Merge #13822: bench: Make CoinSelection output groups pass eligibility filter...
< ossifrage>
I didn't think bitcoin would be using select(), I thought it would be using epoll() and friends?
< bitcoin-git>
[bitcoin] MarcoFalke closed pull request #13822: bench: Make CoinSelection output groups pass eligibility filter (master...fix-out-groups-bench) https://github.com/bitcoin/bitcoin/pull/13822
< ossifrage>
it looks like bitcoin-qt is using up 800 FDs for open files (chainstate, txindex, etc...)
< ossifrage>
509 for txindex and 288 for chainstate
< sipa>
ossifrage: longer term we'll probably switch to libevent
< sipa>
which uses the most efficient polling mechanism on every platfor.
< ossifrage>
sipa, any suggestions on reducing the number of FDs used by txindex?
< sipa>
ossifrage: don't use txindex
< sipa>
:)
< ossifrage>
I wonder how painful of a project it would be to make the move to libevent?
< Dizzle>
We're already using it for RPC and some other networking, right?
< ossifrage>
Hmm, I wonder what bounds the number of FDs used by the txindex ldb files
< sipa>
leveldb has a max open fileslimit
< sipa>
how many connwctions do you have?
< ossifrage>
sipa, 160
< ossifrage>
(right now, but it looks like has been having troubles with new connections for a while due to the number of ldb files open)
< sipa>
bitcoind tries to raise its max fd limit at startup
< ossifrage>
sipa, the problem is with the FD_SET size which is 1024 :-(
< sipa>
yes
< ossifrage>
it looks like ldb is using 1000 fds (as per comment)
< ossifrage>
I think I'll try rebuilding with the default set to 500 or so (I wonder if that is global or per database)
< ossifrage>
However, I suspect it will take another month of runtime to accumulate that many open ldb files to test...
< sipa>
ossifrage: no, it's a runtime testing
< jimpo>
Is there an option for shared (read/write) locks w/o C++ 14?
< jimpo>
looks like sigcache uses boost::shared_lock. is that acceptable or discouraged?
< sipa>
i think as long as we don't have c++14, boost::shared_lock looks like a good solution
< gmaxwell>
Is there really no read/write lock before C++14?! Posix has had them for like 20 years!
< gmaxwell>
BlueMatt: See above FD_SET stuff. Is there any way I could bribe you to PR the patch that changes the net to use poll on Linux?
< gmaxwell>
We have 1001 windows specific things in the codebase, having a linux+bsd specific thing wouldn't be the end of the world.
< gmaxwell>
And we've delayed doing the simple thing for eons, in favor of a grand change that has been slow in coming.
< gmaxwell>
IIRC the patchs that you and patrick had done were like a dozen lines of code, and except for the fact that they added platform IFdefs, were totally clean.
< sipa>
gmaxwell: before c++11, c++ didn't have any form of threading or synchronization primitives at all :)
< gmaxwell>
sipa: I know but why the heck didn't C++11 catch up with basic features from decades ago? :(
< sipa>
gmaxwell: my guess is that c++11 was already an enormous change
< gmaxwell>
aside, <3 RW locks
< sipa>
yes!
< sipa>
the c++14 shared mutex doesn't support upgradable locking
< gmaxwell>
I can see why upgradability would be useful for us, but I don't recall ever using it in my own code.
< gmaxwell>
(maybe posix rw locks aren't even upgradable?)
< sipa>
you can implement upgradable locks on top of shared and exclusive locks
< gmaxwell>
for us I think the big win for rw locks is that we can make all the things that just need readonly access to the blockchain not block each other...
< gmaxwell>
and that doesn't need an upgradable lock.
< sipa>
upgradable locks would be nice in that they let us lock/unlock exclusive access during block validation
< gmaxwell>
block processing could in theory use one, e.g. take a read lock to decide if the incoming block is already known to us, then promote to write if it isn't... but thats rare, might as well take the write lock.
< sipa>
at times when the chainstate is consistent but not done
< gmaxwell>
okay thats a point.
< sipa>
basically you just need to upgrade when flushing the per-block utxo cache to the chainstate cache
< gmaxwell>
I'm always a little warry about peppering the code with a lot of fine locking, since they're slow (potentially very slow if contended)
< gmaxwell>
(and it's hard to tell that it's slow)
< sipa>
uncontended locks are really fast (at least on linux)
< gmaxwell>
It's all relative. I think they're always as slow as a cache line bounce, so equal to probably thousands of multiplies.
< sipa>
the number i had in my mind was 100 cpu cycles for an uncontended loc
< sipa>
but i can't remember actually ever validating that number
< gmaxwell>
hm. okay so an order of magnitude between our beliefs.
< gmaxwell>
I don't see how 100 is physically possible though.
< gmaxwell>
not without doing some kind of transactional crazyness that can rollback the cpu on conflicts.
< sipa>
i think an uncontended lock is a few compare-and-swaps
< gmaxwell>
so per that a few hundred cycles, seems fast to me, though thinking more about it, I guess it no longer strikes me as physically impossible to take a lock in 100 cycles IF you are the same core that last had the lock.
< gmaxwell>
If you aren't then I don't think it's physically possible, another core could take the lock at the same time, and at 100 cycles, it's potentially outside of your lightcone. (13 cm)
< sipa>
just wrote a small c++11 benchmark that creates a single thread which grabs a mutex, and increments a variable, and releases again
< sipa>
14 ns per iteration
< sipa>
agreed, "always on the same thread" is a stronger condition than just "uncontended"
< gmaxwell>
makes sense. yea, 14ns is just not physically possible if it has to consult with a far away core. but I guess it doesn't in common cases.
< sipa>
now trying the same with 2 simultaneous threads
< sipa>
just 100 ns per iteration?
< sipa>
wtf
< sipa>
i was expected several orders of magnitudes more
< gmaxwell>
well, hyperthreading? :)
< gmaxwell>
use taskset to force it into different sockets. :P