< Luke-Jr>
now git just needs to allow signatures after-the-commit
< GitHub9>
[bitcoin] laanwj opened pull request #7821: init: allow shutdown during 'Activating best chain...' (master...2016_04_shutdown_during_activate_best_chain) https://github.com/bitcoin/bitcoin/pull/7821
< wumpus>
just curious, did anyone (or know of anyone that tried) to replace leveldb with lmdb and do benchmarks with regard to chainstate sync yet?
< wumpus>
I'd really like to know this but I don't get around to it, I know there are lmdb versus leveldb comparison sites but should be under our specific workload
< wumpus>
I'm seeing awful latency on arm64 with leveldb
< btcdrak>
no testing was done with lmdb, only sqlite (which was ghastly).
< wumpus>
yes I know about sqlite, sqlite would be an ok format for the wallet but it's too high level/high overhead for chainstate use
< wumpus>
what this needs a very fast, close-to-hw, key/value store, Ideally for this experient I'd just map the entire thing into memory but the device has only 2GB.
< sipa>
wumpus: lmdb uses mmap for everything afaik, so it's hard to use on 32-bit platforms
< wumpus>
I don't care about 32 bit platforms for this experiment
< wumpus>
just want to compare x86_64 versus arm64
< wumpus>
mmapping everything sounds perfect
< jonasschnelli>
wumpus: what device are you targeting? Odroid? PINE A64?
< wumpus>
odroid C2
< jonasschnelli>
I have ordered a PINE A64. ~same sepcs..
< wumpus>
how much memroy?
< jonasschnelli>
2GB
< jonasschnelli>
(which is essential)
< jonasschnelli>
I think we might finally have tiny devices that can run a fullnode and cost <40USD.
< wumpus>
yes I'd preferred one with 8GB but at least it's not abysmal :)
< jonasschnelli>
A fullnode next to your router...
< wumpus>
right, it's promising, too bad leveldb is letting me down on this, I'm not sure why, most profiling tools are broken with this experimental kernel
< jonasschnelli>
imdb sounds promissing...
< jonasschnelli>
wumpus: what this needs a very fast, close-to-hw, key/value store <--- I agree!
< jonasschnelli>
C based
< jonasschnelli>
wumpus: How fast is the "disk" (flash) access on the Odroid C2? You think its enough for UTXO queries?
< wumpus>
exactly!
< wumpus>
well I have a microsd in it of 32gb, but it's not very fast, I only use it for the OS, have attached an external USB harddisk for storing the block chain and utxo set
< jonasschnelli>
wumpus USB2.0 should probably fast enought... though, in theory (IIRC) gbit ethernet should be faster.
< wumpus>
so it could be a problem with slow storage, it looks like a latency problem, block validation fires a lot of queries and some of those end up taking ~1.2ms per input
< wumpus>
well the fastest setup I've been able to use up until now is to put the blocks on the USB harddisk, and the utxo set on a network block device mounted over 1gbit ethernet
< wumpus>
of course, that's kind of cheating :)
< jonasschnelli>
hah. true!
< wumpus>
and it still doesn't manage to max out the CPU on most blocks, so something curious is happening with leveldb
< wumpus>
I've heard about bitcoin over nfc, but no never over IRyet :-)
< Luke-Jr>
turn gameboy colour into a hw wallet?
< wumpus>
almost all those ARM boards have an IR receiver, but never a transmitter
< wumpus>
yes you could use it for communication with a gameboy color probably :-)
< wumpus>
not sure about wavelengths, frequencies etc
< wumpus>
I have an FPGA board (ICEstick) with a IR led that could probably be taught to speak the right protocol, but I'm too lazy :p
< GitHub118>
[bitcoin] jpdffonseca opened pull request #7822: [qa] Add test to RPC listunspent (master...support/add-test-listunspent) https://github.com/bitcoin/bitcoin/pull/7822
< GitHub154>
[bitcoin] jpdffonseca opened pull request #7823: [Wallet] Add index of unspent transaction outputs (UTXOs) (master...enhancement/cache-unspent-outputs) https://github.com/bitcoin/bitcoin/pull/7823
< GitHub62>
[bitcoin] jonasschnelli opened pull request #7824: Add uncontroversial RBF base features (master...2016/04/rbf_uncontroversial) https://github.com/bitcoin/bitcoin/pull/7824
< GitHub56>
[bitcoin] pedrobranco opened pull request #7825: Prevent multiple calls to ExtractDestination (master...enhancement/prevent-multiple-calls-extractdestination) https://github.com/bitcoin/bitcoin/pull/7825
< jtimon>
MarcoFalke: told me that some people told him they kind of like #7779
< jtimon>
wumpus: but nobody commented on the PR...I'm not sure what should I do. Comment the example commit and remove the "Discussion: " label? just close it and keep waiting for someone to hopefuly comment?
< wumpus>
heh we have kind of a PR storm at the moment
< GitHub54>
bitcoin/master fa24456 MarcoFalke: [qa] httpbasics: Actually test second connection
< GitHub54>
bitcoin/master 3bc71e1 Wladimir J. van der Laan: Merge #7802: [qa] httpbasics: Actually test second connection...
< GitHub144>
[bitcoin] laanwj closed pull request #7802: [qa] httpbasics: Actually test second connection (master...Mf1604-qaTestHttp) https://github.com/bitcoin/bitcoin/pull/7802
< wumpus>
jtimon: maybe ask people to look at it during the meeting
< jtimon>
quote from MarcoFalke "Actually we were discussing on IRC with wumpus morcos and sduftar.
< jtimon>
They seemed unanimous about using one flag and not removing it from the method because it might be needed later." when talking about #7779 and #7574
< jtimon>
sdaftuar:
< jtimon>
I'll just leave it here
< wumpus>
we should also inform about the c++11 transition in the upcoming meeting, I want to use nullptr @sipa :)
< paveljanik>
jtimon, I guess it is because it slipped to the second page of the PRs.
< jtimon>
paveljanik: not in a hurry, but it seems a bit useless as a discussion PR if the discussion happens elsewhere (and without me, I opened it to know what people think)
< jtimon>
you don't reproduce it on current master?
< wumpus>
can't try rght now
< jtimon>
ok, if anybody else can try "python ./qa/pull-tester/rpc-tests.py mempool_limit" on the current master to confirm is not a local/cache thing, I would appreciate it
< paveljanik>
jtimon, mmnt
< jtimon>
paveljanik: awesome thanks
< sipa>
wumpus: ack on nullptr :)
< jtimon>
wumpus: sipa: suggestions for more C++11-ish "extern boost::scoped_ptr<CPolicy> globalPolicy;" ?
< paveljanik>
jtimon, OK, 3s
< jtimon>
paveljanik: no hurry
< paveljanik>
no, it is finished -
< paveljanik>
Tests successful
< paveljanik>
Duration: 3 s
< wumpus>
a global scoped ptr? no, I don't like that
< wumpus>
if you need them at all, please explicitly create/initialize and teardown global objects
< paveljanik>
jtimon, what does it print for you?
< paveljanik>
ah mempool full
< wumpus>
we've had enough (de)initialization order problems to last a lifetime
< jtimon>
wumpus: where's the right place for deleting global pointers ?
< jtimon>
people seemed to like boost::scoped_ptr<CChainParams> for #6907
< wumpus>
jtimon: shutdown()
< jtimon>
ok, then I can use a simple pointer that initializes in AppInit2() and gets deleted in shutdown(), fine with me. maybe we should do that with the chainparams globals too
< wumpus>
sure, if you really need something that lasts the entire program you could use global scoped_ptr, but make sure there areno dependencies on anything
< wumpus>
and ideally the global pointer would be restricted within one module, not shared via external, but passed where necessary
< sipa>
ideally all of the node operation in main is encapsulated in a class, which has a pointer to the utxo cache, the policy, the mempool, ...
< sipa>
and there are no globals at all
< * sipa>
dreams
< wumpus>
yes, no globals would be best
< jtimon>
what do you think about eventually moving all globals to globals/server (most of them currently in main.o), globals/util, globals/common, etc
< jtimon>
?
< wumpus>
but if you really need them at least be as careful as possible with them, restrict their scope/accessibility as much as possible, etc
< wumpus>
doesn't sound good to me
< wumpus>
keep the globals with the code that uses them
< wumpus>
keep them as local as possible
< wumpus>
this makes it easier to make it into a class, too
< jtimon>
well, I was thinking that maybe not including all the globals in everything that needs to include main.h for other reason would be a step forward, but thanks for the feedback
< jtimon>
you could just grep #include "globals/ and find functions that could take parameters explicitly (the only true and slow path to removing globals)
< jtimon>
I mean, I've mostly thought about this only for the globals in util.o, chainparams.o and globalPolicy, probably would move globals from main.o to globals/server.o the last thing, since it will be clearly the most disruptive
< jtimon>
on "keep the globals with the code that uses them" well, in my opinion I will use a new global in init.cpp, and everywhere else basically shouldn't be using it (it should be passed down explicitly, but you cannot do that at once)
< jtimon>
where should Params() be called from? ideally, nowhere
< jtimon>
but only from init and one other place would be almost as ideal
< wumpus>
I don't like the globals/ idea. If you need globals from another implementation unit there could be a function that returns (a pointer or reference to) it
< wumpus>
similar to private: or protected: in classes
< wumpus>
where should Params() be called from? <- yes, ideally just once to create the object, after that it just gets passed on
< wumpus>
and stored in appropriate places within classes, so they can pass it on
< jonasschnelli>
sipa: ping
< sipa>
jonasschnelli: pang
< jonasschnelli>
You remember the SetMerkleBranch PR for the wallet:
< sipa>
that's outdated; it's not necessarily in the mempool
< jonasschnelli>
I think the GUI needs an aditional mempool check then
< jonasschnelli>
Somehow the wallet does not detect conflict with a transaction that is not in the mempool
< jonasschnelli>
*conflicts
< sipa>
for detecting what case?
< jonasschnelli>
If a replace a transaction (RBF), I have two transaction in the transaction list, neither of them is marked conflicted.
< gmaxwell>
$ ./bitcoin-cli stop
< gmaxwell>
error: couldn't parse reply from server
< gmaxwell>
0_o
< jonasschnelli>
(the first transaction is removed from the mempool)
< sipa>
jonasschnelli: known to conflict is only for conflicting with confirmed transactions
< sipa>
jonasschnelli: you can use IsTrusted ?
< jonasschnelli>
sipa: okay. Let me see..
< sipa>
jonasschnelli: i don't know the gui, and i don't really know what you're talking about
< gmaxwell>
hm. seems to be deadlocked
< jonasschnelli>
hah. Yes. Forget about it. I'll work on it and show you some code... thats better understandable.
< GitHub105>
[bitcoin] laanwj closed pull request #6774: IsSuperMajority() moved to separate soft forks unit (master...ISM_to_softforks_unit) https://github.com/bitcoin/bitcoin/pull/6774
< sipa>
gmaxwell: going to run with that to monitor
< sipa>
gmaxwell: i think the inv sorting is suboptimal, in that it always puts transactions without unconfirmed dependencies first, even if they have lower individual feerate than dependent ones
< gmaxwell>
Yes, it does.
< sipa>
gmaxwell: after CPFP i guess we'll want to change it to sort by ancestor-combined-feerate, and build a set to submit based on it
< sipa>
gmaxwell: but i doubt it matters at all now
< gmaxwell>
I was aware of it, but didn't think it was worth the more core/computationally expensive handling. (issue being that the parent tx may already have been sent, so you can't just sort by feerate then move down all without parents-- or you'd just get ~what I have)
< sipa>
i guess you'd just want to run CreateNewBlock on the subset of the mempool that overlaps with vInventoryToSend, with the reduces max size, and then send that :p
< * sipa>
ducks
< gmaxwell>
cached createnew block and intersect with the inventory. :P
< sipa>
do you have any statistics for how this affects orphan tx?
< sipa>
i guess you'd need a test setup with multiple nodes
< gmaxwell>
not yet, though with only two nodes running it, I couldn't measure.
< gmaxwell>
I believe it will completely eliminate them; except for garbage being sent by confused node, and except for the fact that we don't sync the mempool at start.
< Chris_Stewart_5>
is the only difference between SCRIPT_VERIFY_STRICTENC & SCRIPT_VERIFY_DERSIG the fact that the former requires the hash type to be defined?
< sipa>
Chris_Stewart_5: strictenc also disallows hybrid public key encoding
< Chris_Stewart_5>
why isn't the hash type being defined in the bip66 specification?
< sipa>
because it's not necessary
< sipa>
the hashtype is included in the sighash, so you can't modify it
< sipa>
or it would invalidate the signature
< GitHub37>
[bitcoin] jonasschnelli opened pull request #7826: [Qt] show conflicts of unconfirmed transactions in the UI (master...2016/04/ui_mem_cflct) https://github.com/bitcoin/bitcoin/pull/7826
< wumpus>
hmm in lmdb reads happen inside a transaction as well, this makes it harder to fit into the current dbwrapper, resetting the transactino for every read is probably inefficient
< sipa>
wumpus: not necessarily, i think
< sipa>
otherwise, you could have a read transaction that persists, and is just closed whenever a write is performed, and then reopened
< Chris_Stewart_5>
sipa: Wrt to our discussion yesterday about executing scriptSigs & scriptPubKeys individually, was OP_CODESEPARATOR initially trying to allow this?
< Chris_Stewart_5>
i.e. it indicates the ending of the scriptSig & beginning of the scriptPubKey?
< sipa>
Chris_Stewart_5: it is assumed that it waas intended to allow delegation
< sipa>
someone signing over the right to sign to someone else, under other conditions
< Chris_Stewart_5>
it seems that you could have some sort of separator like that to say push ops can't transcent separator boundaries... not sure what the value added is though :-/
< GitHub50>
[bitcoin] jtimon opened pull request #7828: Trivial: Globals: Explicitly pass const CChainParams& to ProcessMessage() (master...0.12.99-chainparams-trivial) https://github.com/bitcoin/bitcoin/pull/7828
< jtimon>
newbies and full grown programmers get out of your dark conrners and comment on #7829 so that I can see you. (this is not part of the funded stuff, just you doing stuff for me and me giving advice to you if you want that)
< jtimon>
the mission is not a priority or important, but if you want to get familiarized with github or whatever I'm happy to teach you with simple examples I care about
< kanzure>
jtimon: why do you want pull request 7829 "never merged"?
< jtimon>
kanzure: well, I'm happy merging all those TODO lines in the last commits, but it would be easier for me to keep track about what's been done and what is not if the PR is not merged. Besides that, I'm happy to merge a lot of TODO comments, no problem with me
< jtimon>
kanzure: does that make any sense?
< kanzure>
almost; the PR text could be more clear about what that PR is.... is this a good summary? "There is a set of changes that I would like to make. There's a lot of changes involved in this. To organize this work and to coordinate with others interested in helping make these changes, I have labeled TODOs in this pull request's source code. You are welcome to fork this and fix individual issues, and I will regularly rebase against the ...
< kanzure>
... master branch to keep everything updated."
< kanzure>
(however, it's unclear to me if that is an accurate summary)
< jtimon>
kanzure: I'm happy to change the description of the PR, do you think you get the general intend? offering myself as a "tutor" only for small things I want done? no money involved in either direction
< kanzure>
if your intention was tutorship then that was not clear to me reading the PR text :)
< jtimon>
I don't want to promise too much, but I think I could offer some free guidance with stupid examples I know all about
< jtimon>
kanzure: that is good feedback
< kanzure>
one other minor point of feedback is that if you want to attract novices to your pull request, saying "Probably nobody with bitcoin development experience will think this is a priority. I don't either." will not make it clear to novices that you think these changes are good or worth doing at all :)
< jtimon>
but my intention is not really to start a tutorial, just to make people work for me on simple things I want done, and offer any help I can offer for them, whatever is more attractive without lying
< kanzure>
right, you are offering mentorship/tutorship for how to make simple contributions to the project, in exchange for feedback and advice on a newcomer's other pull requests. but this is not clear from the PR text.
< jtimon>
kanzure: more good feedback, but I feel that's the truth (nobody is nervous about this not happenning soon enough), and it may actually be a relief for someone new (ie it doesn't matter if it takes you 3 weeks or 4, it will be welcome whenever you are ready because nobody is working on this urgently)
< jtimon>
kanzure: I'm happy that you got the basic idea, please, don't hesiate from making this kind of feedback on the PR itself
< kanzure>
i would start the PR text with something like: "I opened this PR so that newcomers to the Bitcoin Core project could make simple changes to get familiar with contributing to Bitcoin Core. I have marked a number of TODO comments in the source code of this pull request. The changes listed here are not critical to the project but they are good introductory material. Each TODO is relatively simple, and more experienced developers are busy ...
< kanzure>
... doing other tasks, making this an excellent chance for you to get feedback from me on basic contributions to Bitcoin Core. Hopefully this will help you streamline submissions to Core, please let me know how I can help and I'm also offering some promises/commitments (see below)."
< jtimon>
ok, I pasted your comment on the PR
< jtimon>
very grateful, but...
< jtimon>
"I have marked a number of TODO comments in the source code of this pull request"
< jtimon>
I haven't merged anything anywhere, at most #7828 as an example
< jtimon>
kanzure: never mind, "of this pull request", re-read, all fine, thank you very much
< kanzure>
yep
< jtimon>
you captured the spirit and improved the description
< jtimon>
in summary is that, free review for simple tasks I review, and I will try to push other people to review, etc
< jtimon>
in summary is that, free review for simple tasks I select, and I will try to push other people to review, etc
< jtimon>
I've met some developers that are shy because they are "C++ newbies" (to be honest, never was one because in the uni it was almast all C/C++ but some slides from older teachers still in pascal, but was never afraid of other languages and they shouldn't be either)
< jtimon>
but I have to admit that moving from subversion to git was kind of a shock for me
< kanzure>
bitcoin was your first introduction to git?
< jtimon>
well, I fetch pybrain before that, but basically yes
< jtimon>
never pushed anything to git that wasn't mine before git
< jtimon>
before bitcoin
< jtimon>
and never rebase -i until sipa told me that was possible ("this is what maaku means by re-writing history" I thought)
< jtimon>
no, I'm lying, I did used git before with maaku, but I had never done interactive rebases until I had to contribute for the first time. for some reason I really needed to rewrite history
< jtimon>
probably just squash 2 commits into one or something stupidly simple that just happened to be impossible for me before
< jtimon>
if other people intend to get stuck in the same sptupid things that I did, I won't let them