< sipa> jonasschnelli: you were just returning from Upgrade without shutting down... no wonder you get a corruption :)
< jonasschnelli> sipa: okay. I see. :/
< bitcoin-git> [bitcoin] jonasschnelli opened pull request #10649: Make sure we only mine via the first wallet (master...2017/06/wallet_generate) https://github.com/bitcoin/bitcoin/pull/10649
< bitcoin-git> [bitcoin] jonasschnelli opened pull request #10650: Multiwallet: add RPC endpoint support (master...2017/06/wallet_rpc_endpoint) https://github.com/bitcoin/bitcoin/pull/10650
< bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/d083bd9b9c52...c68a9a69278a
< bitcoin-git> bitcoin/master 5555fa8 MarcoFalke: qa: Add stopatheight test
< bitcoin-git> bitcoin/master c68a9a6 MarcoFalke: Merge #10632: qa: Add stopatheight test...
< bitcoin-git> [bitcoin] MarcoFalke closed pull request #10632: qa: Add stopatheight test (master...Mf1706-qaStopAtHeight) https://github.com/bitcoin/bitcoin/pull/10632
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/c68a9a69278a...1d991f6f18df
< bitcoin-git> bitcoin/master 700d8d8 practicalswift: Remove obsolete _MSC_VER check...
< bitcoin-git> bitcoin/master 1d991f6 Wladimir J. van der Laan: Merge #10642: Remove obsolete _MSC_VER check...
< bitcoin-git> [bitcoin] laanwj closed pull request #10642: Remove obsolete _MSC_VER check (master...undefined-msc-ver) https://github.com/bitcoin/bitcoin/pull/10642
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/1d991f6f18df...8465b68985f4
< bitcoin-git> bitcoin/master 2c3fc51 fanquake: [depends] expat 2.2.1
< bitcoin-git> bitcoin/master 8465b68 Wladimir J. van der Laan: Merge #10628: [depends] expat 2.2.1...
< bitcoin-git> [bitcoin] laanwj closed pull request #10628: [depends] expat 2.2.1 (master...expat-2-2-1) https://github.com/bitcoin/bitcoin/pull/10628
< sdaftuar> sipa: i've finished my functional test for #10148 -- https://github.com/sdaftuar/bitcoin/tree/test-10148
< gribble> https://github.com/bitcoin/bitcoin/issues/10148 | Use non-atomic flushing with block replay by sipa · Pull Request #10148 · bitcoin/bitcoin · GitHub
< bitcoin-git> [bitcoin] laanwj closed pull request #10541: Docs: Update INSTALL.md to add link to build notes (master...patch-1) https://github.com/bitcoin/bitcoin/pull/10541
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/8465b68985f4...87e69c2549c4
< bitcoin-git> bitcoin/master e5c6168 Pavlos Antoniou: Fix instantiation and array accesses in class base_uint<BITS>...
< bitcoin-git> bitcoin/master 87e69c2 Wladimir J. van der Laan: Merge #10530: Fix invalid instantiation and possibly unsafe accesses of array in class base_uint<BITS>...
< bitcoin-git> [bitcoin] laanwj closed pull request #10530: Fix invalid instantiation and possibly unsafe accesses of array in class base_uint<BITS> (master...master) https://github.com/bitcoin/bitcoin/pull/10530
< bitcoin-git> [bitcoin] laanwj closed pull request #8831: Replace CWalletDB::ReadKeyValue with CWallet::LoadKeyValue (master...2016-09-28-cwallet-loadkeyvalue) https://github.com/bitcoin/bitcoin/pull/8831
< earlz> Why is CheckTransaction in CheckBlock just a normal rejection and not a DoS rejection? Is there some valid case where CheckTransaction can fail and it not be a malicious actor?
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/87e69c2549c4...209eef60a9ac
< bitcoin-git> bitcoin/master 6171826 Alex Morcos: Don't create change at the dust limit, even if it means paying more than expected
< bitcoin-git> bitcoin/master 209eef6 Wladimir J. van der Laan: Merge #9343: Don't create change at dust limit...
< sipa> earlz: yes!
< bitcoin-git> [bitcoin] laanwj closed pull request #9343: Don't create change at dust limit (master...noneconomicchange) https://github.com/bitcoin/bitcoin/pull/9343
< sipa> soft forks
< sipa> earlz: the peer may not be aware of a softfork that you are aware of
< def> https://0x0.st/lFn.txt <- i can't get signrawtransaction to work, any ideas? i know this isn't a support channel but i'm becoming increasingly convinced that this might be a bug in bitcoin-qt
< sipa> def: can you show the output you are trying to spend?
< earlz> ah, didn't think about softforks
< bitcoin-git> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/209eef60a9ac...ffce893982d9
< bitcoin-git> bitcoin/master fd369d2 Kalle Alm: Switched httpserver.cpp to use RAII wrapped libevents.
< bitcoin-git> bitcoin/master 1ae86ec Karl-Johan Alm: Changed event RAII helper functions to inline to deal with duplicate symbol linker errors.
< bitcoin-git> bitcoin/master ffce893 Wladimir J. van der Laan: Merge #9517: [refactor] Switched httpserver.cpp to use RAII wrapped libevents....
< bitcoin-git> [bitcoin] laanwj closed pull request #9517: [refactor] Switched httpserver.cpp to use RAII wrapped libevents. (master...raii-httpserver) https://github.com/bitcoin/bitcoin/pull/9517
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/ffce893982d9...b750b33c3cea
< bitcoin-git> bitcoin/master 8d4dafd Andres G. Aragoneses: contrib/verifybinaries: allow filtering by platform...
< bitcoin-git> bitcoin/master b750b33 Wladimir J. van der Laan: Merge #10276: contrib/verifybinaries: allow filtering by platform...
< bitcoin-git> [bitcoin] laanwj closed pull request #10276: contrib/verifybinaries: allow filtering by platform (master...filterByPlatformInVerifySh) https://github.com/bitcoin/bitcoin/pull/10276
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/b750b33c3cea...01c4b143a87e
< bitcoin-git> bitcoin/master cf68a48 Pieter Wuille: Deduplicate addrdb.cpp and use CHashWriter/Verifier
< bitcoin-git> bitcoin/master 01c4b14 Wladimir J. van der Laan: Merge #10248: Rewrite addrdb with less duplication using CHashVerifier...
< bitcoin-git> [bitcoin] laanwj closed pull request #10248: Rewrite addrdb with less duplication using CHashVerifier (master...chashverifier) https://github.com/bitcoin/bitcoin/pull/10248
< bitcoin-git> [bitcoin] TheBlueMatt opened pull request #10651: Verify binaries from bitcoincore.org and bitcoin.org (master...2017-06-verify-coreorg) https://github.com/bitcoin/bitcoin/pull/10651
< bitcoin-git> [bitcoin] MarcoFalke pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/01c4b143a87e...4bc853b50fd9
< bitcoin-git> bitcoin/master 999923e MarcoFalke: [qa] util: Check return code after closing bitcoind proc...
< bitcoin-git> bitcoin/master 4bc853b MarcoFalke: Merge #10636: [qa] util: Check return code after closing bitcoind proc...
< bitcoin-git> [bitcoin] MarcoFalke closed pull request #10636: [qa] util: Check return code after closing bitcoind proc (master...Mf1706-qaTraceback) https://github.com/bitcoin/bitcoin/pull/10636
< bitcoin-git> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/4bc853b50fd9...6bef7ca8bc6a
< bitcoin-git> bitcoin/master 0a5a6b9 Dimitris Tsapakidis: Fixed multiple typos...
< bitcoin-git> bitcoin/master 6bef7ca Wladimir J. van der Laan: Merge #10633: doc: Fix various typos...
< bitcoin-git> [bitcoin] laanwj closed pull request #10633: doc: Fix various typos (master...patch-1) https://github.com/bitcoin/bitcoin/pull/10633
< bitcoin-git> [bitcoin] laanwj pushed 4 new commits to master: https://github.com/bitcoin/bitcoin/compare/6bef7ca8bc6a...8c2098ad1209
< bitcoin-git> bitcoin/master c8914b9 Andrew Chow: Have `make cov` optionally include branch coverage statistics...
< bitcoin-git> bitcoin/master 405b86a Andrew Chow: Replace lcov -r commands with faster way...
< bitcoin-git> bitcoin/master d5711f4 Andrew Chow: Filter subtrees and and benchmarks from coverage report...
< bitcoin-git> [bitcoin] laanwj closed pull request #10565: [coverage] Remove subtrees and benchmarks from coverage report (master...lcov-remove-extra) https://github.com/bitcoin/bitcoin/pull/10565
< BlueMatt> meeting?
< wumpus> #startmeeting
< lightningbot> Meeting started Thu Jun 22 19:00:45 2017 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
< lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
< achow101> hi
< jonasschnelli> hi
< wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101
< cfields> hi
< wumpus> topics?
< sipa> ohai
< wumpus> #topic high priority for review
< wumpus> any new ones?
< BlueMatt> should i give up on #10179 (+ #10286)?
< gribble> https://github.com/bitcoin/bitcoin/issues/10179 | Give CValidationInterface Support for calling notifications on the CScheduler Thread by TheBlueMatt · Pull Request #10179 · bitcoin/bitcoin · GitHub
< BlueMatt> at least for 15
< gribble> https://github.com/bitcoin/bitcoin/issues/10286 | Call wallet notify callbacks in scheduler thread (without cs_main) by TheBlueMatt · Pull Request #10286 · bitcoin/bitcoin · GitHub
< sipa> BlueMatt: no
< kanzure> hi.
< BlueMatt> sipa: well i mean point being that is something that really needs to stew in master for a month (or more) before a release
< BlueMatt> so its hitting up against the 15 deadline already
< wumpus> why give up?
< instagibbs> hi
< BlueMatt> not a huge deal to push to 16
< BlueMatt> because i dont want to ship 10286 right after merging it
< BlueMatt> its a real not-gonna-see-the-issues-unless-its-in-master-for-a-month kinda thing :(
< wumpus> that's about the risk - what would be the main gain for merging it for 0.15?
< wumpus> eh it's already in high-priority for review category
< wumpus> we can't bump it anything higher :p
< BlueMatt> well im asking if we should drop it from high priority, unless we still think we can get both in pretty quick i think we should say we wont until its 16
< jtimon> I would like review on #8498
< gribble> https://github.com/bitcoin/bitcoin/issues/8498 | Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
< wumpus> ok, but I mean, what would we lose if we decide to drop it for 0.15. Is there something else would be prioritized before it?
< wumpus> I don't think it collides with e.g. multiwallet
< BlueMatt> just 10286 - the dont-really-block-on-wallet-for-block-connection thing
< BlueMatt> its not itself a huge win, but iterative steps towards better disconnection
< cfields> BlueMatt: I'll try to get through it today, I've put it off long enough :)
< BlueMatt> i know ryanofsky wanted it for the separate-process-wallet stuff he was working on
< sipa> BlueMatt: how about we re-assess it next week?
< BlueMatt> ok, sounds good
< jtimon> and #8994 too, not sure which one is more likely to get it
< wumpus> looks like they're both getting reviews
< sipa> and hope for review in the mean time
< gribble> https://github.com/bitcoin/bitcoin/issues/8994 | Testchains: Introduce custom chain whose constructor... by jtimon · Pull Request #8994 · bitcoin/bitcoin · GitHub
< wumpus> yeah...
< ryanofsky> yeah i'd prefer 10286 not to be delayed for months
< jonasschnelli> #10286
< gribble> https://github.com/bitcoin/bitcoin/issues/10286 | Call wallet notify callbacks in scheduler thread (without cs_main) by TheBlueMatt · Pull Request #10286 · bitcoin/bitcoin · GitHub
< wumpus> ok, any other topics?
< jtimon> wumpus: well, maybe none of my prs should be in project 8, I don't know
< phantomcircuit> pong
< sipa> wumpus: coin selection / effective feerate / bnb
< wumpus> jtimon: I'm not sure that one warrants extra priority pre-0.15
< BlueMatt> random PSA: there is now a copy of binaries hosted on https://bitcoincore.org/bin as well as https://bitcoin.org/bin. This will be the new "official" download site (though the bitcoin.org folks said they were gonna keep their mirror up for those who are used to bitcoin.org to not have to hop to another site)
< wumpus> sipa: that's one topic or three?
< sipa> wumpus: one
< wumpus> #topic coin selection / effective feerate / bnb
< sipa> we have a number of open PRs (and one just merged I think) that interact with coin selection
< jtimon> wumpus: egoistically for elements project, it would be very nice to have #8994 before 0.15 for next elements rebase (not that bitcoin should care)
< gribble> https://github.com/bitcoin/bitcoin/issues/8994 | Testchains: Introduce custom chain whose constructor... by jtimon · Pull Request #8994 · bitcoin/bitcoin · GitHub
< sipa> #10637 #10360 #10333
< jtimon> regarding #8498 it's an optimization (perhaps again "not worth it", like #10339)
< gribble> https://github.com/bitcoin/bitcoin/issues/10360 | [WIP] [Wallet] Target effective value during transaction creation by instagibbs · Pull Request #10360 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/10637 | Coin Selection with Murchs algorithm by achow101 · Pull Request #10637 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/10333 | [wallet] fee fixes: always create change, adjust value, and p… by instagibbs · Pull Request #10333 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/8498 | Optimization: Minimize the number of times it is checked that no money... by jtimon · Pull Request #8498 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/10339 | Optimization: Calculate block hash less times by jtimon · Pull Request #10339 · bitcoin/bitcoin · GitHub
< sipa> are instagibbs, achow101, ... here?
< instagibbs> I'm not sure many are even close to merge.
< instagibbs> yeah
< sipa> i agree
< achow101> i'm here
< gmaxwell> Hellow meeting.
< instagibbs> 10333 is closest, but needs more revision
< sipa> but i think there are two approaches to follow here
< wumpus> jtimon: I really think we should have #8994, but it seems to collide with a lot of other things
< gribble> https://github.com/bitcoin/bitcoin/issues/8994 | Testchains: Introduce custom chain whose constructor... by jtimon · Pull Request #8994 · bitcoin/bitcoin · GitHub
< sipa> one is to aim for overhauling coin selection... which will take a long time, and probably should be merged early in a cycle, so not for 0.15
< gmaxwell> 10333 needs to get in (in some form) because it's an important fix.
< instagibbs> I can do suggested change that gmaxwell gave, which I noted.
< sipa> another is to make changes that 1) fix existing bugs and 2) work as a preprocessing step (in the form "let's see if algorithm X finds a good solution... otherwise fallback to what exists"
< wumpus> agree with doing coin selection changes early in a release cycle
< sipa> if 10333 is considered a bug fix, perhaps it should be prioritized for 0.15?
< jtimon> wumpus: I guess that will wait forever then, previous versions were closed because they "collided with mempool/fee stuff", like more than 1 year ago? I'm happy to help people do the trivial rebases for people if that's the problem
< jtimon> anyway, topic changed...
< gmaxwell> 10333 should do nothing other than prevent serious overpayment in some corner cases.
< instagibbs> sipa, It's marked as such, but if people deem it higher priority I'll make it one
< gmaxwell> (at least once it's twiddled some)
< sipa> would 10333 + 10637 be reasonable to aim for in 0.15?
< instagibbs> It needs rebasing likely due to recent dust change merge
< instagibbs> #10637
< gribble> https://github.com/bitcoin/bitcoin/issues/10637 | Coin Selection with Murchs algorithm by achow101 · Pull Request #10637 · bitcoin/bitcoin · GitHub
< wumpus> I don't think so
< instagibbs> latter seems a stretch imo, as great as it would be.
< wumpus> #10637 would be better for 0.16 IMO
< gribble> https://github.com/bitcoin/bitcoin/issues/10637 | Coin Selection with Murchs algorithm by achow101 · Pull Request #10637 · bitcoin/bitcoin · GitHub
< sipa> to clarify, 10637 is just a preprocessing step that makes it much more likely to find a match without change, reducing UTXO set size
< wumpus> the other one is already marked for 0.15
< sipa> if it fails, it falls back to what we have
< achow101> I don't think 10637 would be good for 0.15. I haven't entirely made sure that it doesn't do anything stupid
< instagibbs> sipa, if the code is correct, sure, earlier it wasn't (I'd have to do lots of review to make sure)
< sipa> ok
< sipa> instagibbs: of course
< gmaxwell> I am not in favor of takin total overhaul as a general strategy. The problem is that each logical change has a complex 'whole system impact' question connected to it, which is really hard to reasona about for "hay guies, I replaced the whole thing!" So I would much rather see a series of 'obviously sane' changes followed by "I reworked it to make the code clearer with no substantive change in b
< wumpus> achow101: agree, no need to hurry it just to make a release
< instagibbs> it's a nasty web of code, not his fault, just a fact
< gmaxwell> ehavior".
< sipa> ok, so let's aim for all coin selection changes in 0.16
< sipa> apart from bug fixes
< wumpus> the problem is that at some point the code is so convoluted that no option remains but 'redesign the whole thing'
< jtimon> yeah, I've been trying to do something like #8494 since at least Jun 10, 2015, but I think actually earlier
< gmaxwell> :( is it so late to the end of 0.15 that we're slipping things out of it?
< gribble> https://github.com/bitcoin/bitcoin/issues/8494 | [init, wallet] ParameterInteraction() iff wallet enabled by MarcoFalke · Pull Request #8494 · bitcoin/bitcoin · GitHub
< wumpus> most of that has been replaced by now, but the coin selection code is arguably some of that
< sipa> gmaxwell: i wish you were right, but i believe at some point we'll need to bite the bullet
< sipa> though a number of steps can be taken before that point
< wumpus> gmaxwell: well it starts to be important, even now, to prioritize things
< instagibbs> With BnB in place, lays decent groundwork for more extensions
< gmaxwell> wumpus: will then we stay where we've been stuck for years then? I think it improved a lot, changes we made in 0.14 cleaned up CreateTransaction a lot.
< jtimon> gmaxwell: agree on that way of doing big refactors, that's the sanest way to review them at least
< instagibbs> making sane change-making policy could come next
< instagibbs> sane steps exist i think
< gmaxwell> sipa: obviously it needs to be reworked, but I cannot review something that is just "here is replacement code that does everything subtly differently" against threats like "will this cause runaway upbidding" or "will this potentially cause the utxo set size to skyrocket relative to the current code" or "will this trash user privacy" (well hard do to worse than the current on that mark. :P). But
< gmaxwell> I can review things like instagibbs's effective rate change against those criteria.
< gmaxwell> I think it's important to seperate intentional behavioral changes from implementation structure.
< jtimon> so why not try to get merged 10333 first which seems the simpler to review and then revisit the other 2 ?
< gmaxwell> jtimon: instagibbs has changes he needs to make.
< gmaxwell> And sure.
< * jtimon> nods
< sipa> gmaxwell: i'm worried about something like 10360 to cause unintended runaway behaviour due to some interaction as well
< wumpus> jtimon: yes
< sipa> gmaxwell: even if it looks perfectly fine by all metrics we considered
< gmaxwell> Though if integrated correctly (not saying it is yet) 10637 is just a preprocessing stage that can't make things worse. (not saying that it's yet integrated correctly).
< instagibbs> Action item is me incorporating suggestion to 10333 make more robust and beg for review again here
< wumpus> so should we add 10333 to high priority for review?
< achow101> imo, yes
< gmaxwell> sipa: I don't think it looks fine, I think it fixes a flaw we depend on due to a ~total lack of other mechenisms to sweep txins.
< sipa> gmaxwell: yes, but i can't reason about its total effect
< gmaxwell> (it's easy to show that 10360 will abandon more txouts than current code; and also the current code will _create_ txouts that an immediate second run under that patch would not consider spending)
< sipa> i like the change as such, but who knows what interaction we missed
< sipa> anyway, probably a discussion for later
< gmaxwell> So the only thing I don't know if the effect would be castrophic or just merely not perfect.
< sipa> yes, i think 10333 should be prioritized
< gmaxwell> ack on 10333.
< wumpus> ok, added
< gmaxwell> sipa: I think a simple invarient is that we must spend everything we create at least under more or less stationary conditions. Anything less leads to runaway utxo set growth (though perhaps slowly...).
< gmaxwell> but thats a minimum and perhaps not sufficient. But still 10360 fails it.
< gmaxwell> to be fair, it's not 10360's fault so much as that we only pass that criteria now due to bugs that 10360 fixes.
< sipa> other topics?
< cfields> quick topic suggestion: add "strongly discourage/forbit default function arguments" to the style guide?
< luke-jr> cfields: eh, I don't think that's a good idea
< wumpus> #topic add "strongly discourage/forbit default function arguments" to the style guide
< cfields> *forbid. forbitting would be too much :p
< BlueMatt> not always
< wumpus> f-orbitting
< gmaxwell> default arguments are a footgun for sure, can we articluate where they are okay-- if they aren't forbidden?
< sipa> f(ire from) orbit?
< wumpus> no, I don't think that's a good rule in general either
< luke-jr> the important thing should be making sure the behaviour doesn't change, when func signatures change
< BlueMatt> if theres 20 arguments to the function or its a big many-callers function, maybe
< cfields> gmaxwell: yes, i think that's the discussion I was looking for.
< luke-jr> "if code calling it before compiles with the new signature, it must behave the same"
< wumpus> if there's 20 arguments to a function there's something wrong in general
< wumpus> please make a rule not to have 20 arguments please
< wumpus> certainly not booleans :(
< BlueMatt> (adds a default arg to CScheduler::schedule())
< BlueMatt> wumpus: lol, yes, indeed
< sipa> "It is strongly encouraged to have 19 arguments to functions."
< wumpus> Foo(false, true, false, true, false, true, true)
< BlueMatt> wumpus: you forgot the NULL for good measure which may or may not be a pointer :p
< wumpus> BlueMatt: oh yes
< wumpus> default arguments are mostly good for one thing: transitions without touching all the code
< gmaxwell> I think the default arg is okay when going from 0 to 1 arguments to add a flag, for example.
< sipa> well i think defaults arguments are often used to reduce code impact of a patch... adding an optional argument at the end (especially directly after a non-optional one, and with a type incompatible with the first optional one if any) is perfectly safe and can strongly reduce the amount of code touched
< sipa> the same can be achieved by adding a fewer-arguments wrapper, though
< wumpus> e.g. if you add an argument to a function that's called from 400 places, it's usually better to add a default argument, at least temporarily
< wumpus> right. I think that's a valid use.
< sipa> i like the approach of creating a new function (with more args, and perhaps even a different name), and changing the old one to be a wrapper around the new one
< luke-jr> indeed, I have been rebasing the rescanblockchain RPC, and this is needed to minimise the conflicts
< gmaxwell> The defaults lead to negative interactions with the fact that arguments aren't named. I like sipa's suggestion.
< wumpus> well if it has the same name there is no effective difference
< luke-jr> (it adds an argument for a stop-scanning index)
< BlueMatt> sipa: thats how you end up with AcceptToMemoryPoolWorker(tx, true, false, false, true, NULL, (default) false)
< BlueMatt> :(
< gmaxwell> wumpus: because when you make further additions to the arguments, you don't mess things up.
< sipa> that wrapper approach also is more restrictive: it _just_ allows the exact old use case, and a new one - not a variety of combinations based on multiple optional args
< wumpus> absent argument naming, I like enums better than booleans
< wumpus> anyhow, no, I don't think we should add a general rule about this
< gmaxwell> We had a near consensus fault in the main split as a result of something related to this. IIRC.
< wumpus> just pay attention at reviews to the specific case
< * luke-jr> wishes there was a tool to audit such changes for us
< cfields> ok
< wumpus> not everything has to be spelled out in the style guide, that just makes for laziness *we've checked all the checkboxes*
< gmaxwell> right
< sipa> also, i wish there was a sed-like took that understood our AST a bit better, so you could add a new argument to a function everywhere
< sipa> but regexps are fundamentally not strong enough to do that :(
< wumpus> if only c++ was easier to parse...
< gmaxwell> though I still think sipa's suggestion is good: if you need to introduce an argument to a function called in many places, wrap it with something that provides the default. Then over time we change call sites to call the new function.
< luke-jr> at least it's not Perl
< wumpus> you can do things like that using python + clang, but it's pretty involved, and not fun
< jtimon> ack on strongly discouraging optional arguments, we have way too many and some are really stupid IMO (ehem, ehem #9717 )
< gribble> https://github.com/bitcoin/bitcoin/issues/9717 | Pow: Remove fCheckPOW from CheckBlockHeader by jtimon · Pull Request #9717 · bitcoin/bitcoin · GitHub
< gmaxwell> then we don't end up with function signatures that are full of a mix of default and non-default arguments.
< wumpus> even if you have a c++ parser you get such a lot of information into your mutation function, so many cases to handle
< wumpus> gmaxwell: I fully agree, just don't think it makes sense as a rule
< gmaxwell> ACK
< jtimon> you can also have simpler wrappers on functions with less parameters and "default values" instead of optional arguments
< wumpus> yes
< gmaxwell> yea, thats what sipa was just saying.
< sipa> jtimon: that's what i was suggestion
< jtimon> sipa: sorry, catching up...
< wumpus> though the advantage of default arguments is that they're self-documenting in the function signature
< wumpus> anyhow, more topics?
< sipa> do we want to bring up multiwallet API again?
< jonasschnelli> yes
< gmaxwell> I don't think I've personally found that too useful except for functions with just a couple arguments with one or two simple flags. FrobThing(&thing,FrobHarder=true).
< wumpus> #topic multiwallet API
< jonasschnelli> can we decide what we want to use? path or query string and or versioning?
< wumpus> path
< jonasschnelli> I think /v1/wallet/<walletid> seems ideal
< gmaxwell> can we at least put a version in first?
< wumpus> we've decided that last time, that wallet will be based on endpoints
< luke-jr> query string <.<
< jonasschnelli> gmaxwell: yes. See above
< sipa> luke-jr: i've been thinking about the advantages and disadvantages of both
< jonasschnelli> luke-jr: what advantages do you see with query strings?
< wumpus> oh sure /v1/something is fine with me
< luke-jr> jonasschnelli: it's not hacking the path into a parameter? it's extensible?
< wumpus> if we can use /v1 as the general node endpoint hten
< sipa> query string has the advantage that it's more flexible - there could be multiple things to direct the RPC, and they don't need to be in a hierarchical pattern
< wumpus> and /v1/walletname for the wallet
< wumpus> don't use query strings with POST
< wumpus> just don't
< jonasschnelli> luke-jr: you can still add a query string later if you need non-hirarichal inputs later
< gmaxwell> path seems fine to me, so long as there is a version, but I'm clueless. I do have some questions though.
< jonasschnelli> what wumpus sais
< wumpus> POST has always been used with the idea that the parameters are in the payload
< * luke-jr> wants to hear the rest of sipa's thoughts. ☺
< wumpus> POST should not have URI parameters
< jonasschnelli> wumpus: exactly
< ryanofsky> i think query strings are analogous to command line options, paths are analogous to positional command line arguments
< sipa> however, most things that would go into the path/query string as well, can just become RPC arguments as well
< wumpus> if you confuse them, it will be hard to use from some languages
< sipa> i think that perhaps the wallet name is different, in that it's actually a different "module" you're sending it to
< jonasschnelli> sipa: indeed... but the /wallet/ endpoint could have a different rpc server once
< ryanofsky> options are better for non required things that tweak the current operation, arguments are better for required values that determine operation
< gmaxwell> I think it's likely in the future that we'll support doing this "create transaction in wallet A but also able using coins from wallet B,C"-- seems clear to me how this could work in the GUI. How could we RPC call that?
< sipa> so i think my preference is path - and for things that don't fit in the normal hierarchical structure, use named RPC arguments
< wumpus> oh no... not commands from multiple wallets
< sipa> gmaxwell: ugh...
< wumpus> can we at least agree on one object to apply commands too
< luke-jr> BTW, did we consider a generic JSON-RPC "wallet" named param?
< gmaxwell> it's not really 'from multiple wallets'.
< sipa> luke-jr: i suggested that before, yes
< wumpus> gmaxwell: in that case you should just use the raw transactions api
< wumpus> you can listunspent on multiple wallets, then use thei r utxos
< sipa> luke-jr: i'd be fine with it, except it's less compatible with a future change that moves it to a different process (which will necessarily have a new endpoint)
< jonasschnelli> luke-jr: generic wallet named parameter seems fine. Is it mixable with the non-named parameter?
< wumpus> no need to do some obscure voodoo calling a method on multiple objects on the same time
< gmaxwell> in any case, without that you are forced to make idiotic pay wallet A from wallet B transactions just to make a large payment. (or as wumpus notes, do something with raw transactions)
< jonasschnelli> I guess we then would only allow wallet selecting with name bases params?
< wumpus> that way madness lies
< luke-jr> sipa: ah, right
< sipa> so i think this guideline may help: arguments that select something that may in the future move to another process, should go into the path
< gmaxwell> wumpus: fair enough that you wouldn't want to do a ?wallet=a&wallet=b voodoo in any case there, so no interaction with this question.
< sipa> other things should be RPC named args
< jonasschnelli> I guess endpoints will also make process separation simpler
< sipa> jonasschnelli: that's what i was saying
< wumpus> sipa: +1
< jonasschnelli> Okay. Lets then go for /v1/wallet/walletID for now
< wumpus> yes
< sipa> ack
< luke-jr> v0? :P
< jonasschnelli> WalletID is the wallet filename for now... not sure if this is a problem
< sipa> luke-jr: v0 already exists :p
< jonasschnelli> v0 is not what users are expecting..
< wumpus> no, v1, v0 is what we had before :)
< luke-jr> but this isn't changing the API
< sipa> no, it's creating a new API
< jonasschnelli> it's just a prevention for future api changes
< wumpus> yes.
< gmaxwell> adding /wallet/ is a new API.
< gmaxwell> even though its a trivial one.
< wumpus> I think it's good to add it just in case, we should be able to use /v1 as endpoint for non-wallet methods
< wumpus> and / for both wallet methos and non-wallet methods (wallet will use 'the default wallet')
< luke-jr> oh, that's a point: how do we specify "no wallet"?
< jonasschnelli> Yes. /v1 should do the same / (selecting the default wallet)
< gmaxwell> oh nice, though if we're going to do that, we need to ban wallet methods on /v1/ right away.
< wumpus> gmaxwell: yes please
< sipa> how about we also add /v1/node for all non-wallet RPCs?
< wumpus> /v1 should do away with wallet methods, / should have them for backward compat
< sipa> yes
< jonasschnelli> Wait, do we want to ban non wallet methods sent to /v1/wallet/?
< jtimon> wumpus: why not use query string with POST? we do everything with POST in the rpc, there's no GET, perhaps a link I could read?
< wumpus> jonasschnelli: yes
< sipa> jonasschnelli: yes
< gmaxwell> well we have node rpcs but we also have pure function rpcs. though I don't know how good it is to bother callers with the fine details of how an rpc is implemented.
< luke-jr> :|
< jonasschnelli> Okay. That makes it a bit more complex. :)
< jonasschnelli> And not user friendly.
< jtimon> oh, the params after ?, right, sorry
< sipa> jonasschnelli: how so?
< instagibbs> we should just ban all calls, start fresh :P
< luke-jr> sipa: can't just change the endpoint for existing software..
< jonasschnelli> what about createrawtransaction?
< sipa> luke-jr: then use the old API, which keeps working
< luke-jr> sipa: then you can't select the wallet
< wumpus> jonasschnelli: well... the methods thaat have both wallet and non-wallet functionality are obviously special here
< jtimon> sipa: most things? all things I believe, even for GET
< jonasschnelli> switch over to /wallet when you call fundrawtransaction and back to /node when using signrawtransactionwithkey?
< wumpus> jonasschnelli: they could be on both, I don't care
< gmaxwell> having them track that "x is a wallet rpc" vs "y is a node" rpc is already asking a lot. and following this there should be something for pure rpcs that interact not with the node state or wallet. and so on.
< gmaxwell> I guess I'm raising the same concern as jonasschnelli
< wumpus> gmaxwell: it's a good preparation for beginning to split off the wallet though
< sipa> i see there is a usability issue in doing so
< jonasschnelli> Not sure if we can/should already seperate the calls in endpoints
< wumpus> gmaxwell: when the wallet would be a separate process it also won't have the node methods
< gmaxwell> Yes, and I think the wallet split is reasonable, I'm cautioning about what happens if you continue the logic.
< jonasschnelli> Once we have process separation, then we would need it. Yes.
< wumpus> also it's a bit starnge to have the same non-wallet methods aliased for every wallet
< jonasschnelli> We could warn in the release notes.
< wumpus> aliasing is usually a bad idea
< luke-jr> I suppose if people want backward compatibility, they can just user per-username default wallets, and have multiple users
< gmaxwell> Maybe there really are only three kinds: Things that talk to the node, things that talk to a wallet (and maybe also a node), and things that are pure functions.
< wumpus> what makes /v1/wallet/mywallet getnetworkinfo different from /v1/otherwallet netnetworkinfo
< wumpus> none has anything to do with wallets
< wumpus> gmaxwell: yes, those three kinds exist
< jonasschnelli> Yes. Agree that it looks bad.
< sipa> can you give an example of the middle one?
< wumpus> gmaxwell: we've been trying to get rid of " things that talk to a wallet (and maybe also a node)"
< sipa> oh, signrawtransaction needing UTXO access
< jonasschnelli> hm... wumpus: not sure. The wallet could have it's own network rules in future
< wumpus> sipa: getinfo
< sipa> wumpus: i don't care about getinfo :)
< gmaxwell> it's a little unfortunate to expose the three kinds to users though. decoderawtransaction and createrawtransaction are pure, signrawtransaction is not.
< wumpus> jonasschnelli: sure, then it could have a *different* getnetworkinfo
< wumpus> sipa: there are more, I've listed them in some issue, let me see
< jonasschnelli> I don't think it's wrong to have node calls in /v1/wallet/mywallet .. it allows us to – in theory – have different peering for different wallets. Also chaintips can be different.
< wumpus> sipa: #7965
< sipa> i think i'm fine with making v1 support all non-deprecated non-wallet RPCs as well
< gribble> https://github.com/bitcoin/bitcoin/issues/7965 | Remaining instances of ENABLE_WALLET in `libbitcoin_server.a` · Issue #7965 · bitcoin/bitcoin · GitHub
< gmaxwell> I think a split on anything other than wallet vs non-wallet basically exposes implementation details to users.
< bitcoin-git> [bitcoin] TheBlueMatt opened pull request #10652: Small step towards demangling cs_main from CNodeState (master...2017-06-cnodestateaccessors-1) https://github.com/bitcoin/bitcoin/pull/10652
< gmaxwell> jonasschnelli: those don't sound like very interesting things to accomplish. IMO.
< luke-jr> I think we're adding too much complexity to make 0.15 :<
< wumpus> "RPC still has some calls that vary depending on wallet support. We should split these up. This is the more annoying part as it will involve API changes. No non-wallet RPC call should make an assumption about "a wallet"."
< sipa> can we please at least remove getinfo from v1?
< BlueMatt> YES
< wumpus> YES
< gmaxwell> (and by wallet vs non-wallet, I mean a logical split where the address and transaction manipulating functions which are technically pure would still be wallet functions)
< jonasschnelli> Okay. Should the call split be in the same PR that introduces the endpoint?
< wumpus> jonasschnelli: not necessarily, small steps are fine
< sipa> jonasschnelli: i don't care
< jonasschnelli> I think it may get pretty big.
< gmaxwell> we shouldn't ship a release with /v1/ though that has a lot of calls we intend to remove, however.
< jonasschnelli> And ideally the endpoint is in 0.15 (otherwise multiwallet is not really useful)
< wumpus> at this point it's better to make progress at all
< luke-jr> let's get rid of BTC amounts while we're at it! :x
< sipa> gmaxwell: agree
< wumpus> gmaxwell: if it's in a release it means it needs to be /v2
< sipa> jonasschnelli: we have time, i think
< wumpus> gmaxwell: simple as that
< gmaxwell> luke-jr: I'd like that but switching to integers could be a v2 thing too...
< wumpus> sigh
< sipa> wumpus: sure; but i think we can merge endpoints now, and do another PR before 0.15 to remove some of the unnecessary stuff from the wallet endpoints
< gmaxwell> wumpus: well will we want to continue supporting v1 for a long time in th case?
< jtimon> I would not apps anything in the uri, just /v1/wallet/ with {"wallet": "mywalley", "otherparam" : "other value"}
< sipa> jtimon: the downside of that is that doesn't prepare downstream apps well for a future process split
< jonasschnelli> jtimon: having it in the JSON layer would make process seperation harder...
< wumpus> (sorry, just tired of the whole 'switching to integers' thing)
< sipa> wumpus: i think the current solutions which accepts strings everywhere is perfectly fine
< gmaxwell> wumpus: oh I thought other people wanted to do that, but the inability to change the api was holding it off.
< sipa> PLOINK
< jtimon> luke-jr: super ACK on finally moving from BTC to satoshi once and for all, but last time I tried I had to close the PR
< luke-jr> sipa fell in the ocean?
< instagibbs> that's dutch for "meeting over"
< wumpus> gmaxwell: I mean the problem is that JSON doesn't have integers
< wumpus> #endmeeting
< lightningbot> Meeting ended Thu Jun 22 20:01:02 2017 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< sipa> instagibbs: indeed; clearly at least wumpus got it
< gmaxwell> wumpus: yea, I thought the suggestion was integers in strings. sorry if I'm desynced there.
< BlueMatt> sorry cfields, I didnt want to hold off on PRing #10652 anymore....I only started it last dec :p
< gribble> https://github.com/bitcoin/bitcoin/issues/10652 | Small step towards demangling cs_main from CNodeState by TheBlueMatt · Pull Request #10652 · bitcoin/bitcoin · GitHub
< luke-jr> JSON has Numbers, which can be integers just fine
< wumpus> if only we used a *SANE* RPC mechanism that support uint64 values
< jtimon> I see, the reason to have the wallet id in the uri is for processes, thanks
< cfields> BlueMatt: np, great to see
< sipa> wumpus: ASN.1
< cfields> haha
< gmaxwell> wumpus: the issue that its trying to address is that some json libraries coerse json numbers to 32-bit floats and other horrors and do corrupt our values.
< wumpus> we'd haved save hours, no, days of discussion
< jonasschnelli> cfields: interested if you know why I got again dependencies issues on travis only: https://travis-ci.org/bitcoin/bitcoin/jobs/245674276#L2089
< gmaxwell> luke-jr: if you're willing to support proper handling of number we don't need to change anything-- current thing is fine.
< wumpus> gmaxwell: we currently accept strings as BTC amounts
< gmaxwell> wumpus: yes, but we don't return them.
< luke-jr> gmaxwell: current thing confuses internals with externals
< wumpus> ASN.1 hehe
< jtimon> unrelated: I would add #9271 to project 6 (libconsensus), although I don't manage to fix the tests if I change the enums for the consensus part. Removing that last part doesn't feel right either
< gribble> https://github.com/bitcoin/bitcoin/issues/9271 | Theres two types of flags: consensus and script by jtimon · Pull Request #9271 · bitcoin/bitcoin · GitHub
< gmaxwell> I'm sorry for commenting. perhaps it would just be better addressed by offering an RPC that isn't json in the future.
< cfields> jonasschnelli: will take a look
< sipa> gmaxwell: XML?
< jtimon> I gave up on verifyHeader, but not on verifyTx
< jonasschnelli> cfields: thanks!
< wumpus> gmaxwell: no, don't be sorry, I'm sorry for reacting so heavily to it, it's just that it's been discussed so many times, and every time it goes the same
< jtimon> sipa: SOAP :p
< luke-jr> wumpus: because we never change the API..
< wumpus> some people want solution A, some want B, it never changes
< gmaxwell> My skin crawls a bit about the big footgun json gives our callers here. :( (though seems luke wasn't even concerned about that)
< cfields> jonasschnelli: btw, I pinged you a few days ago for discussion about the policy change, but I guess I missed you. I suspect it's too late now to pester you about it for a min?
< wumpus> two? three? years ago I had a PR open that allows, with a command line argument, to change the encoding of amounts to four different things
< wumpus> both for accepting and returning
< wumpus> but no, no one was interested
< wumpus> we rather just keep discussing it
< gmaxwell> changing the API from a commandline seems unsafe, however. I think it came up here because we're doing api versions.
< wumpus> yeah yeah...
< wumpus> never mind
< gmaxwell> (e.g. you set the commandline one way then run joinmarket that doesn't know about that, and Bad Things Happen (tm) ... though I apologize if your PR actually addressed that.)
< wumpus> so, we could switch between just btc amount and "btc amount" and drop the integer variants, also fine with me
< * jtimon> ended up in https://zeroc.com/products/ice when talking about rpc...remembers he used it for robots to talk to computers with more computing power
< luke-jr> wumpus: that would just make the problem worse
< wumpus> no way that's dangerous as it would just result in incompatible values that wouldn't be parsed instead of 10^8 blowup
< wumpus> oh yeah of course it just makes things worse
< wumpus> OF COURSE
< gmaxwell> wumpus: Thats a good point
< wumpus> jtimon: it's not just bitcoin, no one ever agrees about RPC mechanisms, ever, anywhere :)
< luke-jr> lol
< gmaxwell> actually what effective numeric value does "1.0000" have in javascript? (a string with a 1.00 in it)
< wumpus> that's why hundreds different ones exist
< wumpus> 1.0000 if you interpret it as number
< jtimon> wumpus: right, I just remembered with gmaxwell comments what I had used i the past, I hated soap and liked ice
< gmaxwell> okay, that looks vaguely safe. I was worring for a moment that it would be 0 or something.
< wumpus> gmaxwell: javascript:alert("1.0000"+0)
< bitcoin-git> [bitcoin] achow101 closed pull request #10511: [Tests] Include branch coverage info in coverage test (master...lcov) https://github.com/bitcoin/bitcoin/pull/10511
< gmaxwell> I don't know how anyone can stand programming in this language because "blart"+0 = "blart0"
< wumpus> gmaxwell: I don't know either, it's just madness
< wumpus> but yes, that probably means that changing a number to a string is going to result in funny results instead of errors immediately
< sipa> Can I haz some review on https://github.com/bitcoin-core/leveldb/pull/2 ?
< luke-jr> sipa: but I thought we all trusted you to do the LevelDB review? :P
< sipa> luke-jr: not of my own code
< luke-jr> oh, that's not just the version bump
< wumpus> no, the version bump was already done
< wumpus> this just moves the atomic pointer code
< sipa> yes, it makes it prefer the c++11 native construct over the adhoc asm code
< wumpus> yep
< gmaxwell> sipa: unrelated to us reviewing it, but any feedback from upstream about it?
< gmaxwell> I certantly feel much more comfortable using C++ atomics.
< sipa> gmaxwell: it's even code from upstream
< sipa> i just changed the priority of choosing one over another
< sipa> not a single comment in the upstream github repo
< profall> Can two different people mine to different accounts on the same node?
< wumpus> since when can you mine on people?
< profall> Sweat shop with people hashing with pen and paper
< profall> ASIC, Miner, machine do-dad hashing thing. Want me to rephrase the statement?
< sipa> what do accounts have to do with iy
< sipa> what have nodes to do with it
< wumpus> the question just makes no sense, also this is not the place to ask support questions, try #bitcoin
< profall> If Miner #1 is mining to address ABC and Miner #2 is mining to address XZY. However, they have are using the same bitcoin core daemon.
< gribble> https://github.com/bitcoin/bitcoin/issues/1 | JSON-RPC support for mobile devices ("ultra-lightweight" clients) · Issue #1 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/2 | Long-term, safe, store-of-value · Issue #2 · bitcoin/bitcoin · GitHub
< profall> Alright, will ask there.
< sipa> profall: bitcoin core is irrelevant in this question; all core does is provide a block template to mine from
< sipa> it does not care what you do with that, where you make the payout go, or where you submit the solved block
< profall> Alright, thank you sipa
< bitcoin-git> [bitcoin] ryanofsky opened pull request #10653: Simple, backwards compatible RPC multiwallet support (master...pr/multiparam) https://github.com/bitcoin/bitcoin/pull/10653
< achow101> morcos: sdaftuar: what's the best way to get a long term fee rate (e.g. estimatesmartfee for 1008 blocks)? I need this for #10637.
< gribble> https://github.com/bitcoin/bitcoin/issues/10637 | Coin Selection with Murchs algorithm by achow101 · Pull Request #10637 · bitcoin/bitcoin · GitHub
< achow101> right now I am doing this abomination: https://0bin.net/paste/aql1DfVmBRbaZ9ks#P+ZTH6yYhbb4X-+YeCrCU/Mmt85cjNpkBvlqFvxxHFZ since it seems that estimatesmartfee() will return a fee rate of 0 if the confirmation target is out of the data range
< achow101> (getminimumfeerate in that snippet is a function that gets the fee rate with estimatesmartfee and then returns the max of that or the minrelayfee)