< jtimon> never mind, origin/0.12 already has 68/112/113
< jonasschnelli> wumpus: does "getlabeladdress" would still make sense for labels only? https://github.com/bitcoin/bitcoin/pull/7729/files#diff-df7d84ff2f53fcb2a0dc15a3a51e55ceR2506
< jonasschnelli> What if an the same label is used for multiple addresses?
< wumpus> jonasschnelli: I don't think so, I say the same in my OP :) (see the discussion with Luke-Jr in the pull)
< jonasschnelli> Okay.
< wumpus> he's using the functionality for his miner and that was my only reason to make that consession, I suggested another way of working but haven't heard back.
< jonasschnelli> Haven't seen that.
< jonasschnelli> I cloned now the wallet, made it runnable in parallel and removing accounts. I take some parts out of 7729
< wumpus> awesome!
< wumpus> well I wouldn't blame you if you don't take getlabeladdress, it makes very little sense for labels and makes it necessary to keep around a CAccount structure we'd rather get rid of. I honestly think he should find a different solution
< jonasschnelli> Yes. I have removed it.
< GitHub110> [bitcoin] jonasschnelli opened pull request #7830: [Wallet] Add cloned wallet, remove accounts, reset version (master...2016/04/wallet2) https://github.com/bitcoin/bitcoin/pull/7830
< GitHub72> [bitcoin] jonasschnelli opened pull request #7831: [CLI] refactor wallets RPC JSON conversions (master...2016/04/cli_conversion) https://github.com/bitcoin/bitcoin/pull/7831
< wumpus> for anyone that wants to do perf measurements of their own I've open sourced the lmdb experiment, be sure to read README.md https://github.com/laanwj/bitcoin/tree/2016_04_mdb
< sipa> If you manage to
< sipa> +suffer financial or other losses due to this code I demand compensation for
< sipa> +feelings of misplaced guilt.
< sipa> LOL
< wumpus> :-)
< GitHub82> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/3bc71e1572cb...bbaf5976af84
< GitHub82> bitcoin/master 07398e8 Wladimir J. van der Laan: init: allow shutdown during 'Activating best chain...'...
< GitHub82> bitcoin/master bbaf597 Wladimir J. van der Laan: Merge #7821: init: allow shutdown during 'Activating best chain...'...
< GitHub122> [bitcoin] laanwj closed pull request #7821: init: allow shutdown during 'Activating best chain...' (master...2016_04_shutdown_during_activate_best_chain) https://github.com/bitcoin/bitcoin/pull/7821
< GitHub0> [bitcoin] laanwj pushed 1 new commit to 0.12: https://github.com/bitcoin/bitcoin/commit/4226aacdba7d0e1e22555dac69363b3b460a166b
< GitHub0> bitcoin/0.12 4226aac Wladimir J. van der Laan: init: allow shutdown during 'Activating best chain...'...
< GitHub95> [bitcoin] laanwj pushed 3 new commits to master: https://github.com/bitcoin/bitcoin/compare/bbaf5976af84...1ddf0cee679d
< GitHub95> bitcoin/master 2d1d658 Pieter Wuille: Track block download times per individual block...
< GitHub95> bitcoin/master 0e24bbf Pieter Wuille: Self check after the last peer is removed
< GitHub95> bitcoin/master 1ddf0ce Wladimir J. van der Laan: Merge #7804: Track block download times per individual block...
< GitHub85> [bitcoin] laanwj closed pull request #7804: Track block download times per individual block (master...betterkicktimeout) https://github.com/bitcoin/bitcoin/pull/7804
< GitHub39> [bitcoin] laanwj pushed 1 new commit to 0.12: https://github.com/bitcoin/bitcoin/commit/90f1d246d38803eb546c6652ddce5ebea55eec98
< GitHub39> bitcoin/0.12 90f1d24 Pieter Wuille: Track block download times per individual block...
< GitHub161> [bitcoin] laanwj opened pull request #7832: Reduce block timeout to 10 minutes (master...2016_04_reduce_block_timeout) https://github.com/bitcoin/bitcoin/pull/7832
< jonasschnelli> wumpus: did you already try to run it (your LMDB branch) on a standard 64 bit Linux?
< wumpus> yep
< wumpus> passes the tests and seems to work
< jonasschnelli> Okay. Then I'll give it a try on my Debian Server (fresh sync from random peers).
< GitHub3> [bitcoin] laanwj pushed 2 new commits to master: https://github.com/bitcoin/bitcoin/compare/1ddf0cee679d...5851915a006a
< GitHub3> bitcoin/master 62b9a55 Wladimir J. van der Laan: Reduce block timeout to 10 minutes...
< GitHub3> bitcoin/master 5851915 Wladimir J. van der Laan: Merge #7832: Reduce block timeout to 10 minutes...
< GitHub61> [bitcoin] laanwj closed pull request #7832: Reduce block timeout to 10 minutes (master...2016_04_reduce_block_timeout) https://github.com/bitcoin/bitcoin/pull/7832
< jonasschnelli> wumpus: is there a reason to reserve 8GB for the chainstate2/data.mdb db?
< jonasschnelli> or is that just on my end...
< wumpus> well it's bound to not be too little, for now
< jonasschnelli> Ah. The readme was shortened on my smartphone. Nice.
< wumpus> you can patch the code to set it lower, but I didn't feel like finding the lower bound
< GitHub32> [bitcoin] laanwj pushed 2 new commits to 0.12: https://github.com/bitcoin/bitcoin/compare/90f1d246d388...cada7c2418ef
< GitHub32> bitcoin/0.12 4c3a00d Wladimir J. van der Laan: Reduce block timeout to 10 minutes...
< GitHub32> bitcoin/0.12 cada7c2 Wladimir J. van der Laan: Fill in rest of release notes
< jonasschnelli> > Only 64-bit. Sorry, life is not fair.
< jonasschnelli> hah
< wumpus> doing some last-minute tests, going to tag 0.12.0rc1 in a bit
< assder> 0.12.1rc1?
< wumpus> yes, .1
< * jonasschnelli> is slowly dust his gitian builder
< wumpus> :D
< sipa> hmm, wallet.py rpc test doesn't work here, even though i didn't change anything wallet related
< * sipa> rebases
< wumpus> on 0.12?
< wumpus> Passed here: Running testscript wallet.py ... Tests successful Duration: 62 s
< sipa> no, master
< sipa> Unexpected exception caught during testing: No JSON object could be decoded
< sipa> let me try actual master
< wumpus> seems to work fine here, too
< sipa> hmm, master fails for me
< sipa> actually, all rpc tests fail
< * sipa> blames himself
< sipa> eh, HTTP 403
< sipa> how is the test framework supposed to authenticate?
< MarcoFalke> rpcuser, rpcpassword
< wumpus> there's only one way to get forbidden: if your IP isn't allowed to connect
< sipa> ah, it writes a bitcoin.conf file
< sipa> with username/password rt
< wumpus> wrong user or password will get you HTTP_UNAUTHORIZED (401)
< wumpus> no idea how an RPC test could end up connecting through a non-allowed IP though, afaik none of the tests sets rpcallow
< sipa> 2016-04-07 12:38:02 Received a POST request for / from 192.168.44.1:58356
< sipa> why is it using my LAN address...?
< wumpus> yes that seems wrong
< wumpus> why is it binding on your LAN address
< sipa> i played with iptables a while ago, perhaps i haven't rebooted yet
< sipa> sorry for bothering you with it
< wumpus> well I'd still like to know what caused this
< sipa> iptables -t nat -A POSTROUTING -j MASQUERADE
< sipa> iptables -A FORWARD -o eth0 -m state --state RELATED,ESTABLISHED -j ACCEPT
< sipa> iptables -A FORWARD -i eth0 -j ACCEPT
< wumpus> rpc_url, the function to determine the RPC URL to connect to, hardcodes 127.0.0.1 unless variable rpchost is set - the only test crazy enough to set that is rpcbind_test.py
< wumpus> (which isn't ever started automatically IIRC)
< wumpus> so this must be a really strange network configuration indeed :)
< wumpus> maybe connections to localhost got masqueraded to your LAN address
< wumpus> probably the error "Unexpected exception caught during testing: No JSON object could be decoded" could be improved though, it shouldn't really expect a JSON object in the case of a non-OK response
< wumpus> oh fuck it does, JSONErrorReply still converts some JSON RPC errors to HTTP errors, thought that was changed at some point
< wumpus> not 401 or 403 though
< wumpus> easy solution though: actually check that "Content-Type" is "application/json" before starting decoding
< Luke-Jr> why not make it both JSON-RPC and HTTP error?
< wumpus> that's exactly what happens, JSON RPC errors are mapped to HTTP errors here: https://github.com/bitcoin/bitcoin/blob/master/src/httprpc.cpp#L75 I don't know what the JSON RPC spec says I think it's a bit of a level violation
< wumpus> in any case for this it doesn't matter, it just shouldn't be trying to parse responses with the wrong content type as JSON
< sipa> wumpus: the 0.12 travis failure is spurious, i think?
< wumpus> make[5]: Entering directory `/home/travis/build/bitcoin/bitcoin/bitcoin-i686-w64-mingw32/src/secp256k1' random seed = 7810890d4968c3f148a3cdd563177c92 err:seh:setup_exception_record stack overflow 1040 bytes in thread 0009 eip 7bc46e38 esp 00540f20 stack 0x540000-0x541000-0x740000 FAIL: tests.exe
< wumpus> wow that's one I've never seen before
< wumpus> but it must be a transient failure, yes
< wumpus> there haven't been any recent changes to secp256k1
< wumpus> (at least the one in bitcoin)
< GitHub136> [bitcoin] laanwj opened pull request #7833: tests: Check Content-Type header returned from RPC server (master...2016_04_rpc_tests_check_content_type) https://github.com/bitcoin/bitcoin/pull/7833
< wumpus> * [new tag] v0.12.1rc1 -> v0.12.1rc1
< instagibbs> a failed assert for onlyMaybeDeadlock means it's actually deadlocked?
< wumpus> instagibbs: probably not, it's a common intermittent error: https://github.com/bitcoin/bitcoin/issues/7470
< instagibbs> oh, right, i didn't see "Assertion" in the issue before. Thanks.
< sipa> why is that case classified as a potential deadlock?
< sipa> there are not even 2 locks that overlap
< sipa> cs_main is the only one shared
< instagibbs> how am I supposed to read the log?
< jonasschnelli> wumpus: F.Y.I: the lmdb node crashed.
< jonasschnelli> sync.cpp:112: void potential_deadlock_detected(const std::pair<void*, void*>&, const LockStack&, const
< jonasschnelli> LockStack&): Assertion `onlyMaybeDeadlock' failed.
< wumpus> another potential deadlock, ugh
< wumpus> that should be unrelated to lmdb though, I haven't touched any locking
< jonasschnelli> Seems unrelated to lmdb
< wumpus> yes it's the #7470
< wumpus> another such report and I'm going to just reip out the potential deadlock detection
< wumpus> travis also crashes on it frequently
< wumpus> the problem is that at some point the lock debugging was enabled by default with --enable-debug
< wumpus> ever since people have been getting this frequenltly
< instagibbs> wumpus, I just deactivated it in my config :/
< jonasschnelli> Is the crash-assert only active in --enable-debug?
< instagibbs> it was literally firing off each time my wallet rebroadcast a txn
< instagibbs> jonasschnelli, if you enable that it has it, yes
< wumpus> jonasschnell: that's the only way to enable it with autoconf, yes
< instagibbs> - CPPFLAGS="$CPPFLAGS -DDEBUG -DDEBUG_LOCKORDER"
< instagibbs> + CPPFLAGS="$CPPFLAGS -DDEBUG"
< jonasschnelli> hmm.. we should have an extra autoconf -enable for -DDEBUG_LOCKORDER...
< jonasschnelli> IIRC it once was like this.
< wumpus> it used to be that you had to manually provide `CPPFLAGS=-DDEBUG_LOCKORDER` after configure to enable it
< wumpus> yes it was
< wumpus> this seemed like a good idea at the time to get more test coverage
< wumpus> and theoretically it is
< wumpus> but not if the check is broken in the first place
< wumpus> because those things have *never* got me a deadlock when not building without -enable-debug
< wumpus> -not
< wumpus> I'd be ok with a --debug-locks though jonasschnelli :)
< wumpus> or rather --enable-debug-locks, which isn't implied by --enable-debug
< jonasschnelli> I mean we could still enable it by default..
< wumpus> nah
< wumpus> only if we can fix it
< wumpus> we know the drill by now - false alarms are worse than no alarms
< jonasschnelli> Indeed!
< gmaxwell> never having gotten wedged in practice doesn't mean there isn't a lock inversion that needs to be fixed though.
< gmaxwell> but yes, false alarms are worse than no alarms.
< gmaxwell> the historical low accessiblity of the detector means that relatively little testing has been done with it... we should probably get it out of travis but also go beat on it some.
< phantomcircuit> gmaxwell, it detects some stuff where the lock ordering is different in init.cpp from everywhere else
< phantomcircuit> which never causes a deadlock
< jtimon> sorry, the meeting is in 1 h 40 mins, right?
< jonasschnelli> jtimon: in 40mins
< jtimon> jonasschnelli: ok, thank you, I definitely needed to ask :)
< wumpus> 40 minutes, yes
< gmaxwell> phantomcircuit: we shouldn't be doing that in init.cpp then, yes it doesn't cause a deadlock -- but it undermines the simple and useful error detection here and produces false positives. An alernative would be to have the lock detector not activate until there are multiple threads... without looking at the specific cases I don't know which would be easier.
< wumpus> in my experience the deadlock detector code is hard to understand, and changes are hard to review
< wumpus> anything that makes it even harder to debug, like making it check if there are multiple threads, doesn't sound like a good idea to me
< wumpus> I think it already does that though. It keeps a lock stack per thread, and if the order of acquiring the locks differs it signals that
< wumpus> at least it used to do that
< sipa> wumpus: i'll have a look, i think i know how it works and how it should work
< wumpus> it seems pretty broken now
< wumpus> so let's disable it for --enable-debug and travis at least, keeping it as option makes sense
< wumpus> I think where it mainly fails is when try_lock is used
< morcos> wumpus: i'm not familiar with that code at all, but what you are tlaking about changing wouldn't get rid of the AssertLockHeld checks would it? b/c i think its important to keep those routinely tested
< wumpus> yes I think we should keep AssertLockHeld checks, they don't give problems
< wumpus> (I added many of them actually :)
< wumpus> it's only the deadlock detection that gives trouble, so yes makes sense to split that out
< morcos> yeah, right now they're both under -DDEBUG_LOCKORDER
< wumpus> yes they use the same underlying tracking
< wumpus> but maybe sipa can solve it :) I'd say he has enough to do though
< sipa> wumpus: the report in that bug report is wrong, it should not trigger the warning
< sipa> not even if they weren't try locks
< wumpus> yes it seems to be a false alert
< wumpus> and a lot of those happen, all with slightly different locks involved
< wumpus> if there are any real reports it's really hard to narrow down *which* locks are the problem from that output
< wumpus> it's very possible that the detection is just broken and generating spurious reports
< gmaxwell> another option is to stop using it and switch to helgrind for this purpose.
< wumpus> yes
< gmaxwell> (it wouldn't replace assert-lockheld though)
< gmaxwell> though due to performance I don't think helgrind would be usable on arm.
< wumpus> assert-lockheld works so we should just keep that, and disable the lockorder check until someone fixes it
< gmaxwell> But it helgrind will also catch a lot more than potential lock inversions.
< wumpus> yes the drawback of the *grind tools is the performance loss involved in instrumenting the code
< gmaxwell> seems people have not been running helgrind on the codebase.
< wumpus> (-DDEBUG_LOCKORDER also adds overhead, but only on aquiring/releasing locks)
< gmaxwell> and the overhead of being wrong and not getting used? :P
< wumpus> yes and the overhead of tons of open issues on the repository
< * btcdrak> rings the bell
< wumpus> #startmeeting
< lightningbot> Meeting started Thu Apr 7 19:00:58 2016 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
< lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
< wumpus> PSA: 0.12.1rc1 has been tagged, please start your gitian builders
< jonasschnelli> I have three topic proposals
< jonasschnelli> (1) How to proceed with cores wallet (I have a proposal)
< jonasschnelli> (2) CoreDev Hacking convention in Zurich (short announcement)
< jonasschnelli> (3) Dealing with RBF RPC/UI
< petertodd> jonasschnelli: ack, and I need an excuse to visit Zurich :)
< jonasschnelli> http://coredev.tech
< jonasschnelli> (sketch)
< wumpus> for the last meeting we have ACTION: start generating mtp violating transactions (gmaxwell) (wumpus, 19:08:21)
< wumpus> ACTION: #7543 has 5 tested ACKs so far. It should be ready for merge (wumpus, 19:22:52)
< wumpus> that was done
< gmaxwell> RE: mtp violations, I did. I generated a number... none were mined. I am still working on this-- I suspect I need to relay harder.
< wumpus> ACTION: disable bad-chain alert for 0.12.1 (wumpus, 19:38:39)
< wumpus> that was also done
< wumpus> ACTION: disable bad-alert detection in master if it doesn't improve enough by 0.13 (and don't forget!) - well I tagged dgenr8's issue for 0.13, not sure what the progress there was
< wumpus> ok let's start with jonasschnelli's topics
< wumpus> #topic How to proceed with cores wallet
< btcdrak> wumpus ack
< jonasschnelli> My proposal for a concept: extend #7830: copy the wallet, remove backward compatibility (not required), remove accounts, replace BDB with LogDB, add BIP32, add SPV (= process separation)
< jonasschnelli> It would help to merge it as soon as it is usefull (remove accounts and add label RPC calls, reset wallet-version)
< jonasschnelli> If we agree on the concept. I'm happy to write tests, etc.
< jonasschnelli> So we can ensure that the second wallet runs smoothly along with the main wallet
< gmaxwell> This sounds conceptually okay to me, though how will we deal with improvements that flow to the current wallet? apply them twice (as applicable)?
< wumpus> sounds good to me, we're kind of held up on updates on the current wallet, if you want to work on a new one that makes sense
< jonasschnelli> gmaxwell: Yes. Important improvments could be applied on both wallets.
< btcdrak> jonasschnelli: sounds great
< gmaxwell> I think it's much better than a boil the ocean rewrite.
< jonasschnelli> The second wallet should not have a stable API for now.
< jonasschnelli> And we could merge more aggressively there.
< sipa> well if you intentionally break things in it, it may not get sufficient testing
< jonasschnelli> I would also be happy to be the maintainer of that second / currently experimental wallet.
< sipa> but concept ack
< wumpus> or maybe it will get sufficient testing
< wumpus> some people are waiting for things to be broken, especially the account system
< wumpus> but any update to the current wallet goes so slow, partially because every time backwards compatiblity has to be considered
< gmaxwell> it probably won't get 'sufficient' testing for now. But thats okay. It'll get some toying around with, which will be good feed back-- but most importantly we can make incremental progress.
< wumpus> so it's kind of circular
< gmaxwell> we really will need to up the quality and rigor of wallet tests for new functionality there; right now our testing for the wallet (what exists) has many checks that check exact behavior, which makes development hard. That kind of test can be tolerable for consensus code... it's a huge burden for wallet code.
< jonasschnelli> Yes. We could mark it as experimental, don't use it with large amounts.
< wumpus> also updates to the current wallet don't get sufficient testing/review either, there's just no energy behind it
< morcos> is the idea that every important change to the existing wallet will need to be replicated though? if so we maybe shouldn't stay in this state too long. or someone has to volunteer to do those copies.
< sipa> was just about the bring that up
< wumpus> morcos: well I expect them to diverge soon enough that that won't say a problem for long
< GitHub118> [bitcoin] sdaftuar opened pull request #7835: Version 2 transactions remain non-standard until CSV activates (master...CSV-relay-after-softfork) https://github.com/bitcoin/bitcoin/pull/7835
< phantomcircuit> gmaxwell: i can help you with relaying harder
< jonasschnelli> morcos: I think as we proceed, most features will only make sense on one or the other side.
< sipa> we can't put the burden on external contributors to duplicate
< wumpus> features will mostly end up in the new wallet
< wumpus> the old wallet should still receive bugfixes etc
< wumpus> it's a bit like stable versions/master
< morcos> hmm.. lots of things would apply to both, conflicts, abandoned transactions, fee estimates
< jonasschnelli> wumpus: +1
< wumpus> in any case I'd say this is up to the people doing the work
< morcos> wumpus: oh ok. that makes sense, but when would the target be for the new wallet to be production ready
< morcos> 0.14
< morcos> ?
< wumpus> morcos: when it's ready
< jonasschnelli> morcos: sure. But the second wallet should once be independent from the core. So we should work towards a clear interface.
< morcos> i guess i dont' wnat us to be in a state where the old wallet has deteriorated due to lack of attention and the new wallet is not yet stable
< gmaxwell> wumpus: things like the recent performance improvements would apply to both; (sufficiently) poor performance is a bug. ... I think doing this will have a _serious_ cost, but the alternative of continuing to not advance in that area of the software is worse.
< morcos> we need something that is good to use
< jonasschnelli> New wallet without account and without BDB could be in 0.13, stable API in 0.15, ... non-beta in 0.16
< morcos> right, i'm not opposed to doing this, i think we need to do something
< wumpus> morcos: it's a risk, but you can't stop people from working on a new wallet if that's what they prefer
< sipa> yes, i think we should aim to have the new wallet production ready soon, but just no stable onterface
< wumpus> (or maybe jonasschnelli and me should start a fork of bitcoin core *ducks*)
< sipa> *interface
< jonasschnelli> wumpus: I did that already... but the amount of reviewers was... lets say ... extremely tiny.
< jonasschnelli> We can't say SPV is doomed and not improvig the wallet at the same time.
< sipa> also, a lot of the unit tests for non-wallet features rely on having a wallet
< wumpus> yes, the unit tests should be less dependent on the wallet
< wumpus> but that's a different issue
< jonasschnelli> Yes. Hard to send around coins then. But agree.
< sipa> iwe can replicate a wallet in the python framework *ducjs*
< wumpus> (well, the non-wallet unit tests). In any case the unit tests can use the old wallet for now.
< petertodd> sipa: please do; python-bitcoinlib needs a wallet :)
< wumpus> sipa: yes :)
< wumpus> a python wallet please
< wumpus> :D
< sipa> the why do we need a wallet in core? ;)
< btcdrak> !
< paveljanik> for testing...
< wumpus> well one idea was to begin a new wallet outside of core
< jonasschnelli> I agree long term we could remove the wallet... but in tiny steps.
< * gmaxwell> bangs gavel
< sipa> ok
< sipa> who is gavel?
< jonasschnelli> Outside of core does not work for now.
< sipa> indeed
< wumpus> but anyhow the work is currently in the direction of adding a new-wallet in core, so I'd just like to support that
< wumpus> if you want to create a wallet outside of core no one needs any permission at all from anyone here :p
< jonasschnelli> As soon as the wallet can run SPV, we can think about moving it.
< btcdrak> sipa: do I need to call an ambulance?
< paveljanik> can't just work on the interaface/API for wallet and the new wallet to be outside of Core?
< sipa> jonasschnelli: i think it will still share a large part of the codebase, even if it's not runnijg in the same process
< wumpus> sure, feel free to do what you want to do outside of core paveljanik
< petertodd> jonasschnelli: why worry about SPV? write one that scans full blocks, grabbing them from a local bitcoind
< jonasschnelli> sipa: Yes. Once it can run SPV, it shares some base code, also the net stuff.
< sipa> jonasschnelli: please, modularization first, separate process next, and then we can start talking about other repository
< paveljanik> wumpus, that nees Core to provide API/something Jonas is already working on...
< wumpus> it makes sense, e.g. joinmarket has a wallet outside of core, although it's somewhat suboptimal
< sipa> petertodd: that's the idea
< sipa> petertodd: spv does not imply bip37
< wumpus> (still relies on the wallet inside core but using watch-only)
< jonasschnelli> I think it's to early to remove the wallet from core. We can think about it in 2-3 years.
< wumpus> jonasschnelli: yes
< wumpus> no one is actually proposing doing that now, but it's always how this discussion goes
< wumpus> and has gone for years
< morcos> would it be helpful to try to document the plan a bit more clearly (not the whole future long term plan, but exactly what jonas is going to be workign on and how we can best support him)
< morcos> in an issue or something
< jonasschnelli> petertodd: Right. Running Core in full-block SPV is easy. Adapting the wallet so its not depending on a mempool, etc. is a bit harder.
< wumpus> any progress you make is helpful jonasschnelli
< btcdrak> next topic?
< sipa> agree
< jonasschnelli> morcos: I'll write a proposal.
< jonasschnelli> agree next t.
< sipa> (my battery is at 11%)
< morcos> i'm just worried about falling into a position where the new wallet is not ready fast enough and needed improvements / bug fixes / etc are sometimes applied to one and sometimes the other and sometimes both
< wumpus> #topic CoreDev Hacking convention in Zurich (short announcement)
< morcos> the plan doesn't have to be perfect, but it helps if its clear
< wumpus> morcos: it's a risk, but it's how it goes in open source
< jonasschnelli> I think we could all meet together and hack at some important stuff.
< jonasschnelli> http://coredev.tech
< gmaxwell> jonasschnelli: please just propose things for the short/mid term for now.
< phantomcircuit> petertodd: spv in this context really just means that the spv proofs are being saved such that the wallet could operate in a true spv mode, not that it will
< jonasschnelli> Best would be to get SW nailed at the meeting in ZH.
< wumpus> awesome jonasschnelli
< jonasschnelli> I'm also convinced if we do that, we find sponsors pretty easy.
< jonasschnelli> Room, food, etc. is organized.
< jonasschnelli> If someone needs travel subsidy, please tell me.
< morcos> i like the idea, but unfortunately can't attend that weekend
< jonasschnelli> Important: Because of the short timelime, please tell me if you are interested to attend during the next 7 days.
< sipa> is the date fixed?
< wumpus> sure I will
< jtimon> I probably will as well
< jonasschnelli> Date is semi-fixed. :)
< petertodd> jonasschnelli: I can attend
< sipa> i can attend
< sipa> (13 minute tram ride)
< sdaftuar> jonasschnelli: i'm interested, though not sure yet if i can make it
< jonasschnelli> Hah.
< wumpus> i can attend too, nothing else on that date at laest
< jonasschnelli> Its soon. Please try to give me a yes/no during the next 7 days.
< jonasschnelli> Okay. I'll flag wumpus, petertodd and sipa as "confirmed".
< jonasschnelli> and jtimon (he lives in spain and needs to come!)
< gmaxwell> If you want people to come, more advanced planning would help, in the future! :)
< jtimon> jonasschnelli: yeah, unless something unexpected prevents me from going, sounds great to me
< jonasschnelli> gmaxwell: Agree. I just can't in June/Juli/Aug!
< jonasschnelli> If it wont work, I'll organize another one in fall.
< petertodd> gmaxwell: heh, I feel lucky when I have a whole two months of notice to travel halfway around the world - a week is more common :)
< btcdrak> jonasschnelli: seems like you have a lot of takers already
< sipa> petertodd: agree
< jonasschnelli> And Zurich is great! :-)
< jtimon> never been there, happy to visit it
< petertodd> I was in Zug for a week, and it was so beautiful that a cup of coffee cost $10
< jonasschnelli> </topic>
< wumpus> hehe
< sipa> next topic?
< wumpus> #topic Dealing with RBF RPC/UI
< jtimon> maybe I've been in the airport...sorry, yeah, next topic
< michagogo> (null) <jonasschnelli> New wallet without account and without BDB could be in 0.13, stable API in 0.15, ... non-beta in 0.16
< michagogo> Sorry I'm late, but: Bitcoin Core is still in beta, no?
< sipa> haha
< petertodd> speaking of, opt-in RBF is getting used by someone at least: http://p2sh.info/dashboard/db/replace-by-fee
< jonasschnelli> RBF: I dont have a clear vision how the RPC should deal with RBF
< gmaxwell> michagogo: the "beta" was dropped from the description years ago.
< michagogo> Huh, that timestamp broke weirdly
< michagogo> gmaxwell: really? I must have missed that
< gmaxwell> michagogo: and some stupid lable wouldn't excuse shoddy support.
< michagogo> gmaxwell: obviously, I was just commenting on the "non-beta" milestone
< jonasschnelli> Opt-in is one issue to deal with, the second one is: how does the process looks like if someone adds an output/input?
< jonasschnelli> Ensure that the RBF rules are respected (>fee, pay for the bandwith, etc.)
< petertodd> so, I think what you actually need to do here from an RPC point of view is think in terms of what addresses the user wanted to pay, and whether or not known (confirmed) txs succesfully did that - which isn't really how the wallet works right now
< MarcoFalke_> Shouldn't we only support fee bump as a fist step?
< petertodd> MarcoFalke_: ack
< paveljanik> MarcoFalke: +1
< sipa> ack
< jonasschnelli> *exhales*
< gmaxwell> In many ways, adding outputs is more useful... but "fee bump only" was also my initial impression at jonasschnelli's efforts.
< phantomcircuit> petertodd: the wallet does track which outputs are change, so nominally it's also trakcing which outputs where intended as payments
< jtimon> one step at a time sounds good to me
< jonasschnelli> I think we should have a fee-bump RBF in 0.13 (RBF was in 0.11.x and 0.12).
< petertodd> MarcoFalke_: and with fee bump, first implementation should probably *not* add inputs, which complicates things
< jonasschnelli> | How would a feebump over RPC looks like?
< petertodd> jonasschnelli: hmm? I released RBF for v0.11, but Core didn't
< jonasschnelli> sry, mixed up with CLTV... right. 0.12
< petertodd> jonasschnelli: https://github.com/petertodd/replace-by-fee-tools/blob/master/bump-fee.py <- there's my implementation, with is basically bumpfee <txid> <ratio/delta-fees> => <new txid>
< gmaxwell> for adding outputs, I think the best way involves some larger workflow changes. E.g. the ability to queue sends, and auto-sendmany from the queue.. then that could be extended to auto-add-outputs for queued sends where the original is sent.
< gmaxwell> or something like that.
< jonasschnelli> petertodd: bumpfee ... yes. maybe we find a call-name that is more flexible for the future?
< petertodd> jonasschnelli: I didn't handle the case where you'd respent the change, but just having fee bumping error out in that case isn't all that bad; a slightly smarter version could combine the dependent txs (assuming they're all yours)
< sipa> related: cpfp?
< gmaxwell> RE: feebump, I think a different approach should be used.
< phantomcircuit> petertodd: there's significant privacy implications to combining payments
< petertodd> jonasschnelli: AbstractRespendWithSomeThingChangedFactoryBean?
< jonasschnelli> ^^
< petertodd> phantomcircuit: sure, which is why getting estimates right in the first place helps a lot
< sipa> jonasschnelli: it shall be called BeeFump
< petertodd> phantomcircuit: but if fee bumping is a once-in-a-while thing, I'm not that concerned re: privacy
< jonasschnelli> altertransaction was something we once have talked about
< Lightsword> petertodd, your factory needs a factory :P
< petertodd> sipa: +1
< gmaxwell> Instead of having a command to 'bump' we should implement "adaptive fee" where it just precreates the bumps with nlocktimes and queues them. Expecting the user to always need to manually bump will not make for a good expirence.
< sipa> gmaxwell: elaborate?
< sipa> ah
< petertodd> Lightsword: I'm not going to grey-goo the world just to bump fees... :)
< wumpus> sure, that'd be even better, but even a simple 'bump fee' command would be better than nothing
< petertodd> gmaxwell: yeah, ack on that being better, although the code to do feebumping will get reused by the nlocktime version
< gmaxwell> Also, for a direct manual bumprpc. it sould probably have an api that encourages a multiplicative increase. Since otherwise you can end up needing to bump stupid numbers of time.
< jonasschnelli> Okay. Will work on "bumpfee".
< jonasschnelli> How do we estimate fees for replacements?
< petertodd> gmaxwell: my python script does it based on a ratio
< kanzure> #action bumpfee work
< petertodd> jonasschnelli: double every time by default? it's a good start
< gmaxwell> where as 1.5x each time (subject to the protocol constraint) can span all possible values is a fairly small number of bumps with a maximum overpayment of 50%.
< petertodd> jonasschnelli: having to bump a fee implies that your fee estimation isn't working anyway :)
< jonasschnelli> petertodd: indeed!
< wumpus> exponential fee backoff
< gmaxwell> petertodd: well it could be working now, just not precognative. :)
< phantomcircuit> gmaxwell: we'd need to be very careful with automatic fee bumping to ensure that people are aware of it and expect that behavior
< gmaxwell> unrelated, it would be kinda cool to run the fee estimator on all unconfirmed txn in the mempool and display the current estimate on them.
< paveljanik> BTW - when we bump fee, will we just lower the change? Or different way to do so to not help Sybil-network-attackers to identify the change?
< wumpus> phantomcircuit: agreed
< gmaxwell> phantomcircuit: jonasschnelli had a checkbox in the UI for the things he was doing so far.
< petertodd> paveljanik: just lowering the change is by far the easiest; doing better re: privacy is always going to be more expensive, and tricky to be succesful at
< jonasschnelli> Yes. We don't opt-in automatically for now I guess.
< gmaxwell> paveljanik: the main thing to do is to get the estimate right in the first place, though right now change is so throughly identifyable you're closing the barn door after the horse has left. :)
< wumpus> phantomcircuit: automatic behavior in the wallet can be somewhat scary, at least now you have to click confirm to pay a certain fee
< wumpus> phantomcircuit: (although you could have the same, just agree on a certain *max* fee)
< gmaxwell> wumpus: what I suggested on the PR for that was just to list the possible fees (by the checkbox)
< jonasschnelli> wumpus: Agree. No automatic behavior that spends inputs automatically.
< btcdrak> I would just have "increase fee" and give an amount.
< gmaxwell> Fee of x/y/z/q after 0/2/3/4 blocks.
< morcos> Ok so not to beat a dead horse here, but just b/c i want to understand. This kind of change to wallet behavior is something would be only added to new wallet or to both?
< jonasschnelli> I though we could re-use the fee slider (but with ~x2 values).
< petertodd> morcos: fee bumping itself I think we should add to the existing wallet; more complex strategies may be infeasible with the current wallet
< wumpus> morcos: depends on the order in which things actually get implemented, I'd say
< wumpus> yes agree with petertodd there
< gmaxwell> morcos: for example the queuing behavior I described above may be unrealistic to do in the current wallet because all our apis expect to return txid.
< jonasschnelli> morcos: the old wallet could just have a fee bump, ..the new wallet a feature to add outputs.
< gmaxwell> I think new wallet API should break that and instead return a handle, that can be used to get the txid if it's available yet.
< jtimon> jonasschnelli: will the new and the old wallet share a common interface (even if one of them just throws a non-implemented error on some methods or something)?
< sipa> jtimon: imho, no
< sipa> or not initially
< wumpus> the idea is tthat the new wallet can have a completely new interface
< wumpus> throwing out all the old cruft
< jonasschnelli> jtimon: if we want to depracate the old wallet and remove it once, it should probably be independent.
< wumpus> of course if there are things that make sense they can be kept
< morcos> it seems to me that there ought to be an rpc call to jus tincrease the fee by an amount (as btcdrak says) and then there ought to be an rpc call/gui option that says expedite which replaces the fee on an existing transaction wiht the new estimates (assumign the new estimate is higher), you may have changed the target.
< jonasschnelli> Also while writing on the new wallet, we can care more about core interaction. No mempool access everywhere.
< sipa> jonasschnelli: that may be... hard
< wumpus> jonasschnelli: at least make it event driven
< jtimon> mhmm, I'm just worried about calling code for both wallets from the same place
< wumpus> jonasschnelli: no assumption on *immediate* mempool access
< gmaxwell> jonasschnelli: many good features benefit from mempool access.
< sipa> jtimon: is that a problem?
< wumpus> the mistake we made in the GUI is to do a lot of calls directly, which makes it very hard to move things to another process or make them network transparent
< jtimon> with my suggestion the new wallet could have a different interface in practice (as said the old wallet can just throw errors)
< wumpus> and it also holds up the GUI thread for things that shouldn't
< sipa> hey, main.cpp used to call the gui directly
< jonasschnelli> :-)
< wumpus> it's fine to have a query mempool, but it should be regarded as asynchronous
< wumpus> heh true sipa, we made a step forward with bitcoin-qt :-)
< sipa> other topic?
< petertodd> wumpus: once you're talking about async access, probably easier just to keep a copy of the mempool on your local app
< sipa> as we seem to have backtracked to a previous o e
< wumpus> better to have something then make a wild plan which grows so huge it can never be executed
< jtimon> sipa: mhmm, seems that code of the form if(fSomething) { // call old wallet } else { // call new wallet} doesn't really make removing the old wallet easier in the future
< wumpus> petertodd: I don't know, depends on the application
< sipa> jtimon: oh hell no, that isn't allowed
< sipa> jtimon: everything through validationinterface
< petertodd> wumpus: well, unless you're memory constrained, keeping a copy locally and keeping that in sync can't be that hard
< jtimon> sipa: oh, that was my question then, thanks
< wumpus> jtimon: yes, wallets (as well as other modules) register their own RPC calls, there's no need for if(fSomething) logic
< wumpus> any other topics?
< jtimon> great, that was my only concern about jonasschnelli's plan, I knew I was probably misunderstanding something
< jonasschnelli> I'll write a clean concept.
< sipa> oh, small question: are we ok with backporting master's script/tx unit tests to 0.12?
< petertodd> sipa: ack
< paveljanik> why not?
< wumpus> well if the tests are better it always makes sense to backport them
< sipa> testd are not typically something we backport
< wumpus> I'm fine with copying all the tests from master to 0.12 given that the funcitnality is supported
< sipa> but i think it makes total sense as we want softforks backported too
< sdaftuar> wumpus: i wanted to flag #7835 which i just opened for 0.12.1
< petertodd> sipa: agreed
< wumpus> sdaftuar: too late, 0.12.1 was just tagged
< sipa> sdaftuar: just rc1?
< sipa> wumpus: ^
< wumpus> yes
< sipa> or no rc cycle?
< btcdrak> Freudian slip
< jtimon> well, as a not important topic, I would appreciate some feedback on a little experiment in #7829 . I still need to improve the PR description, but the basic idea is helping new people to get their first PR merged and get familiar with git, the review process or whatever, by telling them to do stuff I want done
< MarcoFalke_> lets hope rc1 one is the final release ;)
< wumpus> [21:01:15] <wumpus> PSA: 0.12.1rc1 has been tagged, please start your gitian builders
< morcos> right, so its not too late for 0.12.1 right?
< btcdrak> 20:01:47 <wumpus> PSA: 0.12.1rc1 has been tagged, please start your gitian builders
< wumpus> well there will only be a rc2 if necessary, does this need to block the release?
< sdaftuar> i think so. i think there's a problem with the mempool being able to be filled with things that may not be mined
< sipa> wumpus: we should consider it i thonk
< sdaftuar> without the change, which is simple
< wumpus> ok...
< sipa> sowwy
< wumpus> rc1 is dead folks
< wumpus> don't bother building it
< * btcdrak> stops his gitian builder
< wumpus> long live rc2
< btcdrak> I thought we had discussed this exact thing before regarding V2
< morcos> wumpus you know i save these things until after your release candidates just for the entertainment value right?
< wumpus> morcos: I know, no offense taken :) better now than later, almost no one wasted cycles on rc1 yet
< morcos> btcdrak: yeah i dont' think we properly thought through what happens when things end up in your mempool that won't be reliably mined
< btcdrak> sdaftuar: that's quite an elegant solution.
< sdaftuar> btcdrak: that's because i stole it from sipa's segwit
< wumpus> #action review and ACK https://github.com/bitcoin/bitcoin/pull/7835/files asap
< btcdrak> long live sipa
< morcos> we should add this to our list of standard ways to implement things whenever we relax standardness rules
< btcdrak> morcos: agreed
< sipa> 2% battery left for me; anything else?
< morcos> travel charger?
< jonasschnelli> lol
< wumpus> I don't think so
< wumpus> #endmeeting
< lightningbot> Meeting ended Thu Apr 7 19:55:45 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< achow101> wait, is rc1 actually dead?
< wumpus> yes, dead as a doornail
< btcdrak> "it's dead Jim"
< wumpus> we'll probably tag rc2 tomorrow
< achow101> ok
< wumpus> doesn't make sense to do a gitian process in the mean time