< bitcoin-git> [bitcoin] MarcoFalke pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/8e00a6855240...761fe07ba9b5
< bitcoin-git> bitcoin/master 490da63 Kristaps Kaupe: Make lint-includes.sh work from any directory
< bitcoin-git> bitcoin/master 761fe07 MarcoFalke: Merge #16768: test: Make lint-includes.sh work from any directory
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #16768: test: Make lint-includes.sh work from any directory (master...lint-includes-anydir) https://github.com/bitcoin/bitcoin/pull/16768
< midnightmagic> ah. so the code isn't old. it's pre-warning in gcc saying we're doing it right now, lucky us. what a bizarre warning to emit.
< sipa> if you're linking pre-gcc-7.1 and gcc-7.1 compiled code, things will break
< sipa> the compiler can't know whether you're doing that or not
< sipa> generally ABIs don't change like that
< midnightmagic> Ah, an ABI thing. Okay less weird to me. Thank you.
< bitcoin-git> [bitcoin] MarcoFalke pushed 5 commits to master: https://github.com/bitcoin/bitcoin/compare/761fe07ba9b5...45be44cce4fa
< bitcoin-git> bitcoin/master b21680b Ben Woosley: test/contrib: Fix invalid escapes in regex strings
< bitcoin-git> bitcoin/master 8389207 Ben Woosley: lint: Disable flake8 W504 warning
< bitcoin-git> bitcoin/master 0ef0e51 Ben Woosley: lint: Bump flake8 to 3.7.8
< bitcoin-git> [bitcoin] MarcoFalke merged pull request #15257: Scripts and tools: Bump flake8 to 3.7.8 (master...flake-36) https://github.com/bitcoin/bitcoin/pull/15257
< luke-jr> I don't think we use it, but just in case, GCC miscompiles PPC's hardware random to repeated values: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=91481
< coinmonks> Hey Michael, My name is gaurav,. I run Coinmonks (https://medium.com/coinmonks) publication..
< coinmonks> I also run Coincodecap.com where we track crypto based on their Github activity
< coinmonks> We started a series "Developers in crypto" .. we want to mention you in our blog.. and we have few question
< coinmonks> Can you please help us with them?
< coinmonks> Your background?
< coinmonks> When and how you get involved in Bitcoin?
< coinmonks> What are your main contributions on Bitcoin ecosystem?
< coinmonks> What are new tech innovations you introduced on Bitcoin?
< coinmonks> Any interesting story you might want to share about contributing on Bitcoin?
< jouke> What are your private keys?
< coinmonks> Shit ..I didn't realise I was typing this on general chat :)
< wumpus> heh
< coinmonks> anyone from India here?
< bitcoin-git> [bitcoin] MarcoFalke opened pull request #16806: doc: Add issue templates for bug and feature request (master...master) https://github.com/bitcoin/bitcoin/pull/16806
< bitcoin-git> [bitcoin] laanwj pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/45be44cce4fa...cbde2bc80674
< bitcoin-git> bitcoin/master fae91a0 MarcoFalke: test: Remove incorrect and unused try-block in assert_debug_log
< bitcoin-git> bitcoin/master cbde2bc Wladimir J. van der Laan: Merge #16804: test: Remove unused try-block in assert_debug_log
< bitcoin-git> [bitcoin] laanwj merged pull request #16804: test: Remove unused try-block in assert_debug_log (master...1909-testFix) https://github.com/bitcoin/bitcoin/pull/16804
< bitcoin-git> [bitcoin] laanwj pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/cbde2bc80674...5667b0d758f1
< bitcoin-git> bitcoin/master 2457aea Samuel Dobson: Assert that the HRP is lowercase in Bech32::Encode
< bitcoin-git> bitcoin/master 5667b0d Wladimir J. van der Laan: Merge #16792: Assert that the HRP is lowercase in Bech32::Encode
< bitcoin-git> [bitcoin] laanwj merged pull request #16792: Assert that the HRP is lowercase in Bech32::Encode (master...201909_bech32_hrp_check) https://github.com/bitcoin/bitcoin/pull/16792
< bitcoin-git> [bitcoin] meshcollider opened pull request #16807: Let validateaddress locate error in Bech32 address (master...201909_bech32_error_detection) https://github.com/bitcoin/bitcoin/pull/16807
< bitcoin-git> [bitcoin] meshcollider pushed 5 commits to master: https://github.com/bitcoin/bitcoin/compare/5667b0d758f1...5e202382a987
< bitcoin-git> bitcoin/master a31be09 Antoine Riard: Encapsulate tx status in a Confirmation struct
< bitcoin-git> bitcoin/master 7e89994 Antoine Riard: Remove SyncTransaction for conflicted txn in CWallet::BlockConnected
< bitcoin-git> bitcoin/master 40ede99 Antoine Riard: Modify wallet tx status if has been reorged out
< bitcoin-git> [bitcoin] meshcollider merged pull request #16624: wallet: encapsulate transactions state (master...2019-08-encapsulate-tx-state) https://github.com/bitcoin/bitcoin/pull/16624
< meshcollider> review beg for #15450
< gribble> https://github.com/bitcoin/bitcoin/issues/15450 | gui: Create wallet menu option by achow101 · Pull Request #15450 · bitcoin/bitcoin · GitHub
< meshcollider> its already on high priority list
< meshcollider> I'd like to merge it tomorrow but it'd be great to have at least one more review or tester
< jnewbery> #proposedmeetingtopic: review/merge #16704 or #16713 to remove worrying "unknown new rules activated (versionbit 1)" warning
< gribble> https://github.com/bitcoin/bitcoin/issues/16704 | Dont warn about activated buried BIP 9 deployments by achow101 · Pull Request #16704 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/16713 | logging: Redefine CSV and segwit deployments to avoid unknown softforks warning by jnewbery · Pull Request #16713 · bitcoin/bitcoin · GitHub
< fanquake> meshcollider: just make sure you check/close/merge the base PR as well. I assume it’s still the same in create wallet. Has 1 ACK I think.
< meshcollider> fanquake: I see yeah, why is the base PR not on the high priority list as well/instead?
< fanquake> meshcollider: it is
< fanquake> #16261
< gribble> https://github.com/bitcoin/bitcoin/issues/16261 | gui: Refactor OpenWalletActivity by promag · Pull Request #16261 · bitcoin/bitcoin · GitHub
< meshcollider> Oh I totally missed that, sorry
< meshcollider> Yeah I saw the PR just didn't see it on the list for some reason
< aj> jnewbery: https://github.com/ajtowns/bitcoin/commits/201909-unknown-softforks -- i would've thought something like that would make more sense than reinstating csv/segwit into vDeployments?
< aj> jnewbery: (also, shorter :)
< jnewbery> aj: looks good to me!
< aj> jnewbery: if you like it, put a PR on it? :)
< aj> jnewbery: i mean, feel free to merge it into your PR :)
< jnewbery> aj: testing now
< aj> jnewbery: thinking about it, could just set it to 0 for regtest (since no historical versionbit stuff for segwit etc needed there); which means it could be set straight after the buried heights not after -segwitheight is worked out
< jonatack> meshcollider: re-reviewing #15450 now
< gribble> https://github.com/bitcoin/bitcoin/issues/15450 | gui: Create wallet menu option by achow101 · Pull Request #15450 · bitcoin/bitcoin · GitHub
< stevenroose> Is there already a C++ implementation of the taproot tagged hashes? I'm looking for some example values to test an implementation against.
< sipa> note that nothing about taproot is final
< sipa> (or even guaranteed to make it in)
< stevenroose> sipa: I realize, but it doesn't hurt to start experimenting with some implementation already :)
< sipa> of course
< fjahr> I would like to ask for feedback on my proposal for a rolling UTXO set hash (originally proposed by sipa) at the meeting today. If possible please take a look at my write up: https://gist.github.com/fjahr/fa4892874b090d3a4f4fccc5bafa0210
< fjahr> #proposedmeetingtopic Rolling UTXO set hash
< achow101> #proposedmeetingtopic avoid loading every wallet transaction into memory
< sipa> fjahr: cool!
< jnewbery> aj: done: #16713
< gribble> https://github.com/bitcoin/bitcoin/issues/16713 | logging: Redefine CSV and segwit deployments to avoid unknown softforks warning by jnewbery · Pull Request #16713 · bitcoin/bitcoin · GitHub
< jnewbery> thanks
< instagibbs> any trick to these travis timeouts im seeing
< instagibbs> 10 minutes no output
< sipa> yeah i'm seeing it too
< instagibbs> I used to get these with some secp-zkp stress tests on 32 bit arch, first time seeing them anywhere else really :(
< BlueMatt> last-ditch attempt at https://github.com/bitcoin/bitcoin/pull/16421 for 0.19....already has 2.5 acks....
< bitcoin-git> [bitcoin] MarcoFalke closed pull request #16704: Don't warn about activated buried BIP 9 deployments (master...buried-versionbits-cache) https://github.com/bitcoin/bitcoin/pull/16704
< sipa> i don't understand this appveyor error:
< sipa> c:\projects\bitcoin\src\script\miniscript.cpp(274): error C2220: warning treated as error - no 'object' file generated [C:\projects\bitcoin\build_msvc\libbitcoin_common\libbitcoin_common.vcxproj]
< sipa> 6>c:\projects\bitcoin\src\script\miniscript.cpp(274): warning C4101: 'error': unreferenced local variable [C:\projects\bitcoin\build_msvc\libbitcoin_common\libbitcoin_common.vcxproj]
< sipa> it'd be nice to know what variable is unreferenced
< bitcoin-git> [bitcoin] GChuf opened pull request #16808: GUI: fix and stylize language list (master...translation-list-fix) https://github.com/bitcoin/bitcoin/pull/16808
< MarcoFalke> the travis timeouts when running apt update in docker are known for months
< MarcoFalke> I have a ticket open with them, but no real reply or solution
< sipa> seems i haven't been submitting many PRs lately :)
< MarcoFalke> Sometimes the warnings don't come for a week or two, but at some point they are back ..
< MarcoFalke> I think you can fix it by removing ' error' from that line
< sipa> huh
< sipa> oops, i was looking in the wrong branch :(
< sipa> thanks
< jb55> I might have missed the convo but has anyone looked at github actions for builds?
< sipa> yes
< bitcoin-git> [bitcoin] dongcarl opened pull request #16809: depends: zlib: Move toolchain options to configure (master...2019-09-improve-zlib-pkg) https://github.com/bitcoin/bitcoin/pull/16809
< dongcarl> jb55: I believe MarcoFalke talked about it a little on #bitcoin-builds, don't remember exactly but you could search the logs
< MarcoFalke> github actions is in beta and does not accomodate our use-case (for now)
< bitcoin-git> [bitcoin] dongcarl opened pull request #16810: guix: Remove ssp spec file hack (master...2019-09-guix-remove-ssp-spec) https://github.com/bitcoin/bitcoin/pull/16810
< achow101> meeting?
< wumpus> #startmeeting
< sipa> meeting?
< lightningbot> Meeting started Thu Sep 5 19:01:14 2019 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
< lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
< jonasschnelli> hi
< sipa> hi
< achow101> hi
< aj> hola
< MarcoFalke> hoy
< meshcollider> hi
< moneyball> hi
< instagibbs> hi
< wumpus> there are three proposed topics in https://gist.github.com/moneyball/071d608fdae217c2a6d7c35955881d8a
< wumpus> - proposed by jnewbery: review/merge #16704 or #16713 to remove worrying "unknown new rules activated (versionbit 1)" warning
< wumpus> - proposed by fjahr: Rolling UTXO set hash
< gribble> https://github.com/bitcoin/bitcoin/issues/16704 | Dont warn about activated buried BIP 9 deployments by achow101 · Pull Request #16704 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/16713 | Ignore old versionbit activations to avoid unknown softforks warning by jnewbery · Pull Request #16713 · bitcoin/bitcoin · GitHub
< wumpus> - proposed by achow101: avoid loading every wallet transaction into memory
< jeremyrubin> hi
< fjahr> hi
< jonatack> hi
< wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball kvaciral
< wumpus> but let's start with the usual
< wumpus> #topic High priority for review
< wumpus> 7 blockers, 7 things chasing concept ACK https://github.com/bitcoin/bitcoin/projects/8
< gleb> hi
< BlueMatt> I think its already there, but https://github.com/bitcoin/bitcoin/pull/16421 is close to landing and I still really want it for 19
< wumpus> note that the feature freeze for 0.19 is in 10 days
< BlueMatt> (and its a small diff!)
< wumpus> so we likely want to prioritize features that are close to ready now
< wumpus> right
< wumpus> would be nice if that makes it in
< gleb> #16702 is done code-wise (I think), but perhaps it's suboptimal to add it to high prio at this point when we're close to the feature freeze.
< gribble> https://github.com/bitcoin/bitcoin/issues/16702 | p2p: supplying and using asmap to improve IP bucketing in addrman by naumenkogs · Pull Request #16702 · bitcoin/bitcoin · GitHub
< wumpus> also #15759
< gribble> https://github.com/bitcoin/bitcoin/issues/15759 | p2p: Add 2 outbound block-relay-only connections by sdaftuar · Pull Request #15759 · bitcoin/bitcoin · GitHub
< wumpus> gleb: yes, we might want to keep that for 0.20
< wumpus> also requires too much review to still make it to 0.19 anyway
< wumpus> still, great to hear you're making progress!
< sipa> yeah, that sounds like too close to make it
< jeremyrubin> Not super critical, and it's relatively new so not much time for review, but would help me to get #16766 as my work on OP_SECURETHEBAG wallet support depends on it
< gribble> https://github.com/bitcoin/bitcoin/issues/16766 | wallet: Make IsTrusted scan parents recursively by JeremyRubin · Pull Request #16766 · bitcoin/bitcoin · GitHub
< sipa> i'll review 15759 again
< BlueMatt> 16702 is probably a bit far review-wise, though it would be nice, but 15759 is also really close and should god
< jeremyrubin> And I think it is a bug
< sipa> bug fixes can go after the feature freeze
< wumpus> yes
< jeremyrubin> wasn't sure as it's a substantial behavior change for wallet, but fine :)
< wumpus> they can go any time ("scan parents recursively" sounds scary to me though, performance wise :)
< sipa> it's cached to avoid exponential blowup
< sipa> but otherwise, yeah
< instagibbs> also depends on how new the bug is. I *think* it's ancient behavior that "normally" never hits
< wumpus> phew
< instagibbs> anyways
< sipa> instagibbs: agree
< jeremyrubin> instagibbs: I think that is correct
< instagibbs> with fancier wallet setups we may actually hit it :)
< wumpus> anyhow I've added it to high priority for review as requested, people can choose for themselves what to review
< wumpus> #topic remove worrying "unknown new rules activated (versionbit 1)" warning (jnewbery)
< wumpus> yes please
< wumpus> so I guess the disussion is: #16704 or #16713
< gribble> https://github.com/bitcoin/bitcoin/issues/16704 | Dont warn about activated buried BIP 9 deployments by achow101 · Pull Request #16704 · bitcoin/bitcoin · GitHub
< gribble> https://github.com/bitcoin/bitcoin/issues/16713 | Ignore old versionbit activations to avoid unknown softforks warning by jnewbery · Pull Request #16713 · bitcoin/bitcoin · GitHub
< achow101> 16713 is the simpler, easier to review fix
< achow101> But I think it would be better to eventually get rid of these deployment parameters and have a more permanent solution that lets us reuse bits in the future
< wumpus> what disadvantages does it have compared to the other one?
< MarcoFalke> bits can be reused as long as they don't overlap in time
< aj> 16713 is updated as of a few hours to be even simpler; i think it lets us reuse bits fine?
< wumpus> e.g. why are there two open at all?
< wumpus> MarcoFalke: yup
< MarcoFalke> I already closed the one by achow101 *hides*
< aj> MarcoFalke: smooth
< wumpus> ok, that concludes the discussion then I guess :)
< achow101> my understanding of how consensus.vDeployments worked was that you couldn't define two forks with the same bit since they'd have to occupy the same index position and that's not possible
< wumpus> oh
< achow101> but 16713 has changed since I last reviewed it and it looks very different
< wumpus> sorry
< MarcoFalke> oh, maybe you are right when it comes to the implementation. Though, BIP9 does allow it
< wumpus> BIP9 definitely allows it, that was an important part of the design
< aj> achow101: vDeployments just matches the enum, the actual bits used are independant, and just have to not overlap per their corresponding timestamps
< sipa> i haven't looked at the code in a while, but it was certainly intended to permit reuse of bits
< sipa> the deployments array index is independent from the bip9 bit position
< instagibbs> yep
< aj> achow101: see VersionBitsConditionChecker::Mask in versionbits.cpp, it pulls out the .bit field from BIP9Deployment struct
< achow101> i'm probably wrong
< instagibbs> It was confusing the first time I read it, I came to same wrong conclusion
< aj> achow101: there's also a unit test for overlapping bit usage on mainnet in src/test/versionbits_tests.cpp, "Verify that the deployment windows of [...]"
< wumpus> if even experienced developers get confused by it, some documentation/comments might help in that case
< wumpus> #action please review #16713 so that it can be merged asap
< MarcoFalke> I think it shouldn't matter in practice, hopefully there are less than 27 softforks in the future
< gribble> https://github.com/bitcoin/bitcoin/issues/16713 | Ignore old versionbit activations to avoid unknown softforks warning by jnewbery · Pull Request #16713 · bitcoin/bitcoin · GitHub
< wumpus> given that 'the future' is unbounded, that's a difficult statement
< wumpus> #topic Rolling UTXO set hash (fjahr)
< fjahr> Did anyone have time to look at the proposal? Any questions?
< achow101> link?
< instagibbs> fjahr, pitch it to us in a few sentences :)
< sipa> fjahr: it's not clear to me exactly what you're proposal :)
< fjahr> I have picked up Pieter Wuille's proposal from 2017 to use a rolling hash for the UTXO set hash. It deals with the problem of a long computation time of the UTXO set hash which results in a slow RPC call gettxoutsetinfo (can take several minutes depending on hardware). I investigated three hash functions: MuHash, ECMH and LtHash and started implementing them in Bitcoin Core for comparison. However only MuHash
< fjahr> has a rolling hash implementation so far and my LtHash code is not as optimized as MuHash and ECMH. I am looking for feedback on concept, which choice to make for the hash function and implementation details before filing a PR to Bitcoin Core.
< sipa> it's a complicated question, as the right design may depend on how we intend to use the construction
< wumpus> fjahr: thanks for picking it up!
< MarcoFalke> I think one use case is assumeutxo
< fjahr> sipa: by construction you mean the hash, right?
< sipa> fjahr: right
< sipa> assumeutxo (at least with a from-network-sync approach) will probably need more than just a single hash, as you want to be able to verify chunks etc
< sipa> fjahr: if i recall correctly, the biggest question i had was how to prioritize computation time vs use-from-cache time
< sipa> ECMH is slower to compute, but very fast to use from cached values (e.g. if you have precomputed the "diff" ECMH hash a transaction has, applying to a global sum is super fast)
< sipa> but MuHash is faster is overall computation time
< fjahr> From my benchmarks ECMH was faster overall but I am not sure why MuHash did not perform as you expect in you Mail
< sipa> where do you see a discrepancy? in the hash-to-group-element operation, or in the multity?
< sipa> *multiply?
< sipa> actually i think the discussion of what hash to pick is less important for this meeting
< sipa> we should probably focus on ways to integrate
< fjahr> ok, and also if there is enough interest for this
< aj> it's super cool
< jeremyrubin> Are both constructions one-way?
< jeremyrubin> How does that interplay with reorgs
< fjahr> In terms of integration I chose to implement this as an index, any feedback on this?
< jeremyrubin> Maybe I should read more out of band of here...
< sipa> fjahr: that really depends what for
< fjahr> jeremyrubin: hashes can also be removed again so no problem with reorgs
< sipa> i think as an index it's hard to use precomputation
< sipa> which you really want if you want the rolling part
< fjahr> sipa: what would you suggest in terms of integration?
< sipa> that really depends on what we want to use it for
< aj> sipa: could have it rolling, but in it's own slightly delayed thread like tx indexes, without worrying about precomputation, at least if you don't want to enforce it in consensus
< wumpus> there's 20 minutes to go and still a topic left, let's move on?
< sipa> fjahr: let's talk more after the meeting
< fjahr> sipa: sure
< wumpus> #topic avoid loading every wallet transaction into memory (achow101)
< wumpus> thanks
< sipa> btw, the secp ECMH code you have looks great
< achow101> This is a wallet topic, and probably better for the wallet meeting, but that's next week..
< wumpus> is there any hurry? :-)
< achow101> I was thinking about ways to reduce the memory footprint of loaded wallet files
< wumpus> concept ACK anyhow
< jonasschnelli> jup. also ack
< sipa> achow101: seems hard
< achow101> I was wondering if anyone who was more familiar with the wallet tracking part of the wallet knew if this was attempted before or would majorly break something?
< wumpus> it always seemed unnecessary to me to keep all transactions, even historical ones with all outputs spent, in memory
< jonasschnelli> There is a PR where all wallettxns where kept in mem
< jonasschnelli> +different DB formst
< wumpus> but yes, it's such a part of the current wallet design, it's definitely not going to be easy
< jonasschnelli> #5686 (very old)
< gribble> https://github.com/bitcoin/bitcoin/issues/5686 | [Wallet] replace BDB with internal append only (logdb) backend by jonasschnelli · Pull Request #5686 · bitcoin/bitcoin · GitHub
< sipa> achow101: you mean not _show_ them anymore
< sipa> or just not load them in memory?
< achow101> not load them into memory, so it would read from disk when the full tx is needed
< wumpus> right
< achow101> I plan on just keeping UTXOs and txids in memory
< sipa> achow101: i wish you good luck :)
< jonasschnelli> heh
< wumpus> outward behavior probably shouldn't change
< jonasschnelli> Why not just loading everything into memory?
< jonasschnelli> I think 1000+ wallets are OOS
< * jeremyrubin> wonders how big the largest wallet is
< sipa> OOS?
< jonasschnelli> out of scope
< jeremyrubin> out of scope
< wumpus> there are some heavy users of bitcoin core which have to re-cycle their wallet once in a while because it becomes to big
< sipa> i don't think memory usage is the problem there
< achow101> big wallets also take a while to load, although I don't expect this to effect that
< sipa> just the size of the maps that's being traversed for a multitude of operations
< wumpus> loading time is likely the problem, yes
< wumpus> then again that's directly related
< achow101> ideally this will also reduce the time it takes to calculate things like balances since not every single transaction will be iterated over
< wumpus> yes, exactly
< sipa> achow101: i'm very very scared of things like that
< jarthur> Are exchanges and other large-scale wallet-holders recommended to use Core purely for networking, and maintain their own non-Core wallets?
< sipa> it needs a completely new design i'm afraid
< achow101> sipa: how so?
< sipa> everything is transaction oriented in the current wallet
< wumpus> jarthur: some use core, but can't mention any names
< sipa> new transactions that can others to become conflicted etc
< sipa> changing that to a UTXO model and keeping it in sync with the list of transactions... sounds very hard
< achow101> right, but I think those can still be done by just a list of txids, spent prevouts, and utxos
< jeremyrubin> is it a more important goal to reduce the number of txns or the amount of data each one is storing?
< sipa> but you still need all the dependency tracking between transactions to compute the utxos
< sipa> you can cache the utxo list; that would probably be a worthwhile performance improvement
< wumpus> that would likely be a better first step
< sipa> but getting rid of the transaction entirely... i don't see how
< achow101> also, maybe just loading a neutered transaction without input scripts, because we don't need those
< sipa> achow101: yeah that can work
< jeremyrubin> achow101: that's what I was getting at :)
< sipa> achow101: also, don't let me discourage you if you see a good way to implement it :)
< sipa> i'm happy to be convinced otherwise
< wumpus> there's definitely some cut-off possible, I mean, transactions from years ago can't really become conflicted anymore
< jeremyrubin> achow101: a good first step might be to cache the scriptPubKey of inputs in a WtX
< sipa> i suspect thay in native descriptor wallets the IsMine check will be a lot faster than it currently is
< achow101> anyways, that's all. just wanted to get some opinions before diving in
< wumpus> at some point spent transactions deep enough in the chain can be permantly archived
< wumpus> might be non-trivial to come up with criteria but I don't think every transaction needs to be potentially active forever
< wumpus> achow101: good luck !
< wumpus> #endmeeting
< lightningbot> Meeting ended Thu Sep 5 19:53:03 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< jarthur> MarcoFalke: glad to see the flake8 update finally made it in and you won't have to keep rebasing it. :)
< jeremyrubin> achow101: you can save *some* memory by repacking the CWalletTx struct I bet :)
< wumpus> if you repack it you might as well re-retrieve it from the database
< achow101> I want to avoid touching the wallet database, although that may not be possible
< jeremyrubin> I meant just the fact that there's a lot of fields in the struct that are char then uint64 or something
< jeremyrubin> which adds a lot of interior padding
< wumpus> suure
< jeremyrubin> I did say *some*
< achow101> just a bit
< achow101> At this rate, I think I'll have rewritten the entire wallet by this time next year
< sipa> if done incrementally, i think that'd be great :)
< wumpus> if you manage to do that you're the first ever person to achieve it, people have been saying that since 2011 or so though
< sipa> haha
< sipa> in 2011 the wallet was still part of main.cpp :p
< wumpus> but I'm happy to see the work picking up lately on the wallet
< sipa> we've come a long way
< wumpus> yes, we've come a long way, for a long time there was hardly any interest in it and some devs were even arguing about removing it because it was just too bad
< wumpus> it's definitely not like that anymore
< bitcoin-git> [bitcoin] martinus closed pull request #16801: faster & less memory for sync: bulk pool allocator for node based containers (master...2019-08-bulkpoolallocator) https://github.com/bitcoin/bitcoin/pull/16801
< bitcoin-git> [bitcoin] martinus reopened pull request #16801: faster & less memory for sync: bulk pool allocator for node based containers (master...2019-08-bulkpoolallocator) https://github.com/bitcoin/bitcoin/pull/16801
< instagibbs> for all the hate on the wallet, the only other solutions are things like Electrum personal server, and other more ad hoc solutions. It's a project worth iterating on :)
< instagibbs> > jeremyrubin wonders how big the largest wallet is
< instagibbs> I've seen a multi GB testnet wallet, but that likely doesn't count
< instagibbs> if you end up using Core wallet for "industrial" wallet it indeed slows significantly. rhavar probably has some anecdotes
< sipa> change IsMine() to return true; and watch your wallet.dat explode :p
< instagibbs> I guess the slowdown is just iterating through all txns anyways, so meh :)
< phantomcircuit> instagibbs, it does count, don't down play my entirely valid usecase
< * phantomcircuit> runs away
< warren> instagibbs: I've seen a very large Bitcoin exchange in Asia that had suffered for years with a single wallet.dat for all customer wallets. It got to the point where ordinary RPC queries against that wallet would take several minutes. As a workaround for faster queries I told them to setup many parallel servers with bitcoind -txindex and watch-only wallets so that they could parallelize their lookups. I also warned that they check for block
< warren> height agreement before querying anything.
< bitcoin-git> [bitcoin] ch4ot1c opened pull request #16812: doc: Fix whitespace errs in .md files, bitcoin.conf, and Info.plist.in (master...docs/lint-markdown) https://github.com/bitcoin/bitcoin/pull/16812
< jb55> might be handy to have a very large test wallet for performance testing