< bitcoin-git> [bitcoin] fanquake opened pull request #16339: doc: add reduce-memory.md (master...doc-reduce-memory-usage) https://github.com/bitcoin/bitcoin/pull/16339
< bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/dfdcb3dfe535...1088b90cbae3
< bitcoin-git> bitcoin/master 1ac454a Hennadii Stepanov: Enable ShellCheck rules
< bitcoin-git> bitcoin/master 1088b90 fanquake: Merge #16327: scripts and tools: Update ShellCheck linter
< bitcoin-git> [bitcoin] fanquake merged pull request #16327: scripts and tools: Update ShellCheck linter (master...20190702-shellcheck) https://github.com/bitcoin/bitcoin/pull/16327
< luke-jr> re AVX512, someone mentioned using it causes CPUs to overheat and clock down? so even a simple benchmark might not work out :/
< sipa> luke-jr: using avx512 instructions on some cpus causes it to clock down; nothing related to overheating
< luke-jr> hm, even just using it once?
< luke-jr> so it'd break parallel benchmarking, but not serial (not sure parallel was even viable to begin with)
< sipa> how much it clocks down depends on how many cores you're using it simultaneously
< jb55> why does it do that... overheating protection?
< jb55> seems weird
< luke-jr> sounds like overheat protection to me too :x
< bitcoin-git> [bitcoin] fanquake pushed 2 commits to master: https://github.com/bitcoin/bitcoin/compare/1088b90cbae3...4f378ac30cf6
< bitcoin-git> bitcoin/master 08c721d nicolas.dorier: [MSVC] Copy build output to src/ automatically after build
< bitcoin-git> bitcoin/master 4f378ac fanquake: Merge #16308: [MSVC] Copy build output to src/ automatically after build
< bitcoin-git> [bitcoin] fanquake merged pull request #16308: [MSVC] Copy build output to src/ automatically after build (master...build/copy-in-msbuild) https://github.com/bitcoin/bitcoin/pull/16308
< bitcoin-git> [bitcoin] achow101 opened pull request #16341: Introduce ScriptPubKeyMan interface and use it for key and script management (aka wallet boxes) (master...box-the-wallet) https://github.com/bitcoin/bitcoin/pull/16341
< achow101> meshcollider: ^^
< meshcollider> \o/
< meshcollider> 86 commits lol
< meshcollider> That'll be really fun to review
< achow101> A lot of them are really small
< achow101> So I'll be squashing them
< jb55> https://github.blog/2019-07-01-mark-files-as-viewed/ haven't tried this yet, I wish it was for hunks/commits instead of files
< tryphe> jb55, i noticed that feature and was wondering if they get unmarked as viewed if there are modified changes
< hebasto> tryphe: yes, they get unmarked.
< tryphe> hebasto++
< fanquake> probably worth asking jamesob if he's seen any issues with CPUs overheating while benchmarking the AVX512 PR.
< fanquake> I've had a look over the build system changes in that PR, https://github.com/bitcoin/bitcoin/pull/13989#issuecomment-508623934, but don't have a machine that supports AVX512 to test any performance improvements on.
< jb55> achow101: fwiw I don't think those need squashed, they seem like reasonable sized commits to me
< fanquake> sipa It'd be great to have you or cfields take another look over, or Concept ACK / NACK that PR.
< jb55> achow101: actually I appreciate that they are cleanly separated, makes review easier, at least for me.
< bitcoin-git> [bitcoin] ajtowns opened pull request #16342: Backport #16250 for 0.18 (0.18...201906-signrawerror-regression-018) https://github.com/bitcoin/bitcoin/pull/16342
< fanquake> promag: thanks
< promag> fanquake: np!
< fanquake> wumpus could you take a look at 13989, even just a concept ACK / NACK ?
< wumpus> fanquake: sure!
< jamesob> fanquake: nah, benchmarked the avx512 PR mayber twice (for a total of ~2 hours runtime) and didn't see any cpu heat problems. But that was on a desktop machine that's well cooled
< wumpus> where does that concern come from? is that a known issue that using AVX512 heats up CPUs more than other instructions?
< fanquake> wumpus: 09:55:22 <luke-jr> re AVX512, someone mentioned using it causes CPUs to overheat and clock down? so even a simple benchmark might not work out :/
< fanquake> jamesob: cheers 👍
< wumpus> ok
< fanquake> wumpus: although see the replies above, supposedly it’s actually just clocking down.
< wumpus> I think in practice it's hard to ascribe heating to certain instructions, bitcoind validation has always been a risky workload for computers with bad heat management
< wumpus> hhmm apparently it's really a thing that people from cloudflare etc have writetn about https://lemire.me/blog/2018/08/13/the-dangers-of-avx-512-throttling-myth-or-reality/
< wumpus> thanks intel !
< promag> what is the min qt currently?
< wumpus> in any case if you disable AVX512 on a system this will cause bitcoind to not use it too
< fanquake> promag: should be 5.5.1
< wumpus> and newer / properly cooled chips might still benefit from it
< promag> thanks
< wumpus> then again, I'm always a bit skeptical with using the newest instruction set extensions for hashing etc, probably, the kind of CPUs that have this are fast enough to not be bottlenecked by hashing in the first place
< wumpus> it probably pays off more to use special instructions on lower end hardware
< promag> wumpus: good point
< wumpus> why is no one doing this for ARM? :)
< bitcoin-git> [bitcoin] Sjors closed pull request #15424: [wallet] wallet-tool: command to remove key metadata (master...2019/02/wallet_tool_remove_metadata) https://github.com/bitcoin/bitcoin/pull/15424
< provoostenator> Random review beg #15457 for fans of sandboxed operating systems.
< gribble> https://github.com/bitcoin/bitcoin/issues/15457 | Check std::system for -[alert|block|wallet]notify by Sjors · Pull Request #15457 · bitcoin/bitcoin · GitHub
< wumpus> provoostenator: thanks, kind of lost track of that one
< bitcoin-git> [bitcoin] laanwj pushed 4 commits to master: https://github.com/bitcoin/bitcoin/compare/4f378ac30cf6...8c69fae94410
< bitcoin-git> bitcoin/master c1c91bb Sjors Provoost: [build] detect std::system or ::wsystem
< bitcoin-git> bitcoin/master cc3ad56 Sjors Provoost: [build] MSVC: set HAVE_SYSTEM for desktop apps
< bitcoin-git> bitcoin/master f874e14 Sjors Provoost: [build]: check std::system for -[alert|block|wallet]notify
< bitcoin-git> [bitcoin] laanwj merged pull request #15457: Check std::system for -[alert|block|wallet]notify (master...2019/02/std__system) https://github.com/bitcoin/bitcoin/pull/15457
< provoostenator> wumpus: thanks!
< emilengler> I currently have problems with bitcoin-cli to read an integer value on a new command I've implemented
< emilengler> With the JSONRPC however it works fine
< emilengler> bitcoin-cli says "JSON value is not an integer as expected"
< emilengler> Can someone help me?
< sipa> emilengler: bitcoin-cli knows which arguments to treat as strings and which as json
< bitcoin-git> [bitcoin] Sjors opened pull request #16344: [build]: use #if HAVE_SYSTEM instead of defined(HAVE_SYSTEM) (master...2019/07/system-ifdef) https://github.com/bitcoin/bitcoin/pull/16344
< provoostenator> ^ sorry for the case of "this refactor can't possibly hurt"
< sipa> emilengler: if that information is wrong, if you write 5 on the command line, it will be sent in json-rpc as the string "5" rather than the integer 5
< emilengler> sipa: Thanks for letting me know but how does "generatetoaddress" solves the problem?
< emilengler> There the first argument is an integer
< sipa> bitcoin-cli knows that the first argument to generatetoaddress is to be interpreted as json instead of as a string
< emilengler> sipa: Ok, thank you I will take a look at it
< provoostenator> emilengler: it does that here: https://github.com/bitcoin/bitcoin/blob/master/src/rpc/client.cpp#L31
< sipa> yup
< emilengler> provoostenator, that was exactly what I was looking for, thanks
< emilengler> Thanks, it works now :)
< bitcoin-git> [bitcoin] emilengler opened pull request #16345: RPC: Add getblockbyheight (master...getblockbyheight) https://github.com/bitcoin/bitcoin/pull/16345
< wumpus> provoostenator: I had kind of assumed you'd have tested #15457 on a sandboxed system (I didn't have any available)
< gribble> https://github.com/bitcoin/bitcoin/issues/15457 | Check std::system for -[alert|block|wallet]notify by Sjors · Pull Request #15457 · bitcoin/bitcoin · GitHub
< provoostenator> wumpus: I did, but apparently not after that last change (or I did, but somehow missed the issue anyway).
< provoostenator> Meanwhile I have a bit more experience linking C(++) libraries in iOs, which means with a bit of luck I might be able to actually test running stuff, not just compiling.
< wumpus> provoostenator: awesome!
< meshcollider> There's a wallet meeting scheduled for today but considering the low turnout for the meeting yesterday, does anyone want to discuss anything?
< wumpus> it's no longer freedom day at least
< sipa> i could give a bit of an update on miniscript if people are interested
< achow101> wallet meeting?
< meshcollider> #startmeeting
< lightningbot> Meeting started Fri Jul 5 19:00:58 2019 UTC. The chair is meshcollider. Information about MeetBot at http://wiki.debian.org/MeetBot.
< lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
< meshcollider> #bitcoin-core-dev Wallet Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball
< achow101> all I have to say is that #16341 does the wallet restructure and is very large
< gribble> https://github.com/bitcoin/bitcoin/issues/16341 | Introduce ScriptPubKeyMan interface and use it for key and script management (aka wallet boxes) by achow101 · Pull Request #16341 · bitcoin/bitcoin · GitHub
< provoostenator> hi
< provoostenator> achow101: the infamous box? I'll add to my review list.
< meshcollider> #topic miniscript update (sipa)
< schmidty> hola
< sipa> hi!
< sipa> so me and some colleagues of mine have been working on a way to make more complex bitcoin scripts more accessible, in a project called miniscript
< sipa> this is about the bitcoin scirpt language as it exists today, without any softforks or other changes to consensus needed (but obviously prepared for those)
< sipa> the idea is that we can define a subset of script which is composable (so things can be arbitrarily nested, say you can replace a public key anywhere with a "subexpression" which corresponds to an arbitrary policy), has properties like correctness and nonmalleability that can be easily reasoned about, and can be targetted by a compiler that converts policies into scripts
< sipa> with this, we could make software/hardware wallets support basically arbitrary policies, without even any extensions to psbt
< sipa> there's a website which is still actively being changed: http://bitcoin.sipa.be/miniscript
< sipa> does this make sense?
< achow101> sipa: is your website the only documentation of miniscript?
< sipa> achow101: andrew and sanket are also working on a rust implementation, which is public i think
< sipa> but as we're actively changing things yet i prefer to not really have any production code available
< meshcollider> This seems very nice
< sipa> so the different moving parts would be (a) signing logic in core/other software which is miniscript compatible, allowing signing/updating of any valid miniscript-compatible script (b) extension to descriptors that lets you include miniscript expressions, which are kinda engineer-readable but maybe a bit less than descriptors are now... (c) a compiler from policies with probabilities for all
< sipa> branches/choices to miniscript descriptors
< jb55> I see the aor was replaced with explicit probabilities, is that overkill or are there use cases in mind?
< sipa> jb55: it was basically impossible to write a 3-way or where each branch is equally likely
< jb55> ah
< sipa> oh we also came up with a neat algorithm that lets you compute the worst-case satisfaction cost for a miniscript, with annotations on various branches saying "this key is certainly available, this key will not be available at signing time, this key may be available at signing time"
< sipa> which is useful for fee estimation
< bitcoin-git> [bitcoin] sipsorcery closed pull request #15995: systemd service script: set usable permissions on /etc/bitcoin config dir (master...systemdconfig) https://github.com/bitcoin/bitcoin/pull/15995
< sipa> and the compiler is also good enough now that it can construct scripts that are strictly better than the bolt 3 htlc scripts... but obviously incompatible with them
< sipa> it'd be nice to support all actual script people have come up with in the past, but it would require adding a whole bunch of ad-hoc extensions for weird tricks
< achow101> cool! how close to being finalized do you think it is?
< achow101> (aka when pr?)
< sipa> i think we're very close to finalizing the design of the language... but we've been saying that for a few weeks now
< sipa> it'll be some work to convert the hacky code to something PR'able
< sipa> but i hope soon
< sipa> so a motivation for this is for example being able to use a 2-of-3-timelocked-escrow-cool policy as a "participant" in a multisig, rather than being restricted to just keys in the multisig
< sipa> more questions? otherwise end topic
< achow101> the policy language is descriptors, right?
< sipa> nope
< sipa> the thing i envision becoming part of descriptors is a 1-to-1 mapping with script itself
< sipa> so there is no "compilation" step between the descriptor and the script, as that compilation is pretty complex
< achow101> is it policy -> descriptor -> script?
< sipa> so the policy language is similar, but it has probabilities in it, and only has and/or/threshold/key/hash/time functions
< sipa> policy -> descriptor <-> script
< achow101> ah, ok
< sipa> while the descriptor extensions have a ton of different ways of encoding an and/or, depending on context, probabilities, ...
< sipa> i don't expect the policy language and the compiler to become part of core,
< sipa> though maybe some analysis functionality may be (e.g. we can extract a policy without probabilities back from the script etc)
< meshcollider> So what are you planning to PR then?
< sipa> ability to sign/update/psbt arbitrary miniscript compatible scripts
< sipa> and extensions to descriptors
< achow101> basically replacing the solver and ProduceSignature?
< achow101> so instead of pattern matching to figure out how to sign and finalize, interpret miniscript/descriptors
< sipa> right
< sipa> maybe all of the signing code can become delegated to a miniscript module
< sipa> maybe the existing rawtransaction-compatible ones should just stay as legacy code, and only miniscript code gets invoked for more complex things
< achow101> and with native descriptor wallets, people will be able to track whatever weird scripts they want and be able to sign for them
< sipa> the conversion from script to miniscript structure is basically an LALR(1) parser (we have a handwritten one that's easier to read, but tools like bison/yacc can generate code to do this trivially)
< sipa> yup
< sipa> and with psbt, you can have even hardware wallets in theory participate in signing
< sipa> though they may freak out with the unconventional scripts, and change detection becomes a lot harder
< jb55> sipa: is the conversion from script to policy ever ambiguous?
< jb55> if that makes sense
< sipa> jb55: if you define an "abstract policy" which is a policy but without probabilities in it, you can always derive the abstract policy from a script
< meshcollider> Ah that LALR parser point answered my next question
< sipa> in such a way that it's equivalent to the policy fed to the compiler
< jb55> sipa: and that is guaranteed to be unique?
< sipa> jb55: though if the compiler gets a lot smarter, this may not be possible anymore (e.g. complex reasoning over booleans/threshold gates to simplify the policy before compiler)
< sipa> jb55: testing whether two different encodings of the same abstract policy are identical for arbitrary policies is an intractable problem
< sipa> like and(a,thresh(2,a,b,c)) is identical to and(a,or(b,c))
< sipa> things like that cannot be generally compared
< sipa> but the compiler (so far) won't ever make such transitions
< jb55> right
< meshcollider> Alright this seems very nice, thank you sipa, I need some time to read the website in more detail
< meshcollider> Does anyone have any other topics to discuss today?
< achow101> please review wallet boxes and the 3 PRs it depends on
< meshcollider> Yes ^ sorry I haven't reviewed much yet, it's been a crazy week
< meshcollider> #action review wallet box PRs
< meshcollider> #endmeeting
< lightningbot> Meeting ended Fri Jul 5 19:36:41 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< pinheadmz> I have a question about RBF implementation / BIP125 rules: Just seems like these two cases are redundant:
< pinheadmz> bad-txns-spends-conflicting-tx
< pinheadmz> replacement-adds-unconfirmed
< pinheadmz> The first error is thrown if you spend an output that will be replaced
< pinheadmz> but the second doesnt allow the replacement to spend any unconf outputs anyway - already
< pinheadmz> so if the second case was jsut checked first, there'd be no need for the first case ...?
< luke-jr> pinheadmz: they're fundamentally different issues
< luke-jr> pinheadmz: the second one is (hypothetically, in a PR, and in Knots) overrideable
< pinheadmz> luke-jr: thanks, I get it now. The first is definitely invalid, the second is more like a policy
< luke-jr> right