< Jmabsd> code question, https://github.com/bitcoin/bitcoin/blob/master/src/script/interpreter.cpp#L1162 - when generating the sign text for OP_CHECKSIG, the code to produce the signtext with respect to one transaction output, is just " ::Serialize(s, txTo.vout[nOutput]);".
< Jmabsd> Where is the "Serialize" logics for the transaction output (CTxOut)?
< Jmabsd> Where in Bitcoin Core's code are the constraints on what scripts are allowed, implemented? E.g., gossiped transaction's lock scripts must be either of P2PK, P2PKH, multisig, nulldata or P2SH in order to be considered valid and be gossiped?
< aj> Jmabsd: src/policy/policy.cpp IsStandardTx might be what you want
< Jmabsd> aj: ah, so Bitcoin Core nodes will accept a nonstandard tx when it's in a block, but it will not accept them in gossiping, right?
< Jmabsd> aj: do you have any idea where Serialize for CTxOut is?
< wumpus> Jmabsd: that's the definition of "non-standard", it is relay/mining policy not consensus policy
< Jmabsd> wumpus: a miner is free to receive nonstandard transactions, right?
< Jmabsd> and include in blocks, and all nodes will accept those blocks.
< Jmabsd> Where is the "Serialize" logics for the transaction output (CTxOut)?
< aj> Jmabsd: see primitives/transaction.h and serialize.h
< wumpus> a miner is free to do everything accepted by consensus
< wumpus> so yes, they can include non-standard transactions as long as they're valid in blocks
< Jmabsd> wumpus: interesting. cool.
< Jmabsd> aj: ahh, CTxOut has a ADD_SERIALIZE_METHODS. got it.
< wumpus> indeed--classes implement their own serialization/deserialization
< aj> wumpus: in case i'm not awake for the meeting, topic suggestion: lower tx fees, pr #13922 :)
< gribble> https://github.com/bitcoin/bitcoin/issues/13922 | Lower default relay fees by ajtowns · Pull Request #13922 · bitcoin/bitcoin · GitHub
< Jmabsd> wumpus: are only *output scripts* having the standard/nonstandard classification, or also *input scripts* and *P2SH redeem scripts* need to fit a particular template?
< sipa> Jmabsd: there are various standardness rules
< sipa> certain scriot opcodes are nonstandard when executed in a spend
< sipa> there are resource limits like sizes and signature check limits
< sipa> Jmabsd: https://bitcoin.stackexchange.com may be a better place to ask these things
< Jmabsd> sipa: hm
< jonasschnelli> sipa: I that correct that the Cores existing ChaCha20 implementation is for RNG only? It doesn't do the XORing, right?
< gmaxwell> sipa: results from running 13907?
< pierre_rochard> Hello, if anyone here has done a clang AST dump of the project, what command did you use? I'm interested in using python with libclang to traverse the AST
< pierre_rochard> I've tried export CXXFLAGS="$CXXFLAGS -ast-dump" && make, to no effect, not sure if I'm going down the wrong rabbit hole and google doesn't have salient results
< pierre_rochard> searching the repo turned up this recent comment from skeees https://github.com/bitcoin/bitcoin/pull/13815#issuecomment-409795628 which makes it sound like a material undertaking
< wumpus> pierre_rochard: I did some analysis of the bitcoin source using clang a long time ago, using the python clang bindings (clang.cindex)
< wumpus> hooking any kind of analysis tool in the build system is a pain, so I manually provided include paths and such
< pierre_rochard> wumpus: that's what I was hoping to use, after reading through this guide https://eli.thegreenplace.net/2011/07/03/parsing-c-in-python-with-clang
< pierre_rochard> ah I see
< wumpus> cindex doesn't need it (it can parse on the fly through the API), but to add an analysis tool into the build system one way to do so is by creating a compiler wrapper that outputs both normal output and the metadata output (ast, bitcode, stats, etc). If it doesn't do the former it will fail when trying to link.
< wumpus> I don't know if there's an official autotools-supported way
< wumpus> like "call this tool as if you'd be calling a C/C++ compiler but don't touch the output"
< wumpus> yes, that idea
< pierre_rochard> ok in that case I'll try my hand at using cindex directly, instead of doing an ast dump and parsing that
< wumpus> might be I even used the same guide :)
< wumpus> yes, that's easier, I think, certainly initially
< gmaxwell> sipa: grep logs!
< sipa> jonasschnelli: right
< sipa> gmaxwell: no 'locator' message in my log
< gmaxwell> hurrah.
< gmaxwell> sipa: you have net logging enabled, right?
< sipa> gmaxwell: oh, no
< sipa> -log=net ?
< sipa> oh, -debug=net
< gmaxwell> Right
< pierre_rochard> wumpus: got cindex working directly, thank you!
< wumpus> pierre_rochard: awesome
< achow101> it seems like travis isn't triggering for some PRs
< wumpus> which one?
< sipa> meetung?
< luke-jr> hi
< instagibbs> #meeting
< gmaxwell> Hi.
< provoostenator> hi
< jnewbery> hi
< achow101> hi
< meshcollider> Is wumpus here?
< gmaxwell> now we must hunt the wumpus
< Chatturga> hello
< sipa> i'll act as pseudowumpus
< sipa> #startmeeting
< lightningbot> Meeting started Thu Aug 9 19:05:13 2018 UTC. The chair is sipa. Information about MeetBot at http://wiki.debian.org/MeetBot.
< lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
< sipa> topics?
< sipa> we still have quite a few PRs marked 0.17
< sipa> should we discuss low-R sigs?
< achow101> +1
< jonasschnelli> hi
< meshcollider> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr btcdrak sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator
< meshcollider> hi
< jnewbery> 01:59 < aj> wumpus: in case i'm not awake for the meeting, topic suggestion: lower tx fees, pr #13922 :)
< meshcollider> +1
< gribble> https://github.com/bitcoin/bitcoin/issues/13922 | Lower default relay fees by ajtowns · Pull Request #13922 · bitcoin/bitcoin · GitHub
< sipa> perhaps first let's ask around if there are things for 0.17 that deserve the milestone
< sipa> i guess that's a no
< sipa> #topic low-r signatures
< Murch> hi
< sipa> pull request #13666
< gribble> https://github.com/bitcoin/bitcoin/issues/13666 | Always create signatures with Low R values by achow101 · Pull Request #13666 · bitcoin/bitcoin · GitHubAsset 1Asset 1
< wumpus> ohh sorry!
< luke-jr> wumpus: too late, you're fired :p
< wumpus> luke-jr: :p
< sipa> one issue with the approach is that we need to guess whether signatures created by other software will be assumed to be 71 bytes as well
< sipa> as brought up by wumpus in the PR
< sipa> (for size estimation purposes)
< wumpus> yes the size estimation is the only controversial part there imo
< achow101> this is only a problem when we select watching-only inputs
< sipa> as we're getting closer to branching off, i wonder if we shouldn't just punt on that
< wumpus> obviously it's better to generate smaller signatures where possible
< gmaxwell> well we could easily make the size estimate 71 for our keys, and 72 otherwise, and worry about the difference later.
< sipa> achow101's code right now will assume 71 byte signatures if we have the keys for everything, and 72 in any other case
< Murch> That seems safe.
< wumpus> that heuristic makes sense I think
< gmaxwell> that seems reasonable enough.
< gmaxwell> even if we always estimated 72, I'd rather make the smaller signatures.
< wumpus> exactly, it's kind-of independent
< sipa> yes, i don't think the discussion is about whether or not to produce 71 byte signatures
< sipa> we should always do so
< achow101> imo estimating 72 bytes kind of defeats the purpose of producing 71 byte sigs
< Murch> it doesn't
< wumpus> it still takes less space in the block chain.
< gmaxwell> achow101: might defeat _your_ purpose. :P
< luke-jr> achow101: nah, you get value in feerate
< Murch> You're still getting a higher effective fee rate and block space savings
< wumpus> and you get better feerate, yep
< gmaxwell> what wumpus said. it still economizes use of a public resource, and you still get higher priority.
< sipa> in any case,
< sipa> #action review 13666
< sipa> it adds a bit of complexity by adding a second dummy signer
< Murch> might save some 3kB per block or so.
< Murch> If everyone did it
< wumpus> yes
< * luke-jr> wonders if we should look into getting rid of the Segwit dummy/flags too
< provoostenator> You can always add a param to the RPC that says "assume 71 "
< luke-jr> ofc that doesn't affect the consensus limits, so not incentivised :/
< sipa> well, thinking ahead, we may need that anyway - in case more complicated scripts are supported, the size may depend on what subset of signers is present
< sipa> but that seems too late for 0.17
< wumpus> provoostenator: I'm... not sure it's good to add such little implementation details to RPC, but maybe a more general fee estimation strategy can be exposed such way
< wumpus> yes, definitely
< provoostenator> Luckily the RPC is experimental
< gmaxwell> rpc would be a bad place for it
< sipa> or rather, we may need to add a way to pass in information about other signers to transaction creation RPS
< sipa> RPCs
< Murch> @provoostenator: I don't think that there is sufficient use of multi party transactions and savings to add a param for that.
< gmaxwell> the right place would be as metadata on the keys/outputs in the wallet.
< sipa> gmaxwell: how so?
< sipa> ah yes, but the 71/72 thing - agree
< sipa> luke-jr: i don't know what you're referring to?
< gmaxwell> sipa: just that the size of an scriptpubkey's signature is just a property of the private key(s).
< sipa> gmaxwell: yes, indeed
< gmaxwell> just like the pubkey part of the scriptsig being 65 or 33 bytes.
< sipa> do we have other topics?
< gmaxwell> in any case, for now its fine to just assume 72 unless we know otherwise. In the future we can be smarter.
< Murch> the problem will be permanently solved by the new signature encoding that is proposed for Schnorr signatures anyway.
< luke-jr> sipa: just that the dummy 2 bytes in every Segwit tx could be eliminated on the network/disk level, if we're trying to focus on micro-optimisations of space
< gmaxwell> luke-jr: far far better than that can be done.
< sipa> luke-jr: sure; and much more than that, we should look into compressing all data on the wire
< sipa> we have someone looking into that
< meshcollider> You had a whole write-up on a compressed transaction format earlier didn't you
< sipa> meshcollider: yes, there's a bunch of improvements and simplifications
< sipa> which we haven't published
< sipa> 19:07:19 < jnewbery> 01:59 < aj> wumpus: in case i'm not awake for the meeting, topic suggestion: lower tx fees, pr #13922 :)
< gribble> https://github.com/bitcoin/bitcoin/issues/13922 | Lower default relay fees by ajtowns · Pull Request #13922 · bitcoin/bitcoin · GitHub
< sipa> #topic lower tx fees, pr 13922
< Murch> "it changes the min tx fee to 200 s/B, and the incremental relay fee to 100 s/B" doesn't he mean s/kB?
< gmaxwell> and is that kVB not B?
< jnewbery> (I don't have any input on this topic except to concept ACK the PR. Only relaying aj's earlier message)
< Murch> yeah, kvB
< gmaxwell> I feel pretty meh about dropping feerates. I guess 200 isn't that much of a change but at some level we get back in spamflood level land.
< sipa> i have no opinion on this topic
< luke-jr> it's low enough I don't care; I'd prefer to see users adopt changes manually rather than top-down default modifications, but meh
< Murch> gmaxwell: I think that reducing the fee could drive some UTXO consolidation
< gmaxwell> or cause waiting until it's dropped further.
< luke-jr> Murch: if that's the goal, maybe it should explicitly look for that
< Murch> I've observed that time preference signaling has improved a lot this year. One of our customers explicitly waited for us to allow a fee rate of 2 sats/B before consolidating.
< Murch> If 0.2 sats/B were possible, I'd not be surprised to see some more of that.
< gmaxwell> Well lets try it, but if things go wonky it can be set back.
< Murch> We do have small fee spikes every now and then again. I think that we'd get a backlog of consolidations at least part of the time, hopefully consistently in the future.
< Murch> The biggest issue is that most miners don't mine anything below 1 sats/B
< gmaxwell> in terms of the patch, I'm not sure it's doing it correctly as is. Basically we must lower relay and mining behavior before wallet... changing the estimator to allow low estimates before we know they reliably propagate will cause stuck users.
< Murch> (showing as a solid backlog below 1 sat/B, even when blocks have space left)
< luke-jr> Murch: well, if that's the case, it would be bad to drop relay fees lower
< Murch> I don't follow.
< gmaxwell> luke-jr: the theory is if the defaults change the behavior will too.
< sipa> luke-jr: i'm sure miners don't allow below 1 vsat/B _because_ the network doesn't relay them
< Murch> Ah I see.
< gmaxwell> Murch: luke's point is that we shouldn't be relaying things that won't get mined.
< Murch> Yeah, miners tend to lag behind on adoption of new versions
< luke-jr> has anyone discussed this with any miners?
< Murch> It would already help if we found a few percent of the hash rate that tells us they would support it
< luke-jr> Murch: this is a matter of policy, not versions. defaults are just defaults.
< sipa> so 1) reduce the minimum feerate the mining code works at 2) if/when that gets adopted, start relaying below 1 3) if/when that gets adopted, make fee estimation work with it
< Murch> My suspicion is that most miners don't much fiddle with defaults.
< Murch> seems like a reasonable order of things
< luke-jr> Murch: they should. this is their job, not developers'
< gmaxwell> can we not have this debate here and now.
< provoostenator> My main concern with allowing sub 1 satoshi / vbyte fees is that - when fees are low - there's going to be a good chance the users transaction doesn't propagate.
< gmaxwell> it's really tedious.
< Murch> luke-jr: IF they were engaging at that level of the technology, they wouldn't be running mining operations of RaspberryPis, lag behind multiple versions causing orphans, and similar things.
< gmaxwell> provoostenator: that was my comment above, relay/mining behavior needs to change ahead of wallet behavior, they can't both change at once.
< provoostenator> gmaxwell: ah I see, by keeping the wallet minimum at 1 sat / byte, that makes sense.
< phantomcircuit> gmaxwell, mining needs to change, then relay, then wallet
< phantomcircuit> but none of the first two have a reason to change without the later
< phantomcircuit> sooo
< achow101> gmaxwell: so at best, this would need to be deployed across two versions
< gmaxwell> achow101: yes.
< wumpus> at least
< achow101> which means, at this point, it's a year and a half out
< achow101> and by then, who knows what fees will be like
< luke-jr> they can be minor versions
< phantomcircuit> achow101, and realistically miners do change the defaults so...
< wumpus> it depends, we can see the adoption
< wumpus> and decide when (if ever) the wallet part can go in
< gmaxwell> as luke-jr says. also users can locally override their wallets once they know that lower rates will confirm.
< wumpus> yes
< gmaxwell> phantomcircuit: we can handle some things getting relayed that don't get mined at least... (now, not so much years ago)
< Murch> phantomcircuit: It's enough if a portion of the hashrate adopts the change to make it's use a viable strategy for low time-preference txes.
< Murch> Talking to miners would be a good idea
< gmaxwell> I'd really like it if there wouldn't need to be these thresholds at all, but unfortunately, solving making it all automatic seems excessively hard.
< gmaxwell> In any case, I'd review and ack a split up that moved the relay/mining defaults and left the wallet parts up for a later version.
< sipa> any other topics?
< Murch> So, pseudowumpus sipa, anything else on your agenda?
< luke-jr> gmaxwell: I thought the current PR already left wallet alone
< phantomcircuit> gmaxwell, automating all these levels would be vastly simplified by efficient mempool sync
< gmaxwell> luke-jr: ah, haven't checked.
< phantomcircuit> which im sure you know :P
< sipa> phantomcircuit: you can't automate the decision on incremental relay feerate
< sipa> as it's not mempool dependent but bandwidth cost dependent
< sipa> wumpus: you have any other topics?
< luke-jr> sipa: not your own nodes cost directly, though? since you're not getting paid for it; so measuring bandwidth availability may be sufficient?
< wumpus> nope!
< sipa> luke-jr: if you'd set it to 0, you can be pushed into arbitrarily high bandwidth usage by an attacker
< wumpus> the 0.17 list basically
< gmaxwell> luke-jr: for incremental, it's relatively important that the behavior be consistent. Thats largely at odds with automating it, unfortunately.
< sipa> yeah, let's just take further discussion to the individual PRs
< sipa> #endmeeting
< lightningbot> Meeting ended Thu Aug 9 19:44:38 2018 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< wumpus> #13808 certainly needs more review
< gribble> https://github.com/bitcoin/bitcoin/issues/13808 | wallet: shuffle coins before grouping, where warranted by kallewoof · Pull Request #13808 · bitcoin/bitcoin · GitHub
< luke-jr> sipa: right, but if you set it to some formula based on measured bandwidth available..
< wumpus> if not, I'd be inclined to postpone it
< gmaxwell> luke-jr: measuring bandwidth is basically intractable. :)
< luke-jr> gmaxwell: could also query the network router for that info in theory
< gmaxwell> which is will virtually never know...
< sipa> and your ISP's cost, and the bitcoin exchange rate?
< luke-jr> sipa: you're not receiving the tx fee, so just a general cost estimate should be good enough I would think?
< luke-jr> maybe not perfect, but probably better than manually adjusting defaults on the current level
< gmaxwell> sipa: so, now that you're logging right, any results?
< sipa> 2018-08-09T18:55:00.534227Z getblocks locator size 101 > 64, disconnect peer=414
< sipa> 2018-08-09T18:55:00.301077Z receive version message: /bitcoinj:0.14.7/Bitcoin Wallet:6.28/: version 70001, blocks=535894, us=, peer=414
< gmaxwell> So, as evoskuil said, 101 though.. not 100. Moronic behavior. So, should I just back the PR off to 101?
< gmaxwell> [ot, an outright backdoor in VIA C3 cpu] https://github.com/xoreaxeaxeax/rosenbridge