< gmaxwell>
I've been chewing on that data, and I don't think it really supports the 144 block target very much.
< gmaxwell>
sipa: do you have more detailed data?
< sipa>
i have timestamps + depth for each block transfer
< gmaxwell>
logically we'd want to put our breakpoints at peaks in the second derivative of the total blocks served (the cumulative version of that data). Which would in peaks of the derivative of your data.. but it's measurements so getting a useful derivative is hard. working from a smoothed spline it looks like the peaks are at 29, 199, and then there is so much periodic behavior that its hard to see
< gmaxwell>
any other trend. What I think I'm seeing is 24 hour-ish cycles, but they're spread out in block count due to difficulty changing.
< gmaxwell>
e.g. if you want to fully capture a 24-hour sycle then perhaps you need 200 blocks, because there can be 200 blocks in a day.
< petertodd>
gmaxwell: when have I last responded to a post that you think wasn't worth responding too?
< gmaxwell>
petertodd: I can't think of a recent example, remark wasn't intended to criticize your behavior. Sorry if it came off that way.
< gmaxwell>
sipa: so from this data, I'm not seeing a lot of justification for anything but a single level at around 200 blocks +plus a dozen or so blocks for headroom (so, e.g. 288 would be fine). If I use a cumulative maximum from the end to estimate how many transfers are due to fullsync-- there are 107, if then subtract those off, 90% of the cumulative transfer is met by depth 244, and 99% by 410.
< jonasschnelli>
I guess the 26bytes may be padded to 32... so I guess ryanofsky is right here.
< SopaXorzTaker>
jonasschnelli, hey
< SopaXorzTaker>
could I ask for some offtopic help?
< vedochiaro>
ciao
< vedochiaro>
musica anni 80
< jonasschnelli>
SopaXorzTaker: shoot
< jonasschnelli>
If its offtopic use a on-topic channel or PM me
< cchadwicka>
i need to find an online wallet that allows me to buy bitcoin from within the wallet with a credit card, like coinbase, but a different one
< cchadwicka>
any ideas
< instagibbs>
cchadwicka, #bitcoin
< jonasschnelli>
cchadwicka: please no cross posts... use #bitcoin
< cchadwicka>
i am banned from bitcoin
< cchadwicka>
im new here and they kicked me out yesterday
< cchadwicka>
because i was off topic
< sipa>
and if you keep posting offtopic things here, i'll ban you too
< sipa>
jonasschnelli: absolutely, you need to pack the bytes if you want to save space on it
< sipa>
jonasschnelli: but don't bother, saving 2 bytes in such a structure is negligablr
< kanzure>
jonasschnelli: fundrawtransaction complains about minimum fee policy when i set feeRate to 0; i have an output with the fee and i was just going to delete the output later...
< kanzure>
"Note that all inputs selected must be of standard form and P2SH scripts must be in the wallet using importaddress or addmultisigaddress (to calculate fees)."
< kanzure>
does that mean that if i'm spending p2sh inputs that my wallet doesn't know about (except by importaddress p2sh-address) in my fundrawtransaction input transaction, do i need to first run importaddress redeemscript?
< kanzure>
really i just want it to ignore my existing inputs-- i could easily pass my estimate of the final transaction size, if necessary.
< instagibbs>
kanzure, ah, yes that's a check I've always wondered about, that sounds like a sharp edge
< instagibbs>
It's checking the fee you're getting against the entire transaction size once constructed
< kanzure>
sure.
< instagibbs>
I don't totally get the check's reasoning tbh.
< instagibbs>
/ If we made it here and we aren't even able to meet the relay fee on the next pass, give up
< instagibbs>
// because we must be at the maximum allowed fee.
< kanzure>
workaround: create a fake transaction of the same size as the estimated final size of my unsigned watchonly-p2sh-spending transaction, and then use fundrawtransaction and i'll pass a realistic feeRate value.
< instagibbs>
I think the check is just supposed to be reversed.
< kanzure>
and then i'll delete the fake inputs/outputs from the fundrawtransaction output.
< kanzure>
specifically my problem is that fundrawtransaction gives me "Signing transaction failed" when i directly pass my unsigned watcholy-p2sh-spending transaction as the input to fundrawtransaction. so i figured hey i'll just set feeRate to 0 on a dummy transaction, add a change output that i'll delete later, and then use fundrawtransaction... which also has problems.
< kanzure>
instagibbs: okay here is my new workaround. createrawtransaction with a change output spending back to myself the total fee that i would like to spend. fundrawtransaction with reasonable feeRate. using fundrawtransaction output dictionary, i'll switch the "change" output amount to be the fundrawtransaction fee amount, so ultimately there is no extra fee and my original "fee" request shou...
< kanzure>
...ld be satisfied.
< kanzure>
and the final transaction will have two change outputs (unless i consolidate).
< paveljanik>
Are GetDustThreshold and IsDust expected to be called with dustRelayFee other than ::dustRelayFee?
< instagibbs>
It seems surprising behavior to me to have the wallet simply reduce the fee if it hits maxTxFee, other than panic and abort.
< instagibbs>
rather than*
< kanzure>
huh, signrawtransaction also gives me an error ("Operation not valid with the current stack size")
< Chris_Stewart_5>
Hmm, I think I have had that problem when I forgot to call fundrawtransaction first?
< kanzure>
i have definitely called fundrawtransaction; i excised the inputs and outputs, and added them to my unsigned p2sh-spending transaction.
< Chris_Stewart_5>
with watch-only p2sh spending txs the redeemScript is imported right?
< kanzure>
i am spending watch-only p2sh transactions, however i did not pass includeWatching to fundrawtransaction -- i supplied an unsigned transaction to start with.
< kanzure>
also, i did not import the redeemscripts, is that important
< kanzure>
(why would that be important for signrawtransaction? i want to sign only the inputs i'm able to sign.)
< kanzure>
oh, that's a misleading error message, that's how it reports the transaction is incompletely signed?
< Chris_Stewart_5>
Why are you adding a watch-only output to a tx you are creating? You can't fullfill the spending conditions of it by definition
< Chris_Stewart_5>
fundrawtransaction will add inputs to the tx until it *fully* funds the outputs of that tx
< kanzure>
the transaction is fully funded by my inputs, to my knowledge, prior to calling fundrawtransaction with the exception of any extra fee i'm trying to add.
< kanzure>
Chris_Stewart_5: yeah i was overly focused on "errors" that i overlooked "complete: False" and that it had a new scriptsig :)
< Chris_Stewart_5>
so fundrawtransaction isn't giving you a large enough fee?
< kanzure>
fundrawtransaction is giving me an error "Signing transaction failed" if i try to directly call it with my transaction
< kanzure>
probably because it's using dummy signatures somewhere-- i dunno- and yeah i have not imported the redeemscripts (if that's necessary- which is still unclear to me)
< Chris_Stewart_5>
Well, if you (or some one else) is trying to spend a p2sh output the redeem script must be provided in the scriptSig
< kanzure>
i may not have put the reedemScripts in the scriptSigs yet.
< Chris_Stewart_5>
But since it seems like the p2sh output isn't yours, your counterparty will have to sign the p2sh output and provide the redeem script
< instagibbs>
kanzure, that would explain it as it's expecting more stack items
< Chris_Stewart_5>
^
< kanzure>
i'd like to add fee first, sign for the fee, then sign the other outputs-- it's exceedingly inconvenient for me to do this in another order.
< kanzure>
*sign for the other inputs
< Chris_Stewart_5>
You should be able to do it that way, then just pass the partially signed tx to your counterparty
< kanzure>
i think we've already confirmed that i can't because of the error?
< Chris_Stewart_5>
oh, yes.. hmm..
< kanzure>
anyway, my workaround seems to be working for me, where i use createrawtransaction with an output that represents my fee, then later i remove the output, copy the inputs and any extra outputs added by fundrawtransaction to my actual transaction, and then i call signrawtransaction.
< kanzure>
(also i'm adding the "fee" reported by fundrawtransaction to the change output reported by fundrawtransaction, since i have deleted the output that has my actual fee)
< Chris_Stewart_5>
kanzure: Can't your counter party just add the p2sh output to the transaction? If you remove that outpoint your wallet can sign the tx right?
< kanzure>
er, maybe. but keep in mind that i was calling fundrawtransaction not signrawtransaction-- it's something about dummy sigs.
< sipa>
is the problem that fundrawtransactrion can't determine the feerate of your overall transaction as it does not know how large the scriptSigs of some inputs will need to be?
< kanzure>
so you're saying that i have a partial outpoint and that fundrawtransaction should not report "Signing transaction failed" once i remove the incompete outpoint?
< Chris_Stewart_5>
but in the hex transaction you provided to fundrawtransaction you had specified the watch-only p2sh output?
< sipa>
and as a result can't determine how much fee to use?
< kanzure>
sipa: right, that seems likely to me. i am also interested in confirming/denying with someone whether i need to "importpubkey redeemscript" to overcome that?
< sipa>
or importscript
< sipa>
if fundrawtransaction doesn't know what kind of redeemscript a P2SH output that's being spent has, it can't determine feerate
< sipa>
it may need both, unsure
< kanzure>
i added a comment here explaining that perhaps a better design would be to let user specify the final transaction size (if the user knows it) or allowing feeRate=0 if the user plans to delete an output later https://github.com/bitcoin/bitcoin/pull/10202#issuecomment-301134038
< kanzure>
perhaps CreateTransaction is overloaded at this point :p
< sipa>
just slightly
< SopaXorzTaker>
sipa, can you provide some secp256k1 test cases?
< SopaXorzTaker>
such as, "for private key 0xdeadbeef, z=0xabcd, k=0x1337, r=0x1234 and s=0x5678
< sipa>
general question: how useful is gettxoutsetinfo's serialized_bytes field?
< sipa>
it does not correspond to actual disk usage, and is highly database dependent
< sipa>
it's also impossible to even give a reasonable estimate for after pertxout, except by iterating over the whole database (which I'd like to get rid of)
< gmaxwell>
I think it's not useful, or rather the only thing I've ever used it for is taking about how big the utxo set is on reddit, and an on-disk size would be _better_ for that.
< BlueMatt>
sipa: heh, I have some reivew comments noting it seems out of sync with everything....
< BlueMatt>
sipa: gah, why did you kill Accessors?
< sipa>
BlueMatt: i didn't
< sipa>
how do you mean out of sync?
< BlueMatt>
ehh, meant Modifiers not Accessors
< BlueMatt>
sipa: what is serialized_bytes /supposed/ to show? it seems to not represent anything that actually means something (both before and especially, as you note, after)
< sipa>
BlueMatt: i'm very happy to get rid of Modifiers :D
< sipa>
they're so complicated to reason about
< BlueMatt>
they were there to prevent footgun incidents :(
< sipa>
yes, if you have modifiable references
< BlueMatt>
now you can take a AccessCoins, then add something to the cache, then blow your face off
< sipa>
AccessCoins returns a const reference
< BlueMatt>
AccessCoins gives you such a reference, though?
< sipa>
the only way to modify is through AddCoin and SpendCoin (and BatchWrite)
< BlueMatt>
yes, but if you add something to the cache the reference is still invalid? or are unordered_map refs still valid?
< BlueMatt>
ofc if you remove from cache the same element you've still blown your face off
< sipa>
adding something to the cache may always invalidate references
< sipa>
Modifiers didn't help with that
< BlueMatt>
didnt add assert(!fHasModifier)?
< BlueMatt>
which is what you want if you're returning refs from inside the object
< BlueMatt>
(or, it should have, if it didnt)
< sipa>
yes, there could be at most be one modifier at a time
< sipa>
but that didn't prevent any concurrent references
< BlueMatt>
well we could trivially fix that - reintroduce modifiers and allow multiple ones, but assert when you actually modify the cache that there are no other ones
< sipa>
that won't do anything
< BlueMatt>
they did prevent some footguns, though, like you couldnt take two at a time because that implies you're possibly modifying the cache
< BlueMatt>
sipa: I meant modifier here as accessor, really, from AccessCoins
< BlueMatt>
since its returning a possibly-invalidated-by-other-action reference
< sipa>
they prevented exactly the same footguns as are now prevented by just having explicit modification methods
< BlueMatt>
easy to accidentally have one stick around too long
< sipa>
yes, we could make AccessCoin return something like a modifier... but that seems an independent change
< BlueMatt>
fair
< BlueMatt>
I'd like to see that (or just not return a ref, have you checked the performance overhead of doing that copy everywhere?)
< sipa>
i really don't like copying potentially large data structures where it's unneeded
< sipa>
it's also imcompatible with reducing some of the duplicate output lookups during block/tx validation
< BlueMatt>
fair, I am worried about the footgun potential there, though
< BlueMatt>
maybe we can do the copy for now and introduce an accessor in a separate pr?
< sipa>
please
< * BlueMatt>
doesnt want to audit for this case :(
< sipa>
you don't have to
< sipa>
nothing has changed wrt that case
< sipa>
nothing that was prevented is no longer prevent
< sipa>
and nothing that wasn't prevent is prevented now
< BlueMatt>
hmm, maybe i missed part of how it used to work, though i vaguely recall you doing some AccessCoins in place of old modifiers
< BlueMatt>
maybe im wrong
< sipa>
modifiers were only used when a modification was expected
< sipa>
ok, so in the rpc signing code and in bitcoin-tx, there used to be a Modifier that was both used check whether an output already existed, and then add it
< sipa>
which is now turned into an AccessCoin + AddCoin
< BlueMatt>
yea, i mean its probably no big deal, just more shit to review, would be good to also add some kind of checker later so that it doesnt blow up in our face
< BlueMatt>
cause it'd be easy to slip a change in later that breaks that
< sipa>
FWIW, i'm adding a commit that adds a disk_size to gettexoutsetinfo instead, which just asks LevelDB for the actual size
< BlueMatt>
that sounds much better
< sipa>
BlueMatt: std::unordered_map references are only invalidated when the corresponding entry is deleted
< sipa>
they remain valid under addition, including when rehashing occurs
< BlueMatt>
oh, heh, i thought you said no earlier, ok
< BlueMatt>
yea, makes sense
< sipa>
iterators are invalidated, but iterators from the underlying CCoinsViewCache are never exposed anymore