< bitcoin-git>
[bitcoin] sipa opened pull request #13471: For AVX2 code, also check for AVX, XSAVE, and OS support (master...201806_avxossupport) https://github.com/bitcoin/bitcoin/pull/13471
< kanzure>
wom 3
< kanzure>
error. hm.
< jonasschnelli>
sipa: The polynominal operation in Bech32X requires 256bit bitwise AND operation, right?
< sipa>
jonasschnelli: 135 bit XOR
< jonasschnelli>
okay.. I see
< sipa>
on languages that don't support big integers, split it up in 3 64-bit integers
< jonasschnelli>
sipa: I guess it makes sense to have a straight decode() _and_ a correct() in case decode fails?
< sipa>
jonasschnelli: the correction code is less efficient in realizing there are no errors
< jonasschnelli>
okay
< sipa>
it could have a short circuit check if all syndromes are 0, and in that case just straight decode
< sipa>
(the syn variable, after the "for v" loop)
< jonasschnelli>
Okay. That makes sense...
< bitcoin-git>
[bitcoin] kristapsk closed pull request #13464: RPC: Allow to specify rescan start timestamp for importaddress, importprivkey and importpubkey (master...rescan-from) https://github.com/bitcoin/bitcoin/pull/13464
< sipa>
no need to invoke solver etc in that case
< jonasschnelli>
Also, what made me think a bit: dropping a char will not be detected which seems like a likely (error) case
< jonasschnelli>
s/detected/corrected
< sipa>
huh
< sipa>
that should be detected
< gmaxwell>
sipa: he means a frameshift, and correction, not detection.
< gmaxwell>
i think for these things its useful if the length is explicit and known, so then you can trigger dropped character repair more usefully.
< jonasschnelli>
Yes. Length is known,.. but the correction would need to be different
< gmaxwell>
yes, there isn't a simple algebraic correction for that, but you can insert a dummy at each position and run the normal correction. I think it's sufficient to know it's possible to do that.
< jonasschnelli>
agree
< sipa>
i wonder if ability to detect such errors is something we can optimize the code for
< gmaxwell>
even placing two characters in a 64 character string is only about 2k possibilities.
< gmaxwell>
sipa: I started to write a fancy bech32 hinter with the idea of list decoding out to 3 errors, including searching for patterns of up to a couple drop and inserts, then using that recent data base of password entry error probablities (which include probabilities for dropped, transposed, and inserted characters) and ranking a probablity for each option.
< jonasschnelli>
I'm manly worried about that people will think with "it can correct up to X characters", it does include missing chars
< sipa>
it's hamming distance, not levenshtein distance :)
< gmaxwell>
well it does correct up to missing characters with unknown positions, assuming a sutiable decoder.
< sipa>
jonasschnelli: oh, something that i haven't mentioned - that bech32x code can correct up to 14 errors if you know their positions
< sipa>
jonasschnelli: which means that if you have explicitly unreadable characters (as opposed to characters that may be wrong), its error correction ability is much stronger
< sipa>
this is called erasure, not correction
< gmaxwell>
by 'correct up to' sipa means "with no more remaining error detection power"
< sipa>
or in general, you can correct with M known erasures and N addition errors in unknown places as long as M+2*N <= 14
< sipa>
that's not implemented in that demo
< jonasschnelli>
I see...
< sipa>
but it's not very hard to do
< sipa>
M+2*N < 15 is why this is called a distance 15 code
< gmaxwell>
sipa: are those figures really all that useful for this application? e.g. if you can't externally tell that the key is correct (by checking against the blockchain) you can't go safely to that bound, and if you can tell via external information, you can go further.
< sipa>
why can't you safely go to that bound?
< gmaxwell>
because you can't tell if the result is right or not.
< gmaxwell>
if you corrupt 15 characters, but think you have corrupted 14, you'll "recover" something but it'll be the wrong one.
< gmaxwell>
Of course, if you can check against the blockchain you're good to go, but in that case in we could go beyond 15 errors.
< sipa>
right
< sipa>
in practice you always have both, i guess
< sipa>
but the without-the-blockchain inner "loop" is much more efficient
< wumpus>
meeting time?
< jonasschnelli>
jup
< wumpus>
#startmeeting
< lightningbot>
Meeting started Thu Jun 14 19:00:32 2018 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
< gribble>
https://github.com/bitcoin/bitcoin/issues/13425 | Moving final scriptSig construction from CombineSignatures to ProduceSignature (PSBT signer logic) by achow101 · Pull Request #13425 · bitcoin/bitcoin · GitHub
< gribble>
https://github.com/bitcoin/bitcoin/issues/12136 | Implement BIP 174 Partially Signed Bitcoin Transactions serialization and RPCs by achow101 · Pull Request #12136 · bitcoin/bitcoin · GitHub
< wumpus>
unloadwallet from promag seems almost ready for merge
< achow101>
12136 can be removed for now
< wumpus>
achow101: ok
< achow101>
it depends on 13425
< wumpus>
dropped
< promag>
wumpus: I think so, I have to fix last jnewbery points
< sipa>
#13425 is pretty much all of the PSBT internal changes that are needed, excluding serialization and RPCs
< wumpus>
it was unfair for you to have two entires on the list, anyway
< gribble>
https://github.com/bitcoin/bitcoin/issues/13425 | Moving final scriptSig construction from CombineSignatures to ProduceSignature (PSBT signer logic) by achow101 · Pull Request #13425 · bitcoin/bitcoin · GitHub
< jnewbery>
I think since the last change, unloadwallet no longer removes the unloaded wallet from the dropdown menu
< wumpus>
(just kidding, no idea how it came that way)
< promag>
jnewbery: you are right
< promag>
needs signal unload
< wumpus>
right
< jnewbery>
yep. Seems to work with that method declaration readded
< wumpus>
should we add anything to the list this week?
< achow101>
the basic point is that in the current coin selection, we prefer exact matches over confirmations. However the current implementation of srd fallback is that we prefer confirmations over exact matches
< instagibbs>
altnernating between BnB(Exact match) and then fallback, rather than Bnb of all sorts then fallback
< sipa>
so this is a bit of a question on what our coin selection algorithm should prioritize
< achow101>
yes
< sipa>
confirmed coins, or (immediate) fee
< instagibbs>
and privacy
< instagibbs>
imo privacy puts it over the top and we should try really hard to do it
< instagibbs>
but obviously it's not a slam dunk
< sipa>
hmm, unclear how privacy is affect here?
< instagibbs>
change-less outputs mess with coin analysis
< instagibbs>
to a large degree
< sipa>
that's a good point
< sipa>
sdaftuar, morcos: present, opinions?
< sipa>
gmaxwell: ?
< instagibbs>
IIRC we converged on being ok with current behavior, because it breaks the long chains if it works
< achow101>
I also did a simulation (so take it with a grain of salt) that showed that the number of utxos with exact match over confirmations was higher than not
< instagibbs>
was an intentional design decision
< sipa>
what do you mean by "current behaviour"?
< instagibbs>
in master
< sipa>
master or the SRD PR?
< sipa>
ok
< sipa>
there is another question here on what the criteria for SRD merge should be
< sipa>
because it seems it results in somewhat higher average UTXOs per wallet in simulations
< instagibbs>
merge as in code? or?
< achow101>
merge as in merge the pr
< instagibbs>
ah
< sipa>
yes, i'm wondering what our bar for deciding to change tbe logic should be
< achow101>
it doesn't seem to perform as well as the current coin selection in master w.r.t mean number of utxos in the wallet in my simulations
< meshcollider>
Significantly higher?
< instagibbs>
did you try filtering for using only coins lower than target value? using coins with negative effective value?
< achow101>
from ~20 utxos to ~90 utxos
< instagibbs>
allowing a single negative effective value?
< instagibbs>
Core is an extreme UTXO cop currently. I don't think we're going to be able to match it.
< wumpus>
it's good for the selftest one to be seperate, I think that one can be merged
< wumpus>
I haven't really looked at the other ones yet in detail yet
< cfields>
sipa: I have a follow-up PR as well to build a lib-for-each-arch
< sipa>
cfields: ah yes
< sipa>
i'll leave things like this
< cfields>
I figured I'd just wait until everything settled for that one, but let me know if you'd prefer something else
< wumpus>
re: 13442, didn't you first say that made things a few % *slower* on 64 bit?
< sipa>
wumpus: i made more changes, it's faster now
< wumpus>
great, no problems with it anymore then
< sipa>
but it's very heavily compiler dependent... rearranging two lines can have 5% effect on speed
< wumpus>
seems preferable in every way then
< sipa>
or making a constant static
< sipa>
how so?
< wumpus>
both more readable and faster
< sipa>
ah yes, but probably less reliably faster
< sipa>
perhaps on clang it's slower
< cfields>
sipa: also worth considering (I read this just yesterday), apparently gcc switched the way that 256bit loads are done, somewhere around gcc6, I believe.
< sipa>
or with particular gcc versions
< cfields>
so, worth considering compiler age as well.
< wumpus>
right
< wumpus>
if it becomes faster with new compilers it's good
< wumpus>
if slower, not :)
< cfields>
(see gcc's "mavx256-split-unaligned-load" option, which had its default value changed)
< ryanofsky>
cd
< sipa>
wumpus: another benefit is that this lets us compile the exact same code with -mavx, and get a slightly faster version for AVX capable machines
< cfields>
~$
< wumpus>
sipa: that's really nice
< wumpus>
sipa: so we compile it twice, the same code?
< sipa>
wumpus: yup
< sipa>
cfields is working on build system changes for that
< wumpus>
almost for free
< sipa>
i wonder what percentage of our binary will be SHA256 implementations...
< wumpus>
a very small part
< cfields>
heh
< wumpus>
though I understand the sentiment :)
< wumpus>
as for the source code, a larger part, which reminds me I still need to add ARM
< jamesob>
maybe not meeting-worthy, but something I've been curious about is whether there are other indexes people'd like to see (now that we have this nice framework for building them)
< jamesob>
I was thinking address-to-related-txns might be nice
< luke-jr>
AFAIK the goal was to move indexes out of Core
< jamesob>
I think in some cases there can be a compelling argument for additional opt-in indexes
< wumpus>
at least the indexing functionality has been factored to a generic class now
< jamesob>
jonasschnelli: cool, I'll take a look
< jonasschnelli>
jamesob: I tried to build an address-to-related-txns,... but figured out its a source for general evil things like "central validation". :)
< jamesob>
hah
< jonasschnelli>
I think using the wallet more for "selective indexing" makes more sense.
< jonasschnelli>
But IMO an address-to-related-txns is probably better kept external...
< jamesob>
it'd be nice to, e.g., back trezor's web interface with your own bitcoind node instead of a bitcore instance and afaik part of that is because bitcore maintains indexes that bitcoind doesn't
< jonasschnelli>
I understand this use case... but indexing everything for this is just silly..
< jonasschnelli>
You only want to index a certain xpub or range of keys
< jamesob>
yeah, makes sense
< jonasschnelli>
The only use case is probably if you want to recover a backup...
< jonasschnelli>
But even for that, my new scantxoutset PR makes more sense and works also with pruned peers...
< jonasschnelli>
it just can't reproduce the transaction history,... just recover the funds.
< jonasschnelli>
For bitcore / Trezor,... I'm pretty sure one can do the same thing belcher did with stratum (his personal electrum server)
< jonasschnelli>
Create a bitcoin interface where only a defined set of keys (or xpubs) are indexed.
< jonasschnelli>
In the background, you could use a watch-only core wallet for this...
< jonasschnelli>
If you don't need backup-recovery, it would work with pruned peers as well
< gmaxwell>
instagibbs: I don't think it would be hard to beat bitcoin core in terms of tx out cleanup.
< gmaxwell>
For example, if it agressively spent all outputs to a scriptpubkey when it spent from any, it would likely sweep more txouts than the current code, AND improve privacy.
< jamesob>
jonasschnelli: interesting, thanks
< instagibbs>
gmaxwell, yeah yeah i meant within the constraints of the discussion, but agreed!
< gmaxwell>
instagibbs: my thought on it was that we'd offset more txout prolific behavior with explicit txo consuming behavior.
< instagibbs>
gotcha
< gmaxwell>
the two kinds of extra consuming behavior I know of are (1) grouping by scriptpubkey (which helps privacy), (2) spending more small inputs when fees are low.
< bitcoin-git>
[bitcoin] TheBlueMatt closed pull request #11856: [RFC] I Have a Hammer! (Replace parts of ui_interface with validationinterface) (master...2017-12-remove-cvblockchange) https://github.com/bitcoin/bitcoin/pull/11856
< BlueMatt>
heh, havent had time to rebase anything in a while :(
< BlueMatt>
maybe I'll get back to them in a few weeks or so
< promag>
\o/ < 270 prs!
< sipa>
MarcoFalke: feature request: in the conflict checker, if one PR is just a prefix of another one's commits, don't call it a conflict
< sipa>
BlueMatt: :(
< sipa>
BlueMatt: oh, some are still open
< BlueMatt>
lol ffs, I was just closing stale crap I'm not gonna have a chance to rebase for a month
< BlueMatt>
or, more likely, longer
< sipa>
okay!
< bitcoin-git>
[bitcoin] Empact closed pull request #13462: scripted-diff: Simplify common case of CHashWriter and drop SER_GETHASH & SerializeHash (master...serialize-hash-type) https://github.com/bitcoin/bitcoin/pull/13462
< bitcoin-git>
[bitcoin] TheBlueMatt closed pull request #11639: Rewrite the interface between validation and net_processing wrt DoS (master...2017-10-dos-rewrite) https://github.com/bitcoin/bitcoin/pull/11639
< cfields>
gitian builders: v0.16.1 detached sigs are pushed
< cfields>
achow101: ^^
< wumpus>
thanks
< bitcoin-git>
[bitcoin] Empact reopened pull request #13462: scripted-diff: Simplify common case of CHashWriter and drop SER_GETHASH & SerializeHash (master...serialize-hash-type) https://github.com/bitcoin/bitcoin/pull/13462
< bitcoin-git>
[bitcoin] Empact closed pull request #13462: scripted-diff: Simplify common case of CHashWriter and drop SER_GETHASH & SerializeHash (master...serialize-hash-type) https://github.com/bitcoin/bitcoin/pull/13462