< cfields>
luke-jr: for future reference, when behind cloudflare, apache needs mod_cloudflare, otherwise incoming ips are actually cloudflare proxy ips
< cfields>
who knew
< luke-jr>
XD
< luke-jr>
cfields: btw, can you set someone else up with access to raise our bus factor?
< cfields>
luke-jr: sure, i think there are several though
< luke-jr>
eh?
< cfields>
i believe wumpus/btcdrak/bluematt have access
< luke-jr>
wumpus said he didn't, and the other two didn't speak up IIRC
< cfields>
oh, hmm. ok, will check
< cfields>
there was an issue with dns after the move, maybe they're missing the new info
< cfields>
mm, it may be worth setting up some fancy deployment scripting stuff. Not sure what the latest/greatest is. chef/puppet/etc.
< sipa>
maybe wumpus has access but does not know it :)
< luke-jr>
heh
< cfields>
i think that's probably the case. i thought i lost access until btcdrak figured out the dns weirdness
< phantomcircuit>
cfields, he's going to run you over with a bus, quick run away!
< luke-jr>
…
< cfields>
heh
< GitHub152>
[bitcoin] sdaftuar opened pull request #8312: Fix mempool DoS vulnerability from malleated transactions (master...mempool-malleability) https://github.com/bitcoin/bitcoin/pull/8312
<@wumpus>
cfields: I don't think I ever received login details for the 'new' server, at least I don't remember
< jonasschnelli>
Ping MarcoFalke
< jonasschnelli>
I don't see how you could lost coins with the new HD feature...
< jonasschnelli>
*lose coins
<@wumpus>
right - it just works, it exports/imports the keys. What it doesn't do is restore the HD seed, but you do lose the information needed to generate new addresses using the old seed after a export/import. That means the backup gnerated in such a way is not forward-compatible as wallet.dat is
< MarcoFalke>
Then asserts that the last derived key is identical to the last derived key of the first run
< MarcoFalke>
But somehow it does not update the balance
< jonasschnelli>
hmm...
< jonasschnelli>
MarcoFalke: if you call getnewaddress, you get the same address but not the received coins to this address, right?
< jonasschnelli>
I guess it requires a rescan (after getnewaddress)
< MarcoFalke>
jup
< MarcoFalke>
I tried rescan and reindex
< jonasschnelli>
reindex should not be necessary... rescan works?
< MarcoFalke>
no
< jonasschnelli>
let me test it locally... thanks for testing / writing the test!
< jonasschnelli>
I knew that the backup restore is not something users can easily do.
< MarcoFalke>
Correct, but there should be some way to do it
< jonasschnelli>
Not sure if we should have something more convenient (backup restore) for 0.13
< jonasschnelli>
Yes.
< jonasschnelli>
Maybe we discuss this tonight at the IRC meeting
< MarcoFalke>
oh
< MarcoFalke>
I got it
< MarcoFalke>
importprivkey breaks all future backups
< MarcoFalke>
:(
< MarcoFalke>
This should be fixed I assume
< MarcoFalke>
Maybe a small fix in the rescan logic
< jonasschnelli>
What's the reason for this?
< MarcoFalke>
no idea
< jonasschnelli>
Why does it breaks all future backups?
< jonasschnelli>
*break
< MarcoFalke>
Maybe only legacy keys are rescanned when there is at least one?
< MarcoFalke>
Need to look into the code...
< jonasschnelli>
If you are able to recreate the address you had before you restored your backup you should be capable of restoring the transactions with -rescan
< MarcoFalke>
More likely it is just a getbalance issue. I remember there were "some" issues lately
<@wumpus>
what's wrong with getbalance?
< jonasschnelli>
MarcoFalke: I just create a regtest hd wallet (keypoolsize=1), did a backup right after creation, generated 100 blocks, stopped, deleted the wallet,.. then...
< jonasschnelli>
I reloaded the old wallet only containing 1 key, called 100 times getnewaddress, stopped, started with -rescan and could successful reconstruct the balance/wtxs
< jonasschnelli>
Seems to work...
< jonasschnelli>
Maybe need to check in conjunction with importprivkey...
< GitHub155>
[bitcoin] yurizhykin opened pull request #8313: Use std::move() instead of copying/removing in TxMemPool (master...tx-delete) https://github.com/bitcoin/bitcoin/pull/8313
< MarcoFalke>
IsMine returns false for the txs involving the HD keys
< zooko>
Hey Core Devs: here's another patch we've been working on for Zcash which you might want upstream in Bitcoin Core: https://github.com/zcash/zcash/issues/915
< zooko>
We'll put in the time to merge it if desired.
< gmaxwell>
cfields: ^
< zooko>
I've asked Zcash engineer Taylor "earthrise" Hornby to support you on that if you want it.
< zooko>
The _other_ one that I mentioned a few days back turned out to have a bug — we were
< cfields>
zooko / gmaxwell: thanks
< zooko>
trying to measure time to validate a block packed full of worst-case quadratic signature.
< zooko>
And we were accidentally measuring time to check that the signature was already cached. ;-)
< zooko>
But we've fixed that, and now we have some results.
< jonasschnelli>
gmaxwell, sipa: Bip151: what do you think by using HKDF instead of a single SHA_HMAC to derive the symmetric cipher key from the ECDH secret?
< sipa>
i have not looked into HKDF, but we should
< jonasschnelli>
I think there is no "real" HD issue
< michagogo>
wumpus: Broken in what way?
<@wumpus>
michagogo: please read the issues
< MarcoFalke>
jonasschnelli: Could you look into the travis failure
< MarcoFalke>
?
< jonasschnelli>
Yes. Will do..
< gmaxwell>
MarcoFalke: It's unclear to me if we understand the issue you've encountered completely, if we do not understand it, I think it is a blocker.
< michagogo>
(Sorry, I've not had much free time for this stuff in the past months)
< * michagogo>
goes to check
<@wumpus>
if HD is broken, we have to disable it for 0.13
< michagogo>
(Gitian issues or Bitcoin Core?)
<@wumpus>
or at least rc1
< MarcoFalke>
gmaxwell: You can see the issue on travis
< jonasschnelli>
Sure. Let me check MarcoFalke HD test
< MarcoFalke>
It is either an error in my test code
< gmaxwell>
Disabling it indeed would be an option.
< MarcoFalke>
or some issue with IsMine() or something else
< gmaxwell>
This may be a current design limitation if my understanding is correct. But I'm happy to discuss elsewhere/elsetime.
< gmaxwell>
lets talk about the other things then move on to the hd issue as it's own topic.
< cfields>
thanks
<@wumpus>
planning was to do rc1 yesterday, but it was clear it wasn't possible
<@wumpus>
btcdrak: some things can be merged as-is I think
< MarcoFalke>
Also there are some issues open by sdaftuar
< sdaftuar>
#8305 (headers sync issue), #8312 (mempool DoS post-segwit merge), and #8295 (mining code fixes post segwit-merge) are all meant for 0.13.0
<@wumpus>
#7678 (release notes) is a TODO for -final, not rc1, could use some help there though
<@wumpus>
are they ready for merge?
< btcdrak>
sdaftuar those should be tagged with the milestone then
< sipa>
i will write some release notes for the tx relay changes
< sipa>
(but not now)
< sdaftuar>
i believe all could be merged now. after discussing with sipa, i think i'll make #8312 simpler even than it is now, since no one has reviewed anyway
<@wumpus>
right, release notes is not urgent now, just needs to be done before final
<@wumpus>
let's worry about rc1 now
< sdaftuar>
#8305 and #8295 are done as far as i know
< sdaftuar>
(no review though)
<@wumpus>
yes the problem is review - I don't think people are very active at the moment, probably tired from all the big merges such as segwit
< gmaxwell>
I've been very busy with testing, in fact.
<@wumpus>
testing is good
<@wumpus>
releasing rc1 will result in a lot of testing, too
< sipa>
wumpus: i was less active the past week; i'll be back tomorrow
<@wumpus>
ok I see jonasschnelli already added 0.13 milestone to sdaftuar's pulls, thanks
< jonasschnelli>
Yes.
< cfields>
I've been inactive as well. Nice and refreshed now, I can help with review of 8295/8312 once the build issues are worked out
< petertodd>
i'll try to look at 8312 this weekend
<@wumpus>
cfields: I wish we could get rid of the as-root patching requirement for arm linux, it'd make things much easier build-ssytem wise
< cfields>
wumpus: i can look into a quick-hack work-around for rc1. but iirc, it's hard to avoid until ubuntu fixes
<@wumpus>
cfields: well for 0.14 we can switch to 16.04, which I guess has fixed this, but for 0.13 we're stuck with it. I don't mind the gitian prepare script change, but I'm not sure what the gitian people think about it, devrandom hasnt' commented on it
< cfields>
wumpus: quick alternative would bt c/p the descriptor to linux-arm.yml for now
< gmaxwell>
will we be able to distribute arm linux binaries for 0.13?
<@wumpus>
gmaxwell: yes, that was done, but it has added an awkward requirement to the gitian build descriptor
< jonasschnelli>
yes
< cfields>
wumpus: would you like me to just copy the descriptor for now? The issue is the conflicting toolchains, so that would solve it. It would just mean an extra gbuild.
<@wumpus>
in any case, to conclude this topic: https://github.com/bitcoin/bitcoin/milestone/20 is an accurate reflection of what has to be done before rc1? (apart from the release notes issue)
< petertodd>
wumpus: ack
<@wumpus>
cfields: I'll try to contact devrandom about the gitian change, if we can't speed that up, I think splitting up the descriptor makes sense. It does mean temporary changes to the release process, I preferred it this way :)
< MarcoFalke>
The rpc-test is pretty simple: usehd=1, then backup wallet.dat
< jonasschnelli>
The addresses are re-generated deterministic... but somehow reindex fails..
< MarcoFalke>
Do some transactions and discard wallet.dat
< gmaxwell>
If I understand the issue correctly there is a design limitation that we can easily work around. I believe the limitation is that it only scans the keys currently in the pool, and when the pool expands it doesn't go back at all, and it doesn't expand the pool based on used keys as it goes. This means that recovery will miss things without a recan in many cases.
< gmaxwell>
But I could misunderstand.
< jonasschnelli>
Yes. The simples HD feature does not cover restores...
< MarcoFalke>
I am running rescan and reindex. None of it fixes it
< gmaxwell>
If I am not incorrect here, then simply setting the keypool default really large when usehd is set would be an adequate workaround, I believe.
< jonasschnelli>
Restore is manual work (including a rescan)
< gmaxwell>
if a rescan isn't fixing it, thats a more serious issue than I was thinking.
< MarcoFalke>
I was logging the result of IsMine on each tx and it retured false, so the transactions were never added to the wallet in the second run (rescan)
< jonasschnelli>
Yes... I currently trying to find out why MarcoFalke test fails even after a rescan.
< gmaxwell>
In any case we cannot RC it with a serious doubt about it not working. It could be disabled in an RC if needed.
<@wumpus>
but if you set keypool really large then it will generate and store all those keys, defeating the purpose of using HD in the first place, or not?
< jonasschnelli>
But IMO it's unrelated to HD,... the keys are recreated.
<@wumpus>
yes, I think this scary, I don't think this is ready for a release
< gmaxwell>
wumpus: it's the same keys every time, no defeating.
< jonasschnelli>
Sure. Let me first try to find the root cause before we consider a revert
< MarcoFalke>
agree
<@wumpus>
rever wouldn't be necessary, just default the flag to false
<@wumpus>
it defaults to true for new wallets now which is what makes it scary if it would be somehow broken
< jonasschnelli>
Sure. But we probably don't want to ship a (per default disabled) feature which _could_ includes bugs. :)
< gmaxwell>
In any case we cannot RC it with a serious doubt about it not working. It could be disabled in an RC if needed. If the issue can be reduced to needs rescan I described then I think increasing the pool is adequate.
< petertodd>
wumpus: is the flag documented in --help?
< gmaxwell>
We need to understand the issue at a minimum or make it unavailable, too risky as even an option. IMO. But I don't have any doubt that we will understand the issue in a day or so.
< jonasschnelli>
MarcoFalke: have you tried the test with HD disabled (just import your created keys)?
< gmaxwell>
I haven't looked at it yet myself except passively watching the discussion.
< sipa>
likewise
< sipa>
i will look soon
< MarcoFalke>
no
< jonasschnelli>
I think without knowing the root cause of the problem actions are useless.
<@wumpus>
agreed
< sipa>
agree; we need to understand it
< jonasschnelli>
I will know more about it in a couple of hours.
< sipa>
and i believe there is plenty of time for that
<@wumpus>
before when?
< gmaxwell>
Okay well the action is to determine the issue(s). :) Though if wumpus decides to cut an RC before we do, I'd ask that the option be disabled in it.
<@wumpus>
yes, need to disable it completely then, I agree
< jonasschnelli>
Yes. But it could also be a serious import/rescan issue (not related to HD)
<@wumpus>
the non-HD rescan test passes, right?
< MarcoFalke>
yes
< sipa>
i believe that rpc test is less elaborate
<@wumpus>
did you try your new test without HD MarcoFalke?
< jonasschnelli>
I think we would need the same test (that fails) without HD
< MarcoFalke>
You can't create keys deterministically without HD
< jonasschnelli>
I can see the exact keys in the wallet after the restore but I can see rescan failing to reconstruct the wtxs.
< sipa>
no need to discuss the details here
<@wumpus>
oh sure, you can't do exactly the dsame thing
< jonasschnelli>
MarcoFalke: you could create 100 keys and reimport them...
< jonasschnelli>
anyways,... I'll have a look at it and will report on the PR
< gmaxwell>
in any case, I think this can move outside the meeting now, we known what general things we need to do to make progress.
< jonasschnelli>
ack
<@wumpus>
right
<@wumpus>
any other topics?
< gmaxwell>
Sure, an announcement:
< sipa>
*crickets*
< jonasschnelli>
oO
< * petertodd>
braces for an incoming text wall
< gmaxwell>
Matt has announced his new relaynetwork work that uses UDP and FEC, http://bluematt.bitcoin.ninja/2016/07/07/relay-networks/ the current not really fully cooked software gets worldwide block propagation 99% of the time in less than 100ms over the fiber-path distances.
< petertodd>
+1
< jonasschnelli>
Saw this. Impressive work! Well done.
< gmaxwell>
(actually 98% of the time in the last day)
< gmaxwell>
And his deployment uses only cheap VPSes.
< sipa>
wow, i saw the website but hadn't appreciated how good the numbers were
< sipa>
i just looked at how oretty the graphs were
< gmaxwell>
may be of some interest to some people, he's trying to get other people to setup parallel networks, and put up an extensive guide, including tricks for getting access to low latency connectivity between asia and europe. :)
< petertodd>
wumpus: what? don't you have a few tonnes of heavy water laying around?
< gmaxwell>
Including the actual speed of light delays, he gets worldwide sync in 98% of the time in under 238ms, 80% in under 165ms. So thats all fun.
< sipa>
i believe production of heavy water may be more decentralized than sha256 asic mining production.
< gmaxwell>
onto other subjects, I'd still like to see a testnet-defaulting binary during the RCs. My contribution to that effort was getting master working correctly on testnet again. :) but I haven't done anything else. :(
<@wumpus>
petertodd: of course I do, but not enough shielded undergorund space to put it
< gmaxwell>
there are trillions of gallons of heavy water not so many miles from me; ... it's just unfortunately mixed with lots of light water and salt.
< petertodd>
wumpus: that's why I took up caving
< cfields>
gmaxwell: what's the reason for an additional binary?
< cfields>
gmaxwell: oh, you mean default to testnet until final release?
<@wumpus>
gmaxwell: we could do that now, eg before the rc1
<@wumpus>
well at least if I could build releases, dang
< gmaxwell>
wumpus: yea, I think we should too, we just couldn't do it when testnet was getting stuck for many, but it's fixed now.
<@wumpus>
cfields: not the rc1 itself, more like a master-build-with-only-testnet-enabled
< gmaxwell>
cfields: I've found that a lot of people would like to play with it are actually thrown off by setting an option. It's not so intutive for GUI users. I think this would greatly increase testnet usage to have builds that work more like bitcoin/altcoin installs.
<@wumpus>
rc1 should definitely have mainnet by default, like any rc
<@wumpus>
gmaxwell: did you see #8285? it's super-easy now for windows GUI users
< cfields>
hmm
< gmaxwell>
one possibility I considered is that we could just sniff the binary name and have -testnet and -testnet-qt change the default, and ship a link. :P
< petertodd>
gmaxwell: I wonder if a wrapper script with -testnet would help those users?
< gmaxwell>
but perhaps thats too much for now. :)
< gmaxwell>
pfft. plenty of people have gone after hours and spent the rest of their lives there.
< jonasschnelli>
self.node_args[1].extend(['-rescan']) should be self.node_args[1] + ['-rescan']
< MarcoFalke>
Wow
<@wumpus>
jonasschnelli: wow, very good news, and you found it within the meeting :D
< petertodd>
jonasschnelli: oh, is node_args[1] a tuple?
< gmaxwell>
that may just leave us with the surprising behavior that it doesn't fill the pool on its own. which would be good news, since thats easy to workaround.
< jonasschnelli>
I'm not familiar with extend() but seems to be the wrong function for that purpose.
< jonasschnelli>
But I guess we want MarcoFalke's RPC test in 0.13! :)
< petertodd>
jonasschnelli: extend() is list only, and modifies in place
<@wumpus>
yes, extend is in place and returns nothing
< jonasschnelli>
At least we tested again the HD feature...
< sipa>
6 minutes
< sipa>
are we done?
< gmaxwell>
sipa: until we lose you to the catacombs?
<@wumpus>
I think so?
<@wumpus>
#endmeeting
< lightningbot>
Meeting ended Thu Jul 7 19:55:31 2016 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< sdaftuar>
wumpus: can you advise how to handle the warning string in #8295, if -blockminsize is given? i left it untranslated inside an InitWarning, wasn't sure if that was the right thing to do
< petertodd>
fyi, I can't dig it up right now because webbtc.com is broken, but I noticed the other day that we had the first CSV redemption on mainnet!
<@wumpus>
sdaftuar: agree with leaving it untranslated
< jonasschnelli>
*server
< sdaftuar>
wumpus: ok thanks!
< gmaxwell>
jonasschnelli: Not unless it knows the public key.
< jonasschnelli>
I just try to find a/the concrete reason for the believed_server_pubkey in H(session_id || "1" || believed_server_pubkey)
< gmaxwell>
jonasschnelli: If the server has no match for X1 the protocol is just filled with padding at that point.
< jonasschnelli>
How could the client fingerprint when believed_server_pubkey would not be there?
<@wumpus>
sdaftuar: I think some of the InitWarnings are translated, but anyhow, nothing new is going to get translated for 0.13.0
< gmaxwell>
jonasschnelli: As I said, it prevent fingerprinting. The server does not learn what public key the client is expecting; the client cannot learn what public key the server is using (unless it already knows it)
<@wumpus>
sdaftuar: and it's going to be a rare message for people to see, so even otherwise probably a waste of translators time :)
< gmaxwell>
jonasschnelli: if that was not there the client would send the AUTHCHALLENGE and the server would reply with a signature in the AUTHREPLY, which the public key can be extracted from.
< sdaftuar>
wumpus: makes sense to me, just wasn't sure if we were allowed to put untranslated strings into things that might hit the gui
< jonasschnelli>
gmaxwell: ah... even for standard compact normalized signatures?
< sdaftuar>
versus doign a LogPrint() or something
< jonasschnelli>
gmaxwell: I thought in order to do this, it would require a recoverable signature (something like libsecp256k1s secp256k1_ecdsa_recoverable_signature_serialize_compact)
<@wumpus>
sdaftuar: normally would prefer not, but for (detailed) errors can make an exception - not translating them makes them easier to google, for example, and very technical things tend to get translated badly
< gmaxwell>
jonasschnelli: no, without the extra data you just have two possibilties (well 4 with very very low probablity), so no resistance to fingerprinting at all.
< gmaxwell>
jonasschnelli: Does it make sense now?
< jonasschnelli>
gmaxwell: Okay. Got it. Thanks for the patience to explain.
< bsm117532>
Maybe I'm missing something...why does BIP 144 not define this transaction format as nVersion=2?
< sipa>
because it does not add anything
< sipa>
nVersion is part of the transaction data
< sipa>
so you can't modify it upon relay
< sipa>
bip144 adds an extra serialization flag that is not hashed into the txid
< gmaxwell>
jonasschnelli: no problem.
< bsm117532>
So I've got python-bitcoinlib serializing and deserializing according to what I read in BIP 144, but decoderawtransaction doesn't like my serialization. Does anyone have a moment to take a look at this thing and tell me what I've done wrong?
< bsm117532>
So, my input script isn't empty, it's OP_0 <hashed pubkey> as BIP 141 says. But in your test vectors in BIP 143, the input script for P2WPKH is empty.
< sipa>
OP_0 <hashed pubkey> goes into the output
< sipa>
the scriptSig spending segwit outouts is always empty
< bsm117532>
I'm also missing the array length in the witness.