< gmaxwell>
Patches to AFL that let you target specific parts of code, e.g. to fuzz test a patch: https://github.com/aflgo/aflgo
< ossifrage>
FYI the twitching "Reindexing blocks on disk..." did not damp out as I made progress, now it is at 76% and twitching between 7 and 30 weeks
< sipa>
question: how to deal with existing tests that use getnewaddress/sendtoaddress? they often don't work without modification, as spends from segwit outputs don't work before segwit activates (in block 432 on regtest)
< sipa>
1) force all tests to stick with legacy addresses
< sipa>
2) make getnewaddress etc fall back to legacy addresses before segwit activation (but that's not something you want on mainnet, as it could lead to accidentally creating a legacy address if you're too fast
< sipa>
3) have an cmdline argument to make segwit activate from genesis in regtest (which would be on by default, excepr for tests that actually test the transition)
< sipa>
4) use -prematurewitness liberally, for cases where it's only wallet logic that is being tested (not validation/consensus)
< sipa>
5) adapt all tests to mine enough blocks up front (but that's a lot of changes... getbalance calls everywhere)
< jl2012>
sipa: I think 3) is the best, as it could also show all existing tests pass with the segwit getnewaddress
< meshcollider>
I agree, 2 is quite nice in theory but I think 3 is best practically
< MarcoFalke>
5 is an ugly workaround, lets not do that.
< gmaxwell>
5 is strictly worse than 3. I like 3. technically we could make mainnet activate segwit at the same time as p2sh, which would also result in regtest being born-segwit... but it would break transistion tests.
< gmaxwell>
I don't really think we need transition tests anymore, but they're probably also incidentally testing other useful things so probably not great to just drop them.
< wumpus>
thinking about it, I don't really think "The change as written now allows to test the existence of any file." is a concern that should have prevented it from being merged, it's much better than being able to overwrite every file
< morcos>
sipa: i'm in favor of 3 or even just dumping transition tests entirely. i know sdaftuar was looking at these for his attempts to have segwit active earlier, we should get his opinion
< sdaftuar>
sipa: i agree with 3) as the best idea for now.
< sdaftuar>
gmaxwell: sipa: i was experimenting with moving segwit activation back to p2sh activation for mainnet, to see how much code simplification we could get
< sdaftuar>
it ended up being a lot messier than i thought and i got a little stuck on that project--
< sdaftuar>
moving SCRIPT_VERIFY_WITNESS enforcement back is a straightforward change, but the rules around witness commitments in the coinbase are not really changeable, as far as i can tell
< sdaftuar>
the simplest example -- pre-segwit, someone mines a block with a witness commitment that commits to the transactions all not having a witness -- doesn't validate, because checking the commitment
< sdaftuar>
requires checking the witness nonce, which is not present
< sdaftuar>
never mind if there are blocks which have an incorrect commitment (pre-activation) -- i assume those exist too
< sdaftuar>
so i was left wondering if it was worth splitting segwit activation into two parts -- one where witness commitments are only checked at some height, but SCRIPT_VERIFY_WITNESS is enforced eg from genesis
< sdaftuar>
and my instinct was that was only complicating things, and not simplifying
< sdaftuar>
but i'd be interested in others' opinions
< jnewbery>
sdaftuar: I think it's still worthwhile, even if you need to split SCRIPT_VERIFY_WITNESS enforcement from witness commitment verification. We could then change SCRIPT_VERIFY_WITNES activation height on regtest to 0 and remove all the transition tests (except one test for verifying that witness commitment verification is not enforced before witness commitment verification height)
< jnewbery>
sipa: (3) is best for now. It'd be nice to eventually dump most transition tests if possible
< instagibbs>
how would people feel about removing mempoolreplacement arg? There's no tests, and I have yet to hear a reason to not allow it.(or people setting it to off)
< jonasschnelli>
wumpus: Oh. I completely forgot #9937. We should have merged it, yes.
< jnewbery>
luke-jr: I'm looking at #11383 (nice work btw!). I don't understand the comment about it requiring #10615. It looks like they're basically orthogonal (except a bit of overlap in the RPC console). I think it'll probably aid review if you separate them out so there's no cross-dependency.
< gribble>
https://github.com/bitcoin/bitcoin/issues/10615 | RPC: Allow rpcauth configs to specify a 4th parameter naming a specific wallet (multiwallet RPC support) by luke-jr · Pull Request #10615 · bitcoin/bitcoin · GitHub
< luke-jr>
jnewbery: 10615 has the commit resolving the wallet for RPC earlier, so GUI can assign it
< jnewbery>
luke-jr : That's only used for the RPC console. If I just cherry-pick all the commits apart from that, then it works fine without 10615. There are still several outstanding concerns in 10615, so I think it makes sense to separate the two PRs and allow them to be reviewed separately
< luke-jr>
jnewbery: it should be used by WalletModel too (although maybe it isn't yet)
< luke-jr>
(pretty sure it is tho)
< achow101>
.... wow. I managed to get bitcoin 0.1.0 to start syncing off of Core 0.15.99
< sipa>
achow101: i assume you needed to patch 0.1.0 a bit?
< achow101>
sipa: I patched Core to speak 0.1.0
< achow101>
and apparently bitcoin.org's docs are wrong
< achow101>
are coinbase transactions supposed to have witnesses?
< achow101>
I'm seeing coinbase transactions with witness fields of 32 bytes of 0's
< achow101>
*1 stack item of 32 bytes of 0's
< sipa>
achow101: yes
< sipa>
achow101: they're a field intended for extensibility
< achow101>
sipa: oh, ok. I don't see that in the BIPs though..
< achow101>
oh, nvm. found it in bip 141
< sipa>
it's called 'witness reserved value' in the bip
< thomas__>
Hey guys, new here. Does someone have to work on windows and has a good workflow set up ?
< sipa>
a good workflow for what?
< thomas__>
to dev
< thomas__>
I'll be rebuilding a lot this weekend to see how things work, if someone has a way to make it less painful I'd like to hear from him ^^
< achow101>
thomas__: use linux and read the Bitcoin Core docs
< thomas__>
damit, everyone has the same answer. I have an ubuntu set up on my computer, but I also have to do stuff on windows. I don't see myself switching 3 times a day between both.
< achow101>
thomas__: then use a VM
< achow101>
developing things is infintely easier to do in a *nix environment than windows
< sipa>
how well does bash-on-windows work these days?
< achow101>
sipa: pretty well, but AFAIK, no gui support
< thomas__>
the wsl works if you keep all your file on the linux side
< achow101>
i.e. can't run bitcoin-qt
< thomas__>
I'm cross compiling right now, I'll know after that
< achow101>
thomas__: I suppose you could use wsl and cross compile. note that cross compiling with ubuntu 15.10+ (wsl uses ubuntu 16.04) is a bit flaky
< * luke-jr>
ponders if BIP editor should be enabled to fix spelling/grammar without going through the author
< ossifrage>
That is really annoying chrome decided to eat up all the memory, but the oom killer took out bitcoin (while doing a reindex) and somehow the progress went from 99ish% to 85%, that is quite a bit of rollback
< esotericnonsense>
ossifrage: how long does the entire reindex take? if it's the same mechanism as during IBD then with a high dbcache value it's just every N minutes/hours (don't remember the figure)
< esotericnonsense>
(sorry, that's a flush happening every N minutes/hours)
< gmaxwell>
ossifrage: it'll rollback to the last flush. In the future we'll hopefully switch to more incremental flushing which will roll back less far in the event of a crash during initial sync.
< ossifrage>
gmaxwell, I had a 4G dbcache which was part of the reason the oom killer picked on bitcoin
< gmaxwell>
I wish linux mem handling had a "shrink now or you're gonna get killed" signal.
< gmaxwell>
esotericnonsense: that means it'll force a flush once a day if it hasn't been triggered by the cache filling.
< gmaxwell>
during IBD cache filling triggers the flushes.
< ossifrage>
esotericnonsense, I am generating the full tx index (txindex=1) and it is taking a very long time [I'm not sure when I started it, the debug.log got truncated]
< ossifrage>
>24 hours ago
< gmaxwell>
>24 hours with a 4gb dbcache. damn txindex.
< esotericnonsense>
think my laptop was sub 24hours when I did it and it's not super-fast or anything. though that was syncing over LAN rather than reindex.
< esotericnonsense>
(with txindex on).
< esotericnonsense>
is that on a HDD?
< ossifrage>
gmaxwell, it doesn't seem to be IO or CPU bound, but I am using spinning rust, it seemed wasteful to eat up a large % of my ssd on bitcoin
< esotericnonsense>
ah yeah.
< gmaxwell>
ossifrage: with a large dbcache it doesn't matter if you're on a SSD or rust when txindex is not in use, alas...
< ossifrage>
If I turn txindex off, do I have to start over again when I turn it back on?
< ossifrage>
The last log entry was 96% with a 2446MB dbcache and it rolled back to 84% (my memory of almost done was faulty)
< esotericnonsense>
ossifrage: if you have enough log, you can go back and find the time when dbcache reset, that will be the last flush probably at 84%.
< bitcoin-git>
bitcoin/master 22fd04b Gregory Maxwell: Remove nBlockMaxSize from miner opt struct as it is no longer used.
< bitcoin-git>
bitcoin/master c6223b3 Pieter Wuille: Merge #11362: Remove nBlockMaxSize from miner opt struct as it is no longer used....
< bitcoin-git>
[bitcoin] sipa closed pull request #11362: Remove nBlockMaxSize from miner opt struct as it is no longer used. (master...2017_09_rm_nBlockMaxSize) https://github.com/bitcoin/bitcoin/pull/11362