< MarcoFalke>
The python script was calling invalidateblock... I should try to reproduce this.
< gmaxwell>
sipa: your siphash changed missed CompareNetGroupKeyed.
< sipa>
oh!
< gmaxwell>
(I suppose I should also change all those sorts into heapification too).
< kanzure>
matsuo is asking for information about known minimum requirements, but i think he wants to be using a fork of segnet, so i'm not sure what to tell him.
< BlueMatt>
jonasschnelli: thats not a seed....thats a single host
< BlueMatt>
(which is on a residential connection)
< tErik_mc1>
yesterday, while my windows computer had around "6 weeks" of data left to complete blkchn sync, something happen, which i'm not sure, suddenly the monitor was blank, but OS seems to be still running fine, i can see drive led indicator blinking, and i could here audio from it . 1st time something like this happened ! had to force shutdown the pc. then bitcoin-qt keep on showinh error "Error reading from database, …"
< BlueMatt>
jonasschnelli: but, it generally is incredibly reliable :)
< tErik_mc1>
so i ran : bitcoin-qt -reindex , but its still 27weeks of blkchn remains to be synced :( is it ever gonna finish ?
< MarcoFalke>
It could take longer than the initial download right now.
< tErik_mc1>
according to byte counters, bitcoin-qt.exe has used only 20MB download & 114MB upload data.
< sipa>
your block chain index or chainstate is likely corrupted
< MarcoFalke>
jup, reindex will try to read from the disk first
< sipa>
ah, during reindex it will not download, just process what you already have first
< tErik_mc1>
so i guess, most files are there until yesterday's position of 7 week, until then it will not download anymore, but after reaching 7week position it will start download+sync again.
< sipa>
yes
< tErik_mc1>
hi sipa :)
< tErik_mc1>
i talked with you in openbazaar slack chan.
< sipa>
that is very unlikely
< tErik_mc1>
ah i see, sorry about that, theres another user using similar handle as your there.
< tErik_mc1>
If i quit from bitcoin-qt.exe now, and restart computer to complete a system-update, then do you guys think i will have to start re-index from beginning again ?
< sipa>
you can quit and start again, but don't pass -reindex again
< sipa>
if you pass -reindex again, it will start over
< sipa>
if you don't, it will continue
< tErik_mc1>
ah i see, THANKS for clarifying. i wish i had much powerful processor than what is it now (an i3 with 1 core 4 thread) . i guess, if bitcoin-qt is put into a SSD drive then it will be super fast, right ?
< sipa>
no, but increasing its database cache size will help (start with -dbcache=2000 if you want to give it 2 GB)
< tErik_mc1>
2GB of RAM, dedicated for it ?
< sipa>
yes
< tErik_mc1>
:) awesome :)
< sipa>
or whatever number
< tErik_mc1>
i sure can assign 2GB, this computer is 6GB.
< tErik_mc1>
i was about to get those disk-caching software loaded into this, after reboot.
< sipa>
don't
< tErik_mc1>
you are sub-conscious mind-reader, or we have a link.
< tErik_mc1>
the bitcoin-core supports & uses DNSSEC based verified DNS resolution or simple DNS resolution, i'm using "Unbound" local dns resolver.
< sipa>
it just uses that to find other nodes
< tErik_mc1>
If a bitcoin-core node is using residential ip-address, but it has a dyn-dns address (which resolves into that ip-address), then bitcoin-qt can be given -dns commandline option to connect with it ?
< sipa>
please read the documentation (run bitcoind -help)
< sipa>
-dns has nothing to do with all that, and it's on by default
< petertodd>
jonasschnelli: looks fine here; we've had some dns issues that have been hard to track down...
< jonasschnelli>
petertodd: Yes. Works again!
< petertodd>
jonasschnelli: odd, I didn't change anything... :)
< tErik_mc1>
I restarted pc for system-upd, started bitcoin-qt with -dbcache=2048 option, bitcoin-qt started to verify blockchain, then started to re-index, from position where it was before, where i exited from btc-qt. now ram usage is higher than previous but not 2GB, its using around 630MB, 602MB, 577MB, 1.15GB (Private Bytes, Working Set, Private WS, Virtual Size).
< sipa>
2 GB is just a maximum, and only the db cache
< tErik_mc1>
And I/O total is fluctuating from 25 to 47 MB/sec.
< sipa>
during reindex? that's good
< tErik_mc1>
ya, before restart, during re-indexing, it was using i/o close to 89 MB/s.
< tErik_mc1>
But i guess thats because i assigned "higher" i/o priority and "higher" app priority.
< tErik_mc1>
s/app/process/
< tErik_mc>
bitcoin-qt.exe now using 2.3 GB, 2.15 GB, 2.12 GB, 2.46 GB (Private Bytes, Working Set, Private WS, Virtual Size) memory. my router restarted, after that i now see, I/O total rate has fallen to mere 2 to 5 MB/s :(
< tErik_mc>
I have now re-set bitcoin-qt.exe process priority to "Normal" & i/o priority to "Normal" level, now its shifting in between 5.6 to 14 MB/s.
< tErik_mc>
And, after i/o & process priority set into "High" level, now shifting inbetween 7 to 17 MB/s.
< tErik_mc>
Sync is still "22 weeks behind" :(
< tErik_mc>
Restarted bitcoin-qt with -dbcache=1792 and immediately set i/o priority to "High", then process priority to "High", now total i/o rate is in between 25 to 45 MB/sec during re-indexing (in average i would say 40 MB/s) :) (and before that, during block verifying stage, i have seen 170 MB/s often, though it started with 240MB/s, then average rate was around 120 MB/s).
< tErik_mc>
I think few other windows-services create conflicts or purposefully keeps (i/o r/w rate) speed at lower level, i just closed various "Intel" related stuff (this mobo is Intel chipset based) carefully, and few other apps & services from computer manufacturer too.
< tErik_mc>
… before starting with -dbcache=1792 .
< luke-jr>
tErik_mc: you may be CPU bound
< luke-jr>
earlier blocks have no transactions/signatures to verify
< tErik_mc>
bitcoin-qt is using around 25% cpu . its a i3 1 core with 4 thread 3.3GHz cpu, with 6GB 1333MHz ddr3 ram.
< luke-jr>
maybe IO is low because the cache is sufficient?
< tErik_mc>
actually fluctuating in-between 16% — 25% cpu usage.
< tErik_mc>
yes, this cpu has much much less cache, only L2 256 KB, no L1 , no L3 :( and this hard drive is also a sata-ii ide 2.5".
< luke-jr>
I'm talking about dbcache
< luke-jr>
IO is only likely to occur much when you are overflowing that
< luke-jr>
although I'd think you'd be getting more CPU use if it's just using cache mroe
< tErik_mc>
i can max assign 3GB to this bitcoin-qt process, have other servers on this as well.
< tErik_mc>
but bitcoin-qt seems to exceed what i'm specifying !
< wumpus>
1792 already makes flushes very rare
< tErik_mc>
so lets say 2048+768= 2816 MB would be better than 1792 MB dbcache ?
< wumpus>
hardly
< wumpus>
the next step up would be setting dbcache high enough to fit the entire utxo set, 6000 or so, but I doubt you're limited by the db i/o
< tErik_mc>
what min ram would be better ?
< tErik_mc>
*min dbcache ram
< luke-jr>
tErik_mc: GUI or RPC? are you polling it often?
< tErik_mc>
GUI
< luke-jr>
maybe see if RPC does any better?
< wumpus>
did you change -par setting?
< tErik_mc>
my side block-chain download isn't completed yet :( pc crashed yesterday, so i'm doing re-indexing, i was at "6weeks behind", ow at 18weeks behind.
< luke-jr>
I don't see how that is relevant to our suggestions
< tErik_mc>
no other than, -dbcache param, i haven't specific anything else, i have not created bitcoin.conf either.
< tErik_mc>
no other param was specified other than -dbcache param , i have not created bitcoin.conf either.
< tErik_mc>
by RPC, are you indicating i should run the bitcoind.exe daemon ?
< luke-jr>
yes, after shutting down qt
< tErik_mc>
i have assumed that, bitcoind requires passing bunch of params in commandline, as i dont have an example command, i was using bitcoin-qt.exe
< MarcoFalke>
It should not require any params
< tErik_mc>
so for running bitcoind, i would 1st need to create+set the bitcoin.conf, and then run bitcoind and then bitcoin-cli, right ?
< luke-jr>
just run it like you run qt
< tErik_mc>
ah i see. thanks.
< tErik_mc>
i have started bitcoind, but forgot to specify -dbcache :( should i press Ctrl+C in Cmd.exe shell window, to stop it ?
< wumpus>
bitcoin-cli stop
< tErik_mc>
how do i specify parms to view debug info of category "bench" "net" etc ?
< MarcoFalke>
-debug=net or -debug=1
< tErik_mc>
and i should specify dbcache=6000 <— my max ram, right ?
< tErik_mc>
windows or bios showed max 5.8 usable, so i guess i will specify -dbcache=5888
< wumpus>
specifying your entire usable ram as dbcache is not a good idea, other programs will also need memory
< tErik_mc>
ah i see, so once bitcoind start to occupy more rams, avail rams for other will get reduced, and bitcoind will not release to them ofcourse.
< tErik_mc>
then i will try with 2816, sparable at this moment , on this computer . but i also understand now what you've suggested then, unless its close to 6GB, i will not see much improvement.
< tErik_mc>
My other location has a 24GB computer, i guess, i can try bitcoind with sufficient dbcache there.
< MarcoFalke>
dbcache=1000 or 2000 should be sufficient to have a really low miss-rate
< tErik_mc>
BitcoinD seems to work at slightly higher rate than bitcoin-qt, max 5 MB/s difference (i guess around 8% difference) . both are fast at block-verifying stage, when re-indexing stage "XX Weeks Behind" begins, then they are again same rate . and both slowed down after 10 or so minutes of re-indexing :(
< tErik_mc>
i have by mistake (again) started, bitcoind, but this time with -dbcache=5888 (i pressed enter, before i could change it to 2816), now, bitcoin-cli stop is not working anymore, showing error code: -28 , error message: Verifying blocks…
< tErik_mc>
i guess when it reaches re-indexing stage, then bitcoin-cli stop will work.
< tErik_mc>
And btw, i did specify -debug=1 but its not showing any debug info inside CMD shell window :(
< MarcoFalke>
You can try bitcoind -? to read the help. There is a section "Debugging/Testing options"...
< tErik_mc>
But after specifying dbcache=5888, it seems, bitcoind is very slow at finishing "blockchain-verifying" stage . but bitcoin-qt was able to finish that stage with only 1792 dbcache param, at much faster with less time !
< tErik_mc>
finally it fiished "blockchain-verifying" stage, now its stopping :)
< tErik_mc>
i'm looking at https://en.bitcoin.it/wiki/Running_Bitcoin . so may be bitcoind.exe supports -debug=net -debug=alert parameters, instead of -debug=1 . trying again.
< tErik_mc>
or could it be, that, because of the -debug=1 params was given to bitcoind.exe , it was using slowed debug DLLs, instead of faster DLLs, and so it slowed down much ?
< wumpus>
enabling all debug logging makes bitcoind marginally slower
< tErik_mc>
i tried again, this time with " bitcoind.exe -dbcache=2816 " only, though it began at much slower (8 MB/s) i/o rate & less cpu % usage, but after few mins, it sometime even reached 220 MB/s i/o rate , but averaging at 40 MB/s , and cpu usage remaining steady at 25% level . so in this turn it seems, its similar as bitcoin-qt.
< tErik_mc>
But bitcoin-qt, always remains very fast at "blkchain-verifying" stage, and then initially very fast in "re-indexing", but slowed down few mins later.
< tErik_mc>
Since cpu is most of the time close to 25%, and since this i3 uP has 1-core with 4-hardware-threads (4-sub-core) , i guess bitcoin-core is using only 1 thread. i think i should find out how many threads it really using or just pass a param to use 3 threads.
< tErik_mc>
i see, i think i now understand the "par" param, i think i need to pass -1, so that 1 core remains free.
< tErik_mc>
it seems -par=-1 reduces performance :( i will try other settings different time.
< tErik_mc>
THANKS to all of you : wumpus , luke-jr , MarcoFalke , for your kind help, i will head out for now, be back few hours later.