< sdaftuar>
sipa: fyi i added a few sentences to the release notes wiki page mentioning the getdata randomization change in 0.18, but feel free to change it if you had something else in mind
< CubicEarth>
IBD goes very fast until it *seems like* the DB cache "fills up", then it basically stops until I restart the node. It seems to process about 50 - 70 GB worth of transactions before the 4 GB cache becomes full. Is this expected behavior?
< gwillen>
CubicEarth: one confounding factor is that IBD slows down a lot over time as blocks get more full
< gwillen>
(by 'over time' I mean 'as it proceeds through history')
< gwillen>
I haven't observed the behavior you describe and I'm pretty sure it's not expected, as it would be kind of dumb
< gmaxwell>
CubicEarth: I think you're just seeing chain growth, you're also probably mistaken about it stopping.
< CubicEarth>
I'm aware of the slowing-down phenomenon, and this certainly isn't that. This machine - 2012-ea 2.4 Ghz i5, spinning disk, 8GB ram - can easily do two - 2018-type blocks per second
< gmaxwell>
CubicEarth: what you're thinking is "stopping" is that its can't connect more blocks until a particular peer replies with a requested block (or gets timed out)
< CubicEarth>
gmaxwell: In this case, I only have one peer, and that is another full node on my LAN
< gmaxwell>
during that time it doesn't log updatetips (but its still downloading in the background, at least until there are 1000 out of order blocks waiting)
< gmaxwell>
Or you're just seeing the dbcache flush, and if you've cranked that to 4gb then on a slow drive the flush can take two minutes.
< gmaxwell>
but as soon as its done flushing it'll continue like before.
< CubicEarth>
I think it has something to do with the flush, because when I go to close it down, then it does take 2 (or more!) minutes. I was wondering if for some reason it wasn't initiating the flush on its own
< gmaxwell>
it flushes when the dbcache fills or when you shut down (or after 24 hours, in normal operation)
< gmaxwell>
Someday it will flush incrementally in the background. Historically the requirement for consistency made that impossible but last year we did some redesign that eliminated that issue. But doing the rest of the work to track and flush incrementally, without hurting performance, isn't trivial and no one is working on that right now AFAIK.
< echeveria>
I haven't seen very many blocks recently that miss compactblocks, even when they do its using the extra pool. that's just delicious.
< CubicEarth>
Well, once this I'm done with this sync, I'll get another laptop going, and take more careful notes about what I see. I'm suspecting that something is preventing it from flushing the DB when it gets full
< CubicEarth>
gmaxwell: good to know about the incremental flush concept
< gmaxwell>
CubicEarth: I don't think thats really possible, in any case, turn up debug logging (debug=1) and you should see it log every 16mb flushed.