<bitcoin-git>
[bitcoin] fjahr opened pull request #30880: test: Wait for local services to update in feature_assumeutxo (master...2024-09-au-services-test) https://github.com/bitcoin/bitcoin/pull/30880
<laanwj>
<dzxzg> "I suspect it's probably not..." <- allocation overhead can definitely explain an OS specific difference
Emc99 has quit [Quit: Client closed]
<laanwj>
libc malloc tends to be lots faster than what mingw uses
<dzxzg>
Yeah it doesn't make sense to me either that the allocation is the issue
<dzxzg>
Could it be `ftell`? Either it's own overhead or preventing some optimization of writes
<sipa>
The cost of an `ftell` could differ wildly between platforms.
<laanwj>
yes, and file systems
<sipa>
If there are are many more of those if XOR is enabled vs. not, that could explain things.
<achow101>
each call to AutoFile::write() has a ftell if xoring
<achow101>
if someone can figure out how to profile on windows, that'd be great
<laanwj>
it could cache the current position, right?
<sipa>
yeah, i was just going to suggest that
<sipa>
it's possible that fwrite is cached by the libc layer, while ftell always triggers a kernel call?
<laanwj>
std::fwrite is cached, yes
<laanwj>
and it's likely ftell is not optimized in any way because software uses it only rarely (usually to seek once to the end to determine file size)
<maflcko>
Yeah, ftell being the issue seems the most plausible explanation
<laanwj>
yeah the one-time 4096 byte allocation per write call likely isn't going to be the overhead
<vasild>
sipa: I think no, I do not think it can happen that ftell reports incorrect because it misses a recent fwrite which is cached and ftell asks the kernel directly
<bitcoin-git>
[bitcoin] fanquake merged pull request #30865: build: Skip secp256k1 ctime tests when tests are not being built (master...240910-ctime-test) https://github.com/bitcoin/bitcoin/pull/30865
<maflcko>
Writing code to cache it would be trivial, but then one could no longer use fseek outside the class
<vasild>
It would be simpler to poll(2) all sockets and then omit nodes based on these criteria (eg pnode->fPauseRecv).
<vasild>
Anybody sees or remembers a special reason to do the filtering before poll(2)?
<sipa>
vasild: i think i last touched that code
<vasild>
and? :)
<sipa>
i don't see a problem with always polling and doing the filtering later
<vasild>
\o/
<laanwj>
couldn't polling connections unnecessarily have a potential performance overhead?
<laanwj>
not sure it makes much of a difference, it's one system call either way, but i suspect that's the reason to only add conditionally
<sipa>
laanwj: maybe, but i sort of assume that polling many file descriptors at once is pretty well optimized in the kernel, so the marginal cost for an extra one is low
<sipa>
likely more so than our own code for determining what to poll
<sipa>
anyone have a clue why building block413567.raw.h seems to take forever?
<laanwj>
agree, it's not like we're dealing with ten thousands of extra fds here anyway
<fanquake>
sipa: probably a regression from #30842
<gribble>
https://github.com/bitcoin/bitcoin/issues/30842 | build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake` by hebasto · Pull Request #30842 · bitcoin/bitcoin · GitHub
<fanquake>
a similar issue was also reported there
<sipa>
fanquake: thanks
<fanquake>
ill open an issue to track
<fanquake>
given it was meant to be an optimsation for native windows, which seems to be causing perf issues on linux, should probably be reverted, unless we find a quick fix
<dzxzg>
I tried setting up windows profiling in Visual Studio but could only get the CPU profiler to work, on my system at least Visual Studio just crashes when I try to use the more sophisticated "Instrumentation" profiler that measures call counts and wall clock time taken by calls with bitcoin core. (You have to add the /PROFILE linker flag to use the
<dzxzg>
instrumentation profiler)
zeropoint has joined #bitcoin-core-dev
<maflcko>
PSA, if you receive the "At least one of the CI tasks failed." DrahtBot notification, it is a true issue with the pull request, and not one of the (sadly currently common) unrelated ones.
<sipa>
good DrahtBot
<bitcoin-git>
[bitcoin] hebasto opened pull request #30883: Revert "build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`" (master...240912-slow-string) https://github.com/bitcoin/bitcoin/pull/30883
<bitcoin-git>
[bitcoin] sipa opened pull request #30884: streams: cache file position within AutoFile (master...202409_reduce_ftell_xor) https://github.com/bitcoin/bitcoin/pull/30884
<bitcoin-git>
bitcoin/master fa7087b MarcoFalke: util: Use compile-time check for FatalErrorf
<bitcoin-git>
[bitcoin] ryanofsky merged pull request #30546: util: Use consteval checked format string in FatalErrorf, LogConnectFailure (master...2407-log) https://github.com/bitcoin/bitcoin/pull/30546
aleggg has joined #bitcoin-core-dev
Talkless has joined #bitcoin-core-dev
<bitcoin-git>
[bitcoin] fjahr opened pull request #30885: scripted-diff: Modernize nLocalServices to m_local_services (master...2024-09-localservices-new) https://github.com/bitcoin/bitcoin/pull/30885
dzxzg has joined #bitcoin-core-dev
<bitcoin-git>
[bitcoin] instagibbs opened pull request #30886: rpc: Add support to populate PSBT input utxos via rpc (master...2024-09-updateutxo_psbt) https://github.com/bitcoin/bitcoin/pull/30886
___nick___ has joined #bitcoin-core-dev
<bitcoin-git>
[bitcoin] l0rinc opened pull request #30888: build: Minimize I/O operations in GenerateHeaderFromRaw.cmake (master...lorinc/cmake-header-optimization) https://github.com/bitcoin/bitcoin/pull/30888
<bitcoin-git>
bitcoin/master 19f4a7c Fabian Jahr: test: Wait for local services to update in feature_assumeutxo
<bitcoin-git>
bitcoin/master cf0120f Ava Chow: Merge bitcoin/bitcoin#30880: test: Wait for local services to update in fe...
<bitcoin-git>
[bitcoin] achow101 merged pull request #30880: test: Wait for local services to update in feature_assumeutxo (master...2024-09-au-services-test) https://github.com/bitcoin/bitcoin/pull/30880