< bitcoin-git>
[bitcoin] practicalswift opened pull request #18047: tests: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h) (master...fuzzers-netaddress) https://github.com/bitcoin/bitcoin/pull/18047
< kallewoof>
Seems g++/gcc is super slow on some macs. This is causing ./configure to take like 5 minutes to finish, since it runs a bunch of tiny g++ compiles. [ echo "int main(){}" > tst.cpp; time ( for i in {1..10}; do g++ -std=c++11 -c -g -O2 -g3 -O0 -DDEBUG -DDEBUG_LOCKORDER -DMAC_OSX tst.cpp; done ) ] takes 0.3s on linux but 13s on my mac...
< fanquake>
kallewoof can you dump the verbose output of your test programs?
< kallewoof>
fanquake: I tried -ftime-report and basically only a few ms go into the compilation, the rest goes into the starting of g++. how do i make it more verbose?
< fanquake>
-v
< fanquake>
That should dump the compiler, flags, SDK, include dirs etc
< kallewoof>
No change (even tried copying the clang binary into the ramdisk but no change)
< sipa>
kallewoof: do any of the -D flags matter?
< sipa>
i suspect not
< kallewoof>
sipa: no, same results
< fanquake>
kallewoof do you see similar slowness if you use a different Clang? Could install LLVM via brew and try clang++9
< kallewoof>
will try!
< kallewoof>
tho it's strange if that fixed it since you are using the same clang as i am
< fanquake>
Wondering if you Xcode/Command Line tools install might be broken in some way
< fanquake>
*your
< kallewoof>
fanquake: thing is, i had this problem on my previous machine, so i explicitly chose not to migrate my settings over, but to back everything up and pull files from it and do a fresh install of everything, including xcode. same problem appeared...
< kallewoof>
no change
< aj>
kallewoof: maybe filesystem is being super slow gathering include files and/or libs?
< kallewoof>
aj: i looked around and don't see anything suspect about the dirs listed in the output
< fanquake>
only that i have about twice as much junk on my machine
< kallewoof>
Wait. I have the same issue the guy on twitter does, where I have a second container with the same data (disk3)
< kallewoof>
Oh wait no, that is 1 GB not 1 TB
< gwillen>
the fact that running clang directly eliminates the overhead is very suggestive that whatever the problem is, compilation is not in any way involved
< gwillen>
and that it is somewhere in whatever process the "gcc" stub is using to find the actual compiler
< kallewoof>
gwillen: running clang takes me down to 0.7 seconds from 1.3, but it's still way too slow. this should run in <0.1 second though. (linux -> g++ -std=c++11 -c -g -O2 -g3 -O0 tst.cpp 0.02s user 0.02s system 98% cpu 0.040 total)
< gwillen>
well, this may be multiple unrelated issues
< gwillen>
my 10.13.6 machine takes 0.46 seconds to run your 10-trivial-compiles test
< gwillen>
and I have no trouble running configure or building bitcoin
< kallewoof>
my guess is every time *something* is executed, there's a 0.7s overhead. the above would be shortcutting one of the steps, thus the drop in time
< kallewoof>
gwillen: i can compile, but configure takes 5 minutes-ish.
< gwillen>
that sounds even less like a compilation-related issue and more like a system-level issue
< kallewoof>
the actual compilation (because compiling each file actually takes time) is not that different from other machines
< gwillen>
*nods*
< kallewoof>
gwillen: agreed!
< gwillen>
does running clang even just to get the "no input files" error (with no arguments) take 0.7s?
< gwillen>
(it is 0.018s for me)
< kallewoof>
good point. yes "time clang" -> clang 0.01s user 0.66s system 99% cpu 0.669 total
< gwillen>
wow, okay, that's interesting
< gwillen>
normally the next thing I would try would be a system call trace
< gwillen>
but getting one of those on OS X is usually insanely annoying
< gwillen>
kallewoof: ok just curious, what happens if you "time /bin/echo"
< kallewoof>
other commands (git, ls, etc) all run instantly though.
< gwillen>
huh, okay.
< gwillen>
you don't have any remote filesystems mounted? NFS, Samba?
< gwillen>
Dropbox?
< kallewoof>
btw i installed llvm via brew and "time /usr/local/opt/llvm/bin/clang" gives the same 0.7s delay
< kallewoof>
nope
< gwillen>
0.7s feels too long to be anything other than a timeout
< gwillen>
hmm, do you have anything interesting in 'sudo dmesg' or in Console.app after running clang a few times? Most notably, do you see any I/O errors that might suggest your hard disk is failing?
< gwillen>
Like, maybe you have a bad sector in the clang binary or some dependency of it.
< kallewoof>
gwillen: I had this exact problem on my previous machine and one other coworker has it as well. Will check dmesg though
< gwillen>
huuuh. All Catalina, I assume?
< gwillen>
I was literally at dinner with an Apple clang dev a few hours ago, but I'm afraid this is likely to be outside his area of expertise....
< hebasto>
jonasschnelli: does a draft pr trigger build on bitcoinbuilds.org?
< kallewoof>
gwillen: no mojave, all of them i think
< gwillen>
oh, huh, koay
< gwillen>
okay*
< kallewoof>
nothing in dmesg btw
< jonasschnelli>
hebasto: I don't know. Lets try. )
< hebasto>
jonasschnelli: rebased; let travis check it
< gwillen>
kallewoof: I think I could walk you through using Instruments to try to see what clang is spending all that time on
< gwillen>
but I'm kind of just figuring out how to use it myself, PM would probably be best since it might take some doing
< kallewoof>
gwillen: instruments is a good idea. I'll try it later (dinner etc now)
< kallewoof>
gwillen: I have used instruments on command line stuff in the past so I have a pretty clear idea on how to do it. Will ping you if I fail tho
< gwillen>
ok cool, yeah, it seems to be the only thing that fills the strace niche.
< gwillen>
I can get it to give me syscall times, although not consistently syscall arguments, so it still seems like a clunky and inferior replacement for a good commandline tool :-P
< kallewoof>
gwillen: yeah, that about sums it up :)
< kallewoof>
gwillen: oh, and running it from inside instruments makes it take 14 ms and doesn't provide any details. *rubs head*
< bitcoin-git>
[bitcoin] hebasto opened pull request #18051: [WIP] build: Fix behavior when ALLOW_HOST_PACKAGES unset (master...20200201-fix-allowhostpackages) https://github.com/bitcoin/bitcoin/pull/18051
< hebasto>
fanquake: mind making gitian builds for 18051?
< fanquake>
hebasto: sure. I assume this is an alternative to 18045 ?
< hebasto>
sure
< hebasto>
fanquake: ty
< bitcoin-git>
[bitcoin] hebasto closed pull request #18045: build: Do not use QtDBus for macOS builds (master...20200131-macos-qdbus) https://github.com/bitcoin/bitcoin/pull/18045
< jeremyrubin>
We're holding the CTV Workshop today. We'll be live streaming (check my twitter @JeremyRubin) and using the channel ##ctv-bip-review. Will share more information there!
< gwillen>
kallewoof: I was able to run it on the commandline with "instruments -D result.trace -t 'System Trace' <path_to_clang>"
< gwillen>
which resulted in a file named results.trace that I could then open in Instruments with "open result.trace"
< gwillen>
I don't know if perhaps that method might reproduce the issue
< luke-jr>
fall out from Firefox adding Rust code: now a bunch of stuff is broken and they just force LTO as a "fix", apparently breaking proxy support and Tor.. https://bugzilla.mozilla.org/show_bug.cgi?id=1376621