< bitcoin-git> [bitcoin] practicalswift opened pull request #18047: tests: Add basic fuzzing harness for CNetAddr/CService/CSubNet related functions (netaddress.h) (master...fuzzers-netaddress) https://github.com/bitcoin/bitcoin/pull/18047
< kallewoof> Seems g++/gcc is super slow on some macs. This is causing ./configure to take like 5 minutes to finish, since it runs a bunch of tiny g++ compiles. [ echo "int main(){}" > tst.cpp; time ( for i in {1..10}; do g++ -std=c++11 -c -g -O2 -g3 -O0 -DDEBUG -DDEBUG_LOCKORDER -DMAC_OSX tst.cpp; done ) ] takes 0.3s on linux but 13s on my mac...
< fanquake> kallewoof can you dump the verbose output of your test programs?
< kallewoof> fanquake: I tried -ftime-report and basically only a few ms go into the compilation, the rest goes into the starting of g++. how do i make it more verbose?
< fanquake> -v
< fanquake> That should dump the compiler, flags, SDK, include dirs etc
< kallewoof> 1.31s for "int main(){}"... I suspect this is Apple's new system that checks permissions and stuff.
< sipa> is this actually gcc, or clang?
< kallewoof> clang under the hood
< fanquake> The same invocation for me takes 0.073s
< kallewoof> fanquake: I'm on mojave. You're on latest?
< fanquake> No, 10.14.6. At this point I don't think anyone could convince me to upgrade heh.
< kallewoof> same. wonder what is different.
< fanquake> The apple clang version is the same. clang-1100.0.33.17. A few differences in include search paths.
< fanquake> I see -fmessage-length 101 for you, 131 for me. That's the only difference in compiler options.
< sipa> like the width of your terminal
< sipa> *likely
< fanquake> yea, something to do with debug output formatting
< kallewoof> Running that command directly drops it to 0.66s (both yours and mine are the same)..
< sipa> which command?
< kallewoof> the line that start with "/Applications/Xcode.app[...]
< sipa> can you see if this is time spent in user, system, or waiting for io?
< kallewoof> ( -cc1 -triple x86_64-apple-macosx10.14.0 -Wdeprecated-objc-isa-usage ) 0.01s user 0.65s system 99% cpu 0.657 total
< sipa> all system
< sipa> maybe a weird filesystem configuration?
< kallewoof> I've got encrypted APFS
< kallewoof> I thought fanquake did too, though?
< fanquake> Yea my HD is also encrypted APFS
< fanquake> kallewoof could try mounting a new, different type of volume, and running commands in there?
< kallewoof> I can try with an external HDD when I get to the office, but I'm not going there until I get back from London so it'll be a bit :)
< fanquake> You could create and mount a ramdisk
< kallewoof> oh!
< kallewoof> No change (even tried copying the clang binary into the ramdisk but no change)
< sipa> kallewoof: do any of the -D flags matter?
< sipa> i suspect not
< kallewoof> sipa: no, same results
< fanquake> kallewoof do you see similar slowness if you use a different Clang? Could install LLVM via brew and try clang++9
< kallewoof> will try!
< kallewoof> tho it's strange if that fixed it since you are using the same clang as i am
< fanquake> Wondering if you Xcode/Command Line tools install might be broken in some way
< fanquake> *your
< kallewoof> fanquake: thing is, i had this problem on my previous machine, so i explicitly chose not to migrate my settings over, but to back everything up and pull files from it and do a fresh install of everything, including xcode. same problem appeared...
< kallewoof> no change
< aj> kallewoof: maybe filesystem is being super slow gathering include files and/or libs?
< kallewoof> aj: i looked around and don't see anything suspect about the dirs listed in the output
< kallewoof> aj: not sure how to test your theory
< kallewoof> fanquake: spot any differences between this and your output for 'diskutil list'? https://gist.github.com/kallewoof/e63b51f6b5aab2f104ebebc602f0e1a1
< fanquake> only that i have about twice as much junk on my machine
< kallewoof> Wait. I have the same issue the guy on twitter does, where I have a second container with the same data (disk3)
< kallewoof> Oh wait no, that is 1 GB not 1 TB
< gwillen> the fact that running clang directly eliminates the overhead is very suggestive that whatever the problem is, compilation is not in any way involved
< gwillen> and that it is somewhere in whatever process the "gcc" stub is using to find the actual compiler
< kallewoof> gwillen: running clang takes me down to 0.7 seconds from 1.3, but it's still way too slow. this should run in <0.1 second though. (linux -> g++ -std=c++11 -c -g -O2 -g3 -O0 tst.cpp 0.02s user 0.02s system 98% cpu 0.040 total)
< gwillen> well, this may be multiple unrelated issues
< gwillen> my 10.13.6 machine takes 0.46 seconds to run your 10-trivial-compiles test
< gwillen> and I have no trouble running configure or building bitcoin
< kallewoof> my guess is every time *something* is executed, there's a 0.7s overhead. the above would be shortcutting one of the steps, thus the drop in time
< kallewoof> gwillen: i can compile, but configure takes 5 minutes-ish.
< gwillen> that sounds even less like a compilation-related issue and more like a system-level issue
< kallewoof> the actual compilation (because compiling each file actually takes time) is not that different from other machines
< gwillen> *nods*
< kallewoof> gwillen: agreed!
< gwillen> does running clang even just to get the "no input files" error (with no arguments) take 0.7s?
< gwillen> (it is 0.018s for me)
< kallewoof> good point. yes "time clang" -> clang 0.01s user 0.66s system 99% cpu 0.669 total
< gwillen> wow, okay, that's interesting
< gwillen> normally the next thing I would try would be a system call trace
< gwillen> but getting one of those on OS X is usually insanely annoying
< gwillen> kallewoof: ok just curious, what happens if you "time /bin/echo"
< kallewoof> other commands (git, ls, etc) all run instantly though.
< gwillen> huh, okay.
< gwillen> you don't have any remote filesystems mounted? NFS, Samba?
< gwillen> Dropbox?
< kallewoof> btw i installed llvm via brew and "time /usr/local/opt/llvm/bin/clang" gives the same 0.7s delay
< kallewoof> nope
< gwillen> 0.7s feels too long to be anything other than a timeout
< gwillen> hmm, do you have anything interesting in 'sudo dmesg' or in Console.app after running clang a few times? Most notably, do you see any I/O errors that might suggest your hard disk is failing?
< gwillen> Like, maybe you have a bad sector in the clang binary or some dependency of it.
< kallewoof> gwillen: I had this exact problem on my previous machine and one other coworker has it as well. Will check dmesg though
< gwillen> huuuh. All Catalina, I assume?
< gwillen> I was literally at dinner with an Apple clang dev a few hours ago, but I'm afraid this is likely to be outside his area of expertise....
< hebasto> jonasschnelli: does a draft pr trigger build on bitcoinbuilds.org?
< kallewoof> gwillen: no mojave, all of them i think
< gwillen> oh, huh, koay
< gwillen> okay*
< kallewoof> nothing in dmesg btw
< jonasschnelli> hebasto: I don't know. Lets try. )
< bitcoin-git> [bitcoin] jonasschnelli pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/d104aa0ace36...cadb9d33421d
< bitcoin-git> bitcoin/master 61eb058 Joao Barbosa: gui: Drop BanTableModel dependency to ClientModel
< bitcoin-git> bitcoin/master 3aee10b Joao Barbosa: gui: Drop ShutdownWindow dependency to BitcoinGUI
< bitcoin-git> bitcoin/master cadb9d3 Jonas Schnelli: Merge #18036: gui: Break trivial circular dependencies
< bitcoin-git> [bitcoin] jonasschnelli merged pull request #18036: gui: Break trivial circular dependencies (master...2020-01-utilitydialog) https://github.com/bitcoin/bitcoin/pull/18036
< fanquake> jonasschnelli: Could you also take a look at #17937
< gribble> https://github.com/bitcoin/bitcoin/issues/17937 | gui: Remove WalletView and BitcoinGUI circular dependency by promag . Pull Request #17937 . bitcoin/bitcoin . GitHub
< jonasschnelli> fanquake: will do..
< fanquake> jonasschnelli: thanks
< bitcoin-git> [bitcoin] jonasschnelli pushed 3 commits to master: https://github.com/bitcoin/bitcoin/compare/cadb9d33421d...f05c1ac444e0
< bitcoin-git> bitcoin/master ac3d107 Joao Barbosa: gui: Add transactionClicked and coinsSent signals to WalletView
< bitcoin-git> bitcoin/master cb8a86d Joao Barbosa: gui: Remove WalletView and BitcoinGUI circular dependency
< bitcoin-git> bitcoin/master f05c1ac Jonas Schnelli: Merge #17937: gui: Remove WalletView and BitcoinGUI circular dependency
< bitcoin-git> [bitcoin] jonasschnelli merged pull request #17937: gui: Remove WalletView and BitcoinGUI circular dependency (master...2020-01-remove-walletview-bitcoingui) https://github.com/bitcoin/bitcoin/pull/17937
< hebasto> jonasschnelli: mind looking #17966 ?
< gribble> https://github.com/bitcoin/bitcoin/issues/17966 | qt, refactor: Optimize signal-slot connections logic by hebasto . Pull Request #17966 . bitcoin/bitcoin . GitHub
< jonasschnelli> hebasto: needs a rebase
< jonasschnelli> (due to just merged prs)
< hebasto> jonasschnelli: rebased; let travis check it
< gwillen> kallewoof: I think I could walk you through using Instruments to try to see what clang is spending all that time on
< gwillen> but I'm kind of just figuring out how to use it myself, PM would probably be best since it might take some doing
< kallewoof> gwillen: instruments is a good idea. I'll try it later (dinner etc now)
< kallewoof> gwillen: I have used instruments on command line stuff in the past so I have a pretty clear idea on how to do it. Will ping you if I fail tho
< gwillen> ok cool, yeah, it seems to be the only thing that fills the strace niche.
< gwillen> I can get it to give me syscall times, although not consistently syscall arguments, so it still seems like a clunky and inferior replacement for a good commandline tool :-P
< kallewoof> gwillen: yeah, that about sums it up :)
< kallewoof> gwillen: oh, and running it from inside instruments makes it take 14 ms and doesn't provide any details. *rubs head*
< bitcoin-git> [bitcoin] jakubtrnka opened pull request #18050: wallet: replace boost deprecated method call (master...master) https://github.com/bitcoin/bitcoin/pull/18050
< bitcoin-git> [bitcoin] fanquake closed pull request #18050: wallet: replace boost deprecated method call (master...master) https://github.com/bitcoin/bitcoin/pull/18050
< bitcoin-git> [bitcoin] hebasto opened pull request #18051: [WIP] build: Fix behavior when ALLOW_HOST_PACKAGES unset (master...20200201-fix-allowhostpackages) https://github.com/bitcoin/bitcoin/pull/18051
< hebasto> fanquake: mind making gitian builds for 18051?
< fanquake> hebasto: sure. I assume this is an alternative to 18045 ?
< hebasto> sure
< hebasto> fanquake: ty
< bitcoin-git> [bitcoin] hebasto closed pull request #18045: build: Do not use QtDBus for macOS builds (master...20200131-macos-qdbus) https://github.com/bitcoin/bitcoin/pull/18045
< jeremyrubin> We're holding the CTV Workshop today. We'll be live streaming (check my twitter @JeremyRubin) and using the channel ##ctv-bip-review. Will share more information there!
< gwillen> kallewoof: I was able to run it on the commandline with "instruments -D result.trace -t 'System Trace' <path_to_clang>"
< gwillen> which resulted in a file named results.trace that I could then open in Instruments with "open result.trace"
< gwillen> I don't know if perhaps that method might reproduce the issue
< luke-jr> fall out from Firefox adding Rust code: now a bunch of stuff is broken and they just force LTO as a "fix", apparently breaking proxy support and Tor.. https://bugzilla.mozilla.org/show_bug.cgi?id=1376621
< bitcoin-git> [bitcoin] hebasto opened pull request #18052: Remove false positive GCC warning (master...20200201-false-positive-warning) https://github.com/bitcoin/bitcoin/pull/18052