< vasild>
Looks like fuzz input is not a good input to fill addrman with 1000s of different, routable addresses. Quite often the fuzz bytes are zero which result in a 0.0.0.0 which is not routable and also the fuzzing input is exhausted way too quickly.
< vasild>
What about getting some 100 or 1000 bytes of fuzz input, seeding that into a deterministic random number generator and from that generator produce 1000s of addresses?
< vasild>
This way we will never run out of "random" bytes and also, I guess, there will be not big areas of 0s bytes to generate duplicate or non routable addresses.
< jnewbery>
vasild: you're right about !restore_bucketing. Thanks!
< jnewbery>
I don't know much about the fuzzers but I assume there's a way to make the fuzzer itself a bit more intelligent in generating routable addresses, or giving it a seed that results in more routable addresses being generated?
< stevenroose>
It seems that with my latest libboost (Arch...) I can no longer build bitcoind v0.16.3. I previously had to add the `#include <deque>;` before but now there's some other errors relating to `BOOST_PRAGMA_MESSAGE`.
< aj>
stevenroose: add "--enable-suppress-external-warnings" to your configure command?
< stevenroose>
one sec yeah I just noticed it's not an error about the torcontrol.cpp code but about the boost code
< stevenroose>
aj: Giving me a lot of `error: ‘_2’ was not declared in this scope` errors all over the place. Seems to be some kind of argument templating that is broken
< aj>
stevenroose: oh :( --enable-suppress fixed the qt and boost warnings i was seeing
< luke-jr>
stevenroose: bitrot is real :P
< stevenroose>
hmm. I can retry with the entire autogen but I'm doubtful. Does a depends build fix this?
< aj>
oh, sorry, didn't see that it was an old bitcoind as well as a new boost. depends build sounds worth trying; should get you the old boost
< luke-jr>
does fuzz normally take a LONG time to build?
< sipa>
not as long anymore as before the targets were merged in a separate binary
< phantomcircuit>
luke-jr, define LONG
< luke-jr>
phantomcircuit: it's still building :P
< luke-jr>
sipa: I'm seeing lots of binaries produced - was that something that happened recently on master?
< sipa>
luke-jr: yes, on master it's now a single binary
< roconnor>
luke-jr: let me know if this is off-topic, but can't you build the guix bootstrap tarball on your own system? Or is it not deterministic yet?
< roconnor>
Perhaps the build log for that process would let you rederive those tarballs without the use of guix.
< roconnor>
I don't know. I'm specualing a little based on what I would do if this were NixOS instead of Guix.
< luke-jr>
roconnor: that assuems you have guix already
< luke-jr>
roconnor: I did try :P
< roconnor>
I mean, in theory you just need someone to give you the build log to figure out how they are built.
< luke-jr>
I also need to figure out how to get guix to accept the hash :P
< luke-jr>
at least when I was trying this, they didn't have hashes defined for ppc64le
< roconnor>
oh were you able to build the tarballs?
< luke-jr>
no
< luke-jr>
IIRC I was trying to inject raw binaries
< luke-jr>
the design of Guix is kinda weird
< luke-jr>
they hash the input data rather than the output
< luke-jr>
things could be much simpler if they hashed the output
< roconnor>
Again speculating based on how Nix works, there are two (techinically three) different kinds of outputs. Fixed-derivation outputs hash the ouput, while say "non-determinisitc" packages hash the inputs.
< luke-jr>
Guix is supposed to be deterministic-only IIRC
< roconnor>
And if Guix solved the reproducible bootstrap libraries, they they have a fixed derivation (ie hashed output) at some point in their bootstrap process.
< luke-jr>
I'm not sure at this point how much (if any) overlap it has with Nix
< * luke-jr>
wonders if there is an actual reason behind the weird domain-specific language stuff in Guix
< roconnor>
I would be suprised if Guix were determinisitic in general.
< roconnor>
*lol* Guix is main difference between Nix was that it was supposed to *not* be using a domain-specific langauge thing, but using scheme instead.
< luke-jr>
scheme is a DSL when nothing else uses scheme
< roconnor>
Frankly my undertanding of the reasons Guix exists strike me as silly. But maybe I misunderstand them.
< roconnor>
I think your observation makes it clear that if learning the Nix langauge is a hurdle for using NixOS, then switching to Scheme isn't helping.
< luke-jr>
I suspect if I were to switch to a Nix/Guix-alike for my OS, I would end up writing something from scratch :p
< sipa>
roconnor: i think you may be right, but in the end, the reason for why it started existing isn't as interesting for us as the question for why you'd use it; and the (as i understand it) more deterministic/bootstappability of what is built around the package manager is much more relevant than nix language vs. scheme
< roconnor>
Right I was about to say, for the purposes of this discussion Guix has (AFAIU) made far more headway on minimising the bootstrap and reproducibility.
< luke-jr>
I also find it strange how it wants root.
< luke-jr>
dongcarl: yes, it's possible, but the main design/docs seem to want it
< sipa>
dongcarl: what is a .org file?
< luke-jr>
"If apache depends on an ssl library, the link is fully referenced in the apache binary. There is no way to circumvent the pointer to the library."
< luke-jr>
facepalm
< roconnor>
Ya nixos wants root in order to access /nix/store and to set various sticky bits to protect it. You can run outside of /nix/store, but then you cannot download prebuilt binaries because the RPATHs will be all wrong. Though for you luke-jr you probably don't want to download prebuilt binaries.
< luke-jr>
RPATHs seem like a bad idea anyway :P
< dongcarl>
That's mostly because the guix-daemon was modeled to be compatible with the nix-daemon, which wants root. I speculate one of the reasons is because it needs to do the namespace+cgroups container thing which and distros were having a spat over whether or not to enable USER_NS
< roconnor>
yes. When I learn about how Unix systems work, I feel sad.
< luke-jr>
roconnor: well, most systems just figure out the lib path at runtime
< roconnor>
is that true?
< luke-jr>
no reason Guix couldn't do that too
< luke-jr>
yes..?
< luke-jr>
I mean, there's ldconfig, but it's not embedding the path in the binaries
< roconnor>
I was under the impression the answer was no, but this is also the sort of thing I wouldn't know.
< luke-jr>
RPATHs aren't non-existent, but fairly rare
< roconnor>
Otherwise I don't understand why every binary I download needs patchelf.
< luke-jr>
roconnor: because your system isn't doing the job of resolving it? :p
< luke-jr>
I have never heard of anyone having to patchelf binaries
< roconnor>
well you don't download binaries. :D
< luke-jr>
I don't, but other people do
< sipa>
what is patchelf?
< luke-jr>
roconnor: just take our release binaries for example
< luke-jr>
roconnor: they just run fine on a normal Linux system
< luke-jr>
(while we do static link some libs, it isn't all of them)
< roconnor>
right, but AFAIU they all hardcode paths to /usr/lib or /lib
< roconnor>
and in normal unix systems this works.
< luke-jr>
nope
< dongcarl>
roconnor: Right, nix and guix binaries both share this trait of doing the -rpath and setting a weird dynamic linker. Both these behaviours are thoroughly reverted in the Bitcoin Core guix build
< luke-jr>
the path stored in the binary is the part before the =>
< harding>
sipa: A .org file is a document formatted using Emacs's org-mode syntax.
< sipa>
harding: interesting; any advantages it has over markdown e.g.
< sipa>
?
< roconnor>
luke-jr: oooh, Mabe this doesn't work on NixOS because there are no canonical versions of linked libraries.
< luke-jr>
roconnor: /lib64/ld64.so.2 is stored absolute, but that lib itself does the resolving of everything else
< sipa>
dongcarl: so is it fair to say that guix (and the binaries it builds) are all living in their own little world, and are unconventional in that they're all built to live the... but what we want for the bitcoin core release builds isn't so much building something that lives there, but using the guix-world stuff to build a "normal" binary?
< luke-jr>
roconnor: if I were making NixOS, I would modify the ld*.so* to understand Nix's structure ;)
< roconnor>
I mean, which libraries are you going to link with?
< luke-jr>
roconnor: the ones the binary in question's package depends on?
< luke-jr>
I'm assuming you have dep info for everything installed..
< harding>
sipa: not IMO. Early org mode preceeds markdown and I think markdown is a bit less noisy. However, if you really learn org-mode---which I never did---it makes it very easy to manage even huge .org documents.
< sipa>
harding: okay, won't bother then :)
< luke-jr>
roconnor: and if you really want to run a foreign binary, you could have Nix make a guess which libs to use, or ask you
< dongcarl>
sipa: Correct! Eventually we can introduce a new build system in Guix (https://guix.gnu.org/manual/en/html_node/Build-Systems.html) to build a "normal" binary, but I didn't bother since the current way emulates how we do Gitian builds and I wanted to take things one step at a time.
< dongcarl>
sipa: This is also why we can't just use the bitcoin-core package currently in Guix :-)
< roconnor>
luke-jr: I mean where would you store that meta-information of which exact version (by hash) that the binary wants to use? How about in the binary? :P
< luke-jr>
roconnor: then it's a pain if you want to use that package in a situation it wasn't made for
< luke-jr>
eg, if you want to quickly update a security flaw in a lib
< roconnor>
well I suppose that is a point of design contention.
< roconnor>
replace-runtime-depencency creates a new derviation using the binary of the old derivation as input and does a search and replace on hash values. :P
< roconnor>
This sort of has to be done recurively throughtout the entire system's dependencies, which is why it takes minutes instead of seconds.
< roconnor>
luke-jr: thanks. I didn't realize that ldd was doing the resolution of names rather than extracting the contents of the executible.