< jonasschnelli>
wumpus: regarding the UTXO set cursor you have implemented in #7756...
< jonasschnelli>
once you create the cursor, new changes won't affect itteration?
< GitHub131>
[bitcoin] MarcoFalke opened pull request #8821: [qt] sync-overlay: Don't block during reindex (master...Mf1609-qtSyncReindex) https://github.com/bitcoin/bitcoin/pull/8821
< wumpus>
jonasschnelli: indeed
< wumpus>
it uses a leveldb snapshot
< wumpus>
the idea is that you can do a background backup/downlaod/analysis
< * luke-jr>
wonders if it's worth using a leveldb snapshot for sweepprivkey to avoid holding a lock
< wumpus>
does it use the utxo cursor to iterate over the utxo set? if so, you don't need a lock
< wumpus>
getutxosetinfo or whatever it's called also doesn't take a lock, at least not during the crunching phase
< luke-jr>
it isn't implemented sanely yet afaik, just pondering how the ideal impl would be
< dinao>
What is the purpose of the version-field in transactions?
< Yogh_>
when two blocks at the same height (at the tip) arrive in a Core node, does Core choose the first-seen block or the highest-work block as its chaintip of preference?
< wumpus>
always the highest total work, also this is a general #bitcoin question
< jl2012>
#8499 is trivial. What is the progress of #8393 compact block?
< sdaftuar>
jl2012: i think it's done or close to done. could be merged as-is, or with some tests, or with some minor improvements -- not sure what sipa prefers to do, but i'm fine with any of those options
< jl2012>
sdaftuar: thanks, #8499 is redone based on your suggestions
< sdaftuar>
jl2012: cool, i'll take a look
< sipa>
sdaftuar: i'll go through your suggestions later today
< kanzure>
is there some logs i can look at for why there is a tor client in bitcoind rather than using SOCKS5 for tor?
< wumpus>
kanzure: there is no tor client in bitcoind
< wumpus>
the only way bitcoind can use tor is to connect though the proxy or be connected from a hidden service
< wumpus>
and there is some logic to talk to tor through its control port, to automatically listen on a hidden service
< sipa>
that's a client for tor's control interface
< d4de>
so usually people just have that done outside of their app, in any case, my question stands
< wumpus>
sure, having it happen automatically means that it is easier to do
< wumpus>
it's easy enough to disable if you don't want to use the functionality
< d4de>
yep, worth maintaining and be a potential source of issues?
< d4de>
I understand that wumpus
< wumpus>
everything is a potential source of issues
< d4de>
I'm arguing for the merits of having in the code base to begin with
< d4de>
true, so let's not add more to "everything"? see where I'm coming from
< d4de>
?
< wumpus>
who are you and why are you arguing this?
< sipa>
we as a community of developers are the ones who have to maintain this, and we found it worthwhile to have this regardless of the extra work it brings
< d4de>
I'm a bitcoin user... why, because I believe bitcoind should do one thing and one thing only instead of having the binary be bigger?
< wumpus>
it hardly matters to the binary size
< wumpus>
please go do something else
< d4de>
I'll certainly will, thanks
< wumpus>
it's like you arguing anti-privacy
< wumpus>
strange person
< paveljanik>
maybe he comes from the UNIX world. Do one thing only and do it the best you van. But the world has moved...
< paveljanik>
can
< * wumpus>
comes form the 'UNIX world'
< paveljanik>
people are lazy
< * paveljanik>
too
< paveljanik>
even if it is about their money...
< wumpus>
I've developed for solaris, for openbsd, that doesn't make me a crazy person that goes into other people's project's IRC channels and start silly arguments
< wumpus>
oh HPUX even
< sipa>
we still don't have SMTP support in bitcoind, right?
< paveljanik>
he probably misunderstands the tor principles...
< wumpus>
no LISP interpreter either
< * paveljanik>
hides his Editor 8)
< paveljanik>
M-x wallet RET
< paveljanik>
oops, wrong buffer
< paveljanik>
;-)
< sipa>
bah, in my days we used to compute our public key by hand
< cfields_>
wumpus: to clarify, a typical "./configure --prefix=foo" is enough to trigger the -fstack-protector-all bug for you with xenial mingw?
< wumpus>
cfields_: yes
< cfields_>
(I should say.. build with depends, typical configure, and ./test_bitcoin.exe?)
< wumpus>
cfields_: the default hardening settings trigger it
< wumpus>
sipa: but were the keys longer than 16 bits? :-)
< wumpus>
yes, I did nothing special
< wumpus>
why, you can't reproduce it?
< wumpus>
I may have switched the g++ compiler to posix and back a few times :)
< cfields_>
if that's the case, happy to report that it's fixed after my toolchain work
< cfields_>
wumpus: i've added thin toolchain builders to depends that rid us of the ubuntu toolchain dependency. Only needed once for bootstrap, then we're self-hosted
< wumpus>
but I think it should be reproducible with new xenial vm + sudo alternatives to switch the compiler to posix (so that it gets past the threading problems) + follow instructions in doc/build-windows.md ,
< wumpus>
cfields_: interesting, so that version is newer than the one in ubuntu, I guess?
< cfields_>
I'm not sure if that's what you wanted long-term or not, but I wanted to go through the process to see what it would take
< wumpus>
gcc or clang?
< cfields_>
wumpus: yes. gcc 6.2.0. mingw 4.0.6.
< wumpus>
I don't care, I just want the releases to work :)
< wumpus>
whoa 6.2,0, I'm surprise that works at all
< wumpus>
I don't think I've ever used a gcc newer than 5.x
< wumpus>
but anyhow for macosx we use a custom toolchain too right?
< wumpus>
I don't see a problem with doing the same for windows
< kanzure>
wumpus: i don't think d4de is arguing anti-privacy; he previously proposed other ways of using tor.
< cfields_>
figured i'd start with latest stable versions of everything, then backing down as needed
< cfields_>
wumpus: yes. Linux is self-hosted now too.
< wumpus>
the only problem with our own toolchains is that we have to set what CPU and architecture to target and such
< cfields_>
wumpus: the idea would be: very rarely, build new toolchains with gitian and host them. Essentially the same as we do now with the clang toolchain for osx
< cfields_>
wumpus: yep, plug in your triplet, and it builds you a toolchain that just works :)
< wumpus>
the compiler is also built determinstically?
< cfields_>
yep. Even builds its own glibc and uses it to interpret itself
< wumpus>
kanzure: it was weird anyhow
< cfields_>
(only in one round, though)
< wumpus>
cfields_: so we have to do a gitian round for building the toolchain?
< wumpus>
cfields_: I think it makes sense, although it has to be deterministically reproducible over a longer time than typical bitcoin releases I guess
< cfields_>
wumpus: we wouldn't have to as it should just work anywhere. But for purity, I think we'd want to
< sipa>
cfields_: does this mean we could build reproducible binaries, directly from depends/, without gitian?
< wumpus>
but how does that work, the toolchain is built and the result is cached like the other depends packages?
< cfields_>
sipa: I think it'd be possible, but likely not realistic. Things like $USER and $PATH would be hard to strip out
< wumpus>
yes and tons of unix utilities that are used during the build/configure process which may give slightly differnet results on different OSes, the toolchain is not the only thing that reflects in the binaries
< cfields_>
wumpus: yea. The caching part isn't done yet, but I was picturing a make BUILD_TOOLCHAIN=true or so, otherwise it attempts to fetch the known pre-built one
< sipa>
cfields_: would you consider that to be future work?
< sipa>
(making it deterministic without gitian)
< wumpus>
cfields_: you mean, fetch pre-built binaries? that seems like a security risk
< cfields_>
sipa: could be, but I think it'd require a new approach. It essentially needs a chroot at minimum, I think
< wumpus>
sipa: I think you need some kind of controlled environment for deterministic building, whether it a chroot or LXC container
< wumpus>
sipa: otherwise there's just too much factors that can interfere
< cfields_>
wumpus: right, which is why we'd want them built with gitian. The download would look for a specific pre-generated result and checksum. Same as our osx toolchain download now.
< cfields_>
wumpus: also note that the toolchains are chained. So the old one always builds the new one.
< cfields_>
wumpus: again, I'm not married to it. I just wanted to go through it to see what it would take. If you'd prefer a different approach, or not bothering, I'm fine with that. The busted mingw toolchain from Ubuntu just made me grumpy :)
< wumpus>
cfields_: well I think building the toolchain to build with, certainly for windows makes sense
< cfields_>
wumpus: also worth noting that they're stand-alone and relocatable. So they're still generally useful outside of depends
< wumpus>
I just think that outside of gitian, depends downloading binaries is a bit questionable
< wumpus>
it's not what I would expect at least
< cfields_>
wumpus: well, it could default to self-building I suppose
< cfields_>
no reason not to, other than it takes a long time.
< wumpus>
I think it should default to using the toolchain provided by the user, unless specified otherwise
< wumpus>
I mean *generally* if you cross compile you have a toolchain and want to use that
< wumpus>
the ubuntu situation is kind of fucked up
< cfields_>
wumpus: sure. But the question is: going forward, how do we specify the toolchains used for releases? I don't think we always want to be at the mercy of the latest $distro packages?
< wumpus>
cfields_: I think we should distinguish two cases here a) gitian builds b) user's depends builds. For the gitian builds it's acceptable to use our own toolchain. Also for depends, if so chosen. But I don't think depends should start out fetching/building a toolchain by default
< wumpus>
cfields_: an example, let's say I'm cross-compiling for ARM. I have my own toolchain which I've built myself specifically for the device I'm cross-compiling to. But I use the depends to build bitcoin's dependencies.
< wumpus>
cfields_: I wouldn't expect it to start building a random ARM toolchain, then :)
< wumpus>
or ubuntus with broken toolchain we should document how to use our own, alternative toolchain
< wumpus>
+for
< cfields_>
wumpus: yes. The more reasonable approach would probably be 2 processes. One to build our toolchains, the other to build our depends. One can be plugged into the other for gitian/travis.
< wumpus>
at least until ubuntu has cleaned up their act...
< cfields_>
(just by setting PATH)
< wumpus>
right
< cfields_>
ok. And in that case, we'd want them chained together to eliminate distro reliance, right?
< wumpus>
for the gitian builds, absolutely
< wumpus>
I completely agree with what you're doing, I just don't think it should be default for depends
< cfields_>
ok
< cfields_>
sipa: to answer your question, I'm torn. I think working on a deterministic process like that is interesting, but I think it's more correct and generally useful to do it at a higher level
< cfields_>
higher level meaning: fix the tools as needed to allow determinism, then projects can provide their own minimal environments (or use a de-facto one) for stripping out the rest.
< cfields_>
There are already several projects like that, I think I'd only be adding noise :)
< wumpus>
cfields_: yes we should strongly avoid duplicating other project's efforts
< wumpus>
we have plenty of issues of our own
< wumpus>
and way too few people to handle them...
< arubi>
I'm noticing the same behavior change in my own parser after implementing segwit today: http://paste.debian.net/plainh/40b5ff00 - is parsing supposed to fail on zero inputs txs now?
< sipa>
no valid transaction can have zero inputs
< arubi>
right, and that would be the case forever?
< sipa>
yes, it's a consensus rule
< sipa>
and changing it would introduce replay attacks to bitcoin
< sipa>
bitcoin core uses some heuristics when parsing input for fundrawtransaction and decoderawtransaction to determine whether it's an incomplete non-segwit transaction instead
< sipa>
but i expect that longer term we'll need to move away from using the same serialization for incomplete transactions
< sipa>
it becomes very hard to hack things like partial signatures for more complicated scripts into it
< arubi>
exactly
< morcos>
what were we saying CPU SHA256 speed was in MB/s?
< sipa>
150
< morcos>
k, thx
< arubi>
sipa, I see, so not really losing anything by not parsing transactions like those anymore, not that I ever used it for anything meaningful. I'll try figuring out how to bail early. thanks
< jonasschnelli>
has the FIBER (relay network) any form of encryption or authentication?
< sipa>
BlueMatt: ^
< sipa>
(also, it's called FIBRE)
< jonasschnelli>
Ah.. it's french :)
< BlueMatt>
jonasschnelli: it does
< BlueMatt>
it has a simple mac key which is a constant per connection
< BlueMatt>
no encryption, just auth
< sipa>
BlueMatt: ah nice
< sipa>
no encryption makes sense, there is no private data anyway
< BlueMatt>
sipa: the "addudpnode" stuff requires a "password" for each connection