< wumpus>
interesting, just repeating the gbuild seems to get past it
< wumpus>
lxc is weird
< btcdrak>
wumpus: it happens randomly and you just have to run the command again. It's been like that for as long as i remember
< wumpus>
it's building the dependencies, this couuld take awhile
< wumpus>
btcdrak: yes I also remember seeing it before
< wumpus>
it's half-panic every time, debugging issues with gitian is tiring as everything goes so slow, I use this VM for nothing else to make sure (hopefully) nothing breaks it
< wumpus>
I wonder why we have no "upgrade gitian base image" step? doing a system upgrade before every build seems... overkill
< wumpus>
I know apt-cacher is caching the packages so there is no bandwidth overhead but it is still slow
< wumpus>
jonasschnelli: I'm trying to get a bucket of random testnet hosts by looking up testnet-seed.bitcoin.jonasschnelli.ch manually, but none of the node IP returned seem to have port 18333 open, am I doing something wrong?
< wumpus>
or just incredibly unlucky, one time I even got only ipv6 addresses :)
< wumpus>
oh it must be the address obfuscation, of course
< sipa>
address obfuscation? i thought we never adopted that
< wumpus>
I don't think so either
< wumpus>
I don't know then - using another seed gave me valid addresses, btw
< sipa>
maybe all testnet seeds are short lived?
< wumpus>
well I checked about 30 addresses returned from jonasschnelli's seed, none could connect, the first one I tried from petertodd's seed was a hit
< sipa>
hmm
< wumpus>
*yawn* now my testnet node hangs at 40% verification
< wumpus>
I should go back to bed, everything goes wrong today
< sipa>
awww
< sipa>
i see a stream of questions on stackexchange about unconfirmed transactions (mostly created by old software)... i really hope we can get some "fee increase" mechanism in
< wumpus>
I forget that every time, normally I just start bitcoind in gdb, but thought I was donig something so trivial now that it would be unnecessary - wrong :p
< jonasschnelli>
wumpus: re seeder, I started the testnet seeder with a new database this morning. Not sure if the seeder already results enough data.
< sipa>
jonasschnelli: perhaps you should delete the db and start over?
< jonasschnelli>
yes. I did this this morning ~9oclock
< sipa>
ah
< jonasschnelli>
ah.. wait... I guess it's scanning mainnet. :(
< sipa>
ha
< btcdrak>
lmao
< wumpus>
oh, nm, I was looking at the wrong log
< jonasschnelli>
fooled by the subdomain. :)
< jonasschnelli>
Okay. removed the database and started in testnet mode
< wumpus>
yes, windows build seems to work
< jonasschnelli>
sipa: [...] i really hope we can get some "fee increase" mechanism in
< wumpus>
current master: 291be202508e518646116677cc99cb1333739d52e2c4ae7259359b3bd3729e0c bitcoin-0.12.99-win-unsigned.tar.gz
< jonasschnelli>
sipa: you could review the bumpfee command.
< jonasschnelli>
I'm not entirely sure if taking a txid and autobump it is the way to go though.
< sipa>
if CPFP gets in, there is an easier solution
< sipa>
jonasschnelli: i don't know how to make this easy to use
< wumpus>
how is addnode supposed to work? I added a few (seemingly functional) testnet nodes, but they all stay at "connected": false in getaddednodeinfo (testing #8113)
< jonasschnelli>
I'm planing on extending the HD feature. Importing/starting a wallet with a xpriv key. How should we do that. I guess having a -xpriv= startup argument is very bad.
< sipa>
wumpus: it cycles every 2 minutes
< wumpus>
let's first worry about having some mechanism in, and then about making it easy to use
< sipa>
wumpus: and then tries to connect to one of the unconnected addnodes
< wumpus>
there's only two days left before the feature freeze so we need to make some choices
< wumpus>
sipa: okay, thanks
< jonasschnelli>
would ./bitcoin-wallet be a tool where users could create a wallet(.dat) with a xpriv?
< jonasschnelli>
Everything that goes over RPC is somehow not ideal
< wumpus>
that's the eternal question - command line isn't the ideal interface either though, especially if you're not writing shell script
< wumpus>
I think having wallet creation commands on RPC makes some sense, at least once there is multi-wallet support
< jonasschnelli>
wumpus: Yes. But right now, we create the wallet before the RPC is ready.
< wumpus>
yes, right now, I'm sure that needs to be changed
< wumpus>
is there a hurry?
< jonasschnelli>
No hurry.
< jonasschnelli>
Certenly not something for 0.13
< jonasschnelli>
But need some brainfood for afk time. :)
< wumpus>
decoupling the wallet further would mean that node RPC could be ready, before the wallet even starts loading/creating
< wumpus>
especially if the wallet has its own entry point
< jonasschnelli>
yes. I'm working on removine the splash screen and add this state
< wumpus>
removing the splash screen? I'd think we still need it during initial verification
< jonasschnelli>
wumpus: Yes. But you can access the wallet during this time. Get addresses, list transactions, etc.
< wumpus>
there's something to be said for having a wallet as an external tool/library, but the scope for doing that within bitcoin core is very small
< wumpus>
jonasschnelli: I don't think that's a very good idea, even being able to access the wallet during *synchronization* caused a ton of confused users
< jonasschnelli>
wumpus: but decoupling the wallet would result in such states. I guess all SPV wallets allow address/tx access in out-of-sync state.
< wumpus>
e.g. 'I downloaded bitcoin core and sent from *web wallet provider* to a new address, now my transaction doesn't show!'
< jonasschnelli>
Sure, we might want to add a SPV mode first.
< wumpus>
possibly, but I don't think - right now - there is any benefit to removing the splash screen and showing a (mostly dysfunctional) GUI instead
< jonasschnelli>
But its annoying that you need to go through verification/checkblocks if you want to get a new address
< wumpus>
possibly... checkblocks takes too long by default, that's aother issue
< jonasschnelli>
IMO the (wallet)GUI is fully functional during init-verification
< wumpus>
I think validating the last blocks is a good thing, but the default depth setting is overkill
< jonasschnelli>
Yes. But the node (validation) is not tied to the wallet/GUI.
< wumpus>
that's true...
< jonasschnelli>
But right. We need more prominent warnings if the wallet is not in sync
< wumpus>
but it won't start syncing blocks until that part is done, so people will still complain 'why isn't it synching!!!'
< wumpus>
WHHY IS THERE NO BLOCK SOURCE AVAILABLE
< jonasschnelli>
Heh... yes. probably.
< jonasschnelli>
Step by step...
< jonasschnelli>
For 0.14 i'd like to focus more on the UI
< wumpus>
you're right that creating an address can be done without any node functionality, but then, people can't see transactions to the address without it, or being in sync
< sipa>
i think we should disable that feature by default
< wumpus>
which is arguably the part people care about, actually receiving coins
< sipa>
not show a receive address until sync is done
< wumpus>
well for a start it should show a big red warning in the transaction pane when not up to date
< wumpus>
I don't think you need to actually disable it
< wumpus>
but just warn better
< sipa>
perhaps make it greyed out with a button, and if you click it it first asks you "are you aware that you'll only receive once you're synced?"
< wumpus>
there is no indication right now excapt for the extremely toned-down out-of-sync triangles
< jonasschnelli>
Wouldn't this be to much handholding for expert users?
< jonasschnelli>
(not the warning)
< jonasschnelli>
(just the fact that you can't get addresses when you are not in sync)
< wumpus>
yes, i think that's too much babysitting
< jonasschnelli>
IMO even the fact that you need to wait for the init/checks before you can list transactions or get new addresses.
< wumpus>
lots of people do know that, it's just new users that don't, maybe we should add a tutorial mode on first use
< sipa>
right
< sipa>
yes, just thinking through how a first time user sees all this
< jonasschnelli>
This is the difficulty: make it useable for basic/new bitcoin users but also make the experts happy.
< wumpus>
yes
< wumpus>
in any case, adding a warning wouldn't hurt and is easy to do
< GitHub179>
bitcoin/master 1c2a1ba Francesco 'makevoid' Canessa: Add address label to request payment QR Code (QT)...
< GitHub179>
bitcoin/master fb0ac48 Jonas Schnelli: Merge #7636: Add bitcoin address label to request payment QR code...
< GitHub124>
[bitcoin] jonasschnelli closed pull request #7636: Add bitcoin address label to request payment QR code (master...request_payment_qrcode_address_label) https://github.com/bitcoin/bitcoin/pull/7636
< wumpus>
how exactly is up to the person implementing it, I guess
< wumpus>
there's enough place on the "receive" tab to add a warning, and in the transaction pane I suppose it could show an overlay if there are no transactions yet and it is syncing
< MarcoFalke>
Anyone seeing OOM for gitian builds?
< MarcoFalke>
in main.cpp
< wumpus>
no, though adding of debug symbols in #8167 did increase the memory requirements for gitian a bit
< MarcoFalke>
How does gbuild set up the ram of the vm?
< MarcoFalke>
I could try to increase it, I guess.
< wumpus>
-j5 -m5000 where -j is the parallelism and -m is the memory size in mb
< wumpus>
what is the expected behavior when addnoding a seed?
< sipa>
very fuzzy
< sipa>
i think it won't make any connection if you already have 8 outgoing ones
< wumpus>
for me it works with IPs but not with hostnames
< sipa>
it should work with hostnames
< sipa>
wumpus: is this in #8113?
< wumpus>
the expectation is that it looks up the address for the seed then connect to the first address, I suppose?
< wumpus>
yes
< sipa>
it randonly tries one of the returned ip addtesses every 2 minutes
< sipa>
if one fails, there is another attempt 2 minutes later
< wumpus>
it stays at 'connected': false all the time
< sipa>
do you have 8 outgoing connections already?
< wumpus>
yes
< wumpus>
should I kick some other connections?
< wumpus>
ok disconnected all nodes, let's see what happens
< sipa>
the addnode logic should be integrated into the normal outgoing connection logic, and be protected from eviction
< wumpus>
yes, I suppose so, how badly this works I wonder if anyone was actually using this functionality and whether it makes sense to spend time on it
< sipa>
i think after we have p2p encryption and authentication, we may want to encourage it
< wumpus>
I set up a hostname for a known-working testnet node, lets see if it wants to connect to that...
< wumpus>
evicting all other nodes again
< wumpus>
MarcoFalke: yes, I guess so
< wumpus>
it didn't work automatically, though I could force it to connect with "addnode hostname onetry"
< wumpus>
now it shows up in getaddednodeinfo
< sipa>
over long periods of time, it will end up connecring to the addnode nodes when other outgoing connections die
< wumpus>
but I disconnected all other nodes; I don't see why I would have to wait longer than 2 minutes in that case
< wumpus>
in any case I managed to force it to connect and it shows up in getaddednodeinfo correcty
< wumpus>
I still hope Matt can give #8113 a try
< wumpus>
and check it still does whatever he expects from it
< murch>
sipa: Perhaps you remember that we talked about the Coin Selection algorithm a little bit about a month ago. I'm now indeed writing my Master's thesis about that problem. :) I was wondering if someone could confirm or deny my understanding of the current algorithm's behavior.
< murch>
Do I understand correctly, that this actually uses the sorted list (decreasing) of available UTXO and selects each UTXO with a 50%? That would easily explain the growth of the UTXO set, if I'm not mistaken.
< sipa>
murch: i can't say that i've ever looked at the algorithm in detail
< sipa>
murch: it looks like it does 1000 iterations over all UTXOs twice, stopping when it goes over the maximum
< murch>
sipa: But vValue is sorted in decreasing order by size, and it starts at the front selecting UTXO with a random binary chance. That would significantly increase the chance for large UTXO to be selected as inputs, especially since the maximum should be reached quickly when big inputs are added first.
< murch>
I used to think that it randomly selected from all UTXO with equal chance (that's what I simulated two years ago), but I just realized yesterday, that it doesn't seem to do that (anymore?).
< murch>
Unless I'm jut confused about what's happening there, because that seems like a really easy explanation for the UTXO growth.
< instagibbs>
it used to be a much more ocnfusing rsort....
< murch>
instagibbs: Yeah, I proposed to at least split it in two lines for better readability. :)
< murch>
sipa: Now don't fix it today though, I still want to write my thesis on it! ^^
< sipa>
murch: i'm not going to fix anything; i hope you do :)
< murch>
sipa: But, it would be very helpful to me if you could tell me whether I understood correctly what it does there, and I'm not overlooking something glaringly obvious. Because my next step would be to simulate how much of a difference it would make to pick randomly instead of from the front.
< murch>
Seems like a good idea to check the facts before I spend a bunch of hours on something useless. :)
< sipa>
i think you've looked at it better than i have
< murch>
Yeah, but my overall familiarity with the Bitcoin Core is slightly behind yours. ;) But I guess, I'd count that as an "I haven't seen anything that contradicts Murch". :)
< murch>
Then… coming up next, simulation results, I hope.
< murch>
sipa: Thanks
< MarcoFalke>
murch: Correct
< murch>
MarcoFalke: Thank you
< MarcoFalke>
The first pass is selecting a coin with prob .5 as long as the target is not reached
< sipa>
perhaps you should talk to MarcoFalke instead :)
< murch>
Yeah, I believe we've collaborated on this last time around as well. :)
< MarcoFalke>
The second pass will select anything that is not included as long as the target is not reached
< MarcoFalke>
But in any case it will always "unselect" coins when the target was reached to try to "hit the target better" by trying with smaller coins
< murch>
MarcoFalke: I didn't realize this before, but that approach significantly biases towards selecting large UTXO. Do you remember whether the algorithm used to work differently? That would explain the accelerated growth of the UTXO set, no?
< murch>
Aaah, right. I had forgotten about that.
< MarcoFalke>
I think this is still the way satoshi wrote it
< MarcoFalke>
(you could check git log -S or something)
< instagibbs>
murch, you have to remember that excessive amounts of inputs take longer to verify
< instagibbs>
might be taken into account
< _anthony_>
morning all
< MarcoFalke>
The problem is that zero-fee (free) transaction were more common back then
< MarcoFalke>
Right now transaction with no fee (trying to "pay" with priority) are impossible to get through
< sipa>
if you're not trying to hit the target exactly, i think you can limit yourself to only choosing a certain number of utxos from within a range around the target value (say [0.1*target...3*target]
< MarcoFalke>
but coin selection was not adjusted accordingly
< murch>
instagibbs: I solved that by outsourcing the selection to a binary array that flags selected inputs, instead of directly creating sets. That made it much faster. (or do you mean for confirmation on the network?)
< instagibbs>
murch, signing the txn requires hashing the txn over and over for each input.
< instagibbs>
(pre-segwit)
< murch>
instagibbs: Ah, now I get what you mean. Yes, I'm aware. There are also a bunch of other incentives why you don't want to use too many inputs such as fees, and privacy. I'm working on properly writing up all those constraints and requirements for my thesis. :)
< _anthony_>
I have a question: as far as I can tell, script versioning lets you put arbitrary data in the blockchain (just use an unused version number). Is this right?
< instagibbs>
I'm almost sure it wasn't the original idea though, just saying. As sipa said you can simply make sure it doesn't do that by bounding input size
< instagibbs>
_anthony_, the block cost is still static
< MarcoFalke>
murch, you are welcome to do a presentation on your results during a irc meeting
< MarcoFalke>
they are every thursday
< _anthony_>
hmm, yeah, I guess it isn't really much of an advantage
< murch>
MarcoFalke: Thank you, I'd be happy to. I'll get in touch when I have something to present. Perhaps not this week, but next week.
< instagibbs>
_anthony_, otherwise someone today could send a v16 segwit transaction, and stuff 8GB in it
< MarcoFalke>
no pressure :)
< murch>
MarcoFalke: Well, this time I'm fairly certain that I'll have something to present, as in October, I'll have to present some fifty odd pages of work. :)
< instagibbs>
murch, will it include a PR? ;)
< MarcoFalke>
We should look at the new model first
< murch>
instagibbs: That's the plan, although my advisor scolded me last time that it's not part of my thesis, just an added bonus for myself. :)
< MarcoFalke>
Otherwise the pr will receive no review and bitrot instantly
< _anthony_>
yeah I caught that part....I was thinking of how you could make an altcoin using script versioning, but really it wouldn't provide any advantage over regular anyone-can-spend txes
< murch>
instagibbs: Main goals of the work are a comprehensive overview of the problem, the different parameters and approaches how it could be solved, and an evaluation of said approaches. Whether or not it gets adopted by Bitcoin Core is not part of my thesis, but a personal goal. :)
< _anthony_>
you'd have to have support of >50% of miners or else anyone could spend your altcoins out from under you
< murch>
MarcoFalke: Noted. :)
< Chris_St1>
Is there a reason the user agent inside of a version message would change when connecting to nodes within a 5 second period? On testnet..
< sipa>
the agent you sent send, or receive?
< wumpus>
at least bitcoin core doesn't change the 'user agent' on such conditions, no
< Chris_St1>
I'm sending a version message to a tesnet dns seed, and receiving their version message back. If I do this within a 5 second period I'm getting different versions on that same node
< wumpus>
although you should avoid connecting to nodes too intensively
< Chris_St1>
i.e /Satoshi:0.12.0/ then 5 seconds later /Satoshi:0.11.2/
< wumpus>
'change your user agent' isn't a very useful defense against that
< wumpus>
you're connecting to the seed, as in the DNS name? that resolves to a different host every time
< Chris_St1>
mmm yes that could be it
< sipa>
pretty sure that's it :)
< Chris_St1>
Yep. Doh!
< Chris_St1>
Thansk
< sipa>
no results from x9.testnet-seed.bitcoin.jonasschnelli.ch :(
< jonasschnelli>
sipa: this means no NODE_WITNESS nodes...
< sipa>
indeed.
< sipa>
that's pretty bad
< sdaftuar>
i have one if you want to connect to it
< instagibbs>
sipa, where should ACKs be going for segwit pr
< jonasschnelli>
sipa: I guess in master the dnsseed.dump does not list the service flags?