< gmaxwell>
I think thats returning microseconds, not ms. And I think the now units are .. oh nevermind!
< sipa>
getrand returns a result in [0, 1-1/2^48]
< sipa>
ok
< sipa>
it's in the comments even, i think :)
< sipa>
i've tested it by adding a LogPrintf that writes out the new timestamp
< sipa>
the results looks reasonably random around 5s
< gmaxwell>
sipa: yea, in the header. I'd just gone to look to see what the longest number it would return, and got something that was much larger than I expected.
< gmaxwell>
but thats because for some reason I thought it was using ms not microseconds. :)
< morcos>
gmaxwell: i'm trying to understand what you were talking about yesterday with the tiered fee behavior
< morcos>
are you talking about this is how mempools would behave?
< morcos>
and the target size is how big you wanted to limit your mempool to?
< morcos>
or are you talking about something to do with block assembly?
< gmaxwell>
The latter-- block assembly.
< morcos>
I'm not quite understanding the purpose
< morcos>
you still suggest to add in a fee sorted manner
< morcos>
but you want to have the min fee which is required start going up as the block gets bigger?
< gmaxwell>
Thats what Bitcoin did in 2012 and before, and I think the actual implementation wasn't a good path. It's weird to have such a huge amount of statefulness regarding when transactions come in.
< morcos>
the idea being that before the block has decent size that from a long term perspective you need to be adding at least some txs or bitcoin is useless, but once it gets of a decent size, then you need to worry more about what the profit maximizing strategy is?
< morcos>
well i just don't understand why the min fee would ever go above what the orphan risk equilibrium rate is though?
< morcos>
i think you have a point that we may not have properly been considering the importance of that. and it might make sense to for example add a separate configurable parameter which is the min mining fee rate.
< morcos>
but i'm hesitant to jump to the conclusion that we've been that far off.
< gmaxwell>
Consider, right now imagine we get 4 blocks in a row in 5 minutes. We'd end up with 4 750k (or 1MB blocks) that dig deep into the mempool and mine a bunch of low fee transactions (including maliciously created spam attack crap). The result is that the fee behavior is fairly unpredictable.
< morcos>
does the orphan rate really depend on block size, or is it validation cose for which block size is a semi decent proxy?
< gmaxwell>
morcos: I don't think most people mining have done the math. At least the amounts involved right now are below the threshold where it probably matters; and doubly so w/ pools where the pool operator is mostly not eating the cost of their decision.
< morcos>
yes, but shouldn't there just be a single cut-off point for where its no longer profitable to add txs.. and who cares if you get down closer to that sometimes when there are quick blocks in succession. as long as you're not ever going below that?
< gmaxwell>
morcos: yes, there should be.
< phantomcircuit>
morcos, orphan rate is roughly proportional to the size of the transactions weighted by transaction age
< phantomcircuit>
ie newer transactions are likely to take longer to validate and thus increase orphan rate
< gmaxwell>
phantomcircuit: I've not seen data to support that (though it's plausable), I don't know how to factor it in.
< morcos>
seems like even more accurate would be to look at number of differet txs in the prevouts
< phantomcircuit>
gmaxwell, the effect is marginal at the moment because of gbt latency
< gmaxwell>
morcos: relay network is responsable for carrying most blocks between miners and its hitrate probably makes a much bigger difference than UTXO cache hitrate.
< gmaxwell>
phantomcircuit: I think it wouldn't be imprudent to just skip any transaciton you've known about for less than a few seconds while constructing a block, and doing so would probably eliminate the effect you're talking about.
< morcos>
so actual bandwidth is a limiting factor then
< phantomcircuit>
gmaxwell, yes
< gmaxwell>
phantomcircuit: though a more intelligent approach would adjust the minimum fee based on how long you've know about it.. the 100BTC fee transaction you take right away. :)
< morcos>
ha.. maybe, or maybe you don't want your block reward lost in a reorg. :)
< gmaxwell>
hah, okay 100 is a bit high for that example.
< morcos>
i think spending some time thinking about this is a good idea
< morcos>
maybe we don't find any serious concerns
< morcos>
but with the halvening coming up.. it might make things matter that didn't use to
< phantomcircuit>
gmaxwell, yes but that's slightly harder to calculate :)
< gmaxwell>
morcos: the direction I was more thinking in terms was "control stability"; we have variable fees coming in, variable block times. And then just a max block size wall. It doesn't make for very stable control of the system.
< morcos>
yeah i guess thats what i'm not following
< morcos>
i think that maybe thats true looking at a very short time frame
< morcos>
but the whole point is the block time variability is what lets you pay less for slower confirmation
< morcos>
or are you thinking with respect to block size limits
< morcos>
like if we were to increase them, maybe we don't either by consensus or default inrease them as much when they are in quick succession
< morcos>
i guess thats hard to do by consensus
< phantomcircuit>
gmaxwell, unfortunately the way to fix that is convoluted and a hard fork
< gmaxwell>
My thoughts really aren't clear on where to go from this; but the observation is that the old fee scheme provided backpressure.
< phantomcircuit>
adjust the max block size based on the timestamp (oh and can/will be manipulated by miners)
< morcos>
i guess the way i imagine the system is we should assume that blocks will always be 100% full
< gmaxwell>
in a way that was more gradual that "is load currently over limit yes/no"?
< gmaxwell>
morcos: yes, we should. I agree, long run.
< gmaxwell>
but at the region where we don't quite have enough transaction load paying fees which are high enough to discourage abuse or even high enough to be rational to mine, I think there is a stability problem.
< gmaxwell>
Maybe it's just short term and we shouldn't worry about it.
< gmaxwell>
Part of why I brought up that miners now widely mine txn that are almost certantly irrational to mine was basically to make an argument that we probably could do something that lowered incomes slightly but improved stability.
< gmaxwell>
(and, ultimately, some later dynamic blocksize maximum could make that stability promoting behavior also the income maximizing one)
< morcos>
yeah thats the way i'd look at it, the instability is in the super low fee/free tx region which are essentially bonus anyway. for fee payign txs, they are super stable now.
< gmaxwell>
Okay, I agree with that.
< morcos>
but maybe i'm not understanding stability of what
< morcos>
hmm
< morcos>
yeah so if you're a miner
< gmaxwell>
Basically the system currently encourages you to play block arrival time chicken. Pay massively low fees, get mined fast. hurray! ... next time, mined slowly "Boo bitcoin is broken!"
< phantomcircuit>
gmaxwell, what's the formula you were saying governs the maximum feerate a miner should include?
< morcos>
you could sabotage other miners by feeding them and only them a bunch of txs which will make their blocks slower to mine.. but i guess they'd relay those txs themselves
< morcos>
i'm very curious to see if your stratum timings change by much after 0.12
< gmaxwell>
phantomcircuit: assuming all miners, all transactions are equal. marginal cost in btc/kb for byte N in a block is ...
< gmaxwell>
16:03 < gmaxwell> In any case, formula for marginal feerate in order to overcome subsidy loss; there are an infinite numbe
< gmaxwell>
r of tiny miners who all have the same constant and size sensitive delays is (41.6623*e^(((-0.00166649*size)/bytes_per_sec
< gmaxwell>
)-0.00166649*delay_sec))/bytes_per_sec (this is just the derivative of the orphaning cost for a given size/delay/bandwi
< gmaxwell>
16:07 < gmaxwell> (thats for btc per 1000 bytes, which is usually the unit we use for fees)
< gmaxwell>
16:08 < gmaxwell> For 2.491s/90415 bytes/s which were the straum observed measurements; this function is nearly linear in the domain 0-1mb, 0.0004588 to 0.000450 BTC/kb.
< gmaxwell>
phantomcircuit: it's jut taking the probablity that a block was found before you, given an amount of dealy * 25 btc.
< gmaxwell>
and taking the derivative of it.
< morcos>
when you say block found before you, what does that mean
< phantomcircuit>
and of course if you s/25/0/ it's "build infinite blocks"
< morcos>
that you would have lost the orphan race?
< morcos>
or are you not taking that into account
< gmaxwell>
morcos: they should be; patrick has a stratum server running 0.12 and it's pretty much the fastest stratum server except f2pool mines a block.
< gmaxwell>
morcos: that you would have lost an orphan race.
< gmaxwell>
that figure ignores the fees taken so far; but I thought it was fine to do so consider how small they usually are.
< phantomcircuit>
gmaxwell, it's roughly the same speed when run with the "empty blocks first" patch
< morcos>
2.5 sec seems like so many
< phantomcircuit>
i mean 0.11.2 is
< phantomcircuit>
forgot words there
< morcos>
if you're somehow aware another block has been found in less than that
< morcos>
oh
< morcos>
interesting
< morcos>
nvm, i guess invs tell you it was a block, i was thinking you didn't know that
< phantomcircuit>
warmcache rpc + empty blocks results in basically instantly returning a new template
< morcos>
yeah so you get a block inv. for the next XX ms you should switch your mining hardware to mine an empty block
< morcos>
until you at least have th headers or block to build off of for the next one
< morcos>
that way if you happen to find a block you might beat out the other guy
< phantomcircuit>
morcos, the empty block patch isn't spv/stratum "verification"
< morcos>
yeah i'm suggesting a different kind of empty block
< phantomcircuit>
you actually verify the previous block and then simply short circuit getblocktemplate to return an empty transactions list
< morcos>
you're aware a block has been found, but not even the headers for it yet
< morcos>
you mine empty block on the old tip, so if you happen to find one
< phantomcircuit>
i can think of at least a few reasons that doing that is a bad idea :)
< morcos>
you have a chance of winning the orphan race
< phantomcircuit>
"hey morcor i found a new block!"
< morcos>
yes yea
< phantomcircuit>
repeat every 1 second
< morcos>
but worst case is you give up fees which are small anyway
< morcos>
anyway, in direct headers announcement
< morcos>
you'll have the header so they coudln't do that
< morcos>
and if you don't want to validationless mine
< morcos>
whatever its an over optimization
< morcos>
but 2.5 seconds!
< morcos>
just craz
< morcos>
y
< phantomcircuit>
morcos, huh?
< gmaxwell>
Yea, it's surprisingly high but there are many sources of latency.
< phantomcircuit>
morcos, without fully optimizing im currently 1 second behind f2pool
< morcos>
ok so help me understand that
< phantomcircuit>
im certain i can reduce that more
< morcos>
why?
< morcos>
f2pool is mining on top of a valid PoW header as soon as they have it?
< phantomcircuit>
speed of light beijing to montreal for one
< morcos>
lot less than 1 sec!
< phantomcircuit>
another few hundred ms for TestBlockValidity
< phantomcircuit>
(less with the empty blocks patch)
< gmaxwell>
The switch to empty blocks to optimize the race on the tip may not be so interesting just given how long it takes to retask hardware.
< morcos>
ah, right two validations, you have to validate the new block, and your template
< gmaxwell>
yea, f2pool will mine without validating at all.
< phantomcircuit>
gmaxwell, afaik there's no good reason for the switching time to be so high
< morcos>
i think whoever was trying to patch to move TBV out of path has a good idea
< gmaxwell>
I agree.
< phantomcircuit>
the only hardware i ever directly wrote drivers for switched in 40ms max
< morcos>
no reason for it to be in path, if it fails you're screwed anyway, might as well find out 30 sec later
< gmaxwell>
that was lightsword. It does create a little more risk for other users of the network, but I think its acceptable.
< phantomcircuit>
morcos, huh? if TBV fails you're not screwed
< gmaxwell>
(because you might create an invalid block; and it might escape your infrastructure and get handed to a SPV client.. but .. oh well, hardware/software faults can do that)
< morcos>
phantomcircuit: i've been trying to understand how that works in practice
< phantomcircuit>
morcos, any competently written pool software gets block templates from multiple bitcoind nodes
< morcos>
right ok
< morcos>
and they are presumably running different code? (they should!)
< tulip>
most seem to share code to some degree.
< phantomcircuit>
i'd bet 99% of people are running identical code
< morcos>
and if one fails then it just fails over to using the other one?
< gmaxwell>
he means the different nodes.
< morcos>
right, i mean .11.2 and .12.0
< tulip>
what's a failure?
< gmaxwell>
Eligius has done that in the past, I'm not aware of anyone else doing that. Many large miners replicate their setup and upgrade in chunks but don't have smart failover.
< phantomcircuit>
morcos, i cant answer that definitely but they should be
< tulip>
if two software versions turn out to be consensus incompatible a failover is sort of hard to put into logic.
< gmaxwell>
Our software reliablity is greater than their planning horizon. :)
< phantomcircuit>
(which is also a good reason for soft forks to be backported btw)
< gmaxwell>
Luke added the block proposal support so he could test daemon's block templates against each other.
< phantomcircuit>
tulip, a TBV failure is 99% a mempool problem
< gmaxwell>
But I'm not aware of any other pools using it.
< morcos>
tulip: testblockvalidity failing means there is an error in block assembly logic, and its the fact that the consensus code is not borken that makes TBV say hey, this block is invalid
< morcos>
if your mining software connected to .11.2 and .12.0 would it be smart enough to always get the new tip fast ala the .12.0 node or would you lose some of the speed advantages by having multiple?
< morcos>
i guess maybe .11.2 could just be failover
< gmaxwell>
morcos: right, which requires the 0.12 to stop if it fails; rather than, e.g. continue giving bad results and whining in logs.
< morcos>
to be clear, i'm not particularly concerned that there is a bug in 0.12.0. but i think that now we've changed the logic to require mempool consistency. its inevitable that at some point in the future it'll have a bug
< gmaxwell>
We've had consistency bugs in the past.
< morcos>
hopefully one thats unlikely to be hit
< gmaxwell>
Perhaps we should have all full nodes doing a createnewblock every once in a while.
< gmaxwell>
We're probably much less likely to get reports because so few users (and developers) mine.
< morcos>
gmaxwell: i think the kind of bug that is most likely is a situation thats rare, such as a reorg that eliminated one sibling of a sequence locked tx whose fee rate was now too low and so would be evicted except RBF blah blah. and when it happens its just going to happen to everyone.
< morcos>
but having developers running at least one full node calling CNB all the time is worthwhile
< morcos>
i've been doing that for a couple months now
< phantomcircuit>
morcos, probably not since different maxmempool sizes
< phantomcircuit>
even with the index usage in cnb in 0.12 i'd still have nodes with tiered max mempools
< gmaxwell>
right. but, for example, we discourage mining on git master. Now I mine on git master (well, 0.12 branch right now); but if there is any system dependency in the bug, I may not see it.
< gmaxwell>
we've had bugs that were hit before only with debug options on, plausable we'll have one that is only hit with debug options off.
< phantomcircuit>
gmaxwell, that could be part of making CNB run in a background thread continuously
< phantomcircuit>
the gbt longpoll stuff does a good job of only running cnb when it's necessary
< morcos>
i have a request, i'll direct if at gmaxwell since he runs the project now according to hearn
< phantomcircuit>
(although there's a bug there around timing out normal http libs...)
< gmaxwell>
phantomcircuit: yea, I think it could just do that for everyone. if you never call getblocktemplate it could just run with a 10 minute timer or whatever.
< morcos>
before we can really make CNB run all the time and do many other improvements to the code
< phantomcircuit>
gmaxwell, nah you just run it when there's a new block
< morcos>
we need to rework the entire locking strategy
< gmaxwell>
the locking train wreak you mean? :(
< morcos>
exactly
< phantomcircuit>
morcos, i suspect that's much more complicated than it initially sounds :P
< morcos>
we don't have to go to the perfect solution
< gmaxwell>
Every time I go to add something where I need to add locking, I go to look to see how other things are locked. And then I go looking for a bottle of stiff booze (well not really).
< morcos>
but someone who is a better developer than i should draw up a road map for us
< morcos>
sipa had some good ideas for it, but maybe its not fair to give him every hard project
< gmaxwell>
Sipa has been moving in a useful direction in new code, but I think he has not written down the principles that drives that direction enough to make it easy for the rest of us to predict the rest of the path for it.
< morcos>
yeah. i think overall the project would benefit from people working in a team fashion a bit more so workload could be divided.
< morcos>
i'm happy to take direction
< morcos>
maybe thats always hard to accomplish with open source
< phantomcircuit>
gmaxwell, example?
< gmaxwell>
phantomcircuit: look how the headers first, download tracking stuff works.
< morcos>
i think there is a hesitation for sipa to right down those principles (or anyone else) b/c they don't want to be seen as telling everyone else that this is the way it is
< morcos>
s/right/write/
< gmaxwell>
"No please tell us what to do!"
< morcos>
:)
< phantomcircuit>
morcos, the cnb locking can be improved without too much hastle but requires acquiring cs_main and mempool.cs many many more times
< phantomcircuit>
im not sure if that's worth it or not
< gmaxwell>
I currently have code blocked on "need to fix locking".
< phantomcircuit>
there's lots of random stuff where you need partial bits of the chain state but dont actually care if it changes since you're just going to sync against the changes immediately afterwards
< phantomcircuit>
createnewblock being the most obvious
< phantomcircuit>
there's also the wallet code but that's harder to reason aboue
< phantomcircuit>
about*
< morcos>
even just connecting a new block
< * phantomcircuit>
goes to look at CNB in master
< phantomcircuit>
morcos, lots of things devolve into "make chainstate an mvcc db"
< morcos>
once you grab the state... you could release the lock to do all your validation.. and then grab it again to apply
< phantomcircuit>
but complexity kills
< morcos>
if the tip changed int he meantime, abort
< phantomcircuit>
well there's things where copying the state is more expensive than the operation itself
< morcos>
but connect block already grabs all the state it needs for the script verification threads
< morcos>
it just waits for them to complete before flushing the new coinsview back to pcoinstip
< phantomcircuit>
morcos, the particular case of the script verification queues is interesting
< morcos>
there is no reason not to be accepting txs to your mempool during that time
< phantomcircuit>
it should be easy to add another validation state
< morcos>
as you only care about the old state consistency until the new state is applied and then you clean out the mempool for things which can't be there at that time anyway
< phantomcircuit>
i tried but haven't touched that code enough to be confident it would work
< morcos>
what do you mean another validation state?
< phantomcircuit>
if we acquire the same lock twice in the same thread is that going to context switch?
< phantomcircuit>
or deadlock?
< phantomcircuit>
actually it's going to deadlock since they're not reentrant
< morcos>
they are, which locks?
< phantomcircuit>
morcos, "i've validated everything except the scripts"
< phantomcircuit>
oh is cs_main reentrant?
< morcos>
obviously to call ATMP and connect block concurrently you'd need to redesign the threads as well, since they're both almost always called by the same one
< phantomcircuit>
oh they are
< morcos>
ha, yeah, its pretty rare people bother to do assert_lock_held instead of just calling LOCK again
< phantomcircuit>
do you need cs_main to use a CBlockIndex*
< phantomcircuit>
ie chainActive.Tip() ?
< phantomcircuit>
(im going to assume yes)
< morcos>
thats the interesting question
< morcos>
for the most part no
< phantomcircuit>
morcos, it mostly doesn't matter actually
< phantomcircuit>
it looks like you only need chainActive.Tip() for a small fraction of CNB runtime
< phantomcircuit>
gmaxwell, do you know off hand the rough latency of LOCK(cs_main) ?
< morcos>
phantomcircuit: remember if you lock it multiple times you're going to slow down the function b/c it'll have to keep waiting on the locks
< phantomcircuit>
morcos, im going to change fBuildEmptyBlock to fNewBlock
< phantomcircuit>
and change the locking strategy based on that
< phantomcircuit>
morcos, in general though i think we should be willing to give up the lock in CNB to allow for a new tip
< phantomcircuit>
right now there's a problem where CNB can stall validating a new tip
< phantomcircuit>
although this isn't in practice a huge problem since you reduce the average latency by n if you run n nodes
< phantomcircuit>
divide*
< morcos>
phantomcircuit: agreed, thats a problem. but for now the non TBV part of CNB is fast enough that if we just solve for TBV that would be enough.
< phantomcircuit>
morcos, we could run TBV without signature checking but everything being checked has is probably in sigcache
< phantomcircuit>
so i dont see that making a huge difference
< Luke-Jr>
TBV shouldn't be slow..
< GitHub0>
[bitcoin] pstratem opened pull request #7203: Release cs_main in CreateNewBlock while selecting transactions from mempool. (master...2015-12-12-createnewblocklocks) https://github.com/bitcoin/bitcoin/pull/7203
< phantomcircuit>
(note i haven't tested that PR like at all)
< morcos>
phantomcircuit: ha, just thought of this. when we call ConnectBlock from TBV we call it with fJustCheck.
< morcos>
We should be able to release cs_main before control.Wait()
< morcos>
so we don't have to wait for script verificaiton threads to finish with cs_main locked
< morcos>
not sure exactly how much time that would save in general, but seems like a super easy win
< phantomcircuit>
morcos, probably not much, remember that all the transactions being checked are already in sigcache
< sipa>
phantomcircuit: not guaranteed :)
< phantomcircuit>
sipa, it is when you run with the sigcache size i do :P
< gmaxwell>
BlueMatt: have any way to estimate how long it takes for a transaction to reach all the relay node clients?
< Luke-Jr>
blah, gitian needs trusty now and that needs a vmbuilder upgrade :<
< tulip>
that's weird, somehow my node has got into a state where it has fallen behind the chain.
< tulip>
Luke-Jr: nothing interesting, no rejects or invalid blocks, it's just chilling.
< Luke-Jr>
tulip: for how long?
< phantomcircuit>
tulip, getpeerinfo
< phantomcircuit>
i bet you're waiting for a peer to send you a block
< tulip>
phantomcircuit: woo, there we go. socket timed out, and suddenly we're back at the tip after 26 minutes behind.
< phantomcircuit>
tulip, and that's why my gbt nodes connect to trusted nodes and the relaynetwork only
< tulip>
sorry, an hour and 3 minutes behind.
< CodeShark>
it takes over an hour to timeout a block request?!?!
< gmaxwell>
no, it takes 20 minutes.
< gmaxwell>
but if you have multiple screwball peers you might time out more than once.
< gmaxwell>
I know how to make it safely dynamic but haven't had a chance to implement it.
< gmaxwell>
My thought for making it dynamic: start with the 20m timeout. When you sucessfully fetch a block measure how long the getdata took, extrapolate that to 1MB *2. Set the timeout to that.
< gmaxwell>
Any time you kick a peer off for timing out, increase the timeout (1.5x?).
< gmaxwell>
This should be safe and result in quite low timeouts for most peers most of the time.
< Luke-Jr>
sounds gameable by hostile peers
< tulip>
screwy network conditions caused this one but with alarmingly high amount of time where it sat knowing headers it couldn't update to.
< Luke-Jr>
keep connecting to your victim, and try to increase the timeout to a few days.. then trap real blocks, as you feed competing ones over a separate connection
< gmaxwell>
Luke-Jr: Explain, because I think it's not usefully gamable (except by constant factors)
< gmaxwell>
Luke-Jr: oh I assumed the timeout would still be capped at 20 minutes.
< Luke-Jr>
oh, ok then
< gmaxwell>
Beyond that, if you're actually that slow, you have no hope of keeping up with the network.
< CodeShark>
in that case, shouldn't 10 min be the cap?
< Luke-Jr>
CodeShark: nah, your connection might be bursty
< gmaxwell>
CodeShark: No, because ^
< gmaxwell>
there should be some headroom.
< Luke-Jr>
might be reasonable to do some magic before the calibration though
< Luke-Jr>
eg, if you don't get any traffic on the block within ~30 seconds, start getting it from another peer too and kick one if both start sending
< CodeShark>
that's what I was sorta thinking
< Luke-Jr>
otoh, a hostile actor may try to trickle it to you just to keep you downloading as slow as possible
< tulip>
Luke-Jr: something is a bit ambiguous, if I want to slow you down I'll just send a byte a second.
< Luke-Jr>
tulip: yeah
< gmaxwell>
Yea, that requires low level visibility that we don't really have right now.
< CodeShark>
I was thinking we try from different peers and try to use the best one available - if one starts getting slow, we try others
< gmaxwell>
CodeShark: requires having a concept of what 'slow' means, _and_ having sub-message visibility, .
< CodeShark>
yes, we could check the buffer before we complete the download to get an estimate of the datarate
< gmaxwell>
When we start the block fetch, there is always only one available (the first).
< CodeShark>
we could also download multiple blocks in parallel from different peers when measuring download rate to not waste bandwidth
< Luke-Jr>
CodeShark: fancy!
< Luke-Jr>
but definitely more complicated to implement:P
< phantomcircuit>
gmaxwell, has anybody analyzed how restrictions in bandwidth effect the probability that there has been a fork that you've not seen?
< gmaxwell>
well there is only one block on the tip!
< phantomcircuit>
i guess you'd see headers
< tulip>
do nodes actually forward headers like that?
< CodeShark>
sync on core is headers-first
< CodeShark>
which means you can validate PoW before downloading any actual blocks
< tulip>
sure, but do nodes actively forward headers of blocks they don't believe to be the main chain
< CodeShark>
oh, not that I'm aware of ;)
< CodeShark>
not yet
< CodeShark>
it's really expensive to create a competing chain that is invalid...so chances of such a chain growing long are low (unless something is fundamentally broken)
< Luke-Jr>
tulip: if those nodes are attacking you, they do :D
< tulip>
not invalid, just not best
< CodeShark>
such chains tend to die quickly
< CodeShark>
the main threat with headers sync is the isolation attack
< CodeShark>
having said that, it would be nice to have better fork detection capabilities...as in knowing that hashrate has forked
< CodeShark>
which means something is fundamentally broken ;)
< tulip>
forwarding all headers, always would be nice.
< tulip>
that likely breaks assumptions though, I think the current p2p network assumes that if you inv a block you actually have it.
< CodeShark>
wasn't it suggested before that we stop inving blocks and just send block headers?
< gmaxwell>
We could add a message that says "here is a header, but I don't have the block" but then you have to send a "I have the block" later.
< gmaxwell>
CodeShark: yes, we do that now. but then the header is just the inv. :)
< CodeShark>
we could just add a flag to the headers message
< CodeShark>
might be a good idea to start versioning the p2p serialization structures separately from the client version and the consensus structure versions :)
< CodeShark>
and to stop treating blocks as subclasses of block headers ;)
< gmaxwell>
Versioning suggests linear progressions. :)
< CodeShark>
versionbits for p2p? ;)
< CodeShark>
the alternative is to just name the new p2p structures something else...which perhaps might be simpler in the end
< CodeShark>
better fork detection would be a wonderful thing, though
< Luke-Jr>
uh so
< Luke-Jr>
I can no longer gitian-build :/
< Luke-Jr>
vmbuilder fails on trusty
< Luke-Jr>
what vmbuilder did other people here use?
< CodeShark>
we could go beyond just "here is a header, but I don't have the block" - we could do "here's a header, I checked the block and it is invalid"
< gmaxwell>
gavinandresen: Hi. I'm concerned that I keep hearing reports of you taking actions on behalf of Bitcoin Core while not actually communicating with anyone involved in the project, or being involved yourself. I wasn't sure of how much of this was rumor or miscommunication, but this appears to be you directly confirming it:
< phantomcircuit>
i wont be contributing to core anymore until the license is changed to prevent my work from being used by dickbags making backroom deals
< Luke-Jr>
…
< Luke-Jr>
phantomcircuit: you really think Core is going to become non-free software?
< phantomcircuit>
Luke-Jr, no but i can certainly stop contributing
< Luke-Jr>
phantomcircuit: you can, but all it's going to do is increase everyone else's workload :/
< Luke-Jr>
what's the point in making an ultimatum that you know cannot be met?
< phantomcircuit>
Luke-Jr, the license can be changed to prevent specific people from using the code
< gmaxwell>
phantomcircuit: I share your irritation, but you don't need to go pyrric on this; we'll take care about it.
< gmaxwell>
er take care of it. :)
< Luke-Jr>
phantomcircuit: not without it becoming non-free software, in which case I will have to quit Core and fork it.
< btcdrak>
gmaxwell: it's time we boot gavinandresen
< btcdrak>
gavinandresen: you dont contribute to Bitcoin Core just to creating chaos. While others might be more polite, I'm going right ahead and saying this, please go away, you are not wanted here.
< gmaxwell>
phantomcircuit: that people can take our work and use it in other projects is an important safty valve that guards against our own error, and frees us from having to accomidate every wish and desire; without creating lock in should we misstep.
< gmaxwell>
phantomcircuit: so I think precluding people from exploiting out work, as 'just' and satisifying as it would be is not a good forward path.
< gmaxwell>
s/out/our/
< CodeShark>
if phantomcircuit is specifically referring to actions taken by someone on "behalf" of Bitcoin Core, I think the policy needs to be specifically directed at that...and we need to come out with one voice saying that someone doesn't represent Bitcoin Core
< Luke-Jr>
I certainly wouldn't object to a joint statement. Changing the license to non-free is a big no-no IMO though.
< CodeShark>
as to what the action could be, I'll leave that open to interpretation for now in the interest of being slightly more diplomatic than btcdrak
< gmaxwell>
I really can't take the stress of the non-cooperative, backroom dealing, dram production and the attempts to undermine collaborative work anymore.
< gmaxwell>
To say that I'm fed up with it would be an understatement, and attempts to address it in private have failed.
< midnightmagic>
what is bitcoin's licence anyway.. mit/bsd?
< Luke-Jr>
midnightmagic: yes
< midnightmagic>
in magical parallel universe able-to-contact-everyone land, you could change it to gplv3
< Luke-Jr>
midnightmagic: we could change it easily. MIT is relicensable to GPLv3
< Luke-Jr>
but that wouldn't do what phantomcircuit is asking
< midnightmagic>
mm. I don't think it is. That is, the original copyright is not alterable: only the copyright of modifications going forward, and those of people whom you've contacts and who agree in writing.
< gmaxwell>
MIT is a reasonable choice for the basic infrastructure. Not having it that way would just encourage the creation of alternatives just for licensing sake, which would be preferable to avoid.
< btcdrak>
gmaxwell: I dont think relicensing is the solution. I also dont think gavinandresen has a hope in hell gaining traction. XT was total failure and adoption of a hostile fork has been ruled out by miners and most major exchanges.
< aj>
Luke-Jr: Affero GPL with code added to dump the running node's source via an RPC call could be hilarious though (iirc Affero GPL would prevent you from disabling/removing that feature)
< Luke-Jr>
midnightmagic: only the new code going forward can be relicensed in any case; even if the old code were offered under a new license, the MIT license remains valid for it
< Luke-Jr>
aj: it would need to be over p2p, not RPC
< Luke-Jr>
aj: Eloipool uses it
< aj>
Luke-Jr: quite right
< Luke-Jr>
(and will dump its code over stratum or HTTP)
< gmaxwell>
midnightmagic: phantomcircuit is unhappy that people who seem to be openly working against his own efforts trade on his work, and the name of the project. This is a downside of free software, but it's minor compared to the benefits.
< aj>
Luke-Jr: easier to get at source for a python program than a c++ one too
< Luke-Jr>
we could rename the project and enforce it as a trademark..
< btcdrak>
Luke-Jr: "Bitcoin Core" as a phrase is probably trademarkable, but I really dont see the benefit.
< midnightmagic>
gmaxwell: The number of netbsd core developers who have expressed raging bitterness just to me in private email that people like broadcomm make millions on their code and don't even contribute their modifications to the original source is more than I can count on one hand. And at the time they wrote me I was a nobody.. :-o
< Luke-Jr>
btcdrak: but Hearn coined it
< Luke-Jr>
in fact, is that a risk to us?
< gmaxwell>
No.
< btcdrak>
Luke-Jr: no
< midnightmagic>
Does the U.S. have a pre-existing trademark exception rule too?
< midnightmagic>
I know Canada does.
< Luke-Jr>
midnightmagic: I think the GPL would be a bad choice for node software.
< btcdrak>
The solution to gavin is what I said it was back in January when no-one would listen to me >.>
< midnightmagic>
Luke-Jr: it would hinder its deployment probably. But, ignoring all that I don't think it's realistic anyway. Mostly navel gazing I guess.
< midnightmagic>
btcdrak: I'm listening now. Can you link me?
< midnightmagic>
In Canada, if you run a shop with a name of something that was trademarked *after* your use of the term, for as long as you operate you are excepted from trademark infringement. Unless the law has changed. Which is possible, given our recent experiment in neo-fascism.
< btcdrak>
We need a technical lobbiest, someone who will go to companies and listen to them and discuss the technology with them. There are other things too, but I wont say them in public.
< midnightmagic>
Well that was ideally what the BCF was supposed to do. :-/
< btcdrak>
(because they would be used by gavin against us)
< btcdrak>
The BCF is a useless train wreck.
< btcdrak>
the BCF continues to prop up Gavin with the title "Chief Scientist" which gavin uses to mislead the companies he lobbies.
< Luke-Jr>
problem: we are understaffed technically already..
< midnightmagic>
I was optimistic (hope springs eternal) but I guess phantomcircuit pretty much called it ages ago when they first published the constitution.
< randy-waterhouse>
btcdrak: did you specific companies in mind, or some examples?
< randy-waterhouse>
s/did you have/
< btcdrak>
randy-waterhouse: no, everyone in general. gavin and mike have gone around telling lies, the only way to counter that is to give taregetted presentations etc.
< Luke-Jr>
btcdrak: not just a joint letter saying "you may have been told lies by <people>; we don't know what those lies are, so please take what was said with a grain of salt - we're here to clarify anything"
< Luke-Jr>
?
< btcdrak>
Luke-Jr: Yes, we must actively say that. Bitcoin Core devs (as I have said repeatedly in private) are playing into the hands of disruption by being too politically correct and "nice". We need to be prepared to call a spade a spade.
< Luke-Jr>
btcdrak: a letter is easier than a presentation, is what I mean.
< btcdrak>
Luke-Jr: it's a good start,
< btcdrak>
Luke-Jr: if I was a little richer I would pay someone but I have to preserve my funds because of my health situation which is questionable at the moment.
< Luke-Jr>
btcdrak: hope your health improves :|
< btcdrak>
Luje-Jr: yeah ditto! I wish they would find a diagnosis at least.
< phantomcircuit>
gmaxwell, to be clear i meant changing the license to specifically bar gavin from using the code for any purpose and only gavin
< gmaxwell>
phantomcircuit: pedantically, it would no longer be free software; and doing that would be imposing a huge analysis cost of anyone else using the software; not a good tradeoff.
< Luke-Jr>
so I rewatched the "poisonous people" talk (and concluded it doesn't seem to fit Gavin FWIW), and two recommendations that stood out as applicable would be 1) having well-defined and specific goals/todo lists leading up to 1.0; 2) we should [when possible] try to have multiple people familiar with each part of the code - so jgarzik's MAINTAINERS file probably should be laid out expecting multiple maintainers for each subsection
< Luke-Jr>
I'm not sure either of these are too practical for us, but it might be nice to keep them in the back of our minds
< btcdrak>
in the longer term as the goals of breaking Core into sub projects like wallet, consensus and full node, we'll already have the maintainer infrastructure in place.
< Luke-Jr>
not sure what you mean by a trial.. it's not the first maintainership thing (wumpus began as GUI maintainer, and I've been mining maintainer for a while; I imagine there are others too)
< btcdrak>
trial as in see how it goes for expanding to more maintainers
< btcdrak>
maintainer here means people with push access and authority to merge PRs.
< Luke-Jr>
that's not a matter of authority.. people with push access are supposed to merge only with appropriate ACKs/NACKs, not on personal decision
< btcdrak>
Luke-Jr: yes I know, that's well explained in the referenced document.
< Luke-Jr>
btcdrak: on another note, did you manage to do the trusty gitian update?
< btcdrak>
Luke-Jr: I dont think English has the right words for this, haha.
< btcdrak>
Luke-Jr: ut oh. my mind is blank.
< btcdrak>
Luke-Jr: can you refresh my memory?
< Luke-Jr>
btcdrak: in the past ~week or so, master upgraded the version of Ubuntu used by gitian to "trusty"
< Luke-Jr>
I can't get the base image to build
< btcdrak>
Luke-Jr: OK, let me have a go from scratch
< Luke-Jr>
worse: it appears to be trying to touch my MBR :|
< Luke-Jr>
seems upgrading gitian-builder fixes it
< btcdrak>
Luke-Jr: still setting up my VM. what version of gitian-builder did you use?
< Luke-Jr>
before, it was 9d46e26
< Luke-Jr>
now f9ab587
< GitHub166>
[bitcoin] MarcoFalke opened pull request #7205: Update copyright headers to 2015 (master...MarcoFalke-2015-BumpHeaders-0.12) https://github.com/bitcoin/bitcoin/pull/7205
< btcdrak>
Luke-Jr: finally installed virtualbox and build the Gitian VM worked fine with f9ab587 (which is the HEAD when you clone gitian-builder freshly)
< btcdrak>
Luke-Jr: the only problem I had was sudo didnt work for me, I had to put a manual entry in the sudoers file for some reason.
< btcdrak>
Luke-Jr: 9d46e26 is really old too I see.
< BlueMatt>
gmaxwell: not really...the privacy policy on bitcoinrelaynetwork.org states no information about tx receive time will be logged, though clients of course are free to do so themselves (as I cant stop them)
< BlueMatt>
gmaxwell: so...you can go log that if you want