< phantomcircuit>
cfields: what's the best way to add a new binary to the auto tools stuff? i tried copying bitcoin-tx stuff and renaming it but things exploded
< cfields>
exploded?
< phantomcircuit>
complained about things in script.h being redefined
< phantomcircuit>
i'll try again
< cfields>
phantomcircuit: i'd need to see what you tried
< phantomcircuit>
and of course it worked this time
< phantomcircuit>
only thing i did differently was git clean -fdx
< phantomcircuit>
but i guess there's something cached that messed it up before
< gmaxwell>
phantomcircuit: I told you talking to cfields would fix your problems.
< cfields>
heh
< cfields>
phantomcircuit: mm, there really shouldn't be any way for that to happen though, unless you're building with crazy options
< phantomcircuit>
--without-gui --disable-wallet
< phantomcircuit>
nothing else
< cfields>
i think there are some edge-cases where static libs don't get recreated when you're messing with the Makefiles though. Sounds like maybe you had objects linked in twice as a result of something like that
< cfields>
gmaxwell: any chance you've ever investigated how often the opportunistic write in EndMessage() ends up being hit?
< gmaxwell>
Not that I can recall.
< cfields>
ok
< phantomcircuit>
cfields: in general socket send() calls typically succeed completely or fail completely
< cfields>
phantomcircuit: "completely" meaning it swallows the entire buffer it's handed?
< phantomcircuit>
cfields, as long as the tcp buffer in the kernel is larger than the buffer being sent then it will complete
< BlueMatt>
i mean it will if there is room in tcp buf
< gmaxwell>
by 'general' he means on linux. on windows the default tcp socket buffers are like 8k.
< phantomcircuit>
cfields: why do you ask though?
< cfields>
right, i wouldn't think that would hold true in many cases
< cfields>
phantomcircuit: mainly curious how often we're sending with an empty send queue, regardless of the msg/tcp buffer size
< phantomcircuit>
cfields, yes im curious why you're curious
< cfields>
phantomcircuit: the opportunistic send breaks my abstraction model. The connection manager needs to hold total send/recv stats, as opposed to static vars in CNode as it is now. Just thinking through how best to handle it.
< cfields>
it's actually not a problem at all, but I'm writing some interim code movement for easier review
< cfields>
(put more plainly, the issue is that the opportunistic send causes send() to be called from 2 threads)
< phantomcircuit>
cfields: the opportunistic send thing can go away, it just means you'll have an extra memory copy
< cfields>
phantomcircuit: my concern was that it's helpful as an optim to prioritize immediate sends (version msg after connect, for example) over others. I can't imagine it matters much, though.
< phantomcircuit>
cfields, it means those things are sent after the recv handler completes
< phantomcircuit>
although actually there could be a conditional triggered to get the thread that sends stuff to wake up when there's data available
< cfields>
right
< GitHub89>
[bitcoin] pstratem opened pull request #7907: Optimize and Cleanup CScript::FindAndDelete (master...2016-04-17-findanddelete) https://github.com/bitcoin/bitcoin/pull/7907
< GitHub49>
bitcoin/master de39c95 Wladimir J. van der Laan: test: move accounting_tests and rpc_wallet_tests to wallet/test...
< GitHub49>
bitcoin/master f4eae2d Wladimir J. van der Laan: test: Create test fixture for wallet...
< GitHub49>
bitcoin/master a25a4f5 Wladimir J. van der Laan: wallet_ismine.h → script/ismine.h...
< GitHub112>
[bitcoin] laanwj closed pull request #7905: test: move accounting_tests and rpc_wallet_tests to wallet/test (master...2016_04_accounting_tests_to_wallet) https://github.com/bitcoin/bitcoin/pull/7905
< wumpus>
gmaxwell: no, don't have netdebug logs, but wow that's a huge increase
< wumpus>
that's the number of notfounds sent or received?
< gmaxwell>
thats recieved.
< gmaxwell>
my sending went up too... but I don't know if my sending counts are representative since I'm running #7840.
< gmaxwell>
1236 2016-04-11
< gmaxwell>
3469 2016-04-12
< gmaxwell>
2961 2016-04-13
< gmaxwell>
5494 2016-04-14
< gmaxwell>
6788 2016-04-15
< gmaxwell>
14933 2016-04-16
< gmaxwell>
10706 2016-04-17
< gmaxwell>
10018 2016-04-18
< gmaxwell>
5225 2016-04-19
< phantomcircuit>
gmaxwell: that's kind of weird
< phantomcircuit>
also
< phantomcircuit>
how did you notice this?
< phantomcircuit>
:)
< wumpus>
if it's anything like how I notice these things he just saw a lot of notfounds in his log, wondered what, and then decided to do statistics on them
< gmaxwell>
yea, I normally have debug logs scrolling by, and when I sense a glitch in the matrix I run stats.
< wumpus>
could be a few strangely-behaved peers, it's not sudden enough for a code change
< gmaxwell>
I'm seeing it on two nodes though the pattern is pretty different.
< wumpus>
(e.g. if we had suddenly started waiting longer before requesting a transaction inved to us, you'd see more of a sudden jump)
< gmaxwell>
it could be a side effect of some kind of denial of service happening to peers.
< gmaxwell>
but ... pretty huge delay to see notfounds. It also could be some party trying to deanonymize the network graph by doing some mutually exclusive propagation stunts.
< sipa>
are the tx or block notfounds?
< gmaxwell>
tx.
< sipa>
i don't understand... *your* are the one running 7840
< gmaxwell>
I think it's unrelated to 7840.
< gmaxwell>
On the _sending_ not founds, that could be due to 7840. But recieving, no.
< sipa>
ah
< gmaxwell>
also I've been running 7840 for a while; and this seems to have ramped.
< gmaxwell>
e.g. for deanon, step 1 give all nodes except two nodes TX A while giving those nodes node A'. Then step 2. create B which is a derivative of A', and give to one of them, and you'll learn if they're directly connected.
< gmaxwell>
and I'm sure there are lots of other variations on that, including more efficient ones, regardless, they'll likely cause a lot of NOTFOUNDS.
< gmaxwell>
Also, because GETDATA since the introduction of the mempool RPC will return things that aren't in the relay pool, someone could be using GETDATA to bypass inv delays.
< gmaxwell>
(man, that mempool message is the privacy leak that keeps on giving)
< sipa>
so you think this is a deanonymization technique?
< gmaxwell>
maybe, seeing if I can get more or less evidence for that.
< GitHub199>
[bitcoin] laanwj closed pull request #7625: Bugfix: Check for bench_bitcoin being enabled where needed, and skip UniValue dependency when unused (master...bugfix_bench_checks) https://github.com/bitcoin/bitcoin/pull/7625
< gmaxwell>
I really can't tell, the highest notfound sending peers I have claim to be 0.11.x.
< gmaxwell>
but it's not confined to a few peers.
< gmaxwell>
notfounds could also be cause by blocks which are mining many doublespends.
< gmaxwell>
since the block will evict the transactions from mempools.
< GitHub37>
bitcoin/master de821d5 Jonas Schnelli: [ZMQ] refactor message string
< GitHub37>
bitcoin/master 0b25a9f Jonas Schnelli: [ZMQ] append a message sequence number to every ZMQ notification
< GitHub37>
bitcoin/master a1eb344 Wladimir J. van der Laan: Merge #7762: [ZMQ] append a message sequence number to every ZMQ notification...
< GitHub76>
[bitcoin] laanwj closed pull request #7762: [ZMQ] append a message sequence number to every ZMQ notification (master...2016/03/zmq_seq) https://github.com/bitcoin/bitcoin/pull/7762
< sipa>
that branch has a chance of 1 in 100 to trigger
< sipa>
so there is a chance of 90.1%
< sipa>
that nLocktime equals the current height
< sipa>
and in that case it should still be acceptable to the mempool (which uses chainactive+1 to finality check)
< NicolasDorier>
oh yes indeed... ok I continue searching...
< morcos>
gmaxwell: i took a look at 4 different nodes, i didn't see much consistency in how many notfounds i received. 3 of the nodes were generally quite small, a few hundred a day.
< morcos>
gmaxwell: one of the nodes received several thousand a day and 19k on 4-16, but still received 6-7k on 4-10 and 4-11, so it seems more variance than a clear increase
< morcos>
might be interesting to understand what causes more of them
< morcos>
NicolasDorier: sipa: why is that necessarily risky? it seems to me we should move the other direction. separating wh
< morcos>
woo hoo!
< morcos>
(continued) ether a tx is broadcast or in the mempool from its inclusion in your wallet. And then having some way of informing the wallet of its status
< NicolasDorier>
morcos: there is this bug https://github.com/sipa/bitcoin/issues/73, I'll find out what happen later, but the problem is that the transaction is rejected by mempool but still added to wallet transactions. Then after one block, the transaction is rebroadcasted but this time accepted to mempool. The caller of sendtoaddress got an error and thought the money
< NicolasDorier>
was not sent, and retried the command, resulting in sending money two times
< sipa>
morcos: i guess it could be 1) "probe" the mempool whether it would be accepted 2) add to wallet 3) broadcast
< morcos>
sipa: awesome work on segwit, i guess i know where i'm spending my time this week
< NicolasDorier>
"probing the mempool" could be also useful for verifytransaction
< sipa>
i'm not sure whether i should maintain both the 0.12 and master branch independently now, or just work on master, get that review, and then backport again
< morcos>
NicolasDorier: ah, ok so i agree, that we need to clean up the behavior, hadn't read that issue before. but it definitely shouldn't be ambigious whether a tx is broadcast or not.
< morcos>
sipa: do you have both branches ready now?
< sipa>
yes
< morcos>
i kind of think we should have as much of a merge freeze as possible and just all buckle down to review segwit
< morcos>
so that would speak to getting both branches out now
< morcos>
it would be nice if the 0.12 branch got some review from fresh eyes, instead of people just trying to verify its a correct backport. with this much code thats prone to failure
< sipa>
the reasoning against two branches is that it has a higher chance of resulting in unreviewed mistakes in 0.12
< sipa>
the reasoning against just master-based review is that it may interfere with for example the py3 changes
< morcos>
personally i think it makes sense if some of us concentrate review on 0.12 and then take a look at master to see if we think its a proper forward port, and some of us work in the opposite direction.
< morcos>
whenever a bug is found or a change made in one.. then you can port it to the other
< morcos>
but it should really be up to you and wumpus i guess to figrue out how to best do this
< morcos>
merging other stuff in the middle, just seems like a mistake though... lets get segwit merged
< sipa>
open question: do i "fake" the author dates of the commits to make github show it in the correct order?
< morcos>
sipa: meh
< instagibbs>
sipa, you doubled the "Since github shows" statement
< sipa>
instagibbs: thanks, fixed
< instagibbs>
really glad to see this pull
< btcdrak>
nice work sipa!
< Chris_Stewart_5>
^^^^\
< instagibbs>
can someone share their segnet node ip/seed?
< instagibbs>
also, how to run segnet regtest?
< sipa>
bitcoin.sipa.be is a node, and run with -segnet
< instagibbs>
oh nevermind, it managed to find peers after a bit
< GitHub181>
[bitcoin] theuni opened pull request #7911: leveldb: integrate leveldb into our buildsystem (master...leveldb-integration) https://github.com/bitcoin/bitcoin/pull/7911
< sdaftuar>
Should BIP61 be interpreted to mean that reject messages with code 0x01 (REJECT_MALFORMED) should NOT have a hash payload, even if the message that triggered the reject is "block" or "tx"?
< GitHub79>
[bitcoin] sdaftuar opened pull request #7912: Tests: Fix deserialization of reject messages (master...fix-mininode-reject) https://github.com/bitcoin/bitcoin/pull/7912
< GitHub192>
[bitcoin] yurizhykin opened pull request #7913: Fix for incorrect locking in GetPubKey() (keystore.cpp) (master...getpubkey-locking-fix) https://github.com/bitcoin/bitcoin/pull/7913