< sipa>
Nobody_: i have the impression you're trying to be nice, so i don't want to yell at you, but i have no clue what you're trying to say
< Nobody_>
I truely am Nobody in the system and as the Court of Japan ruled, Nobody own's the Cryptocurrency as Nobody own's the Blockchain so enjoy your holiday ok? make it the Perfect Greatest you ever could have! ;)
< Nobody_>
With even a bit of love from God
< Nobody_>
No Joke!
< sipa>
ok, thank you!
< Nobody_>
Hahaha
< Nobody_>
Time to leave, have a great one and that magnet link will download if you need it
< promag>
is fee estimation an activity? like it should show up in the activity window while it's not ready?
< promag>
luke-jr: k
< sipa>
promag: that's maybe a useful way to present it
< sipa>
though it's not actually acitivity as in doing something - it just needs enough data
< sipa>
so perhaps you can call it 'gather data for fee estimation'
< promag>
waiting is an activity :D
< promag>
or that
< promag>
I like that
< fanquake>
I'm going to delete the "help me hard fork bitcoin for $500" comments and lock the threads they've been posted on. Seem to be kind of alt-coin related anyway.
< sipa>
fanquake: thanks
< jonasschnelli>
[17:12:01] <sipa>promag: on 8-core AMD Ryzen:
< jonasschnelli>
sipa: you are working on a desktop?
< sipa>
jonasschnelli: via ssh
< jonasschnelli>
sipa: heh... yes. I also use a ssh remote build script when I'm on low battery but high bandwidth.
< sipa>
jonasschnelli: no, i don't generally don't build remotely
< sipa>
i just ssh'ed into my desktop
< sipa>
and compiled there
< jonasschnelli>
Yeah.. makes sense...
< sipa>
usually i build just on my laptop
< jonasschnelli>
compiling is just a battery drainer...
< sipa>
i have a power socket nearby :)
< jonasschnelli>
I think it's a travelers work from non-home location problem. :)
< _jadeeUnix>
.
< jonasschnelli>
luke-jr: a complete UTXO scan with sweepprivkeys takes ~3mins on my machine (mainnet)... is there a nice way to show progress?
< wumpus>
promag: you add some lock(cs_main), were these missing before?
< wumpus>
you don't mention any lock bug fixes, which is why I ask
< promag>
wumpus: yes missing locks
< promag>
not sure if all are bugs because some come from the init & load block index etc
< wumpus>
ok, please mention that in the issue, we should mark it as bugfix and not only refactor
< wumpus>
yes the ones in init are probably not so relevant, but there is one in wallet too
< promag>
but FindForkInGlobalIndex I think it's a bug
< wumpus>
good, thanks for finding and fixing
< JackH>
is there a limitation to how many addresses Bitcoin Core can generate for a given wallet? We issue about 500 new addresses per day, some which are used, some which are not, but is there a limit per wallet?
< wumpus>
the software has no limit
< xmsx>
I tested a wallet with 300k addresses once
< wumpus>
at some point you might run into resource issues, things going slower
< JackH>
we thought about 300k actually being where we would see problems
< JackH>
we are soon hitting that
< JackH>
would there be any difference between HD wallets and non HD wallets in terms of speed?
< wumpus>
I'd be more worried about the amount of transactions, not so much the number of keys, keys are small
< JackH>
so about 40% are transactions
< JackH>
meaning roughly 120k transactions are stored right now
< JackH>
do we know the bottleneck?
< wumpus>
bitcoin core's wallet is not optimized for handling huge numbers of transactions, for example it keeps them all in memory, and doesn't at the moment have an utxo index. There has been work in that direction but any help is welcome.
< wumpus>
as for what the limits are for your cpu/amount of memory, try it out on a development machine
< xmsx>
Also, I would suggest discussing these questions from throwaway account (No one really needs to know that you store all addresses in single wallet) :)
< wumpus>
e.g. using regtest, you can generate tons of transactions and blocks in a relatively small time
< promag>
wumpus: JackH: actually the number of addresses has a great impact on performance
< wumpus>
it has an impact on performance, but what he asked is whether there's a limit
< JackH>
we are looking to run multiple daemons as a solution
< JackH>
and retire daemons as they hit performance issues
< JackH>
daemons//wallets
< kabaum_>
Is it possible that there are other yet unknown ways to malleate a signature than the "-S" trick? Or maybe even known ones? I refer only to inherent ECDSA signature malleability.
< xmsx>
Could anyone review changes to segwitsupport.csv (bitcoincore.org repo) please? Using P2SH-P2WPKH addresses atm but will add bech32 soon too :)
< wumpus>
kabaum_: yes that is possible (might be better to ask in #bitcoin-wizards)
< bitcoin-git>
bitcoin/master 3f09e03 Karl-Johan Alm: Remove unused include in hash.cpp
< bitcoin-git>
bitcoin/master 66479c0 Wladimir J. van der Laan: Merge #11884: Remove unused include in hash.cpp...
< bitcoin-git>
[bitcoin] laanwj closed pull request #11884: Remove unused include in hash.cpp (master...20171213-unneeded-include-hash-cpp) https://github.com/bitcoin/bitcoin/pull/11884
< wumpus>
yep
< BlueMatt>
if someone wants to kill uselessly-trivial prs: #10839 could be merged
< gribble>
https://github.com/bitcoin/bitcoin/issues/10839 | Dont use pass by reference to const for cheaply-copied types (bool, char, etc.) by practicalswift · Pull Request #10839 · bitcoin/bitcoin · GitHub
< bitcoin-git>
bitcoin/master 99ba0c3 practicalswift: Don't use pass by reference to const for cheaply-copied types (bool, char, etc.).
< bitcoin-git>
bitcoin/master c66adb2 Wladimir J. van der Laan: Merge #10839: Don't use pass by reference to const for cheaply-copied types (bool, char, etc.)...
< bitcoin-git>
[bitcoin] laanwj closed pull request #10839: Don't use pass by reference to const for cheaply-copied types (bool, char, etc.) (master...dont-pass-by-reference-for-cheaply-copied-types) https://github.com/bitcoin/bitcoin/pull/10839
< maaku>
the implementation of BIPs 98, 116, 117 could use some review:
< maaku>
(together they implement the features needed for MAST, currently policy-only)
< wumpus>
would be good to bring up in the meeting
< maaku>
I believe kallewoof is going to make a PR when he's done reviewing it himself, but no harm in others taking a peek too
< Chris_Stewart_5>
maaku: What's up with the goto in the tail-call-semantics? Isn't that pretty ill advised?
< maaku>
Chris_Stewart_5: goto is a funny thing. this is actually the one application goto is the best solution for, where the semantics are well-defined and is the easiest way to accomplish what is desired
< maaku>
it's not possible to do tail-call recursion otherwise, and to my knowledge that's basically the reason goto still exists all these iterations of the C++ standard later
< Chris_Stewart_5>
maaku: So I might be getting my 'tail-calls' mixed up, but does this goto keep a stack frames allocated during the recursive step?
< Chris_Stewart_5>
like tail recursion in a functional lang
< maaku>
A new stack frame is not created for the call itself. Any changes to the stack are properly unwound, with destructors called for things that fall out of scope, or stack allocations made and constructors called for things you skip over in other applications of goto (not here)
< maaku>
for example, if the `if {}` statement that had the goto had its own variable that was an instance of the class, the goto would have called the destructor and deallocated it before jumping up to the start of the function
< maaku>
or as an actual example, vfExec is destroyed as part of the goto
< maaku>
(and then recreated when execution reaches it again)
< maaku>
from cppreference:
< maaku>
> If transfer of control exits the scope of any automatic variables (e.g. by jumping backwards to a point before the declarations of such variables or by jumping forward out of a compound statement where the variables are scoped), the destructors are called for all variables whose scope was exited, in the order opposite to the order of their construction.
< Chris_Stewart_5>
maaku: so why can't we just recursively call `EvalScript` inside of VerifyScript?
< Chris_Stewart_5>
and have a counter or whatever of how many recursive calls we have made
< Chris_Stewart_5>
now that I re-read that i'm not sure if that made sense.
< maaku>
(1) that would require refactoring the EvalScript API to carry extra parameters (like the alt stack) as arguments; (2) that would not be safe if/when the limitation of single-recursion is dropped
< maaku>
this is an easier, more targetted change that enables just the functionality required without large code changes, with well defined semantics (even if the use of goto is unusal)
< sipa>
why not wrap the part you're jumping over in a loop?
< Chris_Stewart_5>
maybe I am missing some goto functionality here, but it seems like it would be almost identical to just call EvalScritp() instead of using goto
< sipa>
i admit there are use cases for goto, and dismissing it unconditionally is very much a cargo cult thing to do
< sipa>
but i don't think you need it here
< maaku>
Chris_Stewart_5: look at the altstack and nOpCount variables
< maaku>
sipa: a do-while would be equivalent, if slightly more verbose, at the additional review cost of indenting 1000 lines of code
< maaku>
that's up to kallewoof
< Chris_Stewart_5>
maaku: Ah ok I think i see what you are saying. C++ allows for default args right? But I guess that is still pretty unfortunate
< sipa>
review with ?w=1 (in github) or -w (in git) makes indentation changes trivial to review
< maaku>
sipa: I'm sure it would be a stumbling block for many still, unfortunately.
< sipa>
maaku: i expect that a goto would be even more so
< sipa>
the altstack trick is neat; it allows doing this without any new script versioning at all
< maaku>
re: goto I guess it comes down to encoding semantic intent. a do {} while(!done) construct where done is a boolean variable set inside the loop under complex conditions is not a clearer encoding of the semantic intent, which is to restart execution of the function from the beginning
< maaku>
this is why I am handing it over to kallewoof though, who will make the call on making such changes
< sipa>
ok
< maaku>
yes, and I have another simple soft-fork that recovers script versioning inside the witness itself, which would allow us to later remove the altstack trick without needing a new "script version" as presently defined
< sipa>
i'm still very unconfortable with needing code execution before knowing what code is executable, though
< maaku>
?
< sipa>
you don't distinguish code and data
< sipa>
anything can be either, up to the end of the (top level) program
< maaku>
I mean what are your concerns?
< sipa>
static analysis
< maaku>
(as a lisper I have no sympathy for that general concern ;) )
< sipa>
haha
< maaku>
static analysis of what, limits? those are gone in this iteration
< sipa>
can you enumerate all possible code paths ahead of time?
< maaku>
yes
< maaku>
I'm not sure if that's generally provable in the sense that you can show non-decidable scripts aren't possible to write
< maaku>
but the way this is anticipated to be used, and especially with only single recursion, it is straightforward to show all possible code paths
< sipa>
well there are certainly trivial counterexamples, like just "OP_0 OP_TOALTSTACK"
< sipa>
which permits executing anything?
< maaku>
I expect actual deployment to be of the form [root 1 MERKLEBRANCHVERIFY 2DROP 2DROP N*TOALTSTACK] with [argN ... arg1 script proof] as the witness
< maaku>
it's trivial to show all code paths there -- expand the merkle tree
< sipa>
fair enough
< sipa>
i would just be more confortable with having that property guaranteed
< maaku>
(actually [root 2 MERKLEBRANCHVERIFY] for anyone paying close attention)
< maaku>
it's guaranteed in that construct, or related constructs
< maaku>
I guess I'm not understanding why this is a concern -- is there something you are worried about? witness malleability?
< sipa>
i'm not worried, i'm uneasy
< sipa>
just the thought of having code be subject to other code seems so hard to reason about
< maaku>
well keep in mind that tail recursion can only add spend conditions
< sipa>
but counts towards sigops
< maaku>
nope
< sipa>
nope?
< maaku>
tail recursion policy scripts do not add to any aggregate limits, including sigops. see bip 117
< sipa>
ouch
< maaku>
not really. the worst you could possibly do with a 4MB block is 30s of signature verification, which could be dropped to 1-2s with a reasonable fix
< maaku>
so why do we maintain these limits going forward?
< sipa>
that's utterly unacceptable
< maaku>
it's less than other attack vectos we can't do much about, with the fix in place at least, and it costs 1 block's worth of fee income to perform
< gribble>
https://github.com/bitcoin/bitcoin/issues/11639 | Rewrite the interface between validation and net_processing wrt DoS by TheBlueMatt · Pull Request #11639 · bitcoin/bitcoin · GitHub
< wumpus>
I think we managed to merge two high-priority PRs this week, woohoo
< gribble>
https://github.com/bitcoin/bitcoin/issues/11639 | Rewrite the interface between validation and net_processing wrt DoS by TheBlueMatt · Pull Request #11639 · bitcoin/bitcoin · GitHub
< jnewbery>
Quick topic: can I shill for the Coredev tech days in New York, March 5th-7th?
< wumpus>
#topic coredev tech announcement
< BlueMatt>
jnewbery: I think you just did
< cfields>
woohoo!
< jnewbery>
I've sent invites to everyone I think contributes regularly to Bitcoin Core or lightning implementations. If you think you should have got an invite and haven't, plese message me
< BlueMatt>
put it on your calendar! week after fc so book your flights back via ny!
< jonasschnelli>
jnewbery: keep the location discolued
< luke-jr>
maaku: "We're still working on final confirmation for the location, but it's almost certain to be in Lower Manhattan, close to The Battery."
< jonasschnelli>
*disclosed
< luke-jr>
oops
< wumpus>
better to send the address in private
< BlueMatt>
wumpus: yes, I'm aware, I was pointing out that those three are issues which fix current master bugs ie are blocking 0.16, unlike much of the 0.16 tagged things
< maaku>
"Lower Manhatten, close to The Battery" was all I was looking for
< luke-jr>
hopefully that wasn't too much detail
< wumpus>
BlueMatt: yeah if there's things tagged 0.16 that shouldn't be, let me know
< wumpus>
BlueMatt: I'll bump them to 0.17
< BlueMatt>
"whatever makes it in", right? :p
< instagibbs>
oh meeting, hi
< wumpus>
yes, but if it shouldn't hold up the release it shouldn't really be tagged w/ specific release
< jtimon>
so after #11403 gets merged, what's the timeline for "whatever makes it in" ?
< wumpus>
luke-jr: we have 'high priority' which we already discuss every week
< wumpus>
there's no point in other priorities really
< luke-jr>
[19:11:35] <luke-jr> proposed topic: status of meeting summaries on the website
< luke-jr>
wumpus: sure, just pointing out it has its limits
< wumpus>
#topic meeting summaries
< gmaxwell>
I've been seeing highish connection counts, appears to be organic. Non-listening nodes appear to have grown a lot in the last three months.
< achow101>
luke-jr: what about the meeting summaries
< gmaxwell>
Who called this meeting?
< sipa>
?
< luke-jr>
I don't actually know what's up with meeting summaries, but the last one was 2 months ago (with a huge gap before that), and people are wondering
< achow101>
TheSadFace is the person I got to write them
< BlueMatt>
I mean there are ones sitting there ready to merge
< BlueMatt>
so...that sounds like a first step
< TheSadFace>
Hello everyone, yeah sorry about how slow it's going... After finals I plan to catch up to the present time
< achow101>
the last few weeks have been slightly busy because of exams, so meeting notes haven't really gotten written. but hopefully a bunch will be written in the next few weeks
< wumpus>
oh right I'm allowed to merge things on the bitcoincore.org site now
< wumpus>
:D
< BlueMatt>
lol after you broke the site!
< luke-jr>
ok, so basically it's taken care of, just a matter of time
< achow101>
luke-jr: yes
< TheSadFace>
luke-jr: Yeah just had a crazy last bit of the semester
< wumpus>
well the site was still working :)
< Provoostenator>
I think just posting the chat log quickly after the meeting is better than nothing.
< sipa>
TheSadFace: very happy that someone's doing that
< wumpus>
Provoostenator: the bot uploads the chat log
< gmaxwell>
luke-jr: the would it be slow in travis? it would get cached.
< cfields>
luke-jr: the idea would be to host our toolchains somewhere, so that travis just pulls them like anything else
< cfields>
right
< BlueMatt>
will this make bitcoin core the first open source project to do releases using the 1984-suggested toolchain system? =D
< sipa>
BlueMatt: we don't ship with our own CPU microcode yet :(
< cfields>
gmaxwell: also, the clang/gcc thing is kinda moot. We'll want to build them with each-other anyway, so going clang-only wouldn't make things any easier
< wumpus>
1984 toolchain system doesn't have a good ring to it
< luke-jr>
sipa: yet.
< BlueMatt>
sipa: lol, ok, fair
< BlueMatt>
wumpus: all the better to spy on you with
< wumpus>
BlueMatt: :D
< achow101>
is the goal to eventually get rid of gitian?
< gmaxwell>
BlueMatt: IIRC diverse double compilation was not suggested by KT.
< cfields>
BlueMatt: heh yes. The initial toolchain will likely take ages and ages to build. But after it's done once, updates are quick and easy
< luke-jr>
achow101: I would hope not. Gitian is handy.
< wumpus>
good to hear you're making progress cfields
< BlueMatt>
gmaxwell: oh? how did I get that confused...who suggested it?
< luke-jr>
achow101: at least not unless there's an alternative that isn't Bitcoin/Core-specific
< achow101>
luke-jr: same, although it does need a bit of fixing I think
< gmaxwell>
KT made the problem statement, I don't think DDC was suggested until david wheeler's thesis in the mid-oughts.
< BlueMatt>
ah, ok
< wumpus>
I think abstractly gitian as a launcher for deterministic containers that build is a good concept
< luke-jr>
(and also not distro-specific ;)
< jtimon>
what would be the advantageof getting rid of gitian?
< wumpus>
it does have too much overhead (runnig a full ubuntu VM inside, which upgrades everyt ime), and is pretty hard to initially set up
< cfields>
wumpus: agreed that the concept is good. It's just got lots of rough edges
< wumpus>
the advantage is not in getting rid in it, but building something better
< luke-jr>
wumpus: even making the updates persistent might be an improvement there
< achow101>
I think that setting up a new gitian environment has been slowly getting harder
< cfields>
luke-jr: with the toolchain stuff done, we won't need to do the updates anymore
< cfields>
it'll be distro-agnostic
< wumpus>
cfields: what about the windows installer stuff
< luke-jr>
building NSIS should be trivial I'd think
< wumpus>
cfields: I mean NSIS - we should probably build that too, then
< wumpus>
oh no building NSIS is not trivial :(
< luke-jr>
Gentoo has an ebuild, so just port that
< cfields>
wumpus: right, i haven't gotten that far yet. But I suspect we'll just need to suck it up and build it
< wumpus>
the problem is that it's a mix of cross compield windows code and native code, and has a sucky build system
< wumpus>
yes
< luke-jr>
correction: Gentoo *used* to have an ebuild :x
< cfields>
have no alternatives cropped up in the last ~10 years?
< wumpus>
using the one in 12.04 isn't acceptable anymore at least
< wumpus>
eh, 14.04
< luke-jr>
I have a copy of the last ebuild in my overlay, it's only 111 lines
< cfields>
luke-jr: that assumes you already have scons built and working
< wumpus>
windows has a native installer db system that would be preferable to use
< wumpus>
but porting the installer over to that would be... work
< luke-jr>
yes :p
< cfields>
well we can always take the toolchain stuff but still rely on a few bits and pieces from ubuntu until we get it all worked out
< wumpus>
sure
< luke-jr>
doesn't the toolchain stuff depend on a native autotools/make anyway?
< luke-jr>
what harm in using native scons?
< wumpus>
ah yes window's own system is msi
< cfields>
luke-jr: depends how deep we want to go
< wumpus>
thinking of it, might not be that easy to make those in cross-build
< maaku>
wumpus: building a gitian vm can be easily scriptable. I had vagrant scripts for doing this since 2013, but weren't upstreamed out of lack of interest
< wumpus>
maaku: there is a script for that now
< maaku>
maintaining a script to construct a gitian vm solves the accessibilty problem (and gets more people using gitian)
< cfields>
wumpus: msi's? IIRC gnu ld can target them, at least
< sipa>
i also have brief libsecp256k1 update
< wumpus>
contrib/gitian-build.sh
< wumpus>
cfields: ok, so if we can get someone to make a msi installer for us, we could use that instead
< achow101>
maaku: unfortunately sometimes gitian doesn't get setup even with a script
< achow101>
it might fail at some random point in the process for some unknown reason
< cfields>
wumpus: sure, something to look into
< wumpus>
the problem is that it has a lot to accomodate, because setting up the kvm/lxc is slightly different on differnt systems
< wumpus>
anyhow we'll figure it out, time for next topic
< wumpus>
#topic libsecp256k1 update (sipa)
< luke-jr>
doesn't KVM just work on all modern systems?
< maaku>
wumpus: which is why you outsource the vm maintenance to an existing cross-platform vm management tool, like vagrant, and then do lxc-gitian within that vm
< jonasschnelli>
sipa; what are the benefits?
< achow101>
sipa: that's the pippenger thing?
< sipa>
this is the low-level functionality for efficiently computing a*A + b*B + c*C + ..., with a,b,c scalars and A,B,C points
< sipa>
it is not exposed currently (except for tests and benchmarks)
< cfields>
ah, nice :)
< sipa>
but it's the basis for efficiently building signature aggregation, batch validation, bulletproofs, ...
< jonasschnelli>
nice
< wumpus>
maaku: ok, maybe discuss with cfields
< wumpus>
sipa: congrats!
< sipa>
it doesn't currently affect anything in bitcoin, but i thought about mentioning it here as it's under the bitcoin-core repo and all that
< cfields>
sipa: how do you picture that working with threading?
< cfields>
batches of batches?
< sipa>
cfields: split up the problem in N parts, run each part on one thread, and add the results together
< cfields>
(I realize that's still a ways out)
< cfields>
roger
< sipa>
there are algorithms to actually be more efficient than that, but they need high communication across threads
< sipa>
which may or may not be worth it
< sipa>
(and be much harder to do API-wise)
< wumpus>
one step at a time
< sipa>
anyway, thanks to andytoshi and nickler, and peterdettman who all contributed optimizations
< michagogo>
Sorry I'm late. IIRC a while back I made a gitian-building appliance with video instructions for recreating it from scratch
< cfields>
nice work
< sipa>
and of course gmaxwell for all imput and discussions :)
< sipa>
*input
< achow101>
sipa: so what do we need to be able to use this in Bitcoin?
< sipa>
achow101: ECDSA can't really take advantage of it in its current form
< michagogo>
(Wow, apparently that was over a year ago: https://1drv.ms/f/s!AvCguGMVwWzLgxJPeXdvaQVJ2WJq )
< cfields>
a use-case
< BlueMatt>
ok, any last-minute topics/
< BlueMatt>
?
< gmaxwell>
well I tried to talk about connection slot saturation earlier.
< sipa>
achow101: slightly modified ECDSA would permit batch validation, but there's no reason to choose that over some form of Schnorr-based signatures
< maaku>
achow101: bitcoin doesn't do multi-generator arithmetic
< maaku>
but it's useful for CT/CA
< achow101>
oh, ok. cool.
< BlueMatt>
gmaxwell: was there much to discuss? thanks for the notice, encourage people to run nodes/increase -maxconnections if possible?
< sipa>
</endtopic>
< gmaxwell>
We need to start looking into reenabling some kind of port forwarding I think.
< wumpus>
so does anyone really get a lot of connections?
< gmaxwell>
the number of non-listning nodes as increased by 50% in the last six months.
< wumpus>
I have maxconnections at 500 on one node and have never got more than 100
< achow101>
wumpus: I have 120 right now
< sipa>
i'm at 124 connections
< gmaxwell>
wumpus: when I was commenting a day ago I had confirmations of >110 on 6 nodes.
< Provoostenator>
Are we sure UDNP works?
< wumpus>
on the other node I'm happy to get more than 10
< achow101>
Provoostenator: it's currently disabled by default
< sipa>
Provoostenator: UPNP?
< gmaxwell>
you'll see less if you're in a /16 with many other nodes in it.
< jonasschnelli>
does NODE_NETWORK_LIMITED helps in this case?
< sipa>
jonasschnelli: perhaps!
< wumpus>
well the netherlands has lots of nodes so I've heard :-)
< gmaxwell>
probably not much.
< Provoostenator>
sipa: that one
< wumpus>
they're not *all* mine :p
< gmaxwell>
Running short on inbounds is the reasonable and expected outcome from not having automatic port forwarding... it's obviously not critical currently, but I think it's becoming more important.
< achow101>
gmaxwell: do you think it would be safe to re-enable UPnP?
< Provoostenator>
Maybe BitcoinQT can encourage users to use UPnP with a little nag?
< achow101>
IIRC it was disabled because of vulnerabilities
< luke-jr>
Provoostenator: better to just make it default then..
< wumpus>
any plans for bitcoin over udp? much easier to port fw there
< gmaxwell>
achow101: bleh. I dunno. that codebase sucks.
< wumpus>
yes, UPNP is not going to be enabled by default again as long as I have a say, I have no confidence in that codebase
< gmaxwell>
achow101: but there are other portforwarding protocols that we could support that are simple.
< BlueMatt>
I believe wumpus has investigated it the most, sadly :(
< luke-jr>
wumpus: what if someone ports it to another UPnP lib? (are there any good ones?)
< Provoostenator>
Without UPnP, we could at least show some instructions that they need to perform the port forwarding ritual.
< wumpus>
wasn't there a better replacement for upnp gmaxwell?
< luke-jr>
other protocols won't help with routers being UPnP..
< gmaxwell>
Yes, there are several.
< wumpus>
something that didn't rely on variable buffers and xml parsing
< jonasschnelli>
Provoostenator: a "connectable" green/read flag and some instruction is probably simple
< gmaxwell>
not as widely supported as UPNP but e.g. all apple networking appliances support the one whos name I can't remember.
< cfields>
Bonjour?
< gmaxwell>
where the protocol is just a trivial struct sent over usp.
< jonasschnelli>
Bonjour is mDNS (that is different)
< sipa>
DLNA?
< gmaxwell>
I'm thinking of NAT-PMP
< Provoostenator>
As well as a P2P message like "please try to connect to me", so it's easier to see if the port is open?
< luke-jr>
does anyone actually use Apple networking appliances? :/
< sipa>
ah yes, that
< wumpus>
NAT-PMP is quite common these days, not only in apple
< gmaxwell>
luke-jr: yes, though I'm sure they're not the most popular... NAT-PMP has support beyond apple of course.
< Provoostenator>
The current instruction says "go to 21 and enter your IP there"
< gmaxwell>
I just mentioned apple as a concrete example that it is widely supported.
< luke-jr>
would be nice to find a quality library that can do both
< sipa>
Provoostenator: ?
< gmaxwell>
Provoostenator: wtf? what "the current instructions" say that?
< gmaxwell>
luke-jr: there is not really a reason for a library for nat-pmp. You send a dozen byte packet over UDP.
< wumpus>
I'd be ok with NAT-PMP on by default
< gmaxwell>
And if you want to know your IP you listen for a similarly structured dozen by UDP reply.
< achow101>
I think that's what you were looking for
< cfields>
wumpus: yikes!
< sipa>
maaku: multi-multiplication is useful even for cryptographic systems that don't use multiple generators
< Provoostenator>
achow101 yes
< Provoostenator>
So now they just need some browser fingerprinting et voila.
< sipa>
maaku: e.g. Schnorr batch validation
< Provoostenator>
Next time you go to a shop, they'll offer you a bitcoin payment option. Super convenient!
< jonasschnelli>
sipa: thanks...
< jonasschnelli>
wumpus: oh.. due to UPnP?
< achow101>
wumpus: how so?
< wumpus>
cfields: was a combination of a heartbleed-like leak (to get the ASLR addresses) and the known vulnerability, both are patched now in any case
< gmaxwell>
sipa: in fact it would be perfectly useful in bitcoin for batch validation if but for the ecda reduction of R.x mod P and the lack of R's sign.
< sipa>
jonasschnelli: so take the top 64 bits of the txid, and convert it to a double... that should increase at a constant rate
< maaku>
luke-jr: there are alternatives to vagrant, but I think you missed the point. use a vagrant-like tool to make a linux vm (centos, ubuntu, I don't care). on THAT system make the gitian VM using the usual scripts
< gmaxwell>
do not just cast the bits to a double, as that could constract signaling nans and other baddness.
< wumpus>
achow101: there was a buffer overflow bug in the miniupnp library at some point, don't have the CVE at hand
< sipa>
gmaxwell: it also wouldn't work
< gmaxwell>
There was more than one. We'd previously noted that the code smelled.
< jonasschnelli>
sipa: int percentageDone = (int)(high * 100.0 / 65536.0 + 0.5) (can you explain why 65536.0 + 0.5?)
< cfields>
maaku: that's pretty much Gitian's utility though. I'd argue that if you need a vm builder to run the vm builder, one of them needs some love :)
< sipa>
jonasschnelli: i assume that's just looking at the top 16 bits?
< sipa>
jonasschnelli: 16 bits have 65536 possibilities
< luke-jr>
Gentoo let the CVE sit until just a few weeks ago. My workaround was to depend on the fixed version explicitly. :/
< sipa>
0.5 is for rounding
< jonasschnelli>
sipa: okay. I see
< luke-jr>
anyhow, bbl
< maaku>
cfields: how is that gitian's utility? I'm not sure why it should be our business to maintain the gitian setup procedure on every development environment out there
< wumpus>
the vuln was: TALOS-2015-0035 (CVE-2015-6031)
< wumpus>
had to badger ubuntu day in day out to put up an updated libupnpc library
< sipa>
badger badger badger
< wumpus>
bitcoin wasn't the only affected program, but one of the worst, because it had the upnp structures on the stack
< achow101>
what if we just removed UPnP entirely? It's not like people actually seem to be using it.
< cfields>
maaku: right, Gitian's tasked with abstracting that away. But in reality it only behaves on Linux environments that look as it expects. If something like vagrant/docker/etc. does it better, it may make sense to outsource some of it.
< wumpus>
sure, but please implement an alternative first
< wumpus>
NAT-PMP is great but someone has to do it :)
< gmaxwell>
achow101: I think we need at least some kind of internal traversal, perhaps NAT-PMP is enough.
< wumpus>
I'll create an issue for it
< maaku>
cfields: gitian is not designed to work on windows or mac os x or bsd or even non-mainstream linux environments. it expects a standard linux environment. so give it one: use gitian-lxc inside of a vm or container made with more traditional cross-platform support
< maaku>
i'm not saying replace any aspect of gitian. these are totally different problems, and gitian only tackles one of them
< cfields>
maaku: it is intended to work on osx
< sipa>
looking at RFC 6886, NAT-PNP looks absolutely trivial
< gmaxwell>
I think if we didn't care about learning our external IP (we do) it would be a dozen lines of code to support.
< sipa>
oh, you need to know your gateway address
< sipa>
that looks slightly nontrivial, especially across platforms
< wumpus>
lol yes I also had an IPv6 address in the past, but no longer, seems IPv6 support by the big providers in the Netherlands has been postponed indefinitely, they announce and postpone every time
< wumpus>
one provider acquired another provider, cancelling their ongoing ipv6 rollout
< gmaxwell>
lol
< BlueMatt>
lol, monopolistic practices to prevent ipv6 rollout?
< sipa>
apparently my home desktop, behind a NAT, has a global IPv6 address that works :o
< BlueMatt>
my home internet does as well
< gmaxwell>
SURPRISE.
< gmaxwell>
just when you thought you didn't have to worry much about security on random hosts in your home...
< sipa>
it's firewalled of course
< BlueMatt>
InternetOfShitTakesRevenge
< BlueMatt>
heh, my default garbage isnt
< cfields>
maaku: I see
< sipa>
gmaxwell: my machine is reachable in other ways too - i'm just surpised that ISP, router, OS, ... all support IPv6 without any configuration
< wumpus>
yes, hardware and software support for ipv6 support is very good these days
< wumpus>
that's no longer the bottleneck
< Provoostenator>
Wumpus: seems Ziggo is experimenting with IPv6 again though.
< Provoostenator>
Top hit on Google is people trying to downgrade to IPv4 because [something bad].
< wumpus>
Provoostenator: you mean, to IPv6 only?
< Provoostenator>
Yes, I'll just email them to see if that's possible. Then I'll find out how it works and what breaks.
< wumpus>
I'd not be happy with that either, would just like the choice to use IPv6 as well
< BlueMatt>
wumpus: I mean, hey, from a network-management perspective I do get why one might want to drag their feet on ipv6 rollout...the possibility that your users end up joining a DDoS attack because you host garbage IoT devices all over the place that get pop'ed via IPv6 is....much too high. ofc its something they need to deal with *either way*, but...ehh
< wumpus>
BlueMatt: yes, having all devices reachable for incoming connections from outside by default is one of the things I like less about IPv6, although that's not the fault of the protocol but of router's default firewall configuration
< BlueMatt>
indeed
< Provoostenator>
Can't a standard cable modem just block all ports by default?
< BlueMatt>
should, probably, but wont in my experience
< Provoostenator>
So the user just needs to open them, but not forward. Although that's probably still too much of a barrier.
< wumpus>
blocking incoming TCP SYN packets to the entire range would be a start
< wumpus>
but I don't think any routers do that by default
< gmaxwell>
internet of terror
< wumpus>
then again, that would defeat its entire purpose for things like bitcoin and P2P programs, again requiring the users to manually change a setting
< BlueMatt>
gmaxwell: internet of shit!
< BlueMatt>
wumpus: yea, but upnp and auto-forwarding is easy to get right and implement!
< wumpus>
BlueMatt: not quite, though well at least for ipv4 there are protocols for that, I'm not sure what a ipv6 'please allow this port' request would look like
< BlueMatt>
wumpus: I was joking......
< sipa>
wumpus: PCP supports IPv6 IIRC
< wumpus>
in principle it would be the application requesting global scope for a port instead of LAN scope, of course, it's dangerous to allow applications to make those decisions in general
< sipa>
well UPnP/NAT-PMP aren't designed to bypass firewalls, they're to inform the router of port mappings
< sipa>
in IPv6 those should just not be needed anymore
< wumpus>
yeah, 'should'
< sipa>
unfortunately, many people treat the lack of a mapping as a substitute for a firewall, where of course those protocols effectively become a way for application to bypass security
< wumpus>
right
< wumpus>
another problem is that applications and operating systems open all kinds of ports, expecting them to be only open to a LAN instead of to the whole world, whereas most users don't really configure a firewall
< xames>
not possible to use port 80 like skype...
< wumpus>
NAT made developers and users lazy in that way
< xames>
like teamviewer
< bitcoin-git>
[bitcoin] jonasschnelli opened pull request #11903: [trivial] Add required package dependencies for depends cross compilation (master...2017/12/depends_pkg) https://github.com/bitcoin/bitcoin/pull/11903
< meshcollider>
Ah I missed the meeting oops
< BlueMatt>
meshcollider: tsk tsk, no more getting your prs reviewed this week, then :p
< meshcollider>
BlueMatt: how is that different from every other week though ;)
< BlueMatt>
meshcollider: heh, try the high priority review queue, it at least usually gets you one or two reviews in a week :p
< MarcoFalke>
re: jonasschnelli: [OT] what is the fastest way to amend commit the current changes into a non HEAD commit?
< wumpus>
huh autosquash is nice, so it will automatically mark commits whose message start with squash! as squash and same with fixup!, didn't know about that one
< BlueMatt>
heh, cool
< meshcollider>
You can also turn on autosquash by default with git config --global rebase.autosquash true
< meshcollider>
so you don't have to use --autosquash every time
< BlueMatt>
I really want a fixup! "Title of Commit to Squash Into"
< meshcollider>
that is what it does?
< BlueMatt>
oh, heh, nice
< meshcollider>
;)
< MarcoFalke>
promag: re ParseInt64
< MarcoFalke>
Imo it should be done at init and then throw an error
< meshcollider>
MarcoFalke: I'm going to work on some stronger argument checking as soon as I can
< MarcoFalke>
When your node is running for a day and you sendtoaddress, it is too late to crash on "Wrong tx fee set on command line"
< MarcoFalke>
meshcollider: Cool
< wumpus>
that's what I also said, we should have some options mechanism that parses all options at init time,
< MarcoFalke>
wumpus: Jup, like python :)
< wumpus>
it's crazy to parse options every time in a loop anyway
< wumpus>
or every time a RPC command is called
< MarcoFalke>
It is still good to have the GetArg in place, but internally they might use some sort of caching
< wumpus>
just call the getarg in init
< meshcollider>
GetArg just looks up in mapArgs
< wumpus>
then assign to a global
< wumpus>
or have some automatic system to do that
< wumpus>
caching is also unnecessary, it can just be a variable
< wumpus>
no need for any lookup at all
< meshcollider>
wumpus all args are parsed in ArgsManager::ParseParameters
< MarcoFalke>
wumpus: You'd have to sprikle the code with locks in case we allow args to change at run time, no?
< wumpus>
we don't allow that
< MarcoFalke>
There is a pull to do that, though
< meshcollider>
which one?
< wumpus>
well yes in that case you'd have to protect the values with a lock
< wumpus>
it's tsill better than parsing every time
< MarcoFalke>
wumpus: My style preferense would be to put the lock in the GetArg
< wumpus>
but most settings don't really make sense to change at run time
< MarcoFalke>
The getarg can return the variable
< wumpus>
using strings to refer to variables just doesn't make sense when you know what you're accessing
< MarcoFalke>
jup
< wumpus>
there's this whole hash lookup every time that makes no sense
< jnewbery>
wumpus: Perhaps #11415 is ready for merge?
< aj>
wumpus: cool, i'll review the code, add some tests and maybe even docs then
< aj>
wumpus: modules registering their own options/globals would fit well with declaring some options as needing to be specified per network
< wumpus>
aj: yes, that could be one of the registration parameters
< aj>
wumpus: maybe help text could be one of the params too?
< wumpus>
aj: absolutely, yes
< meshcollider>
aj: he already mentioned that ;)
< MarcoFalke>
falls under "documentation"
< meshcollider>
Ok I'll get started then
< meshcollider>
aj's PR should be merged first but I won't base mine on top of it until its been more thoroughly reviewed / less likely to drastically change
< wumpus>
meshcollider: the idea would be that everything about the option is specified at registration, so that it's in one place and that's the module where it gets used
< danklasson>
I don't know if you guys know. But supposedly this guy set up this fund where you can apply for 1 million dollars donation. Would you guys say that could be used to hire a bunch of devs working on several projects related to Bitcoin, such as segwit supported wallets and LN. What's your thoughts regarding this? https://pineapplefund.org/
< meshcollider>
danklasson: I don't think bitcoin core counts as a charity ;)
< danklasson>
meshcollider: I know but I'm guessing they would be willing to make an exception
< wumpus>
yeah, I've had the question before 'how can I pay to help lightning development', I had no idea
< danklasson>
wumpus: I read that few people are working on LN. If we had a bunch of money we could hire a bunch of devs to work on it. I bet I can get a bunch of guys on board to get the ball rolling
< wumpus>
in open source it's individual developers working on things, there's no one dealing out work items, and allocating resources
< ryanofsky>
imo, gflags has a good model for registering options. lets you declare options where used, creates variables to avoid lookups, and combines with documentation: https://gflags.github.io/gflags/#define
< danklasson>
wumpus: not necessarily. a lot of companies contribute to open source too
< wumpus>
sure you could ask individual developers for donation address
< danklasson>
i was thinking something more like setting up a foundation that hires people and allocates them to different projects
< wumpus>
heh that didn't go too well with the bitcoin foundation... anyhow off topic here
< danklasson>
is there a better channel to discuss this?
< wumpus>
#bitcoin
< Deacyde>
/join #bitcoin
< Deacyde>
:) \
< danklasson>
ok, thanks
< wumpus>
ryanofsky: agree, looks decent
< phantomcircuit>
i've got a node in blocks only mode with a peer that claims to be 0.15.1 sending transactions in violation of the protocol
< phantomcircuit>
lol i also have a peer claiming to be 0.15.0.1
< BlueMatt>
yea, thats been a common complaint, dont know that anyone has reproduced it with their own nodes, but people seem to pop up claiming that every now and again
< BlueMatt>
I'm betting its random garbage nodes hacked to relay lots of shit, cause several people have gone looking for a bug there and failed to find one, afair
< BlueMatt>
err...not *common*, but it gets mentioned once or twice every now and again
< BlueMatt>
we should probably just ban peers for that
< phantomcircuit>
possibly nodes that have whitelisted 0.0.0.0/0