< bitcoin-git> [bitcoin] Empact opened pull request #16123: Return error information on descriptor parse error (master...descriptor-parse-error) https://github.com/bitcoin/bitcoin/pull/16123
< bitcoin-git> [bitcoin] Empact closed pull request #16123: Return error information on descriptor parse error (master...descriptor-parse-error) https://github.com/bitcoin/bitcoin/pull/16123
< bitcoin-git> [bitcoin] practicalswift opened pull request #16124: tests: Limit Python linting to files in the repo (master...lint-inside-repo) https://github.com/bitcoin/bitcoin/pull/16124
< stevenroose> Any idea what it could mean when Travis gives this error for the feature_notification.py test?
< stevenroose> node0 2019-05-29T20:16:15.512410Z runCommand error: system(echo > /tmp/test_runner_₿_🏃_20190529_201410/feature_notifications_45/blocknotify/03d5fa58ac73a6b5e2d1baf5849e0142355758746314509e3eff69683ffe9872) returned -1
< stevenroose> Seems like it can't write to the file somehow. Or the directory doesn't exist or something.
< jnewbery> #proposedmeetingtopic - has travis got worse or have we just added so many builds to our job that it times out?
< jnewbery> moneyball: ^
< ryanofsky> MarcoFalke, could drahtbot just automatically restart travis jobs that fail with "Error! Initial build successful, but not enough time remains to run later build stages and tests."?
< wumpus> ryanofsky: I don't think drahtbot queries/scans the travis logs right now
< wumpus> but it's a good topic I think, it definitely seems that travis is overloaded, I'm not sure what changed why it's like that now
< fanquake> Are we at the free limit/tier with Travis? I know the idea of adding "another" CI wasn't liked too much, for risk of increasing the size of the slot machine, but if we need more capacity maybe we could look at Circle CI, and be able to add some BSDs while we are at it ?
< wumpus> right, I'd be for adding a CI only if it improves reliability by lowering the load on travis, the reason I'm against it otherwise is exactly that, even more different ways to fail randomly and different buttons to press to respin
< wumpus> #startmeeting
< lightningbot> Meeting started Thu May 30 19:00:15 2019 UTC. The chair is wumpus. Information about MeetBot at http://wiki.debian.org/MeetBot.
< lightningbot> Useful Commands: #action #agreed #help #info #idea #link #topic.
< moneyball> hi!
< instagibbs> hi
< jamesob> hi
< meshcollider> hi
< promag> hi
< jnewbery> hi
< jarthur> 'lo
< fanquake> hi
< wumpus> #bitcoin-core-dev Meeting: wumpus sipa gmaxwell jonasschnelli morcos luke-jr sdaftuar jtimon cfields petertodd kanzure bluematt instagibbs phantomcircuit codeshark michagogo marcofalke paveljanik NicolasDorier jl2012 achow101 meshcollider jnewbery maaku fanquake promag provoostenator aj Chris_Stewart_5 dongcarl gwillen jamesob ken281221 ryanofsky gleb moneyball kvaciral
< wumpus> any proposed topics? (in addition to moneyball's) ?
< fanquake> Probably any suggestions for short talks at Core Dev? Is there a list of anything like that already?
< instagibbs> kanzure, ^
< fanquake> If so that's fine, let's discuss the other topics.
< moneyball> fanquake be sure to provide kanzure with any topics you'd like to discuss at CoreDev!
< sipa> about to get to the airport, i think we'll have better meeting time in amsterdam
< promag> jarthur:
< wumpus> #topic High priority for review
< promag> (ops)
< wumpus> 5 things on the list now https://github.com/bitcoin/bitcoin/projects/8
< promag> sorry, what is the date for 0.18.1?
< wumpus> anything to add/remove ?
< wumpus> promag: whaa? is 0.18.1 planned at all?
< wumpus> I don't think anything made it necessary yet
< achow101> hi
< achow101> #15450 for hi prio
< gribble> https://github.com/bitcoin/bitcoin/issues/15450 | [GUI] Create wallet menu option by achow101 · Pull Request #15450 · bitcoin/bitcoin · GitHub
< promag> but nevermind, I got it's too soon
< fanquake> added
< wumpus> absent any really important bug fix I think it's too soon, yes, we don't tend to do a lot of minor releases usually
< fanquake> There's a call for some more review on #15976, has 4 utACKs at the moment (already high-prio).
< gribble> https://github.com/bitcoin/bitcoin/issues/15976 | refactor: move methods under CChainState (pt. 1) by jamesob · Pull Request #15976 · bitcoin/bitcoin · GitHub
< wumpus> okay!
< wumpus> #topic Has travis got worse? (jnewbery)
< wumpus> "has travis got worse or have we just added so many builds to our job that it times out?"
< wumpus> I've wondered this, too, travis has been more unreliable on (PRs at least) than it used to be
< jnewbery> In the last couple of months, *a lot* of travis builds time out
< wumpus> while I don't notice this for local tests
< jamesob> hasn't seemed any worse to me recently, though we've had to rekick certain jobs for months
< jnewbery> I don't know if our builds got slower, travis got slower, or we just added to many jobs for travis to handle
< achow101> a lot of things have been added recently, maybe it's too much for them to handle?
< wumpus> at least we should be careful with making the tests even longer now
< fanquake> Also a lot of depends related PRs timing out recently, but not much that can be done about that.
< instagibbs> There is an element of how Travis is feeling that day
< instagibbs> lots of variance in build times
< wumpus> right, it's very possible that it's not our fault entirely though and it's simply the infrastructure becoming worse
< jnewbery> There are currently 10 different test stages. I know it used to be 6 or 7
< wumpus> I haven't noticed the tests nor builds becoming noticably slower locally
< wumpus> jnewbery: hmm might be time to evaluate whether they're really all contributing something useful
< instagibbs> in elements land we've been having weird issues too that might reflect travis being overly taxed, hard to say
< instagibbs> failure to write to files, that kind of stuff
< promag> best case I usually see is around 20min (for longest job 8)
< jnewbery> I know it runs those test stages in parallel
< wumpus> yes weird stuff happens but I don't think we have that often, it's mostly just timeouts
< luke-jr> instagibbs: does the Elements Travis have caching enabled?
< ryanofsky> are people seeing travis errors other than "Error! Initial build successful..."? This is the only travis error i see and restarting fixes it 100% of the time
< jnewbery> ryanofsky: yes, that's the error
< luke-jr> ryanofsky: I've seen cases where restarting *doesn't* fix it
< instagibbs> ryanofsky, that's when dpeends takes "too long"
< promag> ryanofsky: sometimes I see others and I leave a comment in the PR (before restarting) or create an issue
< instagibbs> and it early exits
< luke-jr> ryanofsky: but they mysteriously resolved before I could troubleshoot :/
< instagibbs> luke-jr, I believe so, the restarting thing fixes that issue
< jnewbery> The longest running test stage is "sanitizers: address/leak (ASan + LSan) + undefined (UBSan) + integer". I wonder if the same hardware is shared between different test stages and whatever is running at the same time as that one might time out
< wumpus> it *should* be fixed by restarting, that's the point of that message, it's an ugly hack though
< jnewbery> yes, travis is supposed to save developer time, not add an extra step to opening a PR!
< luke-jr> jnewbery: on that note, it's annoying that AppVeyor doesn't use standard build tools, and requires duplicating changes for it
< meshcollider> Is it possible for a new drahtbot feature to auto-restart builds with that error?
< achow101> apparently travis switched from docker containers on ec2 to vms on gce late last year, maybe that's related?
< jnewbery> has anyone raised this with travis? We have a paid account, right? Can we try to get support?
< promag> does it run multiple "build depends" with the same conf if needed? sounds unnecessary?
< luke-jr> jnewbery: Travis support is pretty useless in my experience :/
< luke-jr> jnewbery: I expect we'd at the very least need a very specific concern
< jnewbery> luke-jr: support is pretty useless in my experience :/
< wumpus> yes, filing an issue 'it is slow' probably won't do much, I'm sure they get that a lot
< jamesob> circleci (https://circleci.com/) execution is very good in my experience but I am definitely not volunteering to migrate our .travis.yml :)
< wumpus> migrating to another CI would definitely be an option if it's better
< meshcollider> jnewbery: Travis is free for open source projects, we don't pay
< wumpus> (and then I really mean migrating not adding another one)
< luke-jr> meshcollider: we do have a paid account, though
< meshcollider> Oh?
< luke-jr> meshcollider: afaik the Bitcoin Foundation set it up way back when
< jnewbery> well, what's the issue exactly? There's some specific job timout on travis, and so we cancel the build before that timeout to cache whatver has been built already? Can we ask them to increase the job timeout for us?
< jnewbery> I believe we have a paid account so we get more parallel builds
< jnewbery> because we were getting a backlog of PR builds a couple of years ago
< luke-jr> jnewbery: it used to also be required for caches (though I'm not sure if they expanded that to free accounts maybe)
< jamesob> meshcollider: Chaincode starting kicking in for Travis a year and change ago
< jamesob> *started
< wumpus> but it didn't use to run into the timeout so often, so it's become slower, that's really the issue, not the timeout itself; increasing the timeout would work, up to a point, but doing that indefinitely makes the testing slower and slower which isn't good either
< wumpus> are we somehow doing more travis builds than before? e.g. how often does drahtbot re-kick builds?
< jnewbery> yeah, someone needs to investigate where the slowness comes from. Is there an API to pull down all the build results from Travis for a project so we can at least get a sense for how often things are failing?
< wumpus> yes, travis has a quite extensive API
< luke-jr> jnewbery: there's certainly an API
< jnewbery> One issue is that restarting a build makes the logs for the failed build unavailable (at least on the website)
< luke-jr> (including one that lets you SSH into the VM)
< wumpus> jnewbery: I don't know if that's the case for the API, or the new log will get a new id
< luke-jr> wumpus: pretty sure it overwrites the existing log
< jamesob> I think there's some amdahl's law at work here though - is speeding up travis really going to make the process materially faster? we're pretty review-bound
< wumpus> ah yes I'm sure too--this is the URL: https://api.travis-ci.org/v3/job/$1/log.txt it's per job, and it will have the same job id
< wumpus> jamesob: *making it fail less* will lower frustration
< jamesob> wumpus: yeah, agree spurious failures are annoying
< jnewbery> yeah, it's not about how long it takes, it's that if you open a PR, most of the time you'll need to log back in an hour later or more to rekick travis
< wumpus> it's really frustrating if tests fail randomly, if that happens too often people take them less seriously which means actual problems might be ignored
< luke-jr> maybe a flag to tell Travis "updated caches, please restart"?
< * luke-jr> ponders if the Travis job can call the API to restart itself
< jnewbery> wumpus: exactly that - inconsistently failing builds/tests lower confidence in the product/tests and hide real problems
< luke-jr> wumpus: well, false failures on tests is another issue IMO
< promag> luke-jr: it needs an auth token, not sure if there's a way to set secrets in travis
< jnewbery> luke-jr: that'd be nice, or to get drahtbot to do it, but this is just a variation on the increase timeouts and kick the can down the road, no?
< luke-jr> promag: there is, but it may be a dead idea since the job would still be running :/
< luke-jr> jnewbery: dunno, it might be nice to restart after cache updates regardless
< luke-jr> just to make it more deterministic for the actual tests
< wumpus> #topic GitHub feedback issue (moneyball)
< fanquake> Should we move onto another topic? Seems like the conclusion here is to go away adn investigate Travis/bot/other CI options and discuss in AMS?
< moneyball> hi!
< moneyball> so we have this issue https://github.com/bitcoin/bitcoin/issues/15847
< wumpus> ""please review GitHub feedback issue and vote with thumbs up on issues you think you make the top 10 https://github.com/bitcoin/bitcoin/issues/15847""
< moneyball> with many responses - thank you!
< moneyball> i would like to suggest folks take a look at the issue and "vote" for the ideas that resonate with you by doing a thumbs up
< fanquake> moneyball is there any timeline for when that meetup might happen?
< moneyball> this will help us figure out a top 10 for the CEO lunch, which is now scheduled for june 21, and sipa, cfields, and i plan to attend
< moneyball> we can also review this next week at CoreDev if there is interest
< moneyball> that's all i had to say, unless there are any other questions right now
< sipa> i think it's most useful to talk about issues with are mostly unique to our usage of github
< wumpus> thank you, yes this was expected to be a fairly short topic
< sipa> as i expect that there are a lot of grievances that are mostly shared between projects
< luke-jr> hmm, it'd be nice to use GitHub's code review stuff but also verify the diff against a local git instance somehow
< sipa> probably best to discuss on the issue tracker
< luke-jr> sometimes I review on GitHub, and therefore leave a commithash out of ACKs because I don't know for sure the commithash is what I reviewed
< moneyball> luke-jr please make sure to add your ideas to the github issue
< luke-jr> right, but I don't know if this is even a viable issue to address :p
< wumpus> ok, any other topics?
< wumpus> thanks everyone, see (some of) you next week in Amsterdam
< wumpus> #endmeeting
< lightningbot> Meeting ended Thu May 30 19:40:32 2019 UTC. Information about MeetBot at http://wiki.debian.org/MeetBot . (v 0.1.4)
< luke-jr> I'm probably going to have to miss Amsterdam after all, FWIW
< wumpus> oh :(
< instagibbs> CScript(byte_vector) vs CScript(byte_vector.begin(), byte_vector.end()) whyyyy
< gwillen> I suspect because certain overloads didn't used to exist or something
< instagibbs> I don't think anyone actually tries using the former?
< instagibbs> intentionally*
< kanzure> also, i'm still seeking topics for amsterdam meeting
< instagibbs> just my nth time running into it, in case my annoyance is justified
< kanzure> and also, does anyone want to take over luke-jr things and be a substitute luke-jr?
< gwillen> instagibbs: wait I'm confused now, which of these do you consider preferable
< gwillen> I can't tell why you'd ever want to use the longer form
< instagibbs> gwillen, the former gives you a script with the first byte being the size of the vector, IIRC
< instagibbs> the latter gives you what you'd expect :D
< gwillen> oh, that seems like a horrifying bug that should be fixed
< instagibbs> CScript is a prevectoryadayada
< gwillen> it's not even clear to me how that would happen
< sipa> gwillen: CScript(vector) is the same as CScript() << vector
< sipa> it's the script that consists of pushing vector onto the stack
< sipa> the confusion is because CScript (used to be) a subclass of vector
< sipa> so CScript(script) would be the script that pushes script onto the stack
< gwillen> mmm, that seems really confusing and errorprone
< sipa> gwillen: yes
< sipa> i've tried to touch the CScript API once and almost introduced a consensus bug
< gwillen> ahhhhhhh, heh. I see.
< gwillen> are those constructors ever used intentionally, do you know?
< gwillen> (specifically the ones I would describe as ambiguous, and used directly and not via << or the like)
< sipa> if they're actually unused we should just get rid of them
< sipa> feel free to try and remove them, but i suspect we'd have caught them already
< gwillen> yeah, understood, I am going to go poke around but you've made me afraid to touch it much now ;-)
< sipa> i also don't want to imply we're forever stuck with this confusing convention; just that yoi really need to be careful about it
< gwillen> yeah, that makes total sense and I appreciate the clarification
< * sipa> ✈️
< jarthur> Have fun out there, everyone.