Everything
php
Learn Community Releases The PHP Project
Home » Project » Newsgroups » php.internals » AppVeyor timeouts -- Migrate to Azure Pipelines? »

AppVeyor timeouts -- Migrate to Azure Pipelines?

php.internals

Nikita Popov

7 years ago
Hi internals, Since some time ago (not sure how long) our AppVeyor builds have been timing out more often than not, to the point that I've started ignoring failing AppVeyor builds entirely. I just learned about the Azure Pipelines ( https://azure.microsoft.com/en-us/services/devops/pipelines/) offering, which offers open source projects 10 parallel builds with unlimited minutes. Assuming there's no other catch here, it might be worthwhile to migrate our Windows CI jobs to Azure Pipelines. Independently of that, it would be great if someone on a Windows environment could look into the issue of AppVeyor timeouts. While PHP is known to be slower on Windows, I find it somewhat odd that our AppVeyor builds routinely take more than 60 minutes, while Travis builds usually finish in 20 minutes. Based on build logs, running tests on AppVeyor often takes ~45 minutes, while it takes ~15 minutes on Travis. Might there be a legitimate performance issue here, that was caused by a recent-ish (last month or so) change? Regards, Nikita

Christoph Becker

7 years ago
On 02.02.2019 at 13:18, Nikita Popov wrote:
> Since some time ago (not sure how long) our AppVeyor builds have been > timing out more often than not, to the point that I've started ignoring > failing AppVeyor builds entirely.
Indeed, this is particularly annoying for RMs.
> I just learned about the Azure Pipelines ( > https://azure.microsoft.com/en-us/services/devops/pipelines/) offering, > which offers open source projects 10 parallel builds with unlimited > minutes. Assuming there's no other catch here, it might be worthwhile to > migrate our Windows CI jobs to Azure Pipelines.
An alternative to consider is using Travis for Windows CI[1], too.
> Independently of that, it would be great if someone on a Windows > environment could look into the issue of AppVeyor timeouts. While PHP is > known to be slower on Windows, I find it somewhat odd that our AppVeyor > builds routinely take more than 60 minutes, while Travis builds usually > finish in 20 minutes. Based on build logs, running tests on AppVeyor often > takes ~45 minutes, while it takes ~15 minutes on Travis. Might there be a > legitimate performance issue here, that was caused by a recent-ish (last > month or so) change?
To my knowledge, Appveyor “always” was way slower than Travis, and maybe we're hitting the timeouts only now, since the size of the test suite has increased. Since our Appveyor CI runs with slow tests enabled, it might be worth a try to disable these. I've just submitted PR #3789[2] to see whether that would make a big difference. [1] <https://blog.travis-ci.com/2018-10-11-windows-early-release> [2] <https://github.com/php/php-src/pull/3789>
-- Christoph M. Becker

Nikita Popov

7 years ago
On Sat, Feb 2, 2019 at 2:06 PM Christoph M. Becker <cmbecker69@gmx.de> wrote:
> On 02.02.2019 at 13:18, Nikita Popov wrote: > > > Since some time ago (not sure how long) our AppVeyor builds have been > > timing out more often than not, to the point that I've started ignoring > > failing AppVeyor builds entirely. > > Indeed, this is particularly annoying for RMs. > > > I just learned about the Azure Pipelines ( > > https://azure.microsoft.com/en-us/services/devops/pipelines/) offering, > > which offers open source projects 10 parallel builds with unlimited > > minutes. Assuming there's no other catch here, it might be worthwhile to > > migrate our Windows CI jobs to Azure Pipelines. > > An alternative to consider is using Travis for Windows CI[1], too. >
It's an option, but I think we're generally better off using multiple CI platforms to increase the number of parallel builds we get. Next to the builds we already have, I think it would be very good to also have a macos builder (this is supported by both Travis and Azure Pipelines, but needs someone familiar with the platform to set things up) and a 32-bit builder (probably a -m32 build on x86_64 Linux). That would bring us to a total of six builds, which would probably make the Travis queue somewhat slow if we used it exclusively. Nikita

Christoph Becker

7 years ago
On 02.02.2019 at 14:31, Nikita Popov wrote:
> On Sat, Feb 2, 2019 at 2:06 PM Christoph M. Becker <cmbecker69@gmx.de> > wrote: > >> On 02.02.2019 at 13:18, Nikita Popov wrote: >> >>> I just learned about the Azure Pipelines ( >>> https://azure.microsoft.com/en-us/services/devops/pipelines/) offering, >>> which offers open source projects 10 parallel builds with unlimited >>> minutes. Assuming there's no other catch here, it might be worthwhile to >>> migrate our Windows CI jobs to Azure Pipelines. >> >> An alternative to consider is using Travis for Windows CI[1], too. > > It's an option, but I think we're generally better off using multiple CI > platforms to increase the number of parallel builds we get. > > Next to the builds we already have, I think it would be very good to also > have a macos builder (this is supported by both Travis and Azure Pipelines, > but needs someone familiar with the platform to set things up) and a 32-bit > builder (probably a -m32 build on x86_64 Linux). That would bring us to a > total of six builds, which would probably make the Travis queue somewhat > slow if we used it exclusively.
Ah, makes perfect sense (a FreeBSD CI would be nice as well). Regarding the execution time of the test suite, it might be sensible to also have a look at PR #2822[1] and/or PFTT2[2]. [1] <https://github.com/php/php-src/pull/2822> [2] <https://git.php.net/?p=pftt2.git>
-- Christoph M. Becker

Stas Malyshev

7 years ago
Hi!
> I just learned about the Azure Pipelines ( > https://azure.microsoft.com/en-us/services/devops/pipelines/) offering, > which offers open source projects 10 parallel builds with unlimited > minutes. Assuming there's no other catch here, it might be worthwhile to > migrate our Windows CI jobs to Azure Pipelines.
Maybe also get Linux and macOS jobs there? We currently don't have CI for macOS at all, AFAIK, and having different Linux environment probably won't hurt either.
-- Stas Malyshev smalyshev@gmail.com

Joe Watkins

7 years ago
Hi Nikita, I can't be helpful here at all, because (shamefully) allergic to and bad at windows, and no experience at all with azure. But I would really like our CI to be restored to a thing that's useful, instead of running two jobs on travis, and more or less nothing on av (most of the time). I remember having CI on travis for many configurations, including the ones we actually develop with. Now we have two configurations and they don't include no-zts-dbg which is what most internal devs are actually running. Anything that restores anything like that former configuration, I'm a huge +1 on ... Anything that means we can run a useful windows CI, even if we can't move any of the jobs that should be on travis over to Azure, I'm still a +1 on ... Anything that means we have a mac build, the same +1 ... Cheers Joe On Sat, 2 Feb 2019 at 19:38, Stanislav Malyshev <smalyshev@gmail.com> wrote: