Opened 12 months ago

Last modified 8 weeks ago

#5574 new enhancement

Travis CI for MediaGoblin

Reported by: anongoblin Owned by:
Priority: minor Milestone:
Component: programming Keywords:
Cc: Parent Tickets:

Description

[This is Michael Lynch (mtlynch.io). Boris suggested I post here under the anon account as new registrations on the issue tracker seem to be down and the mailing list is not accepting new messages]

I'm interested in contributing to MediaGoblin.

Its test suite is very impressive, so I tried to run the tests and ran into some issues with finding the correct dependencies. One good way to solve this would be to have a continuous integration (CI) service that establishes a "known-good" configuration for running tests.

There was discussion a few years ago about setting up CI, but it doesn't look like it ever went through:

https://wiki.mediagoblin.org/20160206_Log

I spent a few hours setting up MediaGoblin on Travis and got it to successfully execute the tests, but it skips 12 out of 183:

https://travis-ci.org/mtlynch/mediagoblin/builds/449333107

If I force the tests to run, the skipped tests fail:

https://travis-ci.org/mtlynch/mediagoblin/builds/449339819

A few questions:

  • Are all tests currently in a good state? (i.e. should I expect to see all 183 tests pass?)
  • If so, what dependencies am I missing? I tried installing every apt package in the HackingHowTo but the tests still fail.
  • Is there interest in setting up official CI for MediaGoblin? There are many free options available, but it would require the project to move (or at least officially mirror) to more modern git hosting (Github, BitBucket, Gitlab, etc).

Here are all my changes:

https://github.com/mtlynch/mediagoblin/compare/mainline-head...master?expand=1

The most relevant files are my Dockerfile to set up the test environment:

https://github.com/mtlynch/mediagoblin/blob/master/Dockerfile.test

and my Travis configuration:

https://github.com/mtlynch/mediagoblin/blob/master/.travis.yml

Subtickets

Change History (1)

comment:1 Changed 8 weeks ago by Ben Sturmfels

Hi Michael,

I agree that CI is essential and would put the extensive test suite to good use. CI would also help us ensure we have reliable installation processes on a range of operating systems (eg. Debian, CentOS, Ubuntu, Fedora). Thanks for your work on this front.

I haven't investigated the skipped tests, but the will be skipped for good reason - likely because they are known to be broken and there isn't an easy fix. So that's good news - the tests that are expected to pass are passing!

When running the tests on Python 3 under Guix (as documented in guix-env.scm on master), I'm seeing 1 failed 174 passed and 8 skipped, with the failure in "TestSubmissionBasics.test_collection_selection".

As to how to fix the 8 skipped tests, these will just need to be investigated one at a time. The first two seem be failing due to a missing "numpy" python package, but the others look harder.

Regarding adopting Travis CI, I haven't used it myself. I gather that the underlying software is technically free software but it appears that it's really only feasible to use it as a hosted service provided by travis-ci.com. This may be an issue for the project, I'm not sure. Let's keep talking about this.

Regards,
Ben

Note: See TracTickets for help on using tickets.