Opened 11 years ago
Last modified 7 years ago
#458 accepted task
Functional testing via selenium, phantomjs, or similar
|Reported by:||Christopher Allan Webber||Owned by:|
|Cc:||Gabriela Thumé, lotusecho||Parent Tickets:|
It would be good to have a way to run functional tests that mimic user behavior and tie them into our unit testing commands.
We should probably use a tool like PhantomJS or Selenium. And then of course we should have some tests. :)
I'm marking the priority as low at the moment because I think it's not as critical at the moment given everything else going on.
Change History (11)
comment:1 by , 11 years ago
|Priority:||minor → major|
comment:2 by , 11 years ago
|Type:||defect → task|
comment:3 by , 10 years ago
|Status:||new → accepted|
I'm cc'ing gabithume who has been doing some work on this recently (though I don't know what the most recent state of her work is). Gabithume: feel free to claim this ticket if you're presently working on this, or otherwise give updates if you have them! (No pressure though)
comment:4 by , 10 years ago
So, I have been studying a little about these framework to create functional test. Selenium is not headless but has the advantage to be browser agnostic while Phantomjs is headless but webkit-specific.
Selenium Webdriver automation tool provides all sorts of features by driving a browser natively as a user would either locally or on a remote machine using the Selenium Server. It supports a lot of platforms .
Selenium is the leading OSS web testing framework. Also, there are negotiations with W3C about making WebDriver a standard or recomendation .
Tests can be written with python, which is convenient for mediagoblin developers. In this way I created a functional test for ticket #689 about a broken link in forgot password form .
What do you think? Should I send a commit and write some Selenium installation instructions for mediagoblin devs?
comment:5 by , 10 years ago
So, we're going to do more discussion next week about testing between LotusEcho and I since she'll be working on testing as part of her summer internship. I know she's finishing exams this week; gabithume: maybe would you mind jumping on IRC next week and talking with the two of us? Maybe together we can find out the right direction for all this.
comment:6 by , 10 years ago
Sure. I will enjoy it :)
comment:7 by , 10 years ago
|Status:||accepted → in_progress|
I will be implementing this fully with a large test suite as part of my OPW internship. Also, there is a new way to use Phantomjs as a "browser" for Selenium to run Selenium tests in a headless manner. :)
comment:8 by , 10 years ago
I'm working on this in this repository:
comment:9 by , 9 years ago
comment:10 by , 9 years ago
So we ran into one big problem with this test suite (see above) that I don't know how to fix. Essentially, Selenium (and any functional testing tool I know of that is FOSS) is highly coupled with the user interface. Change the theme? All the tests might break. Change your language? That might break it too. Essentially it could mean that you might have to change many of the tests just because of one minor change in the user interface.
I tried to modularize the code somewhat so that common actions wouldn't be invoked by a particular test and instead acted more like a function but I couldn't do that for everything without slowing the progress to a halt. That could be one potential way to make it more maintainable.
The double-edged sword on that becomes that it would be very difficult for new contributors to write a test. That's a problem since one of our goals was to have something that new contributors could do that familiarized them with GMG. Normally writing a test in Selenium is simple; it's basically point-and-click.
comment:11 by , 7 years ago
|Status:||in_progress → accepted|
I'm upping the priority on this, since I believe a better testing framework would increase the robustness of the application in general, which to me seems like a very good thing, and perhaps even needed in some cases.
To me, the current issues with unit tests are: