Tests are important but usually they fall down quickly in your priority list. That’s why when I started a new django project last week, I decided to set up my test environment as soon as possible. Even if it means having 3 tests running at the beginning.
Here are my requirements:
- integration tests for my APIs (unit tests are easier to set up in the future) where integration tests need authentication and test the full stack.
- tests are fast
- tests are run automatically
I’ll focus on the last point for this post. My solution is to run my tests on each push to GitHub. For that, I need to find a CI service that would take care of running the tests (instead of having to build a Jenkins CI instance).
I explored 3 options. My criterias end up being: price and documentation.
- Travis CI: pricing ($129) is too much for my simple project even if the documentation seems to be the best. I’ll give a try for an open source project, no question asked.
- Codeship: pricing is better but I got lost in the documentation when I landed there.
- Circle CI: pricing allows me to have one private project for free, and the documentation isn’t as bad as codeship.
So the good compromise is Circle CI. Even if I had to battle a couple of hours with the documentation (which I think contains way too much text, and not enough code). I’ll try to explain the few hiccups I had.
My django project is running with postgres and redis. Circle CI tries to infer your configuration but it couldn’t infer my project configuration due to my files tree:
In this case, you need to create a YAML file:
circle.yml at the root of your project (useful tip is to get familiar is yaml to avoid dummy mistakes).
My first mistake was line 3, Circle CI doesn’t support every python version. If you are running on the latest docker python image, it’s
2.7.10 which isn’t supported yet on Circle CI.
I’m storing settings such as database credentials as environment variables on the server, and access them as follow from my
My suggestion here is to access your variable using
.get(<name>) so in case the variable isn’t defined it will fallback to
None or best a value you’ve pre-defined (such as
localhost for the host).
In my case, just
DB_USER need to be define in my
If like me, you want to run the coverage for your tests, don’t forget to define your
.coveragerc at the same level as your
manage.py file. Otherwise you will run coverage on all dependencies.
After running the tests, Circle CI will execute
post commands. For coverage, it’s useful to define the
report mode (which output in text format instead of html by default).
Maybe the trickiest error I had was
pyflakes failing. If you run the following 2 commands on your code, you get this output:
Did you get it?
pyflakes is exiting with a status
1 which means error and Circle CI doesn’t like that.
Solution: force the command to exit with a
<command> || :.
Also don’t forget the quotes otherwise yaml thinks it’s a map except that the
post section is a list.
I haven’t dig up into more options, right now my tests are running on Circle CI. It’s good enough for now, the foundations are laid and it will be faster to iterate on it and write more tests.