Should test code be in the same repository as app code?

One of the first questions we typically get when helping new clients is, "Should test code be in the same repository as app code?" And for good reason. There are definitely trade-offs.

In general, the answer is "yes, unless you have a good reason not to". But you might have a good reason not to.

Before answering this question, you will first want to address the following questions,

• Who should write the tests?
• What language should the tests be written in?
• What kind of tests will you be writing?

For example, unit tests tests should almost always be written by developers and in the language they are writing the application code. Unit tests are meant to be a fast way of ensuring classes and methods do their job and should run quickly. Experienced developers will run these many times before pushing their code to a repository.

One of the best reasons to put test code in the same repository as app code is if you are trying to encourage developers to take a more active role in writing, running, or troubleshooting the tests. It's just more likely to happen if the tests are written in their language and already on their computer.

Another great reason to put tests is the same repository as the code is to make sure the version of the tests matches the version of the code. This is sometimes easier said than done. Failing to match the appropriate version of the tests to the appropriate version of the code is one of the leading reasons for flaky tests.

So when should I put the tests in the same repository?

• You are writing unit/integration tests.
• The testers and developers are able to work closely together (or are the same role).
• You have a simple enough architecture (e.g. one UI, one API).
• You don't have end-to-end tests that need to span multiple environments, teams, or services (unless your team has a "monorepo").
• You want to encourage developers to write and run the tests.

And when should I put the tests in different repositories?

• The tests would pull in libraries or dependencies that interfere with application code.
• The tests will be end-to-end tests that cover microservices that already live in different repositories and it's not clear which repository would own the tests.
• Your team structure doesn't allow or makes this option difficult.

I'm stuck with different repositories, now what?

Different repositories aren't always avoidable. If you find you have tests that need to live in different repositories, here are some things to consider:

• How will you match the version of the code to the version of the tests --- one of the leading causes of test flakiness?
• Will the test repository follow the same branching strategies as the application repository and if not will that cause problems?
• How will you kick off test runs whenever the application repository is changed?

If you need help answering these questions and implementing them in your organization, the Test Orchestrators at Team Testery can help.