Hiring for Test Orchestration

Chris Harbert -

Test Writing is an important and non-trivial job. It involves selecting the right frameworks, implementing good design patterns for testing, and writing plenty of code. But in today's fast-paced, devops-driven software shops, simply writing the tests isn't enough. Getting the right tests to run at the right time, reliably and at-scale requires a good working knowledge of devops and cloud technologies. This is where the Test Writer becomes the Test Orchestrator.

Test Orchestration brings together process, test execution, and analytics in order to make sure your dev shop is running like a well-oiled machine. Teams that do test orchestration are happier and more productive, because they spend less time waiting for test results, less time troubleshooting flaky tests, and more time focusing on features. Sounds pretty good, right? That's where a Test Orchestrator can help.

Hire for the right skills

Test Orchestrators and Test Writers have very different roles. While Test Writers need to be good at creating the tests, Test Orchestrators need to also be good at everything required to get those tests to run reliably and at the right time.

Skills Needed for Test Orchestration

  • Cloud technologies (e.g. ECS, EKS)
  • Containerization (i.e. Docker)
  • Command line scripting (i.e. bash)
  • Monitoring system performance
  • Process management
  • Configuring CI/CD

Skills Needed for Test Writing

  • Testing framework(s)
  • Writing code
  • Design patterns for testing (e.g. page object model

Ask the right questions

During the interview, you'll want to be sure to ask the right questions. Preparing questions in advance to ensure they know the required skills and technologies is an important step. For example, you could ask "What command in Linux is used to search through the contents of files?" or "How do you set up SSH keys to not require a password on login?" Questions like these will help you know if you're talking to a test orchestrator or a test writer.

Be prepared to pay more

Tech salaries seem to be going up across the board right now, but this is especially true for Test Orchestrators. Keep in mind that the Test Orchestrator has Devops skills which are in short supply and high demand. They could be using these skills to help teams manage and deploy production infrastructure, but then are also required to have specialized knowledge of test automation.

Consider training your current team

Not all Test Writers have what it takes to make the transition to becoming a Test Orchestrator, but many do. If you've got folks on your team who are capable of learning these skills, you'd be doing everyone a disservice to not help them along a better career path.

Don't be afraid to ask for help

There are companies out there (like Testery) who can help. Testery provides a unique blend of training, coaching, services, and technology to help teams develop their Test Orchestration capabilities in-house. If you'd like a free consultation to see if your team is at the right stage for developing test orchestration capabilities in-house, please reach out.