Understanding Parallelism in Playwright

Playwright has great features around running your tests in parallel. Let's walk through some test execution nuances that may or may not be obvious for the framework.

Workers

By default, the configuration file allocates an undefined number of workers for normal runs and one worker for Continuous Integration (CI) runs.

 workers: process.env.CI ? 1 : undefined,

This means that outside of CI, you'll have one worker per processor core; in CI, only one worker will be used. Having one worker will result in the sequential execution of tests across all files, whereas using multiple workers will distribute the load by running test files in parallel.

If you have multiple test files, after a worker finishes one it will pick another up until all files are complete.

The command line option is available to override the default to avoid changing any configuration.

npx playwright test --workers=1

Tests

By default, the tests defined in each file will run sequentially and ignore the results of the tests before them.

Parallelism

To have all tests be truly parallel, you can define a switch at the file/describe level (a good option if you have some tests that can be fully parallel and others that aren't which can be isolated in separate files).

import { test } from '@playwright/test';

test.describe.configure({ mode: 'parallel' });

A second option is to run them parallel by project.

export default defineConfig({
  // runs all tests in all files of a specific project in parallel
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
      fullyParallel: true,
    },
  ]
});

And the final option is to run everything parallel at the config level.

export default defineConfig({
  fullyParallel: true,
});

Stop ifs

As mentioned earlier, the default behavior of tests is to ignore the status of tests before them. However, if you wish to set a cut-off or end tests based on previous statuses, Playwright has a few options.

You can set a hard limit to the maximum number of failures to save time running a full suite of tests when something has gone wrong in the environment. Simply add the maxFailures option.

export default defineConfig({
  // Limit the number of failures on CI to save resources
  maxFailures: process.env.CI ? 10 : undefined,
});

If you wish to fail a test because another test in the same file/describe block failed, the serial option can be added to a test file. (This is best avoided though when following best practices for test writing).

// Annotate entire file as serial.
test.describe.configure({ mode: 'serial' });

This option makes the tests in the file interdependent, and the run for the group will end at the first failure.

Everything above handles parallelism on a single machine, but Playwright also gives us the ability to take parallelism even further by using shards to divide tests across multiple machines.

By passing in the command line option, tests can be divided by x amount of machines. For example, if you have four machines you can send the following commands:

npx playwright test --shard=1/4

npx playwright test --shard=2/4

etc.

fullyParallel vs non-fullyParallel sharding

If you have your fullyParallel option set to true in your config file, then the tests will be split evenly across the total test count. When set to false, it will divide based on the number of files instead. Because the number of tests inside each file can vary, it's recommended to set the fullyParallel option to true to ensure a better balance.

I hope this gave you a better understanding of parallelism in Playwright to enable your team to get the most out of your machine resources and time. For more information, please visit the official playwright documentation pages. For automatic parallelization of your tests with live updates and reports please check out the Testery platform.