Testery vs Postman...what are the benefits of using Testery over Postman when it comes to API testing?  

First things first...

What is Testery? Testery is a continuous testing platform that helps you run your fullstack tests across multiple frameworks and environments at scale (unlimited parallelization!).  It’s a single platform to view what is happening across your teams, applications and CI/CD from a testing perspective.

What is Postman? Postman makes API development easy.  It offers tools to simplify each step of the API building process and streamlines collaboration so you can create better APIs.  Postman has the ability to write and run API tests and manage the API workflow.

So...which tool is right for your situation?  It really depends on what your team is wanting to accomplish.  Testery would be a great choice if wanting a scalable testing solution that easily supports multiple teams, different types of testing, and multiple environments.  Postman, on the other hand, would be a great choice if wanting to do API-first development, debugging API issues or you are wanting to do some quick exploratory testing of the API. For an automated test runner, Testery is going to be much more efficient, maintainable and scalable. For API exploratory testing, Postman is the “man”!  So, it might not be an either or choice.  You may want to use both tools.  

In fact, our Testery professional services team starts out with postman first on api testing projects, then proceeds to automate those postman tests with open source api testing frameworks (Specflow and Restsharp for C#, Cucumber and Supertest for Nodejs, RestAssured and Cucumber for java) while adhering to SOLID programming principles to ensure the test code is scalable and maintainable, and then runs those on Testery.

Here’s a few reasons why you would want to use Testery:

  • Have a single view of testing across teams, projects and applications.
  • To standardize testing using multiple testing frameworks
  • Manually or automatically kick off different types of tests with different configurations across environments (e.g. UI, API, DB tests)
  • Have unlimited parallelization to help with scale
  • Deployment verification
  • Post-production monitoring
  • Load testing
  • Have a professional services team available to help get automation up and running, test coverage in place, and tests running in the CI/CD workflow.

Here’s a few reasons why you would want to use Postman:

  • To do manual or exploratory testing of API (Postman does support automated testing but it’s definitely not a main feature and is not built into the UI)
  • API mocking
  • Quickly generate API tests
  • Easily manipulate test object data
  • Debugging API issues

Here’s a reference chart comparing a few features between Testery and Postman:


Testery

Postman

Testing frameworks supported

Most open source frameworks (Java, .Net, Javascript, Python, Cucumber, Cypress, etc)

Javascript only

Type of testing supported

Fullstack (API, UI, Database, message queue, load)

API only

Scheduled test runs

Yes - can schedule a test run based on time interval or on a deploy of any project in your repository.  Can customize based on environment, project, etc.

Yes - can set up a monitor based on time interval using an existing Collection.

Ability to run tests

Yes - sweet spot of Testery.  Runs tests across frameworks and environments in same platform at scale

Yes - Postman has a Collections Runner that allows you to kick off any Collection.

Test parallelization

Yes. There is no limit to parallelization 

(permission is required to go higher than 60 parallel tests)

Yes but you can only parallelize a single collection and it’s hard to do.  You cannot parallelize across collections.

Multiple environment support

You can create Pipeline Stages to match the stages in your deployment pipeline and unlimited environments within each stage.

You can create environment variables for different environments and run collections per environment.

Test Maintenance

Easy - can maintain test code in a normal dev IDE, version code, etc.

Hard - not easy to maintain tests. Can’t reuse code.  For instance, can’t do a find and replace on code and can’t create methods.

Notifications

Yes - test results can integrate with tools like Slack and OpsGenie, and has webhooks available

Yes - Test results can integrate with tools like Slack, OpsGenie, Microsoft Teams, etc.

BDD

Yes - supports BDD through frameworks like Cucumber

No - doesn’t support BDD

Integrates into CI/CD

Yes - using Testery CLI

Yes - using Newman

Collaboration

Shared test runs and results where team members can view status of test runs, and chat within a test run about any issues.

Shared Workspaces to share collections, environments, integrations, mocks, etc

Pricing

Based on usage.  No per user pricing.

Per user pricing ($12-$69 per user based on plac) + api usage fees


Interested in trying out Testery?  Create a free account today or chat with one of our test automation specialists to help get your team started down a good continuous testing path.