How to setup a test environment with Postman / Newman

Postman is a tool for API development and testing. In one of our recent projects, we decided to use Newman as our testing cli. In this blog post you can read more about our setup, what this setup is able to do, and - most important - what it's not able to do!

This article requires a basic understanding of Postman. It is also not describing how to write tests in Postman, but how to set up a test environment.

One of the great advantages of Postman and Newman as a test engine is that we are able to test each scope with the help of so-called iteration data. The iteration data is a JSON file that consists of an array of objects that provide environment variables ({{example}}) during the test of a collection ("Postman Runner"). This means that the number of test iterations over the whole Postman collection corresponds to the length of the iteration data array.

Following example would iterate all endpoints of the Postman collection two times:

[
  {
    "user": "user1"
  },
  {
    "user": "user2"
  }
]

In the first iteration, all endpoints that contain {{user}} (e.g. in the path or body) will be called with "user1", in the second "user2".

Also, the variables in the iteration data can be set in the tests or prerequisites like pm.variables.set("user", "user3");. This is especially important if you want to work with data from previous requests. Assuming we have 3 calls:

1. POST a news
2. UPDATE a news/{{uuidNews}}
3. DELETE a news/{{uuidNews}}

In the first call, we e.g. want to test that the news has been created by checking success === true and expecting uuid as a response to be a string. In the second call, we want to update the newly created news entry. Therefore we have to set the environment variable of {{uuidNews}} to the newly created one *before* we trigger the second call (= in the test of the first call). In the third call, we are deleting the newly created news entry. Afterward, we want to set the current environment variable back to the variable that was defined in the iteration data (if it exists there) like: pm.variables.set("uuidNews", pm.iterationData.get("uuidNews")); This might also be done before a call in which you want to work with uuidNews.

Considering this flow, your Postman collection should be ordered by the method:

1. POST - to create new entries

2. PUT - to update newly created entries

3. GET - to get the newly created entry and check its value for the correctness

4. DELETE - remove the newly created entry

In the Postman UI, the following steps are required to run your tests:

1. After importing, select Runner in the Postman UI to open the test runner.

2. Select your collection

3. Select the proper environment (if exists)

4. On Data: Select file, select an iteration file

5. Run the tests

This test runner does basically what Newman does. All you need to do to run your tests in your project now is to export your Postman collection and the local environments of Postman (can both be done by using the Postman GUI). Together with the iteration data, you are now able to run your tests with Newman like:

newman run -e postman.environment_data.json -d postman.iteration_data.json postman_collection.json

A short summary of the pros and cons of this setup:

Pros:

• Ability to test multiple per endpoint by using iteration data

• Ability to not only write tests endpoint-by-endpoint but in the context of a collection

• Writing tests in Postman is pretty convenient

• Postman test runner has a nice and clear GUI for tests

• Together with the Postman console, debugging is quite easy

Cons:

• Hard to keep sync between your codebase to Postman (e.g. newly created routes have to be added to Postman collection manually)
• Difficult to work in teams