Skip to content

Test runs

On the very first login to Zebrunner, you will be asked to select one of the two screen modes: Integration via agent (Tests) or Selenium Hub (Sessions). This view will be displayed by default when you open the Test runs/Test sessions page.

If you skip this step, Integration via agent (Test runs view) will be enabled by default. You can easily switch between the 2 modes by using the control on the Test runs/Test sessions page.

Test Runs and Test Sessions

The Test runs page allows you to launch tests from your GitHub repository, see detailed test results in real time and on a test run finish.

To access the Test runs page, go to the side menu and click "Test runs/Test sessions" (Test Runs and Test Sessions icon on the side menu). If the Test runs mode is not set by default, click on the “Tests/Sessions” switch in the upper-right corner of the window.

All the actions with test runs are performed on this page.

Test Runs Page

Test launches

To start running tests, you need to add your GitHub repository. Or get a sample test automation project to your GitHub account while creating your company in Zebrunner (read more in Getting Started).

Add repository

To add a new repository, perform the following steps:

  1. On the Test runs page, click the “Test launcher” button.
  2. The Launcher wizard will be opened.
  3. Click the “+Add repo” button.
  4. Provide valid credentials for your GitHub account (select the organization of your user and the repository that contains automation tests).
  5. Press “Add”.
  6. Select the automation server and the branch.
  7. Press “Scan”. The scanner will pick up all the test XML files and create a launcher template for every XML. This action may take a couple of minutes.
  8. When you see the “Done” message, click “Back to list”.

Multiple launchers will be created for API, web, and mobile tests.

Add Repository

Remove repository

To remove a repository, perform the following steps:

  1. On the Test runs page, click the “Test launcher” button.
  2. The Launcher wizard will be opened.
  3. Click Remove Repository beside the repository name.
  4. Press “Delete repo” at the bottom of the wizard.

The repository will be removed from the list.

Remove Repository

Create launcher manually

To create a test launcher manually, perform the following steps:

  1. On the Test runs page, click the “Test launcher” button.
  2. The Launcher wizard will be opened.
  3. Click Create Launcher beside the repository name.
  4. Create a new template specifying the template name, choosing the platform and the template, adding or removing arguments.
  5. Press “Add”.

The new launcher template will be added with a tag “Manual creation” Manual Creation

Create Launcher Manually

Launch test run

To launch a test run, perform the following steps:

  1. On the Test runs page, click the “Test launcher” button.
  2. The Launcher wizard will be opened.
  3. Choose the necessary test launcher from the list.
  4. the test environment server.
  5. Select the type of a test run, browser name and its version.
  6. In the section below, parameterize your automation code. Select a branch, environment, thread count, emails, and other parameters.
  7. Add additional parameters by clicking the “Edit” button Launch Test Run at the top of the wizard.
  8. Press “Launch”.

The test run will be displayed in the state of compilation. After it switches to “In-progress” (marked blue), you can track the automation results in real time.

Launch Test Run

Abort test run

To abort a test run “In-progress” (marked blue), perform the following steps:

  1. On the Test runs page, go to the Test run settings Test Run Settings.
  2. Press “Abort”.

The test run will become aborted (marked grey).

Abort Test Run

Add favourite launcher

You can mark a launcher as favourite. It will be displayed at the top of the list of launchers.

For this, choose the necessary launcher and click the star icon Star Icon at the top of the wizard. The launcher will be placed at the top of the list of launchers with a star icon.

Favourite Launcher

Remove launcher

To remove a launcher, perform the following steps:

  1. On the Test runs page, click the “Test launcher” button.
  2. The Launcher wizard will be opened.
  3. Select the launcher you would like to remove.
  4. Click the “Edit” button Edit at the top of the wizard.
  5. Press “Delete”.

The launcher will be removed from the list.

Remove Launcher

Test run results

On the Test runs page, you can see all the test runs that have been executed recently. Moreover, view test run details, logs and video recordings, share test results with your colleagues.

On the Test runs page, for every test run you can see:

  1. Test suite name and type.
  2. Environment.
  3. Platform/browser.
  4. The number of passed, skipped, failed, etc. tests.
  5. Execution time and start day.
  6. Settings (the 3-dot button).

Test Runs Page View

Filter test runs

At the top of the Test runs page, you can filter results by different criteria:

  1. Name.
  2. Reviewed/not reviewed.
  3. Test status (passed, failed, skipped, aborted, in_progress, queued, unknown).
  4. Environment.
  5. Platform name.
  6. Browser name.
  7. Locale.
  8. Start date.

Filter Test Runs

Add stored filter

You can create a custom filter that will be used on a daily basis by you and your teammates.

For this, click “+Add stored filter” in the upper-left corner of the page. Tap the name, choose necessary parameters from the list, press “Add”.

You can create up to 30 new parameter sets at once.

After you are done, press “Create”.

Add Stored Filter

Edit/delete stored filter

You can edit a created stored filter by clicking the pencil icon beside its name. Make the necessary changes and press "Save”.

To delete a stored filter, click Pencil Icon beside its name and press "Delete".

Edit and Delete Stored Filter

View test run results

To analyze test run results, click any item in the list. Or go to the Test run settings Test Run Settings and click “Open” (the Test results page will be opened in a new window).

Passed tests are marked green, and failed ones are red.

For every test, you can see the following details:

  • Execution time
  • Test owner
  • Tags
  • Reference to a Jira ticket (if there is one)

Stack trace is provided for failed tests.

View Test Run Results

Filter/group tests

You can filter test results by:

  • Name;
  • Status (passed, failed, skipped, aborted, in progress).

Tests can be grouped by:

  • Class name;
  • Package name;
  • Tags.

And sorted by:

  • Test name;
  • Duration (long - short);
  • Duration (short - long).

Press “Reset” in the upper-right corner to return to the default test run results view.

Filter and Group Test Results

Mark test as passed/failed

To mark a test as passed/failed, on the Test results page, click Test settings Test Settings and press “Mark test as passed/failed”.

Mark Test as Passed or Failed

This option is only available for failed tests.

On the Test results page, click Test settings Test Settings and press “Link issue”.

The red label will appear, the same one will be displayed in test statistics.

After you link a known issue from Jira to a failed test, next time you get the same stack trace, the test run will be automatically identified and displayed with the same label.

Link Known Issue to Test

Analyze test flow

To analyze the test flow and see additional information about test results, click on any test on the Test results page.

There are several areas displayed to cover the most important aspects of your tests.

In the upper area, view your test history with the execution date (how long ago the test was run) and its status (passed, failed, aborted, skipped). By clicking on any item in the test history, you will be directed to the chosen test info to analyze your test progress.

In the test logs area below, all the test steps are tracked. View the visuals, start date and time, and action taken, filter test steps by status (fatal, error, warn, info, debug, trace, or all).

Visuals, or screenshots (if available) can be maximized by clicking on them.

Test History and Screenshots

Turn on the video to view the test flow with comments of what is going on right now in the text script. You can maximize the video to get a better view.

Under the video player, learn info about the test platform/browser, actual test status, its date and duration.

Video in Test Results

Share test run results

To copy the link to test run results, perform the following steps.

  1. On the Test runs page, go to the Test run settings Test Run Settings on the left.
  2. Press “Copy link”.

Copy link

Send as email

To send test results via email, perform the following steps:

  1. On the Test runs page, go to the Test run settings Test Run Settings on the left.
  2. Press “Send as email”.
  3. The “Email” popup will appear.
  4. Fill in the “Recipients” field (unlimited number of emails per time).
  5. Press “Send”.

A popup message “Email was successfully sent” will appear in the lower-right corner of the window.

Send as Email

Export to HTML

You can export test results to an HTML file and send it to your colleagues.

For this, perform the following steps:

  1. On the Test runs page, go to the Test run settings Test Run Settings on the left.
  2. Press “Export to HTML”.
  3. Save the document on your computer.

Export to HTML

Mark test run as reviewed

To mark a test run as reviewed, perform the following steps:

  1. On the Test runs page, go to the Test run settings Test Run Settings on the left.
  2. Press “Mark as reviewed”.
  3. The “Comments” popup will appear.
  4. Provide a comment if necessary.
  5. Press “Mark as reviewed”.

A mark Reviewed will appear beside the test run name, together with Reviewed if any comment was provided.

Mark Test Run as Reviewed

Rerun/build test

To rerun tests with the same parameters, perform the following steps:

  1. On the Test runs page, go to the Test run settings Test Run Settings on the left.
  2. Press “Rerun”.
  3. The “Rebuild testrun” popup will appear.
  4. Choose what tests you want to rerun: Only failures or All tests.
  5. Press “Rerun”.

Rerun Test

To run tests with additional parameters, perform the following steps:

  1. On the Test runs page, go to the Test run settings Test Run Settings on the left.
  2. Press “Build now”.
  3. The “Build now” popup will appear.
  4. Change the proposed parameters.
  5. Press “Build now”.

Build Test Run

Delete test run

To delete a test run, perform the following steps:

  1. On the Test runs page, go to the Test run settings Test Run Settings on the left.
  2. Press “Delete”.
  3. Submit the action.

The test run will be removed.

Delete Test Run