Documentation
Pricing
Sign in
Sign up
 

Test framework developers

Tesults integrations for popular test frameworks make it quick and easy to upload test results data. This makes it possible for developers of tests to configure results upload to Tesults in minutes. Developing integrations like this is possible and this page shows you how.

This page is for test framework developers to understand how to create an integration for transmitting test results data to Tesults. Specifically there are two audiences who may be interested in this information:

  • Developers of publically shared test frameworks intending to develop an integration package to be shared with users outside of their development team. Tesults is always lookings to offer new integrations for emerging test frameworks and update existing ones to provide more powerful functionality. See the links under Test Frameworks in the menu to view examples of existing integrations
  • Developers of custom test frameworks intending to integrate with Tesults for use internally within their organization but not share the integration externally.

Develop a test framework integration

The general approach for developing an integration is to:

  1. Get test results data generated from a test run in your test framework into the Tesults JSON data standard
  2. Pass the data into the results upload function of a Tesults language library (see Language Libraries in menu)
  3. Document and package up the integration for use by others (if sharing the integration externally).

At every step of the way Tesults can help provide guidance, support and answer any questions you have. Please contact help@tesults.com to be assigned a technical consultant.

Ideal experience for users of your test framework

You can ignore this section if you are developing an integration only for internal use within you organization. Follow steps 1 and 2 of the general approach described above under Develop a test framework integration.

  • Integrations only require configuration and no code for submitting test results data to Tesults.
  • Integrations are quick and easy to configure. Installation and setup should take 5-10 minutes at most.
  • Integrations are released under the MIT license and hosted in public repositories.
  • Where possible, integrations are available to install from popular package managers such as npm, PyPI and NuGet.
  • Integrations meet the basic functional requirements outlined below.

For official integrations to be supported by Tesults we will coordinate what Tesults repositories to utilize and we will publish packages to package mangers using a Tesults signed account.

Functional Requirements

You can ignore this section if you are developing an integration only for internal use within you organization. Follow steps 1 and 2 of the general approach described above under Develop a test framework integration.

Integrations should meet these minimum requirements. If you have not used a Tesults integration before these requirements may be difficult to understand and it is recommended you take a look at the documentation for an existing test framework such as for Playwright (Node.js) to gain context.

  1. The integration accepts parameters for:
    1. target token
    2. build arguments

    See Tesults JSON data standard for more details about how the target and build arguments are used. How you accept parameters is dependent on how parameters are accepted by your test framework, often this can be via the commandline when running tests or using a configuration file. See an existing test framework integration under Test Frameworks in the menu for an example such as Playwright (Node.js).

    If the target token is not supplied, the integration should be disabled and output a message to inform the user results upload is disabled.

  2. While uploading results the integration must output to the console the text: 'Tesults results upload...' and after upload is complete the integration must output the status of the response in the following format:
    Success: true
    Message: Success
    Warnings: 0
    Errors: 0

    Success value must be true or false as returned from a Tesults language library. Message should be 'Success' if Success is true, otherwise it should be the failure message returned in the Tesults language library response. Warnings indicates the number of file uploads that failed, Errors indicates the number of results upload failures.
  3. Official integrations will under go testing and security analysis by the Tesults team prior to publishing. A functional test checklist may be supplied to the integration developer.

The Tesults team is here to support integration developers. Contact help@tesults.com for answers to any questions you may have.