RSpec test reporting

RSpec Tesults Formatter makes setup easy. Add one dependecy to your project.

gem install rspec_tesults_formatter

Once the rspec_tesults_formatter gem has been installed your results data will be pushed to Tesults if you follow the steps here to configure RSpec. By default the RSpec Tesults Formatter will be disabled. To enable the formatter you must provide a value for the tesults_target option. There are also other arguments that you can provide to report more detailed information, outlined below.

If you have not done so already in your RSpec folder, run:

rspec --init

This will create a .rspec file and a spec directory containing spec_helper.rb, if they do not already exist. Open up the .rspec file and add:

.rspec

--require rspec_tesults_formatter
--require spec_helper

Then in the spec_helper.rb file add:

RSpec.configure do |config|
  config.add_setting :tesults_target, :default => "token"
  config.add_setting :tesults_files, :default => "/path/to/files"
  config.add_setting :tesults_build_name, :default => "1.0.0"
  config.add_setting :tesults_build_desc, :default => "Build description"
  config.add_setting :tesults_build_result, :default => "pass"
  config.add_setting :tesults_build_reason  , :default => "Failure reason if fail"
end

The token above should be replaced with the token for your Tesults target, available from the Tesults configuration menu. The other settings are explained in detail below in the 'Tesults Settings' section. Now when you run your tests, use the TesultsFormatter and your test results data will automatically be transmitted to Tesults:

rspec  --format TesultsFormatter ./*_spec.rb

Alternatively, you can add this line to your spec_helper.rb:

# This line MUST come after other tesults settings are set such as tesults_target
config.formatter = "TesultsFormatter"

Then to use Tesults you can run your rspec command without needing --format:

rspec ./*_spec.rb

Supporting Multiple Targets

To consolidate test results data from multiple test jobs you can create multiple spec_helper.rb files, name each one after a target, for example spec_helper_target1.rb and spec_helper_target2.rb. Each spec_helper can specifiy a different target token. Then remove the line --require spec_helper from your .rspec file and instead use --require spec_helper when executing rspec in order to choose the file containing the target you want to use. In this case you have:

.rspec

--require rspec_tesults_formatter

Multiple spec_helper for each target

/project
  /spec
    spec_helper_target1.rb
    spec_helper_target2.rb

Command

rspec --require spec_helper_target1.rb ./*_spec.rb

Tesults Target

tesults_targetRequired

The tesults_target is the token value for the target you wish you to upload results to. You can retrieve your token from the Tesults configuration menu. Learn more about targets here

tesults_filesOptional

Provide the top-level directory where files generated during testing are saved for the running test run. Files, including logs, screen captures and other artifacts will be automatically uploaded.

This is one area where the RSpec Tesults Formatter is opinionated and requires that files generated during a test run be saved locally temporarily within a specific directory structure.

Store all files in a temporary directory as your tests run. After Tesults upload is complete, delete the temporary directory or just have it overwritten on the next test run.

The default behavior of the RSpec Tesults Formatter is to set the describe value as the test suite, the context value as the test description and the it value as the test case name. Also be aware that if providing build files, the build suite is always set to [build] and files are expected to be located in temporary/[build]/buildname

Caution: If uploading files the time taken to upload is entirely dependent on your network speed. Typical office upload speeds of 100 - 1000 Mbps should allow upload of even hundreds of files quite quickly, just a few seconds, but if you have slower access it may take hours. We recommend uploading a reasonable number of files for each test case. The upload method blocks at the end of a test run while uploading test results and files. When starting out test without files first to ensure everything is setup correctly.

• temporary folder
• Test Suite A
• Test 1
• test.log
• screenshot.png
• Test 2
• test.log
• screenshot.png
• Test Suite B
• Test 3
• metrics.csv
• error.log
• Test 4
• test.log

tesults_build_nameOptional

Use this to report a build version or name for reporting purposes.

tesults_build_resultOptional

Use this to report the build result, must be one of [pass, fail, unknown].

tesults_build_descOptional

tesults_build_reason=reasonOptional

Use this to report a build failure reason.

How RSpec test cases are reported in Tesults

This example test file includes one describe, three context and three it blocks. Tesults would report the suite as Test Suite A, the test cases would be reported as Test 1, Test 2 and Test 3. The context values will appear as the descriptions for each test case.

class Calculator
  def add(a, b)
    a + b
  end
end

RSpec.describe Calculator do
  describe 'Test Suite A' do
    context 'Test 1 description' do
      it 'Test 1' do
        expect(Calculator.new.add(1, 2)).to eq(3)
      end
    end
    context 'Test 2 description' do
      it 'Test 2' do
        expect(Calculator.new.add(2, 2)).to eq(4)
      end
    end
    context 'Test 3 description' do
      it 'Test 3' do
        expect(Calculator.new.add(3, 3)).to eq(6)
      end
    end
  end
  end

Note that if you omit the context blocks then have at least one context (or an additional describe) around all it blocks in the file to correctly report test suite names.

Result interpretation is not currently supported by this integration. If you are interested in support please contact help@tesults.com.