Integrating SentryOne Test with the Remote Agent

SentryOne Test generated code includes the ability to publish detailed test results to a server system. If published, each of the individual elements within a test becomes visible on the server every time they are run. Detailed results can also be found. Rather than just a message saying "your grid comparison failed" you get to see what that grid actually looked like.

Other benefits available with server integration are the ability to publish tests and run them automatically on a schedule, the ability to annotate your tests to give meaningful business descriptions to them, and the ability to specify which parts of your system are covered by the tests that you have implemented.

Connecting to a Server

Connecting to a Server

To connect to a server, open the project settings file. By default, this is called ProjectSettings.legitestSettings. When opening the project settings file, a view similar to the following is presented:

SentryOne Test Validate Server Integration

Type the URL of the server that you wish to connect to and select Validate. If the server requires credentials (SentryOne Test Online for example) then a popup may be shown requiring you to log in to your account. After the server has been validated, this is shown on the display.

SentryOne Test Successful Server Validation

Important:  Ensure that the scheme (http / https) is set correctly for the server that you are connecting to. The address required for the SentryOne Test Web Portal is the public API address (https://test-api.sentryone.com).

Publishing Results

Publishing Results

Publishing results can be controlled for each SentryOne Test Group. When selecting on the SentryOne Test Integration node, the following display is visible:

SentryOne Test Server Integration

Select Publish test results to SentryOne Test to send information to the configured server in project settings when the test runs.

Important:  Server integration is controlled by parameters. Whether a test group publishes results is controlled by the Publish test results to SentryOne Test parameter, and the server where results are published is controlled by the SentryOneTestServerUrl parameter.

Publishing Test Assemblies

Publishing Test Assemblies

Once you're finished developing tests, you can publish the resulting test assemblies to SentryOne Test to be run on a schedule, or manually through the web interface. Publish the desired test assemblies to SentryOne Test by right clicking on the desired project and then selecting Publish to SentryOne Test.

SentryOne Test Publish to SentryOne TestPublish to SentryOne Test context menu


SentryOne Test Publish Test Assembly windowPublish Test Assembly window

The options available to us are:

OptionDescription
TargetSpecify the server that you are publishing the assemblies to, or specify a file path to copy the built assemblies. Add a new server by selecting on the SentryOne Test Publish Results Plus buttonbutton.

Results ModeThe following options are available:
Result ModeDescription
Set all groups to publish resultsAll groups in the project are set to send their results to a server when they run.
Set all groups not to publish resultsAll groups in the project are set so that results are not sent at runtime.
Do not change group settingsNo settings are changed, each group publishes results according to the setting set on the SentryOne Test Server Integration node.
Remove group settings to inherit global configurationThe setting is removed from each group, and the setting is picked up from a global parameter file.
 Results ServerThe server where results are sent for published assemblies.
ConfigurationSpecify the Debug or Release configuration for the C# compiler to use.

Note:  For most situations, Release is the correct configuration to use for deployed tests.
Run tests immediately after publishWhen this option is checked, tests are run by the server as soon as publishing is complete.
Annotations

Annotations

Note:  The Annotations video makes reference to the older version of the web portal known as LegiTest Server when going between Visual Studio and the web features.

Annotations allow meaningful descriptions to be assigned to tests and appear on the test management page for a test. These can be useful when business users are trying to make sense of test results on SentryOne Test.

Configure the annotations in the SentryOne Test Visual Studio Extension by selecting the Annotations node under a test. You'll then see the following display:

SentryOne Test Annotations

Add a new annotation by completing the following:

Select Add to add a new annotation, choose the annotation type from the drop-down menu, and then enter descriptive text that relates to the desired test. 

SentryOne Test Add New Annotation

Note:  Annotations are saved automatically. 

The built in annotation types are :

  • User Error Message 
  • Description Resolution 
  • Steps Custom 

Add custom Annotation types by selecting the Annotation Types node, which then displays a list of all the annotation types:

SentryOne Test Annotation Types tab

From here you can add new Annotation types or delete custom added ones. Select Sync to synchronize the list of annotation types with the configured SentryOne Test Server.

Note:  The built in annotation types cannot be deleted.