SentryOne Test generated code includes the ability to publish detailed test results to a server system. If published, each of the individual elements within a test becomes visible on the server every time they are run. Detailed results can also be found. Rather than just a message saying "your grid comparison failed" you get to see what that grid actually looked like.
Other benefits available with server integration are the ability to publish tests and run them automatically on a schedule, the ability to annotate your tests to give meaningful business descriptions to them, and the ability to specify which parts of your system are covered by the tests that you have implemented.
Connecting to a Server
To connect to a server, open the project settings file. By default, this is called ProjectSettings.legitestSettings. When opening the project settings file, a view similar to the following is presented:
Type the URL of the server that you wish to connect to and select Validate. If the server requires credentials (SentryOne Test Online for example) then a popup may be shown requiring you to log in to your account. After the server has been validated, this is shown on the display.
Important: Ensure that the scheme (http / https) is set correctly for the server that you are connecting to. The address required for the SentryOne Test Web Portal is the public API address (https://test-api.sentryone.com).
Publishing results can be controlled for each SentryOne Test Group. When selecting on the SentryOne Test Integration node, the following display is visible:
Select Publish test results to SentryOne Test to send information to the configured server in project settings when the test runs.
Important: Server integration is controlled by parameters. Whether a test group publishes results is controlled by the Publish test results to SentryOne Test parameter, and the server where results are published is controlled by the SentryOneTestServerUrl parameter.
Publishing Test Assemblies
Once you're finished developing tests, you can publish the resulting test assemblies to SentryOne Test to be run on a schedule, or manually through the web interface. Publish the desired test assemblies to SentryOne Test by right clicking on the desired project and then selecting Publish to SentryOne Test.
The options available to us are:
|Target||Specify the server that you are publishing the assemblies to, or specify a file path to copy the built assemblies. Add a new server by selecting on the button.|
|Results Mode||The following options are available:|
|Results Server||The server where results are sent for published assemblies.|
|Configuration||Specify the Debug or Release configuration for the C# compiler to use.|
Note: For most situations, Release is the correct configuration to use for deployed tests.
|Run tests immediately after publish||When this option is checked, tests are run by the server as soon as publishing is complete.|
Note: The Annotations video makes reference to the older version of the web portal known as LegiTest Server when going between Visual Studio and the web features.
Annotations allow meaningful descriptions to be assigned to tests and appear on the test management page for a test. These can be useful when business users are trying to make sense of test results on SentryOne Test.
Configure the annotations in the SentryOne Test Visual Studio Extension by selecting the Annotations node under a test. You'll then see the following display:
Add a new annotation by completing the following:
Select Add to add a new annotation, choose the annotation type from the drop-down menu, and then enter descriptive text that relates to the desired test.
Note: Annotations are saved automatically.
The built in annotation types are :
- User Error Message
- Description Resolution
- Steps Custom
Add custom Annotation types by selecting the Annotation Types node, which then displays a list of all the annotation types:
From here you can add new annotation types or delete custom added ones. Select Sync to synchronize the list of annotation types with the configured SentryOne Test Server.
Note: The built in annotation types cannot be deleted.