SentryOne Test Visual Studio Extension Troubleshooting

EOL: SentryOne Test reached its end of life date on June 15, 2022. See the Solarwinds End of Life Policy for more information.

The following is a list of situations and their solutions that you may experience when using SentryOne Test: 

Files Showing as XML

If the .legitest file is opening as XML, then you need to change the exe with which it's being opened. To do so, right click the .legitest file and select open with... to display the Open With-Testsuite.legitest window. Select LegiTest Files, and then set it as the default.

SentryOne Test Open With window

Note:  If the .legitest file is already open, you may need to close and re-open for the changes to be applied.

Mixed Mode Assembly Error

If a test fails with the exception: 

  • Mixed mode assembly is built against version 'v2.0.50727' of the runtime and cannot be loaded in the 4.0 runtime without additional configuration information 

Then a change to the vstest.executionengine.x86.exe.config file is needed. Complete the following steps to make the change: 

  1. Close Microsoft Visual Studio 2012 (and IIS express).
  2. Launch notepad.exe as an administrator.
  3. Open the file C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.executionengine.x86.exe.config.
  4. Add and save the following lines in bold:
<?xml version="1.0" encoding="utf-8" ?><configuration><runtime><legacyUnhandledExceptionPolicy enabled="1" /><assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1" ><probing privatePath="Extensions" /></assemblyBinding></runtime><system.diagnostics><switches><add name="TpTraceLevel" value="0" /></switches></system.diagnostics><appSettings><switches><!--<add key="ExecutionThreadApartmentState" value ="MTA"/>--> <!--<add key="TraceLogMaxFileSizeInKb" value ="10240"/>--> <!-- MsTest Adapter Specific AppSettings --> <add key="TestProjectRetargetTo35Allowed" value="true" /></switches></appSettings> <startup useLegacyV2RuntimeActivationPolicy="true" ><supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.0" /></startup></configuration>

Adding a NuGet Package Reference

You may find that you need to add a NuGet package reference to your project. For example, when switching a test project from MSTest to NUnit you would need to add a reference to the NUnitTestAdapter package. To do this, expand the project node in Solution Explorer, right click on the References node and then select on Manage NuGet Packages...
SentryOne Test Manage NuGet Packages
In the dialog that displays, select Browse, and then type the name of the package you wish to install. Select the package you want when the search results are returned and then select Install in the right-hand pane. The NuGet package is then installed in the project.

SentryOne Test Install NUnitTestAdapter

Ordering Data Sets in Low Memory Mode

When extracting data sets for comparison, they must be orderable. Small data sets are no problem, because they can be stored in memory and ordered simply. For larger data sets you can use low-memory mode which stores the data in a file on disk. However, data sets in low-memory mode are required to be ordered when they are produced. Most of the time a simple ORDER BY clause allows us to order the set in the same order as the comparison keys. 

When using a string as a key, issues can arise from the fact that SQL orders strings according to the collation defined on the source. SentryOne Test uses binary collation, so the order by clause has to be specified with a binary collation. For example:

SELECT * FROM [Person].[Person]ORDER BY [LastName] COLLATE Latin1_General_100_BIN

Important:  It's important to note that the columns specified in the ORDER BY clause must match the keys specified in the comparison manifest, and in the same order. If your comparison manifest has the keys LastName, FirstName, and Age, then the ORDER BY clause must also specify LastName, FirstName, Age.

Migrating from Suites to Groups

Starting with the 2017.4 release, the concept of individual groups was added to SentryOne Test, and is now the preferred method of implementing tests. There are several advantages to this approach:

  • Because a smaller set of tests exists in a single file, conflicts between developers are reduced
  • Similarly, source control is easier to manage
  • Assets can be created at the project level and shared more widely
  • Parameters become project level concerns and are easier to manipulate across a wider range of functions

To migrate to individual groups, right click on a SentryOne Test suite within solution explorer and then select Migrate to individual groups...

Important:  It is always a good idea to ensure you have a backup before performing any sort of migration.