Note: This is an archive of the help guide for LegiTest Server. Please upgrade to SentryOne Test for the latest product.
See the Upgrading from LegiTest to SentryOne Test FAQ for current LegiTest and Workbench customers and Migrating from LegiTest to SentryOne Test video to learn more about the upgrade process.
About LegiTest Server
LegiTest Server gives you an easier glimpse at your tests and the results of the tests. You can drill into any given element to detect where a failure may have occurred. LegiTest Server also allows you to integrate test runs and alerts with the Notification Engine.
Release Notes
An archive of the release notes for LegiTest Server may be downloaded here as a PDF.
Installation
Workbench Server is the main application that LegiTest Server is hosted within.
Note: Features only become available if you have purchased a license for the product you are trying to use.
1. After ensuring you meet the Hardware and Software Requirements, open the installer.
2. Uncheck the box Send feature usage statistics to SentryOne if you don't want to send feature usage statistics. Read the License terms and conditions, then select I agree to the License terms and conditions. Select Next to continue the installation.
Note: The Use previous installation settings option helps you upgrade from older versions by attempting to gather as much of the previous installation's settings as possible. For example, the application pool identities may need reconfiguring.
3. Review the itemized list of components that Workbench Server will install and the total space required on the Select Components page. Select Next to continue the installation.
4. Configure the website where Workbench Server will be accessed. The following configuration options are available:
Configuration Option | Description |
---|---|
Workbench Server URL | The URL that you enter in a browser to access Workbench Server. Note: If you want to associate Workbench Server with a different port number than the industry standard port 80 (or 443 for https), include the port number using the following syntax: http[s]://<ServerName>[:<Port>]/ |
Run application pool as | The user account that the website uses for accessing the database, server files, and resources. Note: The account should already have access to the database(s) and server(s) that you want to access and document. The account should have database access that includes the ability to create, update, and delete tables, stored procedures, and other database objects. The following accounts are selectable:
|
Select Next to continue the installation.
Important: The user account that Workbench Server Website's Application Pool is authorized to use, dictates the option that's used for BI xPress Server Website and DOC xPress Server Website's Application Pools. If the websites are authorized to run as a specific user, unique users can still be selected for each Application Pool.
If Workbench Server needs access to services, databases, or assets on any other computer than the machine where Workbench Server is installed, then a specific domain user must be used by the Application Pool Identities and the DOC xPress Data Handler Service.
5. Enter the connection information for the database that's used to store the user and administration information. Select Next to continue the installation.
Important: If you want the User and Administration Database located on a different system than the Workbench Server website and the setting Windows Authentication, then the Workbench Server Application Pool configured in Step four must be run using a domain user account.
6. If you selected User specified below for the Workbench Server Application Pool, then you can enter a different User name and password. If you did not select User specified below, select Next to continue the installation.
7. Enter the connection information for the database that's used by BI xPress Server within the Monitoring Console. Select Next to continue the installation.
Important: The BI xPress Application Pool configured in Step 6, and the Workbench Server Application Pool configured in Step 4, must be running a domain user account if you want the BI xPress Database located on a different system than the Workbench Server website.
8. If User specified below was selected for the Workbench Server Application Pool, then you can enter a different User name and password. If User specified below was not selected, select Next to continue the installation.
9. Review the settings for how the DOC xPress Server meta data is distributed. Select Next to continue the installation.
Important: The DOC xPress Application Pool configured in Step 8 must be run using a domain user account if you want the DOC xPress Database located on a different system than the Workbench Server website.
10. Single Database: If you want to store all of the meta data in a single DOC xPress Server database, select Use a single database for DOC xPress Server storage. Enter the connection information for the database used to store the DOC xPress Server meta data information. Select Next to Continue the installation.
![]() | ![]() |
Multiple Databases: If you want to store your meta data in up to four separate databases, select Use up to four separate databases for DOC xPress Server. Enter the connection information for the databases that used to store the DOC xPress Server meta data information. Select Next to continue the installation.
Configuration | Description |
---|---|
Command Query Store | The DOC xPress metabase that DOC xPress Desktop Application users connect to in order to manage and create snapshots and data dictionary categories. |
Dictionary Query Store | An optimized database used to store Data Dictionary content for use within DOC xPress Server. |
Generator Query Store | An optimized database used to store metadate regarding DOC xPress Server generated documentation. |
Browser Query Store | An optimized database used to store the structure of objects for use within DOC xPress Server. |
![]() | ![]() |
Important: For management or scalability reasons, all database information can point to the same database, shared databases between any number of DOC xPress Server database(s), or even separate databases for each database required by DOC xPress.
11. Configure the DOC xPress Data Handler Service and the service login settings. Select Next to continue the installation.
Important: The DOC xPress Data Handler Service runs in the background and performs all of the DOC xPress document generation and processing. This service runs after installation, and when the computer starts up by default.
If the DOC xPress Data Handler Service is set to run as a specific user, that user must be contained within the "Log on as a service" local security policy.
If documentation is not being generated, make sure that the service is running, and that the user account running the service has access to create, update, and delete tables, stored procedures, and other database objects within the DOC xPress Server database(s) that were created in Step 10.
12. Configure the server paths to the server directories. The following configuration options are available:
Directory Configuration | Description |
---|---|
DOC xPress Templates Directory | This directory that contains server templates that are used to lend form and reason to the output in DOC xPress Server. |
DOC xPress Output Directory | This is the directory that contains all of the documentation that is generated by DOC xPress Server. |
CHM Compiler Application Path | This is the path to the CHM Compiler that is installed on the server. |
Select Next to continue the installation.
Important: Currently, DOC xPress Server only supports local folder locations for the DOC xPress Output Directory. Without the HTML Help Compiler application (hhc.exe), generating CHM documentation will fail.
13. Enter the connection information for the database that's used to store the LegiTest Server test suite history and assemblies. Select Install to begin the installation.
Important: Selecting Install downloads and installs the necessary software prerequisites. After installing the software prerequisites, Workbench Server begins installing the necessary software. After installation has completed, it may take several minutes while the server self configures before the website becomes operational.
Allowing IIS Access
After the Workbench Server installation has completed, the administrator must allow users and groups within IIS to access the homepage of Workbench Server. Allow access by completing the following steps:
Note: Control over users is handled within Workbench Server's administrative area.
- Open IIS, right-click on the Workbench Server site, and then select Edit Permissions.
- Navigate to the security tab, then select Edit to open the Permissions for Workbench Server Host window.
- Select Add to open the Select Users or groups window.
- Add any individual users and groups needing access to Workbench Server's homepage. Select OK for all of the remaining dialogs to save your changes.
Hardware Requirements
Determine your hardware requirements based on your environment's deployment model. LegiTest Server needs access to several databases to function. This creates the following two deployment scenarios:
Single Machine
In the single machine deployment model, the same machine has both Workbench Server and the databases installed. Workbench Server is the web application that contains LegiTest Server and its associated assemblies while the databases are hosted within a SQL Server Instance.
The following memory and processor requirements apply to all editions of LegiTest Server:
Component | Minimum Requirements | Recommended Requirements |
---|---|---|
Memory | 4 GB | At least 8 GB and should be increased as the LegiTest database size increases to ensure optimal performance. |
Processor Speed | 1.4 GHz | 2.0 GHz or faster |
Processor Type | Not Applicable |
|
Multiple Machines
In the multiple machine deployment model, Workbench Server and the databases are installed on separate machines. Workbench Server is the web application that contains LegiTest Server and its associated assemblies while the databases are hosted within a SQL Server Instance. It is important to differentiate between the Server machine and the Database machine:
- Server machine - the machine where Workbench Server is installed.
- Database machine - the machine containing a SQL Server instance where the LegiTest database is deployed.
The following memory and processor requirements apply to all editions of LegiTest Server:
Server Hardware Requirements:
Component | Minimum Requirements | Recommended Requirements |
---|---|---|
Memory | 4 GB | 6 GB |
Processor Speed | 1.4 GHz | 2.0 GHz or faster |
Processor Type | Not Applicable |
|
Database Hardware Requirements:
Component | Minimum Requirements | Recommended Requirements |
---|---|---|
Memory | 4 GB | At least 6 GB and should be increased as the LegiTest database size increases to ensure optimal performance |
Processor Speed | 1.4 GHz | 2.0 GHz or faster |
Processor Type | Not Applicable |
|
Software Requirements
Before installing LegiTest Server, make sure you meet the following system requirements:
- Internet Information Services (IIS) 7 or later with:
- Dynamic Compression installed
- Windows Authentication installed and enabled
- .NET 4.5.1 or later installed
- PowerShell 3.0 or higher installed
- Access to SQL Server 2008 or later instance
Note: LegiTest Server needs access to a SQL Server 2008 or later instance to install the following database:
- Workbench Server User and Administration database
- LegiTest database
The following operating systems are supported by LegiTest Server:
64-Bit (x64) Operating Systems: | 32-Bit (x86) Operating Systems: |
---|---|
|
|
|
|
|
|
|
|
| |
|
The following optional prerequisites are required to access some resources using LegiTest Server:
- Scheduling Test Assemblies
- MSTest Suites: Microsoft Visual Studio 2013 or Agents for Microsoft Visual Studio 2013
- NUnit Test Suites: NUnit Test Console
- Any software requirements (SSIS, SSAS, et cetera) used by LegiTest needed to execute a deployed test suite
The following browsers are supported by LegiTest Server:
- Mozilla Firefox 29 or later
- Internet Explorer 8 or later
- Google Chrome 35 or later
Note: JavaScript must be enabled within the Web Browser for proper viewing of LegiTest Server.
Software Requirements for other Workbench Server products
The following optional prerequisites are required to access some resources using BI xPress Server:
Security
- Windows User account(s) that can be used to access databases, servers, and documentable objects within the user's organization
- SQL Server User account(s) to use for accessing BI xPress Server databases
Auditing Framework
- Each version of SQL Server Integration Services that the user wishes to apply the auditing framework to.
- Any Third-Party Products (Connection Managers, Tasks, Components, etc.) that are used within the SSIS Package(s)
The following optional prerequisites are required to access some resources using DOC xPress Server:
Security
- Windows User account(s) that can be used to access databases, servers, and documentable objects within the user's organization
- SQL Server User account(s) to use for accessing DOC xPress Server databases
Documentation
- CHM
- HTML Help Workshop and Documentation
The following optional prerequisites are required to access some resources using DOC xPress Visual Studio Extension:
- Any edition of Microsoft Visual Studio 2012
- Any edition of Microsoft Visual Studio 2013
- Any edition of Microsoft Visual Studio 2015
The following optional prerequisites are required to access some resources using DOC xPress Remote Agent:
64-bit Operating Systems: | 32-bit Operating Systems: |
---|---|
|
|
|
|
|
|
|
|
| |
| |
| |
|
Providers:
Hive:
64-bit Operating Systems: | 32-bit Operating Systems: |
---|---|
|
|
|
Oracle:
64-bit Operating Systems: | 32-bit Operating Systems: |
---|---|
|
|
|
|
| |
|
SQL:
64-bit Operating Systems: | 32-bit Operating Systems: |
---|---|
|
|
|
SSAS:
64-bit Operating Systems: | 32-bit Operating Systems: | SSAS Azure Analysis Services: |
---|---|---|
|
|
|
|
SSIS:
- Each version of SQL Server Integration Services (2005 or above) that the user wishes to snapshot.
- SQL Server Data Tools or Business Intelligence Development Studio.
SSRS:
- Each version of SQL Server Reporting Services (2005 or above) that the user wishes to snapshot.
- SQL Server Data Tools or Business Intelligence Development Studio.
- Support provided for the following report locations:
- File System: SQL Server 2005 and above
- Native Web Services: SQL Server 2008 R2 and above
- SharePoint Web Service: SQL Server 2008 R2 and above
Informatica PowerCenter:
If connecting to a SQL Repository:
64-bit Operating Systems: | 32-bit Operating Systems: |
---|---|
|
|
|
64-bit Operating Systems: | 32-bit Operating Systems: |
---|---|
|
|
|
Required Permissions
Important: Many of the Application Service accounts need db_owner. These accounts issue updates to the database that may alter the schema of the database.
The following is a comprehensive list of each account and their required permissions:
Application Pool Identities:
Workbench Server Application Pool:
- User and Administration database:
- db_owner
- LegiTest Server database:
- db_owner
- LegiTest Command Line Execution and Task Scheduler:
- Local Security Policy:
- Log on as batch job
- Any permissions (databases, report servers, file system, et cetera) required to successfully run a LegiTest Test Suite deployed to the server
- Local Security Policy:
BI xPress Application Pool:
- BI xPress database:
- db_owner
- SSIS Monitoring Console:
- For importing package layouts from SSIS Catalogs:
- db_datareader
- For importing package layouts from SSIS Catalogs:
- SSRS Monitoring Console:
- SSRS ReportServer database(s):
- db_datareader
- SSRS ReportServer database(s):
DOC xPress Server Application Pool:
- DOC xPress database(s):
- db_owner
DOC xPress Data Handler Service:
- DOC xPress database(s):
- db_owner
- Local Security Policy:
- Log on as a service
DOC xPress Remote Agent Required Permissions
- Local Security Policy:
- Log on as a service
- Technology specific permissions:
- Hive
- Read permissions for the HDFS files that you wish to document
- Informatica
- View definition permissions on the Informatica repository
- Read permissions within the Informatica repository on any folder and object you wish to document
- SQL Server
- Definition permissions for any object you wish to document
- SQL Server Analysis Services Multidimensional:
- Read definition permission on each database object you wish to document
- SQL Server Analysis Services Tabular:
- Administrator permission on each database object you wish to document
- SQL Server Integration Services
- SSIS Catalog:
- Read and Modify permission for each project within the SSISDB the user wishes to analyze
- SSIS Catalog:
- SQL Server 2008 R2 (or higher) Reporting Services
- Folder Security Settings:
- Minimum Predefined User Roles required:
- My Reports
- Content Manager
- Minimum Tasks required for a custom User Role:
- View Reports
- View Folders
- View Data Sources
- View Models
- Consume Reports
- Minimum Predefined User Roles required:
- Web Service Security Settings:
- Minimum Predefined System Role required:
- System User
- Minimum Tasks required for a custom System Role:
- Execute Report Definitions
- Minimum Predefined System Role required:
- Folder Security Settings:
- Oracle Permissions
- Connect permissions on any schema that you wish to document
- Tableau Permissions
- User must be a part of the Server Administrator role
- Hive
Managing Your Server
Manage your Workbench Server and DOC xPress Server with System Settings. Open System Settings by selecting the gear icon at the top right of the navigation bar.
The system settings are divided into different pages:
Roles
Within Workbench Server's administration page, the Roles page allows you to create, edit, or delete roles that dictate the actions users can take, and the securables they can view within Workbench Server.
Viewing Role Members
View the users within a selected role by selecting the role category you want to manage.
Note: Selecting a role such as Administrators displays a list of all users assigned to that role, their display name, date joined, and last login time. Within the selected role page, you can edit the role assignment for any selected users.
Adding Users to Roles
Add users to a specific role by completing the following steps:
- Open the desired role category.
- Select Add users to (role) to extend the pane and display the user search bar.
- Enter a username and then select the search button, or scroll through the user list and select a user.
- Select the checkbox next to the desired user, then select Add users to add the user to the selected role.
Removing Users from Roles
Remove users from a specific role by selecting the check box next to each user that you want to remove, and then selecting Remove users from (role).
Creating a New Role
Create a new role, by completing the following steps:
- Open the Roles page, then select Create new role to add a new row within the Roles table.
- Enter a name for the role, then select whether to make this the default role to assign new users.
- Select Update to add the role to the list of roles.
![]() Add Role name and select Update
| ![]() New Role added
|
Editing or Deleting a Role
Edit a role by completing the following steps:
- Select edit for the role you want to edit to open the naming box.
- Make the desired changes to the name, and select whether you want to specify the role as the default role.
- Select update to save your changes.
Delete a role by completing the following steps:
- Select delete for the role you want to delete to open the warning prompt.
- Select delete to confirm the deletion.
Note: Select Cancel to stop the deletion of the role.
Users
The User page displays a list of all current users that have accessed Workbench Server. Administrators can use the User page to manually add domain users to Workbench Server or delete users from Workbench Server.
Note: Workbench Server automatically adds users once they open the server URL by default.
To delete a user, select delete next to the desired user. Select Ok to confirm the deletion
Note: Select cancel to stop the deletion of the user.
Administrators can manually add users to the server using the Add Domain Users button.
From here, administrators can filter and select any domain user they want to include.
Actions
The actions page provides a list of predefined actions for each of Workbench Server's Features that detail what each role can perform. The main actions page lists the actions, the feature it pertains to, and a description of that action.
Select an Action Name to open that action's page. On the Action page you can add or remove roles assigned to the action.
Note: By adding a role to an action, you are granting the role permission to perform the action. Removing a role from an action prevents the role from performing the action.
Adding Roles to Actions
Add roles to a specific action by completing the following steps:
- Select the desired action, then select Add roles to to extend the pane.
- Use the Search bar to search for roles to add to the action. If you do not type anything into the search bar, you can search a list of all roles that have been created on Workbench Server.
- Select the check box next to the desired role, then select Add roles... next to the search bar.
Success: A message displays confirming the role was added and the role appears in the list of roles associated with that action.
Deleting Roles from Actions
Select the checkbox for the desired role, then select Remove roles from to remove a role from an action.
Note: Selecting Remove roles from displays a confirmation message, and the role will no longer be in the list.
Role Based Security
Below is a list of all configurable permissions that can be used in LegiTest Server:
Permission | Description |
---|---|
Create Coverage Reports | Creates a coverage report. |
Dashboard | Displays a quick glimpse of your test results. |
Delete Assemblies | Deletes pushed assemblies. |
Delete Coverage Reports | Delete coverage reports from the server. |
Edit Annotation Types | Edit annotations types on the server. |
Edit Testable Paths | Edit testable paths for system coverage |
Manage Parameters | Edit assembly parameters. |
Manage Tag Permissions | Manage permissions based on roles and tags. |
Manage Test Item | Access the test management page. |
Run Hierarchy | Access the test run hierarchy. |
Run History | Access the run history of a test item. |
Run Tests | Run tests from LegiTest Server. |
Schedule Tasks | Schedule a test suite to be ran automatically from the windows task scheduler or to edit an existing scheduled task. |
System Coverage | Access the System Coverage page and view a list of available reports. |
Tag Items | Tag a test item. |
Untag Items | Untag a test item. |
Update Annotation Message | Edit the annotation message on the server. |
View Coverage Report | View an available Coverage Report. |
View Run Details | View specific details and results of a test run. |
Securables
Important: The Securables page is not applicable within LegiTest Server.
Within Workbench Server's administration page, the securables page gives you a more granular level of control over what each role can see within a DOC xPress solution.
Workbench Features
The features page displays all of the installed Workbench Server products. Administrators can do the following:
- Allocate users
- See the version (whether it's a trial)
- Disable or enable beta features
Select a feature to view all of the users that are currently licensed for that particular feature.
Note: Workbench Server hands out licenses to users who log in until all user-licenses have been consumed by default.
Note: Features that are in beta are disabled by default. Enable beta features for general use within Workbench Server by selecting Enable next to the desired feature.
Adding Users to a Feature
Administrators can add users to a specific feature by completing the following steps:
- Select Add users to next to the desired feature to extend the pane and display the search bar.
- Search for a user with the search bar, or leave the search bar blank to search a list of all users with access to Workbench Server.
- Select the check box next to the desired user name and then select Add users next to the search bar to add the user.
Important: When new features are installed to your Workbench Server server, users are automatically added to the new feature from the Default Role. If no role is currently set as the Default Role, users from the Administration Role are added automatically instead.
Removing Users from a Feature
Remove users from a specific feature by selecting the check box next to each user that you want to remove and then selecting Remove users from.
License
The license page displays all licenses associated with Workbench Server. View the license name, issue date, expiration date, maintenance expiration date, number of users, and if it's a trial. Users can also add a license, view details of a license, or update a license from this page.
Important: Users who have been assigned a role that is a part of the Administration action have access to the Web Administration portion of Workbench Server, where License information is found.
Automatically Activating a License
If you already have an activation key and the server where Workbench Server is installed on has Internet access, then you should be able to automatically activate Workbench Server. Complete the following steps to automatically update your license:
- Open the License page, then select Add a new License to begin the process of automatically activating your installation of Workbench Server.
- Enter your Activation key in the text box and then select activate online to automatically activate your license of Workbench Server Feature.
Manually Activating a License
If you require the assistance of the SentryOne's Support Staff to receive your license, or the machine where you installed Workbench Server does not have Internet access, you may need to manually activate your license of Workbench Server Feature. Complete the following steps to manually activate your license:
- Open the License page, then select Add a new License to begin the process of automatically activating your installation of Workbench Server.
- Select activate manually. Copy the Environment Identifier, Environment Key, along with your Activation Key, and e-mail the information to SentryOne Support.
- Once a SentryOne Support Representative has provided you with your license block, paste the license block into the license block text box and then select apply license to manually activate your license for the currently selected Workbench Server feature.
License Detail
Selecting a license's details button provides you with more information about the license currently activated on the server. Update (automatically), manually update, or deactivate your license on the license details page.
Deactivating a License
If you plan to migrate Workbench Server from one machine to another and have already activated your license on the old machine, you need to deactivate your license on the old machine. Deactivate your license by completing the following steps:
- Open the License page, then select details next to the corresponding feature that you want to deactivate.
- Select deactivate at the bottom of the page to begin the deactivation process.
- Within the confirmation text box, type YES and then select the remove license to confirm deactivation and remove the license from the machine.
LegiTest Server Features
The Dashboard is the first view that displays when you enter LegiTest Server. The Dashboard view allows you to navigate through all tests starting from the very top, the containing assembly, to the very end of any given test. The Dashboard view shows a glimpse of the amount of tests that failed or succeeded in their last run. You can also use the Dashboard to set where the notifications go for these items.
Use LegiTest Server to do the following:
- Use the LegiTest Visual Studio Extension to create test suites.
- Configure test suites to publish results and assemblies to a LegiTest Server.
- From the server, configure and run published test suites. These scheduled test executions will publish their results back to the server.
Summary
With a quick glance at the name of the test object, you can see its status from the last run by looking at the symbol next to it. On the right side of the table, Pass/Fail data from the last run is also summarized so you can see how many items passed or failed.
Selecting the icon to the far right allows you to add a notification to the BixPress Server's Notification Engine.
When you select a test object you will be brought to run history page, unless the item you select is a Test. Selecting a Test opens the details page of the first failed assert if applicable, or the run details page if the Test is part of a Data Driven run. This allows you to quickly see all tests run while data driven.
Get more details on any test run by selecting on the run in the graph or the data table. The run detail page shows the entire test tree from the point selected. You can quickly see what items passed or failed from this page for that particular run.
If an item has more detailed information to view, such as an assert, or an error for that particular step was thrown, a View Details link becomes accessible for that item. Viewing a completed assert takes you to a page showing what the assert was comparing, it's actual and expected result, and if it had passed or failed. This view also shows you any parameters that were used if it was a data driven run.
Viewing a failed item allows you to view the error thrown as to get further insight into what went wrong with the test step.
Viewing unexecuted assemblies:
The summary page has two views. The first (and default) is the Summary view. The second view is the Unexecuted Assemblies view. This view lists all assemblies that have been pushed to the server, but have yet to be executed. You can execute these tests from this view, and once complete, their results are pushed over the Summary view and removed from the Unexecuted Assemblies view.
Notifications
The Quality Dashboard can also be used as a shortcut for adding a notification to the Notification Engine that's triggered by a test object. Complete the following steps:
- Select the icon on the far right of the desired test to open the Add and remove notification menu.
- Select add notifications to open a prompt for enrolling in notifications.
- Enter the name of the notification, the event name that triggers the notification, and the severity of the notification. Select enroll to enroll the test object in notifications.
Tagging Test Objects
LegiTest Server allows you to add custom tags to any test object on the Dashboard. Tags are searchable and enable you to show all objects tagged with the desired filtering tag.
Use the test management page for the desired test object to tag a test object.
Filtering By Tag
To filter the summary page and show only items by a specified tag, do one of the following:
- Select any visible tag on the summary page.
- Use the search bar on the summary page.
Filtering by tag displays all tags matching the specified search term and allows you to select any tag in the search results.
Once a tag is selected, you can see which tag is being displayed under the search bar. Select Remove Filter to remove the applied filter.
Viewing Grid Asserts
If an assert contains a grid comparison or validation, any errors are pushed to the server for you to view as a grid. Failed rows are highlighted in red, and rows that are not equal but within a set tolerance are highlighted in yellow.
You can export the table to Excel by selecting Export to Excel above the desired grid. Selecting Export to Excel saves the table as an .xls file that can be opened in Excel.
Note: You may see the following warning: "The file format and extension don't match". The file will open fine as long as you select yes.
You can select Switch View to view the expected and actual grids side by side instead of as one grid.
Grid Assert Hyperlinks
You can add hyperlinks to help manage test(s) that the grid/validation assertion belongs to. For grid comparisons, you can add a hyperlink per side (actual and expected hyperlink.) Validation grids have only have one hyperlink.
To access the Edit Hyperlinks page, drill down to the test, then select the wrench icon.
Note: Selecting the wrench icon displays the Manage Test page and Edit Hyperlink options at the bottom.
Select Edit Hyperlinks to display the following configurable options:
Option | Description |
---|---|
Column Name | Identifies the name you want to call the column. |
Link | Creates the fully qualified link for you to select. The DotLiquid syntax can be used. |
Text | Identifies the text to be displayed for the hyperlink. DotLiquid may also be used to display a column. |
Note: Grid comparisons have one hyperlink per side (actual and expected.) However, validation grids only have one hyperlink. Both are configured in the same manner.
Additional Information: If DotLiquid is used for an object that does not exist, you won't see a return because it's an empty string. For more information about DotLiquid syntax, please see their page at http://dotliquidmarkup.org/
Deleting Tests
Move Assemblies to the trash view of the summary page by selecting the blue trash icon. Select the test objects that you want to delete permanently from the trash view.
After moving the assembly over to the trash, you are presented with the following two options:
Option | Description |
---|---|
Move Back | Move back moves the assembly and all children back to the summary page for standard viewing. |
Delete | Delete permanently deletes the test object and all of that object's runs. Note: This option is available on all test objects. Delete also remove all of that object's children and their runs. |
Empty Trash permanently deletes ALL test objects in the trash view, along with their runs.
Manage Tests
Select the wrench icon from the LegiTest Server Dashboard to open the Manage Tests page.
On the Manage Tests page, you can do the following:
Option | Description |
---|---|
View Run History | Opens the Coverage Tests page. This page displays runtime graphs and other information about past test runs. |
View Latest Run | Opens the Coverage Tests page. This page displays details and results from the last run of the selected test. |
Delete Assemblies From Server | Removes assemblies from the server and prevents future tests from running. You have to push the assemblies from your client. For more information about pushing assemblies, see Publishing Assemblies. |
Schedule New Tasks | Create, edit, and delete test runs. Select Schedule New Tasks at the top of the table to create a new task. Select the pencil icon to edit and the trash can to delete. |
Parameters | Parameters created in the LegiTest client are visible in this section. You cannot edit parameter names, but you can edit parameter content. Select Save after changing the information to save. Select Save and Run to save and run the test using the new configuration. |
Tags | Add and delete markers or Tags that add meaningful information to tests. To add a tag, enter the tag name into the text box labeled New Tag and select Add Tag. |
Notifications | Displays the LegiTest Server notifications configured in BIxPress's notification engine. Select the play icon to send the notification. |
Navigation
The navigation menu to the left displays the latest status for that test object at a glance. Select any of these links to open the management page for that item.
Note: The navigation menu only shows items within the same assembly.
Tagging
Tagging a test object allows you to label your test objects with custom tags. These tags can be used on the Summary page for filtering, and on the Tag Permissions page for setting permissions for individual items. In the Tag section, select from exisiting tags on the server using the dropdown menu, or enter the name for a new tag.
Server Testing
With LegiTest and LegiTest Server you can launch tests uploaded to the server from the LegiTest Quality Dashboard. Any uploaded test suite can also be scheduled to run on a schedule with LegiTest's command line.
Publishing Assemblies
Before running or scheduling tests from the server, you must push the test assemblies to the server.
Note: Ensure that you have added references to all used dlls if you want to run tests from the server.
To push test assemblies to the server, complete the following steps:
- Right click the project you want to push to the server.
- Select Publish to LegiTest Server to open the publish to server window.
- Specify how and where to publish the assemblies and results. The following configuration options are available:
Option | Description |
---|---|
Target | The target is an existing LegiTest Server such as http://localhost or it can be a folder on the system. Setting the target as a LegiTest server instance allows you to run the tests from the server. |
Results Mode | The result mode allows you to set all, some, or no suites to be published. The following options are available:
|
Results Server | The server where results are published. |
Configuration | Allows you to specify if the Release or Debug configuration should be pushed to the Target location. |
Run tests immediately after publish | Runs the pushed assemblies from the server once the assemblies are pushed to the server. If left unchecked, the assemblies are pushed to the server, but will not have any run associated until it's manually run from the server or client. |
4. Once these settings are selected and you select ok, the required parameters for publishing updates and the assemblies push to the server. This gives you a flexible way of choosing the location of all assemblies and results, and managing a project with multiple suites.
Publishing Assemblies with the LegiTest CommandLine
Push assemblies using the included LegiTest CommandLine tool. To do so, the following switches must be used:
Switch | Description |
---|---|
lh | The server in which to push the assemblies. |
ma | The path to the main assembly. This assembly will execute tests when ran from the server. |
af | The path to the folder that contains the assemblies to be pushed. All assemblies required by the main assembly to run successfully. |
Example:
lh "localhost" -ma "C:/ToPublish/LegiTestProject.dll" -af "C:/ToPublish"
Running Test Assemblies
LegiTest server allows you to run a LegiTest Assembly from the LegiTest dashboard. When you log onto LegiTest and view your LegiTest dashboard, the top level items marked as Assemblies have two icons beside its name. The the green play button launches the selected test suite. The green calendar allows you to schedule the assembly to be run at a later time.
Running a test assembly displays a notification prompt about the test status. Once the test finishes, a success or failure message displays based on the outcome of the test. If the test run is successful, you can see the run information in the test runs, and drill down into the assembly.
Command Line
If Workbench Server was not installed in the default location, or the installation is later moved, you need to update the configuration file for LegiTest Command Line. This configuration file is located by default at:
- C:\Program Files (x86)\Pragmatic Works\Workbench Server\LegiTest Command Line\PragmaticWorks.LegiTest.CommandLine.exe.config
The section that needs updating looks like the following example:
<appSettings>
<add key="NUnitTestRunnerLocation" value="%programfiles(x86)%\NUnit 2.6.3\bin\nunit-console.exe" />
<add key="MSTestRunnerLocation" value="%programfiles(x86)%\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" />
<add key="schtasks.exe" value="%windir%\System32\schtasks.exe" />
<add key="LegiTestCommandLine" value="%programfiles(x86)%\Pragmatic Works\Workbench Server\LegiTest Command Line\PragmaticWorks.LegiTest.CommandLine.exe" />
</appSettings>
The value for each respective key needs to be changed if any of the following scenarios occur:
NUnitTestRunnerLocation
- A different version of NUnit is being used.
- the installation of NUnit is moved.
MSTestRunnerLocation
- A version other than Visual Studio 2013 is being used.
- The installation of Visual Studio 2013 is moved.
LegiTestCommandLine
- The default location for Workbench Server is not used.
- The installation of Workbench Server is moved.
Command Switches
LegiTest Server can perform many actions with the command line tool. The tool is found in the directory C:\Program Files (x86)\Pragmatic Works\Workbench Server\LegiTest Command Line. Double-click the file PragmaticWorks.LegiTest.CommandLine.exe.
Actions:
Command | Description |
---|---|
-ArgsForServer, -la | Args of for server initiated run, used for server. |
-AssemblyName, -an | Name of Assembly to run from server. |
-DllLocation, -d | Location of the test suite assembly. |
-Help, -h | Displays the help text. |
-Host, -lh | Host server. |
-MsTestLocation, -ml | MSTest Location. |
-NUnitLocation, -nl | NUnit Location. |
-Server, -ls | Used by LegiTest Server Scheduler. |
-TestGroup, -tg | Specifies which Test Groups to run, separated by a comma. |
-UniqueIdForServer, -li | UniqueId of TestSuite, used for server. |
-MainAssembly, -ma | Used when pushing assemblies to the server. This will be the path to the executing assembly, and stored as the entry point on the server. |
-AssemblyFolder, -af | Specifies a folder containing the assemblies to push to the server. All files in this folder will be compressed and pushed to the server. |
Runners:
Command | Description |
---|---|
-MsTest, -m | Enable for MSTest. |
-NUnit, -n | Enable for NUnit. |
Scheduling Test Assemblies
Select the green calendar button by the desired assembly name on the LegiTest dashboard to run your Test Assemblies on a schedule. Selecting the calendar button opens the scheduling page. After you've set your desired schedule you will be brought back to the LegiTest Dashboard.
Option | Description |
---|---|
Task Name | The name assigned to the task within Window's Task Scheduler. |
Server | The name of the server where the scheduler will run. The default is localhost. |
User | The Windows user where the task is run. |
Password | The password for the Windows username supplied previously. |
Start Date | The first date where the task is run. |
Start Time | The time of day that the task runs in a 24 hour format. |
Interval | Set whether to run every hour, every day, every week, or every month. This interval starts from the supplied start date and start time. |
System Coverage
The System Coverage integration gives you visibility into the parts of your system that have been tested, and the status of those tests.
Note: System Coverage rules are set by the user and are not enforced by LegiTest.
Getting Started
To get started with the System Coverage feature of LegiTest, complete the following steps:
1. Open a LegiTest suite in Visual Studio, and navigate to the Server Integration node.
2. Enter the address of the server that contains the installation of Workbench Server to give access to the Doc xPress metabase, which is what the report will be built from.
3. After setting up your server, navigate to the System Coverage node and select a test.
Note: In this example we have a SQL Comparison, comparing two tables within the same database. The database being tested is called GridStuff, and the two tables are Grid1 and Grid2.
4. In the Doc xPress search section, we typed in GridStuff, because we want to specify that this specific test is responsible for representing the health of the GridStuff object. To do this, we highlight GridStuff, and select Add to add it to this test.
5. Once you have the items you want represented by this test, save the project. Make sure to select Push results on the server integration page. Once saved, run the test and then visit the server to create a report that includes this item.
6. Open the LegiTest Server dashboard to access the System Coverage on the server. Select System Coverage in the navigation bar to open the Reports dashboard.
Note: The Reports dashboard displays any available reports in a list format. You can select any report you want to view.
7. For this example, select Create Report to open the create report page to begin creating a report.
8. Enter a report name. Use the search bar to search the Doc xPress metabase for items to add to the report. For this example, GridStuff was added, because we know that the GridStuff item was added to a test. This now becomes a report item in the report. When an item is added, the reporting for that item starts at the Path level for it, and then calculates downward. For this example we also add the \Databases path as a report item. Once you've added all of the desired report items, create the report.
The report page opens after the report is created. The top report item is the total of all items added to the report. The second graph shown is for the GridStuff item we added.
Note: Because the test failed that represented the GridStuff item, the entire path is considered failed.
The second report item, Databases, shows a percentage of failures, passes, and untested items.
Note: In tests that were not shown in the setup, some tests that tested the health of AdventureWorks passed, and we can see some passes for items under the Databases path, and failures, that represent the GridStuff test added for this example.
Graphs are a quick way to view the status of your environment. Select Tree view to switch to a more detailed drillable view. Drill into a path to see where these failures and passes occurred.
During report creation, you can choose which view to default to. By selecting the Default to Tree View option above the Create Button, the report opens to the tree view.
Search Testable Objects
Use the Search Testable Objects page to find the latest status of a testable object and avoid creating a report to view just a single item. The search function works similarly to the search on the Create Report page because it also searches the Doc xPress metabase. The search function enables you to find out if the object was tested, and if so, if it passed or failed, or if its previous test was inconclusive.
Edit Testable Paths
The Edit Testable Paths page allows you to manage which paths are tested. By default, most well known paths are added. If you need more paths to be tested, such as custom paths, you can add them on this page.
System Dashboard
Items from the LegiTest Server's system coverage can be added to the Workbench Server System Dashboard.
To add a LegiTest System Coverage tile to the System Dashboard, select System Dashboard at the top navigation bar to open the System Dashboard page.
Select an action icon on the dashboard pane to perform one of the following actions:
Action | Description |
---|---|
Information | Toggle your view of the text located between the Workbench Server and tool icons. |
Refresh | Updates the dashboard view with the most current data. |
Edit Dashboard | Opens the System Dashboard Settings page to configure the dashboard. |
You can rearrange the order of tiles by dragging and dropping them within the page.
System Dashboard Settings
System Dashboard Settings allows you to create, edit, and delete tiles to be used in the System Dashboard. To add a LegiTest System Coverage tile to the System Dashboard complete the following steps:
- Select Select tile to add to your dashboard... checkbox and select Legitest System Coverage.
- Select + Add Tile to open the LegiTest System Coverage configuration page.
System Coverage
In the Add New Tile - LegiTest System Coverage page, you can configure the following:
Option | Description |
---|---|
Name | Displays the name of the tile on the System Dashboard. |
Description | Enter helpful information for the tile. Descriptions are only visible in the System Dashboard Settings page. |
Tag Name | Filter the system coverage tests based on an assigned tag within LegiTest Server. |
Include Partial | Select Yes to display coverage of partial runs of a test assembly. Select No to display only the most recent full run. |
Frequently Asked Questions
Custom User Accessible URL
By default, Workbench Server's setup instructions require the machine name or the fully-qualified machine name to be used as the user accessible URL that will be used to access Workbench Server through the web browser.
Note: Additional configuration is required after installation if you want to use a custom user accessible URL.
Warning: The following configuration steps are designed for basic custom user accessible URLs within an intranet and should only be attempted by domain and web administrators. SentryOne is not responsible for any issues that may occur due to modifications to your organization's DNS.
Configure a custom user accessible URL by completing the following steps:
1. Follow the Setup Instructions for Workbench Server. Make sure the user accessible URL and host name are set to the default value as set within the installer. Everything else within the setup instruction can be customized for your environment.
2. Once Workbench Server has been installed, launch Internet Information Services (IIS) Manager on the host server.
3. Open the Connections panel, expand the Sites node and then select the Workbench Server site.
Note: Depending on the settings the user configured during installation, the site's name may be different then what is shown below.
4. In the Actions panel, select the Bindings... action to open the Site Bindings dialog box. Select Add... to open the Add Site Binding dialog box. Type the host header that you want to use to connect within the Workbench Server, and then select OK to continue.
5. Once all of the site bindings have been configured, select Restart within the Manage Website portion of the Actions panel to restart the Workbench Server site.
6. Connect to the intranet DNS Server once Internet Information Services (IIS) Manager has been configured.
7. Launch Server Manager on the DNS Server. Use the left pane console tree to expand the DNS Server name in the Server Manager snap-in. Double-click on Forward Lookup Zones.
8. Select the DNS zone with the host server that Workbench Server is installed on from the displayed zones list. Right-click the DNS zone and select New Alias (CNAME)... to open the New Resource Record dialog box.
9. Use the alias name configured previously and use the fully qualified machine name that Workbench Server was installed on as the target host, then select OK.
Success: Workbench Server should now be configured to use a custom user accessible URL.
Flushing the DNS Resolver Cache
In some rare cases, you may be unable to connect. In this instance, you need to flush your DNS Resolver Cache. To flush the DNS Resolver Cache, complete the following steps:
- Launch Command Prompt .
- Enter ipconfig /flushdns within the Command Prompt window and press Enter.
Managing User Account Information
Select the profile icon in the top right of the navigation bar, then select your username to open the Manage Account page. View your Role memberships (a list of roles assigned to you) and make updates to your profile. Makes changes to your display name or biography (if desired) then select save to update your profile information.
Note: A green saved message displays at the top of the screen when your account has been updated.
Multi-domain Considerations
Warning: SentryOne does not provide support nor do we suggest these scenarios. Almost all of the modifications require altering the operating system, Internet Information Services, and domain settings. Proceed with caution and always make a backup before modifying anything.
The methods described below are for advanced IT administrators who are comfortable with and have been allowed to alter any required settings and/or security protocols. These helpful hints are only for illustrative purposes and each environment has its own unique requirements and hurdles.
Workbench Server is a robust software suite that can span multiple assets within an organization, even those that lie on different Microsoft Windows Domains. Below are some helpful hints and recommendations for implementing a multi-domain installation.
Scenario | Description |
---|---|
Workbench Server and SQL Server Databases are in different domains | Hosting the Workbench Server within a domain other than the SQL Server databases can be accomplished using Windows Authentication or SQL Authentication, assuming that necessary firewalls and ports have been opened and allowed to communicate. |
Users and Workbench Server are in different domains | If you are in a different domain than the Workbench Server computer, then trust between domains needs to be established. |
Workbench Server / SQL Databases
Hosting Workbench Server on a computer that's in a different Microsoft Windows domain than the databases needed to store information can be accomplished by using Windows Authentication or SQL Authentication. Accessing machines across domains carries additional security risks that need to be taken into consideration before implementing this scenario.
Using Windows Authentication to access cross-domain databases
- Ensure that there's a domain trust between the two domains. The following Microsoft TechNet article explains: https://technet.microsoft.com/en-us/library/cc961481.aspx
- Open and secure any necessary ports for communication between Workbench Server and any SQL Server instance. Port 1433 is the most common port assigned to SQL Server.
- Complete the following while installing Workbench Server:
- The domain user used in the next two steps must belong to the domain that has trust. The relationship defined within the Domain Controllers Domain Trusts area will dictate which domain the user must be defined within.
- Ensure that the user installing Workbench Server has access to the other domain's database servers so that the installation SQL scripts can execute and create databases, tables, etc.
- Specify a user for the various application pools which has access to the other domain's database servers. This domain user will be used to run the website and will need to create, update, and delete database objects in the same manner as a single-domain installation.
Important: If the domain user installing Workbench doesn't have the appropriate rights on the other domain, installation may fail.
Using SQL Authentication to access cross-domain databases
- Open and secure any necessary ports for communication between Workbench Server and any SQL Server instance. Port 1433 is the most common port assigned to SQL Server.
- While installing Workbench Server, specify the SQL Server instance and database as usual although a static IP address will most likely be used instead of a friendly name.
- Be sure to include the user name and password as usual.
Users / Workbench Server
Additional Information: Hosting Workbench Server on a computer that's in a different domain than the users that will be accessing it presents some challenges in making sure that both domains have a trust relationship. The following Microsoft TechNet article illustrates Domain Trust and how to implement it: https://technet.microsoft.com/en-us/library/cc961481.aspx. Once a trust has been established between domains, the domain users that should be allowed access must be granted permission to access Workbench Server. This can be done in the Internet Information Services Manager
AlwaysOn Availability Group Considerations
Note: This isn't intended to be a guide for configuring an AlwaysOn Availability Group. This guide is assuming that an AlwaysOn Availability Group has already been properly configured and you want to add the Pragmatic Works databases to an existing availability group as availability database(s).
If you want to install Workbench Server database(s) to an AlwaysOn Availability Group as availability databases, you should consider the following:
Server Installation
Before beginning the installation process of Workbench Server, the users that will be set to run each of the application pools must have permission to connect to the availability group through the availability group listener.
Note: When installing Workbench Server with the intent of using an AlwaysOn Availability Group, the application pools must be set to run as a specific user and cannot be configured to run as either the Application Pool Identity or Network Services.
Once the user(s) that will be running the application pools have been granted permission to connect to the availability group through the availability group listener, installation of Workbench Server can begin. Installation of Workbench Server will follow the default setup instructions with several noticeable differences:
- Ensure that the Workbench Server application pool is set to run as the user that was originally configured to have permission to connect to the availability group through the availability group listener.
- Ensure that the User and Administration database is configured so that its server setting is set to point to the availability group listener.
- Ensure that the BI xPress Server application pool is set to run as the user that was originally configured to have permission to connect to the availability group through the availability group listener.
- Ensure that the BI xPress database is configured so that its server setting is set to point to the availability group listener.
- Ensure that the DOC xPress Server application pool is set to run as the user that was originally configured to have permission to connect to the availability group through the availability group listener.
- Ensure that the DOC xPress database(s) are configured so that their server settings are set to point to the availability group listener.
- Ensure that the DOC xPress Data Handler Service is set to run as the user that was originally configured to have permission to connect to the availability group through the availability group listener.
- Ensure that the LegiTest database is configured so that its server setting is set to point to the availability group listener.
Once Workbench Server installation has been completed, creation of the BI xPress, DOC xPress, LegiTest, and Workbench Server databases will occur automatically on the primary availability replica and then you can continue on to adding the databases to the availability group as availability databases.
Adding the databases to the Availability Group
Once the database(s) have been created, the database(s) can now be added as availability database(s) to the availability group. This is done in the exact same way a user would add any other database to an availability group:
- Within the database properties, ensure that the databases' recovery model is set to Full.
- Create a network share that each availability replica has permissions to access.
- Create a full backup of each database on the network share created previously.
- Once all of the databases have a full backup located on a network share, expand AlwaysOn High Availability within management studio's object explorer.
- Right-click on the availability group that the user wishes to add the database(s) to and select Add Database...
- Walk through the steps within the "Add Database to Availability Group" wizard, ensuring the database(s) selected meet all prerequisites. Select Full and specify the network share created previously as the location accessible by all replicas on the Select Data Synchronization page.
Once the BI xPress, DOC xPress, LegiTest and / or Workbench Server database(s) have been added to the availability group as availability databases, whenever you want to connect to the databases, you should connect through the availability group listener.
Cannot Find MS Test or NUnit Test Runner
When running tests from the server, you may encounter an error stating that the server can not find the MSTest or NUnit test runner. This can occur if you have the these items installed in non-default areas. To change where the LegiTest Server looks for these items, edit the config file found by default here:
C:\Program Files (x86)\Pragmatic Works\Workbench Server\LegiTest Command Line\PragmaticWorks.LegiTest.CommandLine.exe.config
The following sections in bold need to be updated to match the user's install location for MS Test or NUnit:
<appSettings>
<add key="NUnitTestRunnerLocation" value="%programfiles(x86)%\NUnit 2.6.3\bin\nunit-console.exe"/>
<add key="MSTestRunnerLocation" value="%programfiles(x86)%\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" />
<add key="schtasks.exe" value="%windir%\System32\schtasks.exe" />
<add key="LegiTestCommandLine" value="%programfiles(x86)%\Pragmatic Works\Workbench Server\LegiTest Command Line\PragmaticWorks.LegiTest.CommandLine.exe" />
</appSettings>
Addendum: LegiTest Visual Studio Extension
Note: The following two sections on system coverage and viewing reports relate to integrating the Visual Studio extension component of LegiTest with LegiTest Server. These features are not currently part of SentryOne Test.
LegiTest System Coverage
System Coverage displays objects that have been tested within a system. System Coverage has two modes of operation: DOC xPress integrated, and standalone.
DOC xPress Integrated
In DOC xPress Integrated mode, information about the objects available for tests comes from DOC xPress Server. Because there is a solution snapshot present, the system can tell you the items that are available to test. LegiTest Client displays the DOC xPress Metadata Search tool, and you can select items from your DOC xPress Server to mark as tested. There are four available categories of test output:
Test Outcome | Description |
---|---|
Passed | An item is covered by a passing test. |
Failed | An item is covered by a failing test. |
Inconclusive | An item is covered by a failing test. |
Not covered | An item is known in the DOC xPress Server database, but no tests that cover it have been run. |
Standalone
In standalone mode, there isn't any information about the objects available to test. In this mode, System Coverage is only able to show the first three categories of output (Passed, Failed, Inconclusive). The system collates data based on the identifiers you assign to individual items.
Setting up Coverage Information
When setting up system coverage there are two parts that must be configured:
- the binding of items to tests
- the configuration of viewable reports on the server.
In the client, selecting the System Coverage node in a test group displays the following:
On the right is the DOC xPress Metadata Search which allows you to find items to mark as covered. On the left is a tree that represents the content of the current group / suite. Entering a search term returns results from the server.
By selecting an item in the left hand tree and then double clicking the found item in the right hand pane, the item is bound to the test. Now when the test runs, that item is considered covered and has a Passed, Failed, or Inconclusive status depending on the test outcome.
LegiTest Viewing Reports
Select the Reports node under System Coverage to display system coverage reports from the Server in the LegiTest Client. Selecting the Reports node produces a list of the coverage reports available on the server.
Select a report, and then select View this report to display the graph view of the report. The graph view contains a summary graph, and a graph for each coverage report item that's configured.
Select View as tree to change the view to a detailed tree view that displays all the available objects, and their status.