SentryOne Test Release Notes

EOL: SentryOne Test reached its end of life date on June 15, 2022. See the Solarwinds End of Life Policy for more information.

For more information on the additional steps SolarWinds is taking to improve our security posture and policies, please reference this blog by SolarWinds President and CEO Sudhakar Ramakrishna.

2021

Version 2021.8 [July 15, 2021]

Notes
- There's an updated EULA for this version.

2020

Version 2020.23.0 [November 18, 2020]

Fixes

66306: Parameters from published assemblies not showing
- Note: Cloud version only

Version 2020.17.0 [August 25, 2020]

Notes

Default trial duration changed to 15 days

Version 2020.10.0 [May 20, 2020]

Note: This release is for the installed components listed below. See the Upgrading SentryOne Test article for instructions on downloading the latest agent and extension installer files.

Features

Remote Agent and Visual Studio Extension now support TLS 1.2 (for SentryOne Test Cloud version)

Version 2020.225.2 [May 5, 2020]

Fixes

Minor bug fixes

Version 2020.225.0 [February 26, 2020]

Notes

Item 62680: Update EULA

2019

Version 2019.4.6 [December 18, 2019]

Fixes

Fix 60960: Update EULA

Version 2019.3.1 [July 3, 2019]

New Features

Feature 53106: 2019 SSIS support in SentryOne Test

Fixes

Fix 53691: S1T VSX Project templates do not show up in the 'Test' category
Fix 53932: SentryOne Test: Using a parameter inside of another parameter will crash VS
Fix 53992: VS 2019 Community Edition Blue Theme fix for S1T

2018

Version 2018.5.1 [December 4, 2018]

New Features

New 42048: LegiTest Client now integrates with SentryOne Accounts system
New 42092: Copy-Paste Test functionality
- Users can now copy and paste tests from one group to another
New 44453: LegiTest Runtime now identifies hosts using their service identifiers
New 45510: LegiTest Visual Studio extension has been removed from Workbench Installer
- The LegiTest extension can be downloaded here:

Fixes

Fix 42033: LegiTest custom tool generates a warning on generation
Fix 42041: Row Count Recipe fails when row counts match but are from differently named tables
Fix 42047: REST assets in LegiTest cause project to reference Core assembly
Fix 42050: The LegiTest Recipes Console does not support relative paths for the targetFolder argument
Fix 42067: Comparison Manifest reports no key column being defined as an error
Fix 42082: Bulk Insert Asset ignores provided ID for identity columns
Fix 42083: Cannot build LegiTest project when name suffixed with ".NUnit"
Fix 42085: LegiTest Project Templates don't show up under the Visual C# \ Test category
Fix 42087: Grid Validation Manifest comparisons do not substitute parameters
Fix 42418: When adding multiple tests the naming is 'Test 1' and 'Test 1 (2)'
Fix 42419: The setting for 'Treat whitespace as NULL' is not persisted
Fix 42440: Server Settings are not retained
Fix 42441: Intellisense warns of unknown identifier on aliased columns used with UNION ALL
Fix 42445: Interactive Comparison's "Format SQL" button adds a semicolon to the end of the query, which throws an ORA-00911 error with Oracle Queries
Fix 42448: Comparison Editor status indicator doesn't change to indicate current status unless connection is changed.
Fix 42457: Deleting parameters that have already been mapped should ask user for confirmation
Fix 42512: BrowserEmulation is not set correctly if the key is not already present
Fix 42514: Opening Read Historical Result Wizard is causing Visual Studio to crash
Fix 42707: LegiTest picks up incorrect NUnit Test adapter
Fix 42779: Row Count Recipe failures no longer report the table name
Fix 43858: LegiTest throws an Exception when loading an SSIS package from an ISPAC file, then crashes Visual Studio
Fix 44289: Add a note to the LegiTest Recipe documentation that Oracle is not supported.
Fix 45423: Server parameters were being created in group-specific settings files
Fix 45767: ParameterProvider PrintDebugInformation is called before parameter files are loaded

Version 2018.2.3 [April 27, 2018]

Important: The 2018.2.3 release for LegiTest will be the last supported release for SQL Server 2008 and 2008 R2.

New Features

New 3772: Validation results now show up in the Visual Studio Errors tool window
New 6824: Data Profile recipe
- LegiTest now includes a validate data column profile recipe that can test minimum and maximum values, range checks, et cetera
New 12563: Dimension Validation recipe
- LegiTest now includes a dimension validation recipe to ensure a table adheres to standard dimension table rules
New 13468: Fixed Width Text File Asset now includes the capability to skip header rows
New 13504: Added support for generation of NUnit 3 Tests
New 15679: Added 'Send equal rows to results server' to Interactive Comparison
New 16104: Runtime settings used for each execution are now logged
New 16336: Added actual value to value comparison assertion error messages
New 16936: Range Check recipe now includes additional settings to include informational columns
New 16969: Interactive Comparison and Comparison Manifest includes ability to equate white space to NULL values
New 17415: Text Format Validation recipe
- LegiTest now includes a text format validation recipe that checks the validity of data within a specific column: Alpha Numeric, Email Address, North American Phone Number, Numeric, Social Security Number, Country Code (ISO 3166) and State Code (ISO 3166)

Updated Features

Update 8782: Improved messaging around header row parsing failures in flat file asset editors
Update 12417: Data set filtering now allows for NOT BETWEEN, IS NULL and IS NOT NULL
Update 16338: Validation manifests can now validate that values are NULL / NOT NULL

Fixes

Fix 4777: Within source controlled environments, renaming or moving items will not properly clean up generated resources
Fix 11670: Visual Studio Extension uninstaller displays multiple check boxes for the same version of Visual Studio
Fix 13113: Mouse wheel does not work to scroll packages in the Package Test Wizard on the Select Packages page
Fix 13505: Naming a test ValidationManifest results in a compilation error
Fix 15610: Data Validation wizard failed on long running queries
Fix 15694: Execute Report action and wizard do not have a timeout control
Fix 15699: Fail on server publish should be set for all recipes
Fix 15702: Object Reference Error thrown when NULL values are returned in MDX queries
Fix 15762: Table lists do not refresh when switching from an existing connection to a new connection
Fix 16045: Correct path to the custom Plussom assembly
Fix 16105: XPath extraction does not support XPath validation with namespaces
Fix 16284: Wizards that should create multiple groups fail to move one group
Fix 16337: ExecuteScalar on SalesForce does not return the correct result
Fix 16354: Unable to rename singe groups / suites unless renamed through solution explorer
Fix 16360: Query Editor throws an "InvalidOperationException" when attempting to type while the cursor is off screen
Fix 16938: LegiTest does not reinstall NuGet packages with unresolved references
Fix 16983: Adding key or comparison columns could add the wrong column in Interactive Comparison

Version 2018.1.4 [February 8, 2018]

New Features

New 5497: Validation and Comparison manifests now support regular expression matching and asymmetric tolerances
- A new match type of 'RegexMatch' can be used to match an expected regular expression pattern against an actual value
- Percentage and fixed tolerance comparisons can now be set to use asymmetric tolerance values for specific ranges
New 11057: LegiTest now auto selects resources / assets when possible
New 13437: When re-publishing an assembly that is in the recycle bin on the server the assemblies are removed from the trash
New 14542: Visual Studio's test output console now includes LegiTest parameter information
New 14663: LegiTest extensions now support initializing logging through external configuration files
New 14715: Added support for Recipes
- Recipes allow reusable test patterns to be quickly and easily turned into tests, allowing for tests to be created with very little knowledge of unit testing in general
- Several recipes are available - including a uniqueness validation, range check, standard deviation validation etc
New 15190: Added 'Value Comparison' assertion
- Value comparison replaces the 'Integer Comparison' and 'Scalar Comparison' assertions
- Easier configuration for validating a single scalar value
- Wider array of operators for comparison against multiple types
New 15191: Added 'Expression' assertion
- Expression assertion allows LegiTest filter expressions to assert on single or multiple scalar values
- Use complex combinational logic to verify resource values
- Familiar SQL-like syntax for comparison and logical combination through AND / OR
New 15671: Load Text File action
- The Load Text File action allows users the ability to load the content of any text file into a resource for later use

Updated Features

Update 14583: Help Documentation now includes a Frequently Asked Question including detailed code samples for one-time use parameter values

Fixes

Fix 13467: 'Move Up' / 'Move Down' moves all tests to the bottom of the list
Fix 13901: Exception messages in Group Setup and Teardown are difficult to troubleshoot
Fix 13972: Compiler warning in generated projects: Architecture mismatch between MSIL and x86
Fix 13973: LegiTest does not parse user defined parameter syntax in Data Driven test queries
Fix 13978: Enabling an empty data-driven source causes unclear compilation error
Fix 14562: Within Data Driven Source, the "Edit Filter" and "Edit Asset" buttons for File Handling sources do not do anything
Fix 14626: Validation Manifest configured with invalid grid provider crashes Visual Studio on column load
Fix 15116: Project Level Asset file doesn't get marked as dirty in VS after changes
Fix 15129: LegiTest Rest Connection/Query Headers do not replace parameter syntax
Fix 15134: Legitest: "Object cannot be cast from DBNull to other types" error in Legacy Test Suites
Fix 15178: Fixed Width File Reference asset throws "Could not detect file format value cannot be null. Parameter name: values"
Fix 15252: Tests targeting NUnit error with BadImageFormatException when run via LegiTest Runner
Fix 15629: Parameter replacement does not work within the filter of a Data Driven Directory Reference
Fix 15654: Project level connection assets with parameterized connection strings do not work in all scenarios
Fix 15657: Value Comparison fails when ResourceKey contains ( or )
Fix 15658: Comparison Manifest does not recognize global connection assets

LegiTest Server Version 2018.1.4 [February 8, 2018]

Updated Features

Update 14631: Data Driven Test parameters are now included in Notification Engine

Fixes

Fix 14019: LegiTest CommandLine does not capture console output
Fix 14563: Deleting results that includes a published assembly does not also delete the assembly
Fix 14724: Annotations are not transmitted correctly to LegiTest Server
Fix 15166: Notification Engine failures prevent LegiTest assembly run histories from updating
Fix 15187: LegiTest CommandLine doesn't respect spaces within assembly file locations
Fix 15194: Date Time Format for Scheduled Tests does not work in non-US locale settings
Fix 11144: When creating a System Status View, if the selected solution doesn't contain Lineage no message is presented to the user.

2017

Version 2017.4.7 [November 27, 2017]

New Features

New 5476: Improved multi-developer experience
- Assets can now be defined at the project level
- LegiTest suites can be migrated to individual group files
- Added ability to have single groups in single files, enabling better sharing
- Added project-level parameter files to simplify configuration across a project
- Added project-level server integration to enable simpler and more consistent deployment
New 9822: Reduced runtime dependency footprint for LegiTest
- Less DLL references are now required for some testing patterns
New 11951: Added error inspector for looking at warnings at the group/test level
- Warning tooltips are also enhanced with extra detail
New 13116: Runtime dependencies can now be resolved via NuGet
- Most of the runtime dependencies of LegiTest tests can now be resolved via NuGet, making deployment on build servers much easier
New 13435: Generated file names for asset content files are shorter
- Names of existing assets will be shortened upon opening the test suite / group that contains them.
New 13454: Added ability to define test framework type at project level
- Framework type is now defined at the project level, instead of for individual suites/groups
New 13480: Suites, Groups, Settings and Asset files now have appropriate icons
New 13502: Added ability to ignore tasks by name in SSIS Wizard
- Added ability to use wildcards to ignore tasks by name - e.g. *SSISOps*

Updated Features

Update 8753: REST assets now include a "Number of Retries" option that controls the amount of connection attempts upon failure
Update 11156: Added help overlay to the query editor
Update 13492: Improved warning message within SSIS Package Reference asset when pointing to a newer SSIS version

Fixes

Fix 12021: DOC xPress connector and System Coverage panel give no warning when server is unlicensed
Fix 12414: Comparison for Validation Manifest are run in reverse of what the UI displays
Fix 13060: Package Reference Asset throws an SSISCatalogPackageNotFoundException error when a folder is selected
Fix 13072: ReadHistoricalResultsAction can display an interactive dialog during test execution
Fix 13093: LegiTest does not publish to server all needed assemblies when using SSIS Packages Reference
Fix 13098: LegiTest suites / groups do not function correctly in project sub-folders
Fix 13100: LegiTest tests will fail on a data driven run if the server fails and 'fail on server failure' is not checked
Fix 13104: Renaming a Suite or Group does not delete the old output folder
Fix 13120: Actual and Expected values are being flipped on integer comparison when pushing results to server.
Fix 13458: New "LegiTest Project - NUnit" projects do not include a NuGet reference to NUnitTestAdapter
Fix 13935: Within Connection Asset, clicking the Test Connection throws InvalidOperationException if Connection String is empty
Fix 13936: "The type or namespace name 'Dictionary<,>' could not be found" when generating a LegiTestGroup without any tests
Fix 13987: "Member names cannot be the same as their enclosing type" when a test is named the same as its group
Fix 13993: Publishing a Data Driven test that uses a Delimited Content asset misses publishing certain assemblies
Fix 14000: Data Driven tests containing nullable values can throw ArgumentException
Fix 14033: SSIS Package Wizard throws an InvalidDataException when selecting a package that has a higher SSIS version
Fix 14551: Interactive Comparison throws "Comparison no longer exists" error when mapping key columns with different names

LegiTest Server Version 2017.4.7 [November 27, 2017]

New Features

New 13054: Workbench Server documentation now includes required permissions

Updated Features

Update 11979: Online help documentation now includes a rolling list of release notes that include all product release notes within a single file

Fixes

Fix 9260: Report Totals do not show at the top of the Coverage Report
Fix 10328: Exporting assert results to excel throws "The file format and extension doesn't match. The file could be corrupted or unsafe" warning
Fix 11188: Adding a new annotation type throws a "500 Internal Service" error
Fix 11190: Adding a new testable path throws a "500 Internal Service" error
Fix 12566: Attempting to schedule a task throws a "String was not recognized as a valid DateTime" error
Fix 13124: Creating a System Coverage report only allows the adding of one item
Fix 13453: Server reports successful when test runner fails
Fix 13455: System Dashboard's LegiTest Status doesn't show up on the dashboard
Fix 14016: Empty Trash button does not remove assembly
Fix 13073: The Workbench Server installer does not automatically add the "Log on as batch job rights" permission to the correct user
Fix 13525: Within System Dashboard, LegiTest Status tile does not appear even though valid tags are used

Version 2017.3.3 [August 7, 2017]

New Features

New 10820: LegiTest UX Improvements
- "Wizard: Interactive Comparison" is now just "Interactive Comparison" in the context menu.
- When adding a new connection from the Interactive Comparison wizard, the focus will start in the Connection Name box.
New 10825: Added Xml Comparison Assertion
New 11178: LegiTest Extension now supports Visual Studio 2017
- Email support@pragmaticworks.com to request the installer that contains the Visual Studio 2017 supported version of the LegiTest Extension
New 11680: Added support for SQL Server 2017
New 11869: LegiTest Command Line can now push assemblies to the server.
New 12511: LegiTest documentation now includes software requirements for NUnit support

Updated Features

Update 10319: Comparison Manifests now work based on column name instead of column index
Update 10925: Small numbers are now detected as more commonly used types

Fixes

Fix 9248: Double clicking on the scroll bars within System Coverage adds selected item to target list
Fix 10923: "Wrong number of arguments provided" error when additional columns are added to test data source
Fix 11691: Previewing a query in Interactive Comparison fails when using a Data Driven variable.
Fix 11871: Adding two interactive comparisons in the same tests re-uses the same resource keys
Fix 12037: Using a reserved word (params, int, static, etc...) as a project name, prevents LegiTest from compiling
Fix 12415: Data set filtering does not handle NULL values correctly
Fix 12499: When selecting columns to the left of a 'Select *' statement, LegiTest Syntax Highlighter throws ArgumentOutOfRange error
Fix 12513: Parallel execution tracks are not reporting test actions to LegiTest Server
Fix 12520: Editing a query against a database with compressed columnstore indexes could crash Visual Studio
Fix 13081: Get Row Count test elements throws "The given key was not present in the dictionary." error

LegiTest Server Version 2017.3.3 [August 7, 2017]

New Features

New 7357: Annotations are now included in a Notification template.
- Test annotations are now included in the metadata send to the Notification Engine, so they can be incorporated into emails and other messages.
New 11010: "Select All" option now included within execution results
New 12007: Publishing Assemblies can now be restricted with Role Actions
New 2846: Licensing now works within environments that use FIPS compliant algorithms for encryption

Fixes

Fix 10383: Altering an HTML Message Template could remove/exclude data in a Notification when sent/executed
Fix 11113: LegiTest Notifications are not being sent when BI xPress isn't licensed
Fix 11662: The operation has timed out error, when communicating large results to the server
Fix 12038: Command Line request timeout is not configurable.
Fix 12470: Update button within the Edit Annotation Types does not work
Fix 12403: Denied Access to registry key "HKEY_CURRENT_USER"
Fix 11086: Stored procedures CheckIfUserCanAccessSecurable and GetAllSecurableEntities are always being dropped and created at website startup
Fix 12480: Could not load file or assembly 'ChilkatDotNet2' or one of its dependencies.

Version 2017.2.3 [May 1, 2017]

New Features

New 8734: SSIS Get and Set Properties elements now includes current design time values
New 8799: Directories can now be used as data sources for data driven tests
New 9132: Added ability to select colums from flat files
- The user can now select which columns are included in the dataset when sourcing data from a flat file.
New 10294: Fixed Width File References now allow you to choose the file format
- Ragged right
- Fixed width with line breaks
- Fixed width without line breaks
New 10367: Rows with no differences after a grid comparison can now be viewed on the server

Updated Features

Update 8750: SSAS Wizard generates connection assets at the suite level and shares between the groups
- The SSAS Wizard now creates connections at the test suite level instead of the test group.
Update 8800: Datasets can now be filtered
- Datasets from any grid source can be filtered by specifying expressions on a column or multiple columns.
- This filtering determines which rows will be seen by consumers of the grid / dataset.
- This filtering is applied after the initial data retrieval, so it should be used primarily for data sources that don't support filtering at the source, like text files or REST APIs. Databases and similar sources should be filtered through the SELECT statements for best performance.

Fixes

Fix 9159: Interactive Comparison does not save changes to currently selected elements if the editor is open
Fix 9217: Reserved filters list in custom filters section is unreadable
Fix 9225: Flat File connection managers are imported as ADO.NET Connection assets within the SSIS Wizard
Fix 9861: Execute Task element incorrectly reports task execution failure
Fix 10159: File Reference does not extract column headers if all columns are of the data type string
Fix 10917: LegiTest Extension reports version 1.0 within the "Installed Products" section of the "About Microsoft Visual Studio" page
Fix 10987: Editing a newly created interactive comparison before saving could result in an ArgumentNullException
Fix 11151: Interactive comparison always resets the timeout to 30 when editing

LegiTest Server Version 2017.2.3 [May 1, 2017]

New Features

New 6809: Added ability to suppress test failures for a set time frame
New 9870: Tests can now be moved to trash, then deleted from the server.
- Tests can now be moved to trash from the summary page.
- Trash view has been added, this allows users to permanently delete tests from the server.
- Unexecuted assemblies can no be deleted from the server on the unexecuted assembly view on the summary page.
New 9871: Summary Page now has separate views for Run and Not run assemblies.
New 10272: Users can now select existing tags from drop down when tagging a test object.
New 10331: Test management page now shows an item's most recent status in the navigation menu
New 10911: Design system for handling authorization for cloud services
New 11046: Notification Engine now has dynamic html template for LegiTest status

Updated Features

Update 6740: The "Export to Excel" feature within Grid Comparison now provides more details
- The file name for the excel spreadsheet now includes the date and time the test was executed
- The excel spreadsheet now contains a header that includes the suite, group, test and action name as well as the start and end time for the execution
Update 9868: Test Scheduler page has now been simplified for ease of use
Update 9873: Added "Missing Test Runner" topic to FAQ for LegiTest Server
Update 10978: Help documentation for setup instructions now includes details for multi-domain considerations

Fixes

Fix 9169: "Already added" warning appears when adding a new item to a Coverage Report
Fix 9228: Spaces in an assembly name are causing them not to run from the server
Fix 9241: Deleting a test execution fails
Fix 9817: Side by Side table view for errors does not have consistent height on both sides
Fix 9821: Breadcrumb located on View Details Page does not link
Fix 9869: Tests not part of previous run are not represented as such.
Fix 10273: Parameters containing certain special characters are not displaying properly on the test management page
Fix 10330: Long names in management navigation menu extend past frame.
Fix 11021: Managing a test with an existing Hyperlink template, will not load the template area for editing.
Fix 11026: When viewing a grid comparison result on the server, rows with no information still show a hyperlink.
Fix 10346: Some operations within DOC xPress and System Dashboard aren't responsive or appear disabled
Fix 10382: Workbench Server does not populate Action and RoleAction tables in host database

Version 2017.1.3 [February 2, 2017]

New Features

New 6732: LegiTest Command Line Tool can run test assemblies by name
New 7790: Added ability to load 'manual' data grids
- Data grids can now be loaded from delimited content stored within the test suite
- Provides an easy way to inject static data for comparison or data driven testing
New 7795: Added support for multiple delimited file formats
- Delimited file format support allows a variety of flat file formats to be read.
- Automatic detection of delimiters, quotes, header rows and preamble sections.
New 7876: Added Select All / None option in the SSIS Wizard
New 7893: Flat file data types suggestions are provided
New 7894: Flat files support the same data source model as other sources
- Flat file support has been upgraded to use the common data source model in LegiTest.
- Enables flat files to be easily used in interactive comparisons, grid validations and data-driven tests.
New 8298: Interactive Comparison now displays whether preview comparison would have succeeded or failed
New 8307: SSIS Wizard now extracts all connection managers
- Connection managers are now converted to assets by the SSIS Wizard irrespective of whether they become part of a data flow test or not
New 8310: Added ability to load fixed width files
- Fixed width files can be used in data driven testing, interactive comparison and as a data source in normal tests
- After entering the column widths, data formats are detected automatically
New 8798: Added row / line number reporting to grid comparisons
- The row / line number is now reported as part of the error message in case of an error

Fixes

Fix 5500: Timeout when gathering columns for comparison manifest with large data sets
Fix 7877: SSAS validation tests are not being generated in some situations
Fix 8238: Parameter mappings do not work within the SSAS Process actions
Fix 8275: Clicking "Advance" while configuring an OLE DB connection throws assembly error
Fix 8284: Table Row Count action did not respect parameterized table name
Fix 8299: Error changing keys in interactive comparison editor
Fix 8313: Object Reference Error when adding Test Suite to project using Read Historical Results or Interactive Wizard template
Fix 8754: Executing packages in a test does not recognize their associated project
Fix 9151: Error: Keyword not supported: 'provider' for Asset connection properties
Fix 9221: When using the SSIS Wizard targeting ISPAC packages, it does not generate Task level tests

LegiTest Server Version 2017.1.3 [February 2, 2017]

New Features

New 4878: Added "Two Grids" Layout for Grid Comparison Assert View
New 5000: LegiTest Server's System Coverage report now interacts with Workbench Server's (Beta) System Status feature
- Associating a system coverage report to a view found within Workbench Server's new (Beta) System Status allows System Status to display the system coverage for any items included within both the view and the coverage report
- The (Beta) System Status can be enabled by your Workbench Server administrator through the Administration Features page
- By default, the new (Beta) System Status will be disabled
New 6782: Filtered Dashboard now shows all of a tagged item's children even if they are not tagged
New 7309: Users can now add their own hyperlinks to Grid Comparisons and Validations via the test management page
- Hyperlinks can be created on the test management page and viewed on the details page.
New 7399: Test Management page now includes links to other items in the same suite
New 8303: Test assemblies can now be run immediately after publishing
New 8759: Include link to the LegiTest Extension on the LegiTest Server's Dashboard page
New 6256: Users can now delete System Dashboard tiles directly from the dashboard
New 6826: The System Dashboard is now the home page for Workbench Server
New 9267: Workbench Server now includes public beta features that can be enabled or disabled

Updated Features

Update 4882: LegiTest Server now points to NUnit 2.6.4 by default
Update 8739: Improved product version logging at start up

Fixes

Fix 7810: Fixed Legitest Server json errors when NewtonSoft.Json is installed in the GAC
Fix 8304: Can't see error details on server for failed steps
Fix 8717: Notification Engine buttons do nothing if only a LegiTest Server license is being used
Fix 9170: There is no space between the System Coverage page title
Fix 9142: The license expiration date does not appear in the License Manager
Fix 7803: With the addition of the System Dashboard and System Status, the header is disjointed
Fix 7898: Workbench Server does not confirm when deleting users
Fix 7982: License activation and deactivation messages are confusing
Fix 8011: Workbench Server Licenses does not include access to System Dashboard
Fix 8699: System Dashboard drop-down buttons includes erroneous icons and white divider
Fix 8738: If Workbench Server's version falls outside a license's maintenance period, the system settings' licenses page will not display the license
Fix 9285: "Add Domain User" button is missing

2016

Version 2016.4.4 [October 25, 2016]

New Features

New 4776: Query Editor supports Azure SQL DB
- The query editor built in to LegiTest now supports connections to Azure SQL DB
New 4852: Resource key usages are now automatically remapped if a resource key is renamed
New 6729: Usability improvements for the Annotations UI
- Annotation type creation pane is now a separate configuration page.
- The content field supports Markdown for richer formatting when displayed on the server.
- The content field size has been increased, and it now supports multiple lines.
- Annotations for each test / group are listed under the test / group instead of a test-suite level pane.
New 6767: Updated comparison editor UI
- Added an indicator that shows when code completion is available
- Added an indicator that shows when the editor is connected
- Added a Run Query toolbar button to each query editor
New 7345: Increased detail within error reports for Grid Validation assertions
- The Grid Validation assertion now reports row details in a similar manner to the Grid Comparison assertion
New 7377: Data driven testing now supported
- Tests can now be configured to use a data set or grid as input, so that the test executes once for each row in the data set.
- Execute the same test logic for a large number of scenarios very easily
- Use data from any ADO.NET source, SalesForce or REST
New 7855: Improved validation warnings when re-using a resource key name

Updated Features

Update 4768: IntelliSense now available for grid collection asset editor
Update 5498: Validation assertions can now expect empty grids
Update 6215: SSAS Process Item Actions now include option to fail on process failure
Update 6811: Comparison Manifest editor now shows the configured grids for actual and expected

Fixes

Fix 4774: Query Editor picks arbitrary Sql connection for editing
Fix 4904: Query Editor errors are less intrusive
Fix 6194: Editing a comparison column mapping for an invalid column results in a modal error dialog
Fix 6198: VSIX Uninstaller isn't removing extensions during uninstall process
Fix 6625: Visual Studio 2012 crashes when using "Format SQL" button
Fix 6727: SalesForce Query doesn't honor parameters when providing columns for comparison / validation
Fix 6728: Validation Manifest that references SF Query with no rows doesn't allow editing of validation columns
Fix 6731: Executing the LegiTest commandline executable with no parameters throws an unhandled exception
Fix 6743: Help website has grammatical and spelling errors
Fix 6763: Interactive Comparison won't let you finish with an incomplete test
Fix 6764: The Interactive Comparison editor doesn't evaluate parameters defined outside of Test Suite Parameters
Fix 6812: Checking the Show Equal Rows option doesn't trigger an automatic reload
Fix 7326: Syncing does not update annotation description
Fix 7390: SalesForce actions do not return results if they contain three part column names
Fix 7397: Elements with default editor do not save values when another element is selected
Fix 7846: Execute REST actions do not parameterize properties
Fix 7904: Fixed issue with SSRS report calling Parameters

LegiTest Server Version 2016.4.5 [October 25, 2016]

New Features

New 4960: Scheduling tests no longer has a dependency on the LegiTest Command Line
New 6229: Expanded items in LegiTest Server dashboard now remain expanded when done managing tags.
New 6258: Enhanced LegiTest notifications to include grids from comparisons and validations
- Grids are now shown in LegiTest notification emails so that data mismatches or validations show the data in question in the notification email (HTML only)
New 6266: User can now specify the End Time for scheduled test runs
- Allow the user to specify the end time on the end date when scheduling a task. If not specified, default it to end of day instead of using the start time.
New 6278: Added ability to create VSTS work items as a notification engine target
- Notification can now automatically create work items in Visual Studio Team Services
- The template is applied to create a title for the created work item
New 6280: Added ability to run executable as a notification engine target
- Notification engine can now run an executable on the server in response to a notification
- Optionally the templated output is passed to the executable as a file
New 6281: Added ability to make URI GETs/POSTs/PUTs as a notification engine target
- Use notification engine to interact with HTTP APIs
- Configure the payload using the standard templating engine
- Support for basic authentication, as well as OAUTH1 and OAUTH2
New 6282: Added ability to manually run notifications for specific tests from the LegiTest Dashboard
New 6284: Added ability to defer population of notification engine collection objects
- Notification engine now defers some of the work in creating a notification until the notification fires, resulting in higher performance.
New 6286: Implemented Role Based Security
- Manage Access to LegiTest Server actions based on roles.
- Manage access to tests based on roles and tags.
New 6784: Allowed custom name for LegiTest Tile
New 6828: Added Test Management Page
- Manage Parameters, Scheduled Tasks, Tags, and certain actions all from one page.
New 7809: Data driven tests are displayed as separate tests on the details page and as a single item on the summary page.
New 6723: Upgrading Workbench Server will now display a message indicating that all previous versions will be uninstalled during the upgrade process
- Users upgrading Workbench Server will now see a message indicating that all previous versions of Workbench Server will be uninstalled prior to upgrading. This is not a change in behavior from previous releases, only an update to the user interface to make it more clear as to what is happening behind the scenes. This message is only visible when upgrading.
New 7373: Added PowerShell 3.0 or later to the system requirements for Workbench Server

Updated Features

Update 6827: Improvements made to LegiTest Server home page
- LegiTest Server Dashboard is now the LegiTest Server Summary page.
- Clicking on a failed test from the Summary will take you to the details of the failed assert.
- Tags are now listed to the right of the Summary page, with the ability to filter tags moved, and takes up less real estate on the screen.
Update 4718: Activating or Deactivating a license now redirects users to license overview page upon completion

Fixes

Fix 6226: Parameter and Data Type Name no longer overlap when editing parameters.
Fix 6733: Navigation breadcrumb trail is not visible on the details page
Fix 6734: Command line configuration values are not being retained when upgrading LegiTest Server to the newest version
Fix 6735: User is now alerted if the command line can not find a valid test runner
Fix 6738: Summary page does not refresh after a test suite is manually run from the server
Fix 6744: Annotations now show up on the results page.
Fix 6781: LegiTest Server does not display any error if no coverage reports load
Fix 6800: Command Line error message reflects incorrect log location
Fix 7331: Notification events fire multiple times for each Test Group within a suite if configured to an assembly
Fix 6666: Licensing is now more robust and should not issue an error during installation.

Version 2016.3.7 [August 1, 2016]

New Features

New 4611: Ability to specify business metadata for tests
Define annotation types to represent categories of business metadata to apply to tests and groups
Add your own content which is recorded and available to view on the server
Synchronize annotation types between test suites and the server
New 4790: Icons for Rest Action and Rest Connections in Legitest
The icons for the REST actions have been updated.
New 4861: Add Execute REST Scalar Action
REST API calls can be executed that return a scalar value extracted from the response.
New 4922: Implemented Comparison Editor
Interactive comparison editor allows you to configure a comparison between two sources interactively.
Edit the comparison configuration and see the results in real time.
Configure keys, comparisons and aggregates.
Easily edit an existing comparison.
View differences in a highlighted, synchronized grid.
Interactive comparison can use any data source that LegiTest supports
New 5451: Publishing model is more flexible
Publishing of DLLs to server now handled by right-clicking on a project that contains LegiTest suites.
Send the exact DLLs that you want to the server without any accidental publishing during development time.
Publishing of test results to server now handled by setting parameters.
Easily enable or disable publishing of test results without recompilation or regeneration.
New 5460: Added ability to reference resource values using the %keyName% syntax
Now assets and actions can make use of resource values using the same syntax as replaceable parameters in text - %parameterName%
In addition, we can now use DotLiquid syntax for parameter replacement in all places -
New 6007: Add headers to legitest rest actions

Updated Features

Update 4684: Updated LegiTest SSAS support to handle 2016 changes
Update 4747: Added ability to test connections from the connection asset editor
Update 4949: Users can now specify the Process Mode used within the Process Database, Cube, Dimension and Table actions
- Users can now specify between the various Process Modes used within SSAS.

Fixes

Fix 4753: Parameterized connections don't refresh properly in the Grid Collection Asset and Data Comparison Wizard
Fix 4756: Cannot connect to report server using a non-default virtual directory in LegiTest.
Fix 4880: Log4Net is not added to LegiTest Projects as a Nuget reference
Fix 4881: Assembly references need to be refreshed when moving between versions of LegiTest or machines
Fix 4886: Could not load file or assembly 'Microsoft.Data.ConnectionUI, Version=0.0.0.0, Culture=neutral, PublicKeyToken=f4ca07f51760da93' or one if its dependencies. The system cannot find the file specified.
Fix 4925: Obfuscated VS2015 LegiTest Extension throws XamlParseException when loading editors containing vsminipathbutton
Fix 4942: Downgrading ssis package asset causes its name to be reset
Fix 5435: Comparison Manfest Editor fails if you edit a column that no longer exists
Fix 5439: Rename the "Ignore Left-Only Rows" and "Ignore Right-Only Rows" properties to reference Expected and Actual
Fix 5455: Empty resultsets from SalesForce result in an execution error
Fix 5456: REST actions and assertions should be under a category of "Cloud"
Fix 5458: The SSAS Process Actions spin indefinitely when user clicks "connect" on a machine without 2016
Fix 5459: Removing a covered item does not mark suite as needing updating
Fix 5478: Importing a Test Suite does not import parameter mappings
Fix 5489: Improved UIs for SalesForce and Rest Connections & Actions
Fix 5505: Renaming a LegiTest file can lose custom code
Fix 5518: Test wizards cannot be applied at the test suite level
Fix 6185: Data Type combo box within Execute REST Query (Grid) is unreadable
Fix 4932: The VSIX Uninstaller "Uninstall" button acts as a cancel button after initializing uninstallation

LegiTest Server Version 2016.3.7 [August 1, 2016]

New Features

New 4879: Added ability to tag test items, and filter dashboard based on a specified tag.
- Add tags to any test object.
- Filter the dashboard for a given tag.
New 4946: New LegiTest Status Tile
- Quickly view System Coverage for a selected tag.
- Click through the tile to view all tests associated with the specified tag.
New 5449: Added ability to export Difference, Validation, and Aggregation Grids to Excel
- Export data grid results to Excel from the web server
New 5473: Test Runs Detail Page now includes an easy to follow trail to track position in the tree hierarchy.
New 6248: Test Suite notifications now include error details
- Errors from LegiTest tests are now available for use in Notification Framework
- Ability to see error messages from actions and assertions
New 4782: While upgrading Workbench Server, most of the existing settings can be reused.
- Users can now choose whether to use the existing settings from the previous installation or choose to start fresh when upgrading Workbench Server.
New 4923: System Dashboard for Workbench Server.
- Workbench Server's newest feature allows users to create a custom dashboard for displaying status and summary information for BI xPress' Auditing Framework and for LegiTest's System Coverage features. Users can add multiple "tiles" that are each configured for displaying concise data points they may be interested in.

Fixes

Fix 4242: LegiTest Server's Play Suite and Schedule Suite now supports editing and using parameters
Fix 4251: System Coverage search now has a longer delay after typing and initiating search.
Fix 4323: Duplicate TestableObjects no longer being added to table.
Fix 4411: LTSRV is now properly recording all exceptions.
Fix 4444: Testable objects now track partial and full runs and revert to untested once removed from tests.
Fix 5433: When scheduling a test suite to run, it will now default to the current host name.
Fix 5434: Dataset Comparison View properly matches columns in all instances.
Fix 5442: Details Grid within server now properly reflects "ignore left/right rows" options

Version 2016.2.4 [May 4, 2016]

New Features

New 4386: LegiTest supports SQL Server 2016
New 3130: Support mapping tested assets against DOCx Inventory
New 4161: Create LegiTest solution to verify SalesForce against CMS
New 4259: Execute Process action now has ability hide command window on execution

Fixes

Fix 4096: Disable the "Reload Coverage Reports" button when not connected to LegiTest Server
Fix 4097: Expected / Actual Values are reversed in Integer Comparison
Fix 4130: Having CodeSlice installed prevents use of LegiTest files
Fix 4094: SF #90816 - Snapshot Filtering on SQL Server source in IS provider

LegiTest Server Version 2016.2.4 [May 4, 2016]

New Features

New 4386: LegiTest supports SQL Server 2016
New 3130: Support mapping tested assets against DOCx Inventory
New 4161: Create LegiTest solution to verify SalesForce against CMS
New 4259: Execute Process action now has ability hide command window on execution

Fixes

Fix 4096: Disable the "Reload Coverage Reports" button when not connected to LegiTest Server
Fix 4097: Expected / Actual Values are reversed in Integer Comparison
Fix 4130: Having CodeSlice installed prevents use of LegiTest files
Fix 4094: SF #90816 - Snapshot Filtering on SQL Server source in IS provider

2015

Version 2015.3.7 [November 5, 2015]

New Features

New Directory Existence Assert
- Checks whether or not a directory exists at the target directory path
New Execute SalesForce Query (Grid) Action
- Utilizes the Query and SalesForce Connection Assets to extract data from SalesForce Objects for use during comparisons
New Extract Grid Checksum Action
- Quickly generate a checksum for specific grid and stores for later use
New File Existence Assert
- Checks whether or not a file exists at the target file path
New General Usability
- Allow the organizing and scoping of assets at the Test Suite and Test Group level
- Deleting an element will display a list of all dependencies when confirming deletion
- Include ability to import Tests and Test Groups from other LegiTest Suites
- Include support for negative testing, allowing users to create tests the user expects to fail or throw an exception
- LegiTest can now extract connection strings from Packages and Reports
- Quickly find all related sources for a specific action or assert within a Test Suite by right-clicking on them and selecting "Find Sources"
- Quickly find usage of a specific assets within a Test Suite by right-clicking on them and selecting "Find Usage"
New Grid Validation Assertion
- Grid Validation Assertion utilizes the new Grid Validation Manifest to check column and aggregated values within a grid
New LegiTest Server Integration
- LegiTest can now publish LegiTest executables and assemblies to LegiTest Server for execution
New Read Historical Results Action
- Extracts historical results for a specific assertion saved to LegiTest Server and and compares results to current execution
New SalesForce Connection Asset
- Utilized within Execute SalesForce Query (Grid) Actions to connects to SalesForce Objects
New Test Generation Templates
- Quickly generate tests based on target SQL Data Sets, SSIS Packages, SSRS Reports, and SSAS Database Validation
New Test Suite Parameterization
- Create configuration parameters and map them to the properties of specific elements
New Timing Start Action
- Utilized alongside Timing End action to signal the start of a time period for measuring a specific length of time
New Timing End Action
- Utilized alongside Timing Start action to signal the end of a time period for measuring a specific length of time
New Validation Manifest Asset
- Utilized within the Grid Validation Assertion to validate expected comparison values within a Grid

Updated Features

Update General Usability
- Include ability for Test Suites to be renamed
- Include enhanced type checking for more meaningful error messages
- Include ability to enable and disable tests and test groups
Update SSIS Package Reference
- Simplify the SSIS Package Reference asset

Fixes

Fix T-SQL Bulk Insert Asset
- Bulk Insert file path now properly handles environment variables
Fix General Usability
- LegiTest Test Suites now work in conjunction with source control
- Removed LegiTest from list of valid .NET 4.0 Project Templates
- Assets now immediately validate whether or not they are being used by another element
Fix LegiTest Server Integration
- LegiTest now gracefully handles errors returned from LegiTest Server
Fix SSIS Package Reference
- Include Package Reference Assets within toolbox only if the specified version of SSIS is installed and usable
- Fixed issue with password protected packages not loading properly

LegiTest Server Version 2015.3.7 [November 5, 2015]

New Features

New Directory Existence Assert
- Checks whether or not a directory exists at the target directory path
New Execute SalesForce Query (Grid) Action
- Utilizes the Query and SalesForce Connection Assets to extract data from SalesForce Objects for use during comparisons
New Extract Grid Checksum Action
- Quickly generate a checksum for specific grid and stores for later use
New File Existence Assert
- Checks whether or not a file exists at the target file path
New General Usability
- Allow the organizing and scoping of assets at the Test Suite and Test Group level
- Deleting an element will display a list of all dependencies when confirming deletion
- Include ability to import Tests and Test Groups from other LegiTest Suites
- Include support for negative testing, allowing users to create tests the user expects to fail or throw an exception
- LegiTest can now extract connection strings from Packages and Reports
- Quickly find all related sources for a specific action or assert within a Test Suite by right-clicking on them and selecting "Find Sources"
- Quickly find usage of a specific assets within a Test Suite by right-clicking on them and selecting "Find Usage"
New Grid Validation Assertion
- Grid Validation Assertion utilizes the new Grid Validation Manifest to check column and aggregated values within a grid
New LegiTest Server Integration
- LegiTest can now publish LegiTest executables and assemblies to LegiTest Server for execution
New Read Historical Results Action
- Extracts historical results for a specific assertion saved to LegiTest Server and and compares results to current execution
New SalesForce Connection Asset
- Utilized within Execute SalesForce Query (Grid) Actions to connects to SalesForce Objects
New Test Generation Templates
- Quickly generate tests based on target SQL Data Sets, SSIS Packages, SSRS Reports, and SSAS Database Validation
New Test Suite Parameterization
- Create configuration parameters and map them to the properties of specific elements
New Timing Start Action
- Utilized alongside Timing End action to signal the start of a time period for measuring a specific length of time
New Timing End Action
- Utilized alongside Timing Start action to signal the end of a time period for measuring a specific length of time
New Validation Manifest Asset
- Utilized within the Grid Validation Assertion to validate expected comparison values within a Grid

Updated Features

Update General Usability
- Include ability for Test Suites to be renamed
- Include enhanced type checking for more meaningful error messages
- Include ability to enable and disable tests and test groups
Update SSIS Package Reference
- Simplify the SSIS Package Reference asset

Fixes

Fix T-SQL Bulk Insert Asset
- Bulk Insert file path now properly handles environment variables
Fix General Usability
- LegiTest Test Suites now work in conjunction with source control
- Removed LegiTest from list of valid .NET 4.0 Project Templates
- Assets now immediately validate whether or not they are being used by another element
Fix LegiTest Server Integration
- LegiTest now gracefully handles errors returned from LegiTest Server
Fix SSIS Package Reference
- Include Package Reference Assets within toolbox only if the specified version of SSIS is installed and usable
- Fixed issue with password protected packages not loading properly

Version 2015.2 [September 3, 2015]

New Features

New LegiTest Server Integration
- Publish test results directly to LegiTest Server to keep historical records and utilize BI xPress Server's Notification Engine

Updated Features

Update Execute Report Action
- Add the ability to pass parameters through to an SSRS Report executed through the Execture Report Action
Update Directory Copy Asset
- Add "Strict Mode" for Directory Copy Asset
Update File Copy Asset
- Add "Strict Mode" for File Copy Asset
Update General Usability
- Include better design layout when areas are collapsed
- Provide a more detailed description of error for failed tests that better identify where the failure occurred
- Add ability to automatically version test assembly upon test generation
- Better streamline the inclusion and usage of SSIS Packages using the Package Reference Asset

Fixes

Fix Set Properties Action
- Allow properties containing brackets ("[" or "]") to be set
Fix General Usability
- Removed superfluous teardown method created when adding a group level test teardown
- Generated code now automatically updates upon changing the order of the elements within an execution track
Fix SSIS Assets
- Allow the modification of Project level connection strings

LegiTest Server Version 2015.2 [September 3, 2015]

New Features

New LegiTest Server Integration
- Publish test results directly to LegiTest Server to keep historical records and utilize BI xPress Server's Notification Engine

Updated Features

Update Execute Report Action
- Add the ability to pass parameters through to an SSRS Report executed through the Execture Report Action
Update Directory Copy Asset
- Add "Strict Mode" for Directory Copy Asset
Update File Copy Asset
- Add "Strict Mode" for File Copy Asset
Update General Usability
- Include better design layout when areas are collapsed
- Provide a more detailed description of error for failed tests that better identify where the failure occurred
- Add ability to automatically version test assembly upon test generation
- Better streamline the inclusion and usage of SSIS Packages using the Package Reference Asset

Fixes

Fix Set Properties Action
- Allow properties containing brackets ("[" or "]") to be set
Fix General Usability
- Removed superfluous teardown method created when adding a group level test teardown
- Generated code now automatically updates upon changing the order of the elements within an execution track
Fix SSIS Assets
- Allow the modification of Project level connection strings