SentryOne Test Release Notes

2019

Version 2019.3.1 [July 3, 2019]

New Features

  • Feature 53106:  2019 SSIS support in SentryOne Test

Fixes

  • Fix 53691:  S1T VSX Project templates do not show up in the 'Test' category
  • Fix 53932:  SentryOne Test: Using a parameter inside of another parameter will crash VS
  • Fix 53992:  VS 2019 Community Edition Blue Theme fix for S1T
2018

Version 2018.5.1 [December 4, 2018]

New Features

  • New 42048: LegiTest Client now integrates with SentryOne Accounts system
  • New 42092: Copy-Paste Test functionality
    • Users can now copy and paste tests from one group to another
  • New 44453: LegiTest Runtime now identifies hosts using their service identifiers
  • New 45510: LegiTest Visual Studio extension has been removed from Workbench Installer
    • The LegiTest extension can be downloaded here:

Fixes

  • Fix 42033: LegiTest custom tool generates a warning on generation
  • Fix 42041: Row Count Recipe fails when row counts match but are from differently named tables
  • Fix 42047: REST assets in LegiTest cause project to reference Core assembly
  • Fix 42050: The LegiTest Recipes Console does not support relative paths for the targetFolder argument
  • Fix 42067: Comparison Manifest reports no key column being defined as an error
  • Fix 42082: Bulk Insert Asset ignores provided ID for identity columns
  • Fix 42083: Cannot build LegiTest project when name suffixed with ".NUnit"
  • Fix 42085: LegiTest Project Templates don't show up under the Visual C# \ Test category
  • Fix 42087: Grid Validation Manifest comparisons do not substitute parameters
  • Fix 42418: When adding multiple tests the naming is 'Test 1' and 'Test 1 (2)'
  • Fix 42419: The setting for 'Treat whitespace as NULL' is not persisted
  • Fix 42440: Server Settings are not retained
  • Fix 42441: Intellisense warns of unknown identifier on aliased columns used with UNION ALL
  • Fix 42445: Interactive Comparison's "Format SQL" button adds a semicolon to the end of the query, which throws an ORA-00911 error with Oracle Queries
  • Fix 42448: Comparison Editor status indicator doesn't change to indicate current status unless connection is changed.
  • Fix 42457: Deleting parameters that have already been mapped should ask user for confirmation
  • Fix 42512: BrowserEmulation is not set correctly if the key is not already present
  • Fix 42514: Opening Read Historical Result Wizard is causing Visual Studio to crash
  • Fix 42707: LegiTest picks up incorrect NUnit Test adapter
  • Fix 42779: Row Count Recipe failures no longer report the table name
  • Fix 43858: LegiTest throws an Exception when loading an SSIS package from an ISPAC file, then crashes Visual Studio
  • Fix 44289: Add a note to the LegiTest Recipe documentation that Oracle is not supported.
  • Fix 45423: Server parameters were being created in group-specific settings files
  • Fix 45767: ParameterProvider PrintDebugInformation is called before parameter files are loaded

Version 2018.2.3 [April 27, 2018]

Important:  The 2018.2.3 release for LegiTest will be the last supported release for SQL Server 2008 and 2008 R2.             

New Features

  • New 3772: Validation results now show up in the Visual Studio Errors tool window
  • New 6824: Data Profile recipe
    • LegiTest now includes a validate data column profile recipe that can test minimum and maximum values, range checks, et cetera 
  • New 12563: Dimension Validation recipe
    • LegiTest now includes a dimension validation recipe to ensure a table adheres to standard dimension table rules
  • New 13468: Fixed Width Text File Asset now includes the capability to skip header rows
  • New 13504: Added support for generation of NUnit 3 Tests
  • New 15679: Added 'Send equal rows to results server' to Interactive Comparison
  • New 16104: Runtime settings used for each execution are now logged
  • New 16336: Added actual value to value comparison assertion error messages
  • New 16936: Range Check recipe now includes additional settings to include informational columns
  • New 16969: Interactive Comparison and Comparison Manifest includes ability to equate white space to NULL values
  • New 17415: Text Format Validation recipe
    • LegiTest now includes a text format validation recipe that checks the validity of data within a specific column: Alpha Numeric, Email Address, North American Phone Number, Numeric, Social Security Number, Country Code (ISO 3166) and State Code (ISO 3166)

Updated Features

  • Update 8782: Improved messaging around header row parsing failures in flat file asset editors
  • Update 12417: Data set filtering now allows for NOT BETWEEN, IS NULL and IS NOT NULL
  • Update 16338: Validation manifests can now validate that values are NULL / NOT NULL

Fixes

  • Fix 4777: Within source controlled environments, renaming or moving items will not properly clean up generated resources
  • Fix 11670: Visual Studio Extension uninstaller displays multiple check boxes for the same version of Visual Studio
  • Fix 13113: Mouse wheel does not work to scroll packages in the Package Test Wizard on the Select Packages page
  • Fix 13505: Naming a test ValidationManifest results in a compilation error
  • Fix 15610: Data Validation wizard failed on long running queries
  • Fix 15694: Execute Report action and wizard do not have a timeout control
  • Fix 15699: Fail on server publish should be set for all recipes
  • Fix 15702: Object Reference Error thrown when NULL values are returned in MDX queries
  • Fix 15762: Table lists do not refresh when switching from an existing connection to a new connection
  • Fix 16045: Correct path to the custom Plussom assembly
  • Fix 16105: XPath extraction does not support XPath validation with namespaces
  • Fix 16284: Wizards that should create multiple groups fail to move one group
  • Fix 16337: ExecuteScalar on SalesForce does not return the correct result
  • Fix 16354: Unable to rename singe groups / suites unless renamed through solution explorer
  • Fix 16360: Query Editor throws an "InvalidOperationException" when attempting to type while the cursor is off screen
  • Fix 16938: LegiTest does not reinstall NuGet packages with unresolved references
  • Fix 16983: Adding key or comparison columns could add the wrong column in Interactive Comparison

Version 2018.1.4 [February 8, 2018]

New Features

  • New 5497: Validation and Comparison manifests now support regular expression matching and asymmetric tolerances
    • A new match type of 'RegexMatch' can be used to match an expected regular expression pattern against an actual value
    • Percentage and fixed tolerance comparisons can now be set to use asymmetric tolerance values for specific ranges
  • New 11057: LegiTest now auto selects resources / assets when possible
  • New 13437: When re-publishing an assembly that is in the recycle bin on the server the assemblies are removed from the trash
  • New 14542: Visual Studio's test output console now includes LegiTest parameter information
  • New 14663: LegiTest extensions now support initializing logging through external configuration files
  • New 14715: Added support for Recipes
    • Recipes allow reusable test patterns to be quickly and easily turned into tests, allowing for tests to be created with very little knowledge of unit testing in general
    • Several recipes are available - including a uniqueness validation, range check, standard deviation validation etc
  • New 15190: Added 'Value Comparison' assertion
    • Value comparison replaces the 'Integer Comparison' and 'Scalar Comparison' assertions
    • Easier configuration for validating a single scalar value
    • Wider array of operators for comparison against multiple types
  • New 15191: Added 'Expression' assertion
    • Expression assertion allows LegiTest filter expressions to assert on single or multiple scalar values
    • Use complex combinational logic to verify resource values
    • Familiar SQL-like syntax for comparison and logical combination through AND / OR
  • New 15671: Load Text File action
    • The Load Text File action allows users the ability to load the content of any text file into a resource for later use 

Updated Features

  • Update 14583: Help Documentation now includes a Frequently Asked Question including detailed code samples for one-time use parameter values

Fixes

  • Fix 13467: 'Move Up' / 'Move Down' moves all tests to the bottom of the list
  • Fix 13901: Exception messages in Group Setup and Teardown are difficult to troubleshoot
  • Fix 13972: Compiler warning in generated projects: Architecture mismatch between MSIL and x86
  • Fix 13973: LegiTest does not parse user defined parameter syntax in Data Driven test queries
  • Fix 13978: Enabling an empty data-driven source causes unclear compilation error
  • Fix 14562: Within Data Driven Source, the "Edit Filter" and "Edit Asset" buttons for File Handling sources do not do anything
  • Fix 14626: Validation Manifest configured with invalid grid provider crashes Visual Studio on column load
  • Fix 15116: Project Level Asset file doesn't get marked as dirty in VS after changes
  • Fix 15129: LegiTest Rest Connection/Query Headers do not replace parameter syntax
  • Fix 15134: Legitest: "Object cannot be cast from DBNull to other types" error in Legacy Test Suites
  • Fix 15178: Fixed Width File Reference asset throws "Could not detect file format value cannot be null. Parameter name: values"
  • Fix 15252: Tests targeting NUnit error with BadImageFormatException when run via LegiTest Runner
  • Fix 15629: Parameter replacement does not work within the filter of a Data Driven Directory Reference
  • Fix 15654: Project level connection assets with parameterized connection strings do not work in all scenarios
  • Fix 15657: Value Comparison fails when ResourceKey contains ( or )
  • Fix 15658: Comparison Manifest does not recognize global connection assets

LegiTest Server Version 2018.1.4 [February 8, 2018]

Updated Features

  • Update 14631: Data Driven Test parameters are now included in Notification Engine

Fixes

  • Fix 14019: LegiTest CommandLine does not capture console output
  • Fix 14563: Deleting results that includes a published assembly does not also delete the assembly
  • Fix 14724: Annotations are not transmitted correctly to LegiTest Server
  • Fix 15166: Notification Engine failures prevent LegiTest assembly run histories from updating
  • Fix 15187: LegiTest CommandLine doesn't respect spaces within assembly file locations
  • Fix 15194: Date Time Format for Scheduled Tests does not work in non-US locale settings
  • Fix 11144: When creating a System Status View, if the selected solution doesn't contain Lineage no message is presented to the user.
2017

Version 2017.4.7 [November 27, 2017]

New Features

  • New 5476: Improved multi-developer experience
    • Assets can now be defined at the project level
    • LegiTest suites can be migrated to individual group files
    • Added ability to have single groups in single files, enabling better sharing
    • Added project-level parameter files to simplify configuration across a project
    • Added project-level server integration to enable simpler and more consistent deployment
  • New 9822: Reduced runtime dependency footprint for LegiTest
    • Less DLL references are now required for some testing patterns
  • New 11951: Added error inspector for looking at warnings at the group/test level
    • Warning tooltips are also enhanced with extra detail
  • New 13116: Runtime dependencies can now be resolved via NuGet
    • Most of the runtime dependencies of LegiTest tests can now be resolved via NuGet, making deployment on build servers much easier
  • New 13435: Generated file names for asset content files are shorter
    • Names of existing assets will be shortened upon opening the test suite / group that contains them. 
  • New 13454: Added ability to define test framework type at project level
    • Framework type is now defined at the project level, instead of for individual suites/groups 
  • New 13480: Suites, Groups, Settings and Asset files now have appropriate icons
  • New 13502: Added ability to ignore tasks by name in SSIS Wizard
    • Added ability to use wildcards to ignore tasks by name - e.g. *SSISOps* 

Updated Features

  • Update 8753: REST assets now include a "Number of Retries" option that controls the amount of connection attempts upon failure
  • Update 11156: Added help overlay to the query editor
  • Update 13492: Improved warning message within SSIS Package Reference asset when pointing to a newer SSIS version

Fixes

  • Fix 12021: DOC xPress connector and System Coverage panel give no warning when server is unlicensed
  • Fix 12414: Comparison for Validation Manifest are run in reverse of what the UI displays
  • Fix 13060: Package Reference Asset throws an SSISCatalogPackageNotFoundException error when a folder is selected
  • Fix 13072: ReadHistoricalResultsAction can display an interactive dialog during test execution
  • Fix 13093: LegiTest does not publish to server all needed assemblies when using SSIS Packages Reference
  • Fix 13098: LegiTest suites / groups do not function correctly in project sub-folders
  • Fix 13100: LegiTest tests will fail on a data driven run if the server fails and 'fail on server failure' is not checked
  • Fix 13104: Renaming a Suite or Group does not delete the old output folder
  • Fix 13120: Actual and Expected values are being flipped on integer comparison when pushing results to server.
  • Fix 13458: New "LegiTest Project - NUnit" projects do not include a NuGet reference to NUnitTestAdapter
  • Fix 13935: Within Connection Asset, clicking the Test Connection throws InvalidOperationException if Connection String is empty
  • Fix 13936: "The type or namespace name 'Dictionary<,>' could not be found" when generating a LegiTestGroup without any tests
  • Fix 13987: "Member names cannot be the same as their enclosing type" when a test is named the same as its group
  • Fix 13993: Publishing a Data Driven test that uses a Delimited Content asset misses publishing certain assemblies
  • Fix 14000: Data Driven tests containing nullable values can throw ArgumentException
  • Fix 14033: SSIS Package Wizard throws an InvalidDataException when selecting a package that has a higher SSIS version
  • Fix 14551: Interactive Comparison throws "Comparison no longer exists" error when mapping key columns with different names

LegiTest Server Version 2017.4.7 [November 27, 2017]

New Features

  • New 13054: Workbench Server documentation now includes required permissions

Updated Features

  • Update 11979: Online help documentation now includes a rolling list of release notes that include all product release notes within a single file

Fixes

  • Fix 9260: Report Totals do not show at the top of the Coverage Report
  • Fix 10328: Exporting assert results to excel throws "The file format and extension doesn't match. The file could be corrupted or unsafe" warning
  • Fix 11188: Adding a new annotation type throws a "500 Internal Service" error
  • Fix 11190: Adding a new testable path throws a "500 Internal Service" error
  • Fix 12566: Attempting to schedule a task throws a "String was not recognized as a valid DateTime" error
  • Fix 13124: Creating a System Coverage report only allows the adding of one item
  • Fix 13453: Server reports successful when test runner fails
  • Fix 13455: System Dashboard's LegiTest Status doesn't show up on the dashboard
  • Fix 14016: Empty Trash button does not remove assembly
  • Fix 13073: The Workbench Server installer does not automatically add the "Log on as batch job rights" permission to the correct user
  • Fix 13525: Within System Dashboard, LegiTest Status tile does not appear even though valid tags are used

Version 2017.3.3 [August 7, 2017]

New Features

  • New 10820: LegiTest UX Improvements
    • "Wizard: Interactive Comparison" is now just "Interactive Comparison" in the context menu.
    • When adding a new connection from the Interactive Comparison wizard, the focus will start in the Connection Name box.
  • New 10825: Added Xml Comparison Assertion
  • New 11178: LegiTest Extension now supports Visual Studio 2017
    • Email support@pragmaticworks.com to request the installer that contains the Visual Studio 2017 supported version of the LegiTest Extension 
  • New 11680: Added support for SQL Server 2017
  • New 11869: LegiTest Command Line can now push assemblies to the server.
  • New 12511: LegiTest documentation now includes software requirements for NUnit support

Updated Features

  • Update 10319: Comparison Manifests now work based on column name instead of column index
  • Update 10925: Small numbers are now detected as more commonly used types

Fixes

  • Fix 9248: Double clicking on the scroll bars within System Coverage adds selected item to target list
  • Fix 10923: "Wrong number of arguments provided" error when additional columns are added to test data source
  • Fix 11691: Previewing a query in Interactive Comparison fails when using a Data Driven variable.
  • Fix 11871: Adding two interactive comparisons in the same tests re-uses the same resource keys
  • Fix 12037: Using a reserved word (params, int, static, etc...) as a project name, prevents LegiTest from compiling
  • Fix 12415: Data set filtering does not handle NULL values correctly
  • Fix 12499: When selecting columns to the left of a 'Select *' statement, LegiTest Syntax Highlighter throws ArgumentOutOfRange error
  • Fix 12513: Parallel execution tracks are not reporting test actions to LegiTest Server
  • Fix 12520: Editing a query against a database with compressed columnstore indexes could crash Visual Studio
  • Fix 13081: Get Row Count test elements throws "The given key was not present in the dictionary." error

LegiTest Server Version 2017.3.3 [August 7, 2017]

New Features

  • New 7357: Annotations are now included in a Notification template.
    • Test annotations are now included in the metadata send to the Notification Engine, so they can be incorporated into emails and other messages.
  • New 11010: "Select All" option now included within execution results
  • New 12007: Publishing Assemblies can now be restricted with Role Actions
  • New 2846: Licensing now works within environments that use FIPS compliant algorithms for encryption

Fixes

  • Fix 10383: Altering an HTML Message Template could remove/exclude data in a Notification when sent/executed
  • Fix 11113: LegiTest Notifications are not being sent when BI xPress isn't licensed
  • Fix 11662: The operation has timed out error,  when communicating large results to the server
  • Fix 12038: Command Line request timeout is not configurable.
  • Fix 12470: Update button within the Edit Annotation Types does not work
  • Fix 12403: Denied Access to registry key "HKEY_CURRENT_USER"
  • Fix 11086: Stored procedures CheckIfUserCanAccessSecurable and GetAllSecurableEntities are always being dropped and created at website startup
  • Fix 12480: Could not load file or assembly 'ChilkatDotNet2' or one of its dependencies.

Version 2017.2.3 [May 1, 2017]

New Features

  • New 8734: SSIS Get and Set Properties elements now includes current design time values
  • New 8799: Directories can now be used as data sources for data driven tests
  • New 9132: Added ability to select colums from flat files
    • The user can now select which columns are included in the dataset when sourcing data from a flat file. 
  • New 10294: Fixed Width File References now allow you to choose the file format
    • Ragged right
    • Fixed width with line breaks
    • Fixed width without line breaks
  • New 10367: Rows with no differences after a grid comparison can now be viewed on the server

Updated Features

  • Update 8750: SSAS Wizard generates connection assets at the suite level and shares between the groups
    • The SSAS Wizard now creates connections at the test suite level instead of the test group.
  • Update 8800: Datasets can now be filtered
    • Datasets from any grid source can be filtered by specifying expressions on a column or multiple columns.
    • This filtering determines which rows will be seen by consumers of the grid / dataset.
    • This filtering is applied after the initial data retrieval, so it should be used primarily for data sources that don't support filtering at the source, like text files or REST APIs. Databases and similar sources should be filtered through the SELECT statements for best performance.

Fixes

  • Fix 9159: Interactive Comparison does not save changes to currently selected elements if the editor is open
  • Fix 9217: Reserved filters list in custom filters section is unreadable
  • Fix 9225: Flat File connection managers are imported as ADO.NET Connection assets within the SSIS Wizard
  • Fix 9861: Execute Task element incorrectly reports task execution failure
  • Fix 10159: File Reference does not extract column headers if all columns are of the data type string
  • Fix 10917: LegiTest Extension reports version 1.0 within the "Installed Products" section of the "About Microsoft Visual Studio" page
  • Fix 10987: Editing a newly created interactive comparison before saving could result in an ArgumentNullException
  • Fix 11151: Interactive comparison always resets the timeout to 30 when editing

LegiTest Server Version 2017.2.3 [May 1, 2017]

New Features

  • New 6809: Added ability to suppress test failures for a set time frame
  • New 9870: Tests can now be moved to trash, then deleted from the server.
    • Tests can now be moved to trash from the summary page.
    • Trash view has been added, this allows users to permanently delete tests from the server.
    • Unexecuted assemblies can no be deleted from the server on the unexecuted assembly view on the summary page.
  • New 9871: Summary Page now has separate views for Run and Not run assemblies.
  • New 10272: Users can now select existing tags from drop down when tagging a test object.
  • New 10331: Test management page now shows an item's most recent status in the navigation menu
  • New 10911: Design system for handling authorization for cloud services
  • New 11046: Notification Engine now has dynamic html template for LegiTest status

Updated Features

  • Update 6740: The "Export to Excel" feature within Grid Comparison now provides more details
    • The file name for the excel spreadsheet now includes the date and time the test was executed
    • The excel spreadsheet now contains a header that includes the suite, group, test and action name as well as the start and end time for the execution
  • Update 9868: Test Scheduler page has now been simplified for ease of use
  • Update 9873: Added "Missing Test Runner" topic to FAQ for LegiTest Server
  • Update 10978: Help documentation for setup instructions now includes details for multi-domain considerations

Fixes

  • Fix 9169: "Already added" warning appears when adding a new item to a Coverage Report
  • Fix 9228: Spaces in an assembly name are causing them not to run from the server
  • Fix 9241: Deleting a test execution fails
  • Fix 9817: Side by Side table view for errors does not have consistent height on both sides
  • Fix 9821: Breadcrumb located on View Details Page does not link
  • Fix 9869: Tests not part of previous run are not represented as such.
  • Fix 10273: Parameters containing certain special characters are not displaying properly on the test management page
  • Fix 10330: Long names in management navigation menu extend past frame.
  • Fix 11021: Managing a test with an existing Hyperlink template, will not load the template area for editing.
  • Fix 11026: When viewing a grid comparison result on the server, rows with no information still show a hyperlink.
  • Fix 10346: Some operations within DOC xPress and System Dashboard aren't responsive or appear disabled
  • Fix 10382: Workbench Server does not populate Action and RoleAction tables in host database

Version 2017.1.3 [February 2, 2017]

New Features

  • New 6732: LegiTest Command Line Tool can run test assemblies by name
  • New 7790: Added ability to load 'manual' data grids
    • Data grids can now be loaded from delimited content stored within the test suite
    • Provides an easy way to inject static data for comparison or data driven testing
  • New 7795: Added support for multiple delimited file formats
    • Delimited file format support allows a variety of flat file formats to be read.
    • Automatic detection of delimiters, quotes, header rows and preamble sections.
  • New 7876: Added Select All / None option in the SSIS Wizard
  • New 7893: Flat file data types suggestions are provided
  • New 7894: Flat files support the same data source model as other sources
    • Flat file support has been upgraded to use the common data source model in LegiTest.
    • Enables flat files to be easily used in interactive comparisons, grid validations and data-driven tests.
  • New 8298: Interactive Comparison now displays whether preview comparison would have succeeded or failed
  • New 8307: SSIS Wizard now extracts all connection managers
    • Connection managers are now converted to assets by the SSIS Wizard irrespective of whether they become part of a data flow test or not 
  • New 8310: Added ability to load fixed width files
    • Fixed width files can be used in data driven testing, interactive comparison and as a data source in normal tests
    • After entering the column widths, data formats are detected automatically
  • New 8798: Added row / line number reporting to grid comparisons
    • The row / line number is now reported as part of the error message in case of an error 

Fixes

  • Fix 5500: Timeout when gathering columns for comparison manifest with large data sets
  • Fix 7877: SSAS validation tests are not being generated in some situations
  • Fix 8238: Parameter mappings do not work within the SSAS Process actions
  • Fix 8275: Clicking "Advance" while configuring an OLE DB connection throws assembly error
  • Fix 8284: Table Row Count action did not respect parameterized table name
  • Fix 8299: Error changing keys in interactive comparison editor
  • Fix 8313: Object Reference Error when adding Test Suite to project using Read Historical Results or Interactive Wizard template
  • Fix 8754: Executing packages in a test does not recognize their associated project
  • Fix 9151: Error: Keyword not supported: 'provider' for Asset connection properties
  • Fix 9221: When using the SSIS Wizard targeting ISPAC packages, it does not generate Task level tests

LegiTest Server Version 2017.1.3 [February 2, 2017]

New Features

  • New 4878: Added "Two Grids" Layout for Grid Comparison Assert View
  • New 5000: LegiTest Server's System Coverage report now interacts with Workbench Server's (Beta) System Status feature
    • Associating a system coverage report to a view found within Workbench Server's new (Beta) System Status allows System Status to display the system coverage for any items included within both the view and the coverage report
    • The (Beta) System Status can be enabled by your Workbench Server administrator through the Administration Features page
    • By default, the new (Beta) System Status will be disabled
  • New 6782: Filtered Dashboard now shows all of a tagged item's children even if they are not tagged
  • New 7309: Users can now add their own hyperlinks to Grid Comparisons and Validations via the test management page
    • Hyperlinks can be created on the test management page and viewed on the details page. 
  • New 7399: Test Management page now includes links to other items in the same suite
  • New 8303: Test assemblies can now be run immediately after publishing
  • New 8759: Include link to the LegiTest Extension on the LegiTest Server's Dashboard page
  • New 6256: Users can now delete System Dashboard tiles directly from the dashboard
  • New 6826: The System Dashboard is now the home page for Workbench Server
  • New 9267: Workbench Server now includes public beta features that can be enabled or disabled

Updated Features

  • Update 4882: LegiTest Server now points to NUnit 2.6.4 by default
  • Update 8739: Improved product version logging at start up

Fixes

  • Fix 7810: Fixed Legitest Server json errors when NewtonSoft.Json is installed in the GAC
  • Fix 8304: Can't see error details on server for failed steps
  • Fix 8717: Notification Engine buttons do nothing if only a LegiTest Server license is being used
  • Fix 9170: There is no space between the System Coverage page title
  • Fix 9142: The license expiration date does not appear in the License Manager
  • Fix 7803: With the addition of the System Dashboard and System Status, the header is disjointed
  • Fix 7898: Workbench Server does not confirm when deleting users
  • Fix 7982: License activation and deactivation messages are confusing
  • Fix 8011: Workbench Server Licenses does not include access to System Dashboard
  • Fix 8699: System Dashboard drop-down buttons includes erroneous icons and white divider
  • Fix 8738: If Workbench Server's version falls outside a license's maintenance period, the system settings' licenses page will not display the license
  • Fix 9285: "Add Domain User" button is missing
2016

Version 2016.4.4 [October 25, 2016]

New Features

  • New 4776: Query Editor supports Azure SQL DB
    • The query editor built in to LegiTest now supports connections to Azure SQL DB 
  • New 4852: Resource key usages are now automatically remapped if a resource key is renamed
  • New 6729: Usability improvements for the Annotations UI
    • Annotation type creation pane is now a separate configuration page.
    • The content field supports Markdown for richer formatting when displayed on the server.
    • The content field size has been increased, and it now supports multiple lines.
    • Annotations for each test / group are listed under the test / group instead of a test-suite level pane.
  • New 6767: Updated comparison editor UI
    • Added an indicator that shows when code completion is available
    • Added an indicator that shows when the editor is connected
    • Added a Run Query toolbar button to each query editor
  • New 7345: Increased detail within error reports for Grid Validation assertions
    • The Grid Validation assertion now reports row details in a similar manner to the Grid Comparison assertion 
  • New 7377: Data driven testing now supported
    • Tests can now be configured to use a data set or grid as input, so that the test executes once for each row in the data set.
    • Execute the same test logic for a large number of scenarios very easily
    • Use data from any ADO.NET source, SalesForce or REST
  • New 7855: Improved validation warnings when re-using a resource key name

Updated Features

  • Update 4768: IntelliSense now available for grid collection asset editor
  • Update 5498: Validation assertions can now expect empty grids
  • Update 6215: SSAS Process Item Actions now include option to fail on process failure
  • Update 6811: Comparison Manifest editor now shows the configured grids for actual and expected

Fixes

  • Fix 4774: Query Editor picks arbitrary Sql connection for editing
  • Fix 4904: Query Editor errors are less intrusive
  • Fix 6194: Editing a comparison column mapping for an invalid column results in a modal error dialog
  • Fix 6198: VSIX Uninstaller isn't removing extensions during uninstall process
  • Fix 6625: Visual Studio 2012 crashes when using "Format SQL" button
  • Fix 6727: SalesForce Query doesn't honor parameters when providing columns for comparison / validation
  • Fix 6728: Validation Manifest that references SF Query with no rows doesn't allow editing of validation columns
  • Fix 6731: Executing the LegiTest commandline executable with no parameters throws an unhandled exception
  • Fix 6743: Help website has grammatical and spelling errors
  • Fix 6763: Interactive Comparison won't let you finish with an incomplete test
  • Fix 6764: The Interactive Comparison editor doesn't evaluate parameters defined outside of Test Suite Parameters
  • Fix 6812: Checking the Show Equal Rows option doesn't trigger an automatic reload
  • Fix 7326: Syncing does not update annotation description
  • Fix 7390: SalesForce actions do not return results if they contain three part column names
  • Fix 7397: Elements with default editor do not save values when another element is selected
  • Fix 7846: Execute REST actions do not parameterize properties
  • Fix 7904: Fixed issue with SSRS report calling Parameters

LegiTest Server Version 2016.4.5 [October 25, 2016]

New Features

  • New 4960: Scheduling tests no longer has a dependency on the LegiTest Command Line
  • New 6229: Expanded items in LegiTest Server dashboard now remain expanded when done managing tags.
  • New 6258: Enhanced LegiTest notifications to include grids from comparisons and validations
    • Grids are now shown in LegiTest notification emails so that data mismatches or validations show the data in question in the notification email (HTML only) 
  • New 6266: User can now specify the End Time for scheduled test runs
    • Allow the user to specify the end time on the end date when scheduling a task. If not specified, default it to end of day instead of using the start time. 
  • New 6278: Added ability to create VSTS work items as a notification engine target
    • Notification can now automatically create work items in Visual Studio Team Services
    • The template is applied to create a title for the created work item
  • New 6280: Added ability to run executable as a notification engine target
    • Notification engine can now run an executable on the server in response to a notification
    • Optionally the templated output is passed to the executable as a file
  • New 6281: Added ability to make URI GETs/POSTs/PUTs as a notification engine target
    • Use notification engine to interact with HTTP APIs
    • Configure the payload using the standard templating engine
    • Support for basic authentication, as well as OAUTH1 and OAUTH2
  • New 6282: Added ability to manually run notifications for specific tests from the LegiTest Dashboard
  • New 6284: Added ability to defer population of notification engine collection objects
    • Notification engine now defers some of the work in creating a notification until the notification fires, resulting in higher performance. 
  • New 6286: Implemented Role Based Security
    • Manage Access to LegiTest Server actions based on roles.
    • Manage access to tests based on roles and tags.
  • New 6784: Allowed custom name for LegiTest Tile
  • New 6828: Added Test Management Page
    • Manage Parameters, Scheduled Tasks, Tags, and certain actions all from one page. 
  • New 7809: Data driven tests are displayed as separate tests on the details page and as a single item on the summary page.
  • New 6723: Upgrading Workbench Server will now display a message indicating that all previous versions will be uninstalled during the upgrade process
    • Users upgrading Workbench Server will now see a message indicating that all previous versions of Workbench Server will be uninstalled prior to upgrading. This is not a change in behavior from previous releases, only an update to the user interface to make it more clear as to what is happening behind the scenes. This message is only visible when upgrading.
  • New 7373: Added PowerShell 3.0 or later to the system requirements for Workbench Server

Updated Features

  • Update 6827: Improvements made to LegiTest Server home page
    • LegiTest Server Dashboard is now the LegiTest Server Summary page.
    • Clicking on a failed test from the Summary will take you to the details of the failed assert.
    • Tags are now listed to the right of the Summary page, with the ability to filter tags moved, and takes up less real estate on the screen.
  • Update 4718: Activating or Deactivating a license now redirects users to license overview page upon completion

Fixes

  • Fix 6226: Parameter and Data Type Name no longer overlap when editing parameters.
  • Fix 6733: Navigation breadcrumb trail is not visible on the details page
  • Fix 6734: Command line configuration values are not being retained when upgrading LegiTest Server to the newest version
  • Fix 6735: User is now alerted if the command line can not find a valid test runner
  • Fix 6738: Summary page does not refresh after a test suite is manually run from the server
  • Fix 6744: Annotations now show up on the results page.
  • Fix 6781: LegiTest Server does not display any error if no coverage reports load
  • Fix 6800: Command Line error message reflects incorrect log location
  • Fix 7331: Notification events fire multiple times for each Test Group within a suite if configured to an assembly
  • Fix 6666: Licensing is now more robust and should not issue an error during installation.

Version 2016.3.7 [August 1, 2016]

New Features

  • New 4611: Ability to specify business metadata for tests
  • Define annotation types to represent categories of business metadata to apply to tests and groups
  • Add your own content which is recorded and available to view on the server
  • Synchronize annotation types between test suites and the server
  • New 4790: Icons for Rest Action and Rest Connections in Legitest
  • The icons for the REST actions have been updated.
  • New 4861: Add Execute REST Scalar Action
  • REST API calls can be executed that return a scalar value extracted from the response.
  • New 4922: Implemented Comparison Editor
  • Interactive comparison editor allows you to configure a comparison between two sources interactively.
  • Edit the comparison configuration and see the results in real time.
  • Configure keys, comparisons and aggregates.
  • Easily edit an existing comparison.
  • View differences in a highlighted, synchronized grid.
  • Interactive comparison can use any data source that LegiTest supports 
  • New 5451: Publishing model is more flexible
  • Publishing of DLLs to server now handled by right-clicking on a project that contains LegiTest suites.
  • Send the exact DLLs that you want to the server without any accidental publishing during development time.
  • Publishing of test results to server now handled by setting parameters.
  • Easily enable or disable publishing of test results without recompilation or regeneration.
  • New 5460: Added ability to reference resource values using the %keyName% syntax
  • Now assets and actions can make use of resource values using the same syntax as replaceable parameters in text - %parameterName%
  • In addition, we can now use DotLiquid syntax for parameter replacement in all places -
  • New 6007: Add headers to legitest rest actions

Updated Features

  • Update 4684: Updated LegiTest SSAS support to handle 2016 changes
  • Update 4747: Added ability to test connections from the connection asset editor
  • Update 4949: Users can now specify the Process Mode used within the Process Database, Cube, Dimension and Table actions
    • Users can now specify between the various Process Modes used within SSAS. 

Fixes

  • Fix 4753: Parameterized connections don't refresh properly in the Grid Collection Asset and Data Comparison Wizard
  • Fix 4756: Cannot connect to report server using a non-default virtual directory in LegiTest.
  • Fix 4880: Log4Net is not added to LegiTest Projects as a Nuget reference
  • Fix 4881: Assembly references need to be refreshed when moving between versions of LegiTest or machines
  • Fix 4886: Could not load file or assembly 'Microsoft.Data.ConnectionUI, Version=0.0.0.0, Culture=neutral, PublicKeyToken=f4ca07f51760da93' or one if its dependencies. The system cannot find the file specified.
  • Fix 4925: Obfuscated VS2015 LegiTest Extension throws XamlParseException when loading editors containing vsminipathbutton
  • Fix 4942: Downgrading ssis package asset causes its name to be reset
  • Fix 5435: Comparison Manfest Editor fails if you edit a column that no longer exists
  • Fix 5439: Rename the "Ignore Left-Only Rows" and "Ignore Right-Only Rows" properties to reference Expected and Actual
  • Fix 5455: Empty resultsets from SalesForce result in an execution error
  • Fix 5456: REST actions and assertions should be under a category of "Cloud"
  • Fix 5458: The SSAS Process Actions spin indefinitely when user clicks "connect" on a machine without 2016
  • Fix 5459: Removing a covered item does not mark suite as needing updating
  • Fix 5478: Importing a Test Suite does not import parameter mappings
  • Fix 5489: Improved UIs for SalesForce and Rest Connections & Actions
  • Fix 5505: Renaming a LegiTest file can lose custom code
  • Fix 5518: Test wizards cannot be applied at the test suite level
  • Fix 6185: Data Type combo box within Execute REST Query (Grid) is unreadable
  • Fix 4932: The VSIX Uninstaller "Uninstall" button acts as a cancel button after initializing uninstallation

LegiTest Server Version 2016.3.7 [August 1, 2016]

New Features

  • New 4879: Added ability to tag test items, and filter dashboard based on a specified tag.
    • Add tags to any test object.
    • Filter the dashboard for a given tag.
  • New 4946: New LegiTest Status Tile
    • Quickly view System Coverage for a selected tag.
    • Click through the tile to view all tests associated with the specified tag.
  • New 5449: Added ability to export Difference, Validation, and Aggregation Grids to Excel
    • Export data grid results to Excel from the web server 
  • New 5473: Test Runs Detail Page now includes an easy to follow trail to track position in the tree hierarchy.
  • New 6248: Test Suite notifications now include error details
    • Errors from LegiTest tests are now available for use in Notification Framework
    • Ability to see error messages from actions and assertions
  • New 4782: While upgrading Workbench Server, most of the existing settings can be reused.
    • Users can now choose whether to use the existing settings from the previous installation or choose to start fresh when upgrading Workbench Server.
  • New 4923: System Dashboard for Workbench Server.
    • Workbench Server's newest feature allows users to create a custom dashboard for displaying status and summary information for BI xPress' Auditing Framework and for LegiTest's System Coverage features. Users can add multiple "tiles" that are each configured for displaying concise data points they may be interested in.

Fixes

  • Fix 4242: LegiTest Server's Play Suite and Schedule Suite now supports editing and using parameters
  • Fix 4251: System Coverage search now has a longer delay after typing and initiating search.
  • Fix 4323: Duplicate TestableObjects no longer being added to table.
  • Fix 4411: LTSRV is now properly recording all exceptions.
  • Fix 4444: Testable objects now track partial and full runs and revert to untested once removed from tests.
  • Fix 5433: When scheduling a test suite to run, it will now default to the current host name.
  • Fix 5434: Dataset Comparison View properly matches columns in all instances.
  • Fix 5442: Details Grid within server now properly reflects "ignore left/right rows" options

Version 2016.2.4 [May 4, 2016]

New Features

  • New 4386: LegiTest supports SQL Server 2016
  • New 3130: Support mapping tested assets against DOCx Inventory
  • New 4161: Create LegiTest solution to verify SalesForce against CMS
  • New 4259: Execute Process action now has ability hide command window on execution

Fixes

  • Fix 4096: Disable the "Reload Coverage Reports" button when not connected to LegiTest Server
  • Fix 4097: Expected / Actual Values are reversed in Integer Comparison
  • Fix 4130: Having CodeSlice installed prevents use of LegiTest files
  • Fix 4094: SF #90816 - Snapshot Filtering on SQL Server source in IS provider

LegiTest Server Version 2016.2.4 [May 4, 2016]

New Features

  • New 4386: LegiTest supports SQL Server 2016
  • New 3130: Support mapping tested assets against DOCx Inventory
  • New 4161: Create LegiTest solution to verify SalesForce against CMS
  • New 4259: Execute Process action now has ability hide command window on execution

Fixes

  • Fix 4096: Disable the "Reload Coverage Reports" button when not connected to LegiTest Server
  • Fix 4097: Expected / Actual Values are reversed in Integer Comparison
  • Fix 4130: Having CodeSlice installed prevents use of LegiTest files
  • Fix 4094: SF #90816 - Snapshot Filtering on SQL Server source in IS provider
2015

Version 2015.3.7 [November 5, 2015]

New Features

  •     New Directory Existence Assert
    • Checks whether or not a directory exists at the target directory path    
  • New Execute SalesForce Query (Grid) Action
    • Utilizes the Query and SalesForce Connection Assets to extract data from SalesForce Objects for use during comparisons
  • New Extract Grid Checksum Action
    • Quickly generate a checksum for specific grid and stores for later use
  • New File Existence Assert
    • Checks whether or not a file exists at the target file path
  • New General Usability     
    • Allow the organizing and scoping of assets at the Test Suite and Test Group level
    • Deleting an element will display a list of all dependencies when confirming deletion
    • Include ability to import Tests and Test Groups from other LegiTest Suites
    • Include support for negative testing, allowing users to create tests the user expects to fail or throw an exception
    • LegiTest can now extract connection strings from Packages and Reports
    • Quickly find all related sources for a specific action or assert within a Test Suite by right-clicking on them and selecting "Find Sources"
    • Quickly find usage of a specific assets within a Test Suite by right-clicking on them and selecting "Find Usage"
  • New Grid Validation Assertion  
    • Grid Validation Assertion utilizes the new Grid Validation Manifest to check column and aggregated values within a grid
  • New LegiTest Server Integration
    • LegiTest can now publish LegiTest executables and assemblies to LegiTest Server for execution
  • New Read Historical Results Action 
    • Extracts historical results for a specific assertion saved to LegiTest Server and and compares results to current execution
  • New SalesForce Connection Asset
    • Utilized within Execute SalesForce Query (Grid) Actions to connects to SalesForce Objects
  • New Test Generation Templates
    • Quickly generate tests based on target SQL Data Sets, SSIS Packages, SSRS Reports, and SSAS Database Validation
  • New Test Suite Parameterization
    • Create configuration parameters and map them to the properties of specific elements
  • New Timing Start Action 
    • Utilized alongside Timing End action to signal the start of a time period for measuring a specific length of time
  • New Timing End Action
    • Utilized alongside Timing Start action to signal the end of a time period for measuring a specific length of time
  • New Validation Manifest Asset
    • Utilized within the Grid Validation Assertion to validate expected comparison values within a Grid

Updated Features

  • Update General Usability   
    • Include ability for Test Suites to be renamed
    • Include enhanced type checking for more meaningful error messages
    • Include ability to enable and disable tests and test groups
  • Update SSIS Package Reference
    • Simplify the SSIS Package Reference asset

Fixes

  • Fix T-SQL Bulk Insert Asset   
    • Bulk Insert file path now properly handles environment variables
  • Fix General Usability
    • LegiTest Test Suites now work in conjunction with source control
    • Removed LegiTest from list of valid .NET 4.0 Project Templates  
    • Assets now immediately validate whether or not they are being used by another element
  • Fix LegiTest Server Integration
    • LegiTest now gracefully handles errors returned from LegiTest Server
  • Fix SSIS Package Reference
    • Include Package Reference Assets within toolbox only if the specified version of SSIS is installed and usable
    • Fixed issue with password protected packages not loading properly

LegiTest Server Version 2015.3.7 [November 5, 2015]

New Features

  • New Directory Existence Assert
    • Checks whether or not a directory exists at the target directory path
  • New Execute SalesForce Query (Grid) Action
    • Utilizes the Query and SalesForce Connection Assets to extract data from SalesForce Objects for use during comparisons
  • New Extract Grid Checksum Action
    • Quickly generate a checksum for specific grid and stores for later use
  • New File Existence Assert  
    • Checks whether or not a file exists at the target file path
  • New General Usability 
    • Allow the organizing and scoping of assets at the Test Suite and Test Group level
    • Deleting an element will display a list of all dependencies when confirming deletion
    • Include ability to import Tests and Test Groups from other LegiTest Suites
    • Include support for negative testing, allowing users to create tests the user expects to fail or throw an exception
    • LegiTest can now extract connection strings from Packages and Reports
    • Quickly find all related sources for a specific action or assert within a Test Suite by right-clicking on them and selecting "Find Sources"
    • Quickly find usage of a specific assets within a Test Suite by right-clicking on them and selecting "Find Usage"
  • New Grid Validation Assertion   
    • Grid Validation Assertion utilizes the new Grid Validation Manifest to check column and aggregated values within a grid
  • New LegiTest Server Integration 
    • LegiTest can now publish LegiTest executables and assemblies to LegiTest Server for execution
  • New Read Historical Results Action   
    • Extracts historical results for a specific assertion saved to LegiTest Server and and compares results to current execution
  • New SalesForce Connection Asset     
    • Utilized within Execute SalesForce Query (Grid) Actions to connects to SalesForce Objects
  • New Test Generation Templates   
    • Quickly generate tests based on target SQL Data Sets, SSIS Packages, SSRS Reports, and SSAS Database Validation
  • New Test Suite Parameterization  
    • Create configuration parameters and map them to the properties of specific elements
  • New Timing Start Action 
    • Utilized alongside Timing End action to signal the start of a time period for measuring a specific length of time
  • New Timing End Action
    • Utilized alongside Timing Start action to signal the end of a time period for measuring a specific length of time
  • New Validation Manifest Asset
    • Utilized within the Grid Validation Assertion to validate expected comparison values within a Grid

Updated Features

  • Update General Usability  
    • Include ability for Test Suites to be renamed
    • Include enhanced type checking for more meaningful error messages 
    • Include ability to enable and disable tests and test groups
  • Update SSIS Package Reference     
    • Simplify the SSIS Package Reference asset

Fixes

  • Fix T-SQL Bulk Insert Asset  
    • Bulk Insert file path now properly handles environment variables
  • Fix General Usability      
    • LegiTest Test Suites now work in conjunction with source control 
    • Removed LegiTest from list of valid .NET 4.0 Project Templates            
    • Assets now immediately validate whether or not they are being used by another element
  • Fix LegiTest Server Integration
    • LegiTest now gracefully handles errors returned from LegiTest Server
  • Fix SSIS Package Reference
    • Include Package Reference Assets within toolbox only if the specified version of SSIS is installed and usable
    • Fixed issue with password protected packages not loading properly

Version 2015.2 [September 3, 2015]

New Features

  • New LegiTest Server Integration
    • Publish test results directly to LegiTest Server to keep historical records and utilize BI xPress Server's Notification Engine

Updated Features

  • Update Execute Report Action
    • Add the ability to pass parameters through to an SSRS Report executed through the Execture Report Action
  • Update Directory Copy Asset
    • Add "Strict Mode" for Directory Copy Asset
  • Update File Copy Asset
    • Add "Strict Mode" for File Copy Asset
  • Update General Usability
    • Include better design layout when areas are collapsed
    • Provide a more detailed description of error for failed tests that better identify where the failure occurred
    • Add ability to automatically version test assembly upon test generation
    • Better streamline the inclusion and usage of SSIS Packages using the Package Reference Asset

Fixes

  • Fix Set Properties Action
    • Allow properties containing brackets ("[" or "]") to be set
  • Fix General Usability
    • Removed superfluous teardown method created when adding a group level test teardown
    • Generated code now automatically updates upon changing the order of the elements within an execution track
  • Fix SSIS Assets
    • Allow the modification of Project level connection strings

LegiTest Server Version 2015.2 [September 3, 2015]

New Features

  • New LegiTest Server Integration
    • Publish test results directly to LegiTest Server to keep historical records and utilize BI xPress Server's Notification Engine

Updated Features

  • Update Execute Report Action
    • Add the ability to pass parameters through to an SSRS Report executed through the Execture Report Action
  • Update Directory Copy Asset
    • Add "Strict Mode" for Directory Copy Asset
  • Update File Copy Asset
    • Add "Strict Mode" for File Copy Asset
  • Update General Usability      
    • Include better design layout when areas are collapsed     
    • Provide a more detailed description of error for failed tests that better identify where the failure occurred   
    • Add ability to automatically version test assembly upon test generation 
    • Better streamline the inclusion and usage of SSIS Packages using the Package Reference Asset

Fixes

  • Fix Set Properties Action
    • Allow properties containing brackets ("[" or "]") to be set
  • Fix General Usability
    • Removed superfluous teardown method created when adding a group level test teardown
    • Generated code now automatically updates upon changing the order of the elements within an execution track  
  • Fix SSIS Assets 
    • Allow the modification of Project level connection strings