Data Capacity Planning

Note:  See the Apply SQL Server Data Compression article for information and T-SQL scripts to apply data compression to your SentryOne database. 

We also have the SentryOne Scalability Pack for large environments to take advantage of columnstore compression and performance. See the Installation Recommendations article and SentryOne & Microsoft Achieve SQL Server Monitoring Performance Goals blog post for additional information.

Performance Analysis uses the SentryOne database to store all the performance data it collects, utilizing a high performance storage scheme. Users should expect their existing database to approximately double in size if all the existing SQL Servers watched by the Event Manager are watched by Performance Analysis. This is a very rough estimate, because exactly how much space is used by Performance Analysis is directly dependent on:

  • The number of databases on the watched SQL Servers because some of the performance counters collected by Performance Analysis are database specific.
  • The number of physical disks on the watched targets because related counters are disk specific.
  • The Minimum Duration specified for the Top SQL event source. The default global setting is five seconds, meaning that any batches or stored procedures that run for longer than five seconds are collected. If this threshold is lowered, the amount of Top SQL data collected increases. A different Minimum Duration is specified for each SQL Server.
  • Whether Collect Statement Events is set to True for the Top SQL event source. The default is False. If enabled, this may increase the amount of Top SQL data collected by a factor of two or more. This setting is also adjustable for each SQL Server.
  • The Performance Data Retention settings. Different settings can be specified for detailed (or raw) performance data, rolled up performance data, and Top SQL/Blocking SQL/Deadlock data.

For detailed performance data, retention is specified in hours for each performance counter category in the HistoryDataRetentionHours column of the PerformanceAnalysisCounterCategory table. The default may be either 48 or 72 hours, depending on the category. Raw data is shown by default on the Dashboard and Disk Activity tabs when the current date range is less than or equal to 30 minutes. Over 30 minutes, rolled up data is used.                 

  • If you have an unusually large number of databases on SQL Servers monitored by Performance Analysis, consider reducing the retention hours for the SQLSERVER:DATABASES and SQLPERF:VIRTUAL_FILESTATS categories. Data for these categories are stored in the PerformanceAnalysisDataDatabaseCounter and PerformanceAnalysisDataDiskCounter tables respectively.
  • If you have an unusually large number of physical disks per target monitored by Performance Analysis, consider reducing the retention hours for the PHYSICALDISK category. Data for this category is stored in the PerformanceAnalysisDataDiskCounter table.
  • Data for all other categories is stored in the PerformanceAnalysisData table.
  • Keep the retention hours the same for categories that are stored in the same table, otherwise page splitting and fragmentation may result during the pruning process that may eventually affect performance.
  • For rolled up performance data, retention is specified in hours for each rollup level in the HistoryDataRetentionHours column of the PerformanceAnalysisDataRollupLevel table. Rollup data for each break level (specified by the LevelBreakMinutes column) is stored in a separate table, all named PerformanceAnalysisDataRollupXX, where XX represents the ID of the break level. The only rollup table that may get large is the table for the two minute break level, or PerformanceAnalysisDataRollup2. The retention hours for this, or any other break level, can be adjusted as needed.
  • Retention for raw Top SQL, Blocking, and Deadlock data is controlled by the Purge History Older Than setting on the Performance Monitor tab under SentryOne Server > Settings in the Navigator pane. The default is 15 days.
  • If you're using the Event Manager with Performance Analysis that enables viewing Performance Analysis data on the Event Manager calendar, the raw Top SQL, Blocking, and Deadlock data is converted to the native Event Manager storage format and stored in the EventSourceHistory table alongside data for other Event Manager event sources like SQL Agent Jobs. Retention for all Event Manager sources is controlled by the Purge History Older Than setting on the Event History Monitor tab under SentryOne Server > Settings in the Navigator pane.

Expired performance data is pruned by the SentryOne monitoring service every minute or so. The default settings enable you to always have detailed performance data for the last two or three days. However, if you find that you're frequently navigating to date ranges using the Dashboard or Disk Activity tabs where no data is shown, it may mean that you need to increase the retention hours for the detailed and/or rolled up performance data. Balance any changes with the resulting impact it has on database size.

When you start using Performance Analysis, you'll find that your SentryOne database grows quickly at first. After a few days this levels off once the pruning of expired data begins and starts keeping pace with the incoming new data. Get an idea of the mix of Performance Analysis data in your environment by inspecting sizes for the related tables using the following script. Much of the data in EventSourceHistory is likely related to the Event Manager sources.

Performance Analysis Data Script

SELECT
TableName = OBJECT_SCHEMA_NAME([object_id]) + '.' + OBJECT_NAME([object_id]),
[RowCount] = SUM(CASE WHEN index_id IN (0,1) THEN row_count ELSE 0 END),
UsedSpaceMB = SUM(used_page_count / 128),
ReservedSpaceMB = SUM(reserved_page_count / 128)
FROM sys.dm_db_partition_stats
WHERE OBJECT_NAME([object_id]) IN
(
'BlockChainDetail',
'EventSourceHistory',
'MetaHistorySqlServerBlockLog',
'MetaHistorySqlServerTraceLog',
'PerformanceAnalysisData',
'PerformanceAnalysisDataDatabaseCounter',
'PerformanceAnalysisDataDiskCounter',
'PerformanceAnalysisDataRollup11',
'PerformanceAnalysisDataRollup2',
'PerformanceAnalysisDataRollup4',
'PerformanceAnalysisDataRollup6',
'PerformanceAnalysisDataRollup8',
'PerformanceAnalysisTraceData',
'PerformanceAnalysisPlan',
'PerformanceAnalysisPlanOpTotals',
'PerformanceAnalysisTraceCachedPlanItems',
'PerformanceAnalysisTraceDataToCachedPlans',
'PerformanceAnalysisTraceQueryStats',
'MetaHistorySharePointTimerJob',
'PerformanceAnalysisSsasUsageTotals',
'PerformanceAnalysisSsasCubeDimensionAttribute',
'PerformanceAnalysisSsasTraceDataDetail'
)
AND OBJECTPROPERTY([object_id], 'IsUserTable') = 1
GROUP BY [object_id]
ORDER BY TableName;