Title: How to Identify Unused Data
This article applies to:
- Usage Monitoring
- Product edition: N/A
- Feature Category: N/A
Overview
Data ingested is useful if it is also being used. The main way to use data in VMware Aria Operations for Applications is to query for it, whether it be in charts or dashboards or in alert conditions. This article covers some strategies for identifying data that is not being used.
Strategies
Using WFTop
One of the columns of the WFTop output is % Acc. This indicates the percentage of data within each namespace that has been accessed via queries (whether it is in charts or dashboards, alerts, API calls, etc.) over the last 7 days (note: the number of days is configurable in WFTop configurations). As always with WFTop, let the tool run for several minutes for more reliable results. Start with namespaces that have high ingestion rates but low access rates.
Using the Access API
The Access API allows you to provide a specific metric name (or histogram name, span name, etc) and see whether or not it has been accessed in a query over the last 7 days (by default, but configurable up to 60 days).
A common strategy is to start with metric namespaces that contribute the most to the overall ingestion rate and then create a script to determine all of the metric names within those namespaces and feed each of those metric names to the Access API. The Metric Browser lists non-obsolete metric names. There is an underlying (undocumented) API that the Metric Browser uses that you can try to take advantage of (hint: use your browser's developer tools to see the underlying API calls made). While it is possible to list all metric names, it is recommended to focus on specific namespaces one at a time due to the possible sheer number of metric names.
Examining Metrics Usage in Dashboards and Alerts
Besides the previous approach, another strategy to discover metrics that are not being used is to check whether the metrics within a namespace are actually being used in dashboard charts or in alerts. The general steps are to determine all of the metric names within a namespace, check whether each metric name is included in any chart queries for all dashboards, and check whether each metric name is included in any alert queries. To find all the metric names within a particular namespace, use the process as described in the Using the Access API approach. API endpoints exist for retrieving all dashboards and for retrieving all alerts.
The only caveat here is that there's a chance that some metrics are only queried for in ad hoc charts. While this is possible, it's more likely that important data is already used in dashboards and alerts.
See Also
How to Reduce Your Ingestion Rate (PPS)
Find Actionable Usage Information
Comments