A primary goal of VMware Aria Operations for Applications is to provide end users with fast, real-time answers to their data questions. One of the ways we accomplish this goal is by only scanning relevant and recent data by default. However, this has led customers to ask the following questions of VMware Aria Operations for Applications support:
- My query is correct and data was previously present, but now it's not appearing. Where is my data?
- I am seeing inaccurate data results in my chart/dashboard. Did my data get deleted?
Provided that data was indeed ingested to VMware Aria Operations for Applications and the query is correct, then the unexpected data results associated with the questions above can typically be attributed to the exclusion of "Obsolete Metrics".
When a metric or time series stops reporting data and hasn't reported at least one data value within the following 14 days, then it gets marked as obsolete in VMware Aria Operations for Applications. By default, obsolete data is not scanned or rendered in order to promote optimal query performance for real-time data. If obsolete data begins to report again after 14 days, then it will no longer be classified as obsolete.
In order to view obsolete data in your query results, you will need to enable the “Include Obsolete Metrics” property in your chart (Under Advanced tab) as shown below.
If you are programmatically querying, then add includeObsoleteMetrics=true parameter in your api call.
For example;
https://your_cluster.wavefront.com/api/v2/chart/api?q=my_query&s=start_time&e=end_time&includeObsoleteMetrics=true&<optional other parameters>
Now let me answer the common questions previously noted.
My query is correct and data was previously present, but now it's not appearing. Where is my data?
I’ll explain this with an example. Let’s say you have a metric named “app.request.latency.test” which shows latency of a request in a test environment. You only ingested this metric for a period of time while you were testing and then you stopped sending it. After 14 days from the last ingested point, the associated time series will get marked as obsolete in VMware Aria Operations for Applications. If you open your chart for the time period when your test data was reported, then you will see no data.
However, data will appear once you enable the “Include Obsolete Metrics” property under the Advanced tab in your chart.
Hence, if you see no data in your chart and you know that your query is correct and data was indeed ingested, then enable include obsolete metrics property and data should appear as expected.
I am seeing inaccurate data results in my chart/dashboard. Did my data get deleted?
With obsolete data being excluded from query results by default, this can also have an impact on the way data appears once you apply advanced functions to your query. Such as aggregation functions like sum() or avg(). Let’s take the previous example of “app.request.latency.test” and assume that it was comprised of 10 total time series reported by 10 unique sources.
Now imagine that 5 out of 10 unique sources stopped reporting data and haven’t reported any data in the last 14 days. If you open a chart for a time window when all 10 time series used to report, then only the 5 time series that aren't obsolete will be displayed. You will have to enable the “Include Obsolete Metrics” property on the chart to visualize the other 5 time series which have been classified as obsolete.
If you're aggregating data, then the resulting aggregate values will not include the obsolete time series unless the property is enabled and hence it can lead to inaccurate results. The screenshot below shows an average of request latency without the "include obsolete metrics" property being enabled on the chart:
In the example above, the aggregate values range between .01 and .03 because it's only aggregating on the subset of test data that is not marked as obsolete. When we enable "Include Obsolete Metrics" in the following chart, then the query results are based on our entire test data set. Now the range of aggregate values is between 1 and 3.
When looking at historical data, it can be tricky to identify if all data is being included in your query results. In this case, we would suggest enabling the “Include Obsolete Metrics” property to ensure all data is included in your query results.
Summary:
If you see no data or inaccurate data in your chart/dashboard, specially if viewing time windows more than 14 days in the past, then it could be due to obsolete metrics. If you're viewing data in the VMware Aria Operations for Applications UI, then make sure you have enabled the “Include Obsolete Metrics” property. If you're querying programmatically, then ensure you have included the includeObsoleteMetrics=true parameter in the api call.
If enabling this property is still not returning results as you'd expect, then please contact Support for additional help.
Comments