Skip to content

Metrics Monitoring

Metrics monitoring is different because it does not assume that a situation can be explained in fixed states. Instead, it brings you inside your system and provides dizens of metrics that you can then analyze and understand.

Even if you do not need all the metrics, it is better to collect them to look further what they look like.

What is a metrics

  • Name
  • Timestamp
  • Labels
  • Value
  • Type

What are the 4 types of metrics prometheus knows?

  • Gauge
  • Counters
  • Histograms
  • Summaries

The number of http requests is expressed in ...

The duration in ...

The number of active sessions is a ...

What are the labels added by prometheus ?

What are the labels prometheus knows but does not add?

Labels matching

For prometheus to do calculations and comparisons of metrics, labels must match one each side (except name)

Functions

Some important functions:

  • Math: = - / *

  • rate

  • sum
  • deriv
  • delta
  • increase
  • count

In the list above, which ones should be used with counters, and which ones with gauges?

What is the difference between irate and rate? idelta and delta?

Aggregation

How can I get the sum of scraped metrics by job (2 ways)? -- exclusion and inclusion.

How can I get the % of /federate over the other prometheus_http_request_duration_seconds_count ?

Solution
rate(prometheus_http_request_duration_seconds_count{handler="/federate"}[5m])
/ ignoring(handler) group_right sum without (handler)
(rate(prometheus_http_request_duration_seconds_count[5m]))

Over Time

What is the difference between max and max_over_time?

Max, Min, Bottomk, Topk

What is the difference between max and topk?

Time functions

day() day_of_week()

How to use the optional argument of day_of_week?

What is the timestamp() function? How can it be useful?

Tip

You can use Grafana Explore feature to get help and autocomplete on Prometheus (currently still a "beta" feature).

And/Or

Can you think of any usecases for and/or/unless?