Jenkins problems usually surface as complaints: "my build is stuck", "no agents available", "everything was green yesterday". The Jenkins UI shows the current state, but not how the queue grew or when the free executors ran out. This post covers the metrics that answer those questions and five alert rules in PromQL.
The examples use a free Grafana dashboard, Jenkins Lite: https://grafana.com/grafana/dashboards/26040/ . Every metric name below comes from the dashboard JSON.
Expose the metrics
Install the Prometheus metrics plugin in Jenkins. It serves metrics at /prometheus, with the default prefix default_jenkins_. If you changed the prefix in the plugin settings, the dashboard queries need the same change.
scrape_configs: - job_name: jenkins metrics_path: /prometheus static_configs: - targets: ["jenkins:8080"] If your Jenkins requires authentication, add credentials to the scrape job (use placeholders in anything you share). Per-job panels rely on the jenkins_job label, which the plugin adds by default.
What to watch
Queue. default_jenkins_queue_size_value is the total. It splits into _buildable_, _blocked_, _pending_ and _stuck_ (all default_jenkins_queue_*_value). A short queue after a burst of commits is normal. A growing stuck count usually means no agent matches the job's label.
Executors and nodes. default_jenkins_executor_in_use_value, default_jenkins_executor_free_value and default_jenkins_executor_count_value, plus default_jenkins_node_online_value and default_jenkins_node_offline_value. Zero free executors is only a problem when the queue is not empty.
Build success and duration. The plugin exposes counters default_jenkins_builds_success_build_count and default_jenkins_builds_failed_build_count. In the JSON, the success rate over the selected range is:
sum(increase(default_jenkins_builds_success_build_count[$__range])) / ( sum(increase(default_jenkins_builds_success_build_count[$__range])) + sum(increase(default_jenkins_builds_failed_build_count[$__range])) ) These counters restart from zero when Jenkins restarts, so right after a restart the window has little data. Average duration comes from the summary metric (milliseconds):
sum(rate(default_jenkins_builds_duration_milliseconds_summary_sum[$__rate_interval])) / sum(rate(default_jenkins_builds_duration_milliseconds_summary_count[$__rate_interval])) For triage the JSON has top-N panels: failed builds by job, slowest jobs (default_jenkins_builds_last_build_duration_milliseconds) and lowest job health (default_jenkins_builds_health_score). In the JSON, the table of jobs whose last build did not succeed uses default_jenkins_builds_last_build_result_ordinal.
Health. default_jenkins_up, default_jenkins_health_check_score, and the plugin problem counters default_jenkins_plugins_failed and default_jenkins_plugins_inactive.
Five alert rules
Thresholds and for durations are starting points; tune them to your load.
groups: - name: jenkins rules: - alert: JenkinsDown expr: min by (job) (default_jenkins_up) == 0 for: 3m labels: {severity: critical} - alert: JenkinsQueueStuck expr: sum by (job) (default_jenkins_queue_stuck_value) > 0 for: 15m labels: {severity: warning} - alert: JenkinsExecutorsSaturated expr: | sum by (job) (default_jenkins_executor_free_value) == 0 and sum by (job) (default_jenkins_queue_size_value) > 0 for: 10m labels: {severity: warning} - alert: JenkinsNodeOffline expr: sum by (job) (default_jenkins_node_offline_value) > 0 for: 15m labels: {severity: warning} - alert: JenkinsHighFailureShare expr: | sum by (job) (increase(default_jenkins_builds_failed_build_count[1h])) / ( sum by (job) (increase(default_jenkins_builds_success_build_count[1h])) + sum by (job) (increase(default_jenkins_builds_failed_build_count[1h])) ) > 0.5 for: 10m labels: {severity: warning} Notes: if Jenkins stops answering entirely, the series may disappear, so pair JenkinsDown with up{job="jenkins"} == 0 or absent(default_jenkins_up). The failure-share rule returns nothing when no builds ran in the hour (the division yields NaN, which the comparison drops). Lower the 0.5 threshold if your builds are normally almost all green.
Install the dashboard
Grafana -> Dashboards -> New -> Import, enter ID 26040 (https://grafana.com/grafana/dashboards/26040/) or upload the JSON, and pick your Prometheus data source. Grafana 10.0 or newer is required. Variables job, jenkins_job and topn narrow the view.
I keep ready-made alert rules and Telegram routing in a Telegram bot: https://t.me/devkit_shop_bot?start=gf_jenkins_article
Summary
Five questions on one screen cover most of it: is Jenkins up, is the queue growing, are executors free, are all nodes online, and what share of builds fail. Everything else is per-job triage. If you use different metrics or thresholds, tell me in the comments.
