Configure Prometheus on a Sub-Path behind Reverse Proxy

Posted on Sep 24, 2020

While configuring my new Prometheus and Grafana setup, I had to spend some time figuring out how to get the Prometheus UI to correctly serve on a sub-path, e.g. example.com/prometheus.

I won’t bore you with all the intermediate steps I tried (and failed!), but rather jump straight to the conclusion: You need to set the --web.external-url flag to /prometheus/:

1
prometheus --web.external-url=/prometheus/

From the Prometheus manpage:

1
2
3
4
--web.external-url=<URL>
    The URL under which Prometheus is externally reachable (for example, if Prometheus is served via a reverse proxy). Used for generating relative and absolute links back to Prometheus itself. If the URL has a path portion, it will be used to prefix all HTTP endpoints served by Prometheus. If omitted, relevant URL components will be derived automatically.
--web.route-prefix=<path>
    Prefix for the internal routes of web endpoints. Defaults to path of --web.external-url.

(By the way: I’m not sure what’s so hard about just documenting the command line flags on the offcial documentation pages instead of an unhelpful “just run prometheus -h”)

Then configure the reverse proxy so that it forwards requests for /prometheus/ to the address Prometheus is listening on, but without stripping the prefix (!).

Traefik v2:

1
traefik.http.routers.<my-router>.rule="Host(`example.com`) && PathPrefix(`/prometheus/`)"

Traefik v1:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
prometheus:
  image: prom/prometheus:v2.20.0
  command:
     - '--web.external-url=/prometheus/'
     - '--web.route-prefix=/prometheus/'
  # ...
  deploy:
    labels:
      - traefik.frontend.rule=Host:example.com;PathPrefix:/prometheus/
      - traefik.port=9090

Nginx:

1
2
3
location /prometheus/ {
    proxy_pass http://localhost:9090;
}

Thanks also to this issue for some inspiration.

Hope this helps, happy monitoring!