Grafana / Helios setup

In-depth map of how Grafana is deployed on moneta — compose stack, auth, datasources, scrape mesh, dashboards, and the garage Postgres path.

What Helios is

Grafana is not a standalone install. It lives inside /dock/helios on moneta (10.0.0.10), alongside Prometheus, Homarr, Unpoller, and node-exporter. Public entry is Caddy at grafana.gliffy.tv.

13/13Prometheus targets up

7Provisioned dashboards

2Datasources

30dProm TSDB retention

Public URLs

Homarr: helios.gliffy.tv

Grafana: grafana.gliffy.tv

Local binds only — Caddy reverse-proxies 127.0.0.1:7575 / :3000. Prometheus (:9090) stays LAN/localhost.

Auth model

Grafana: local admin + Authentik OIDC (GF_AUTH_GENERIC_OAUTH_*).

Homarr: AUTH_PROVIDERS=oidc,credentials against auth.gliffy.tv.

Sign-up disabled for Grafana local users; OAuth allow-sign-up enabled for Authentik accounts.

Compose services (localhost binds)

Container	Image	Bind	Role
helios-grafana	grafana/grafana:latest	127.0.0.1:3000	UI + provisioning
helios-prometheus	prom/prometheus:latest	127.0.0.1:9090	TSDB · 30d retention
helios-homarr	homarr-labs/homarr	127.0.0.1:7575	Front door
helios-unpoller	unpoller/unpoller	127.0.0.1:9130	UniFi → Prometheus
helios-node-exporter	node-exporter:v1.9.1	127.0.0.1:9100	Moneta host metrics

Source: /dock/helios + Prometheus /api/v1/targets · snapshot 2026-08-21

Data paths

Two independent ingest paths feed Grafana. Metrics go through Prometheus scrapes. Garage vehicle data is written via the telemetry API into Postgres; Grafana queries those views with a read-only role.

Caddy→ Grafana→ Prometheus→ Unpoller / exporters

garagepi / app→ telemetry-api :8120→ Postgres← Grafana (read-only)

Path A — Prometheus

Exporters / Unpoller → Prometheus (:9090, 15s scrape, 30d) → Grafana datasource prometheus (default).

Covers host, GPU, Ollama tok/s, ComfyUI queue, UniFi network.

Path B — Postgres

garagepi / Android → telemetry-api :8120 → Postgres telemetry → Grafana datasource telemetry-postgres.

Apps never talk to Grafana or Postgres for writes; only the API writes, Grafana reads.

Hosts in the scrape mesh

Host	IP	Role	Exporters
moneta	10.0.0.10	storage / LLM	node, ollama :9102, nvidia :9104
glifftop	10.3.119.177	gaming / LLM	node, amdgpu :9101, ollama :9102, comfy :9103
bigscreen	10.4.199.235	gaming / Comfy	node, amdgpu :9101, ollama :9102, comfy :9103
UniFi	10.0.0.1	network	via Unpoller → :9130

Desktop exporters run as systemctl --user with linger. Templates under /dock/helios/*-exporters/.

Provisioned datasources

Files under /dock/helios/grafana/provisioning/datasources/ mount read-only into Grafana at /etc/grafana/provisioning.

Prometheus (default)

UID prometheus · type prometheus

URL http://prometheus:9090 (Docker DNS on helios network)

Access: proxy · timeInterval 15s · HTTP POST

Telemetry Postgres

UID telemetry-postgres · postgres plugin

Host 10.0.0.10:5432 · db telemetry

User grafana (read-only) · SSL disabled (LAN)

Password from GRAFANA_DB_PASSWORD in /dock/postgres/.env

Why two datasources

Time-series infra metrics belong in Prometheus. Arrival/OBD session rows are relational event data already stored for the ingest API — SQL views are the right surface for garage dashboards.

Prometheus scrape mesh

Config: /dock/helios/prometheus/prometheus.yml · global scrape/eval 15s · retention 30d · live probe 13/13 up.

Port convention

Port	Exporter	Where
9100	node_exporter	moneta + desktops
9101	amd-gpu-exporter	glifftop + bigscreen
9102	ollama-exporter	moneta + desktops
9103	comfy-exporter	glifftop + bigscreen
9104	nvidia-gpu-exporter	moneta (2× T4)
9130	unpoller	Helios compose

Live targets

Job	Instance	Host	Health
prometheus	localhost:9090	—	up
unpoller	unpoller:9130	—	up
node	node-exporter:9100	moneta	up
node	10.3.119.177:9100	glifftop	up
node	10.4.199.235:9100	bigscreen	up
amdgpu	10.3.119.177:9101	glifftop	up
amdgpu	10.4.199.235:9101	bigscreen	up
ollama	10.0.0.10:9102	moneta	up
ollama	10.3.119.177:9102	glifftop	up
ollama	10.4.199.235:9102	bigscreen	up
comfyui	10.3.119.177:9103	glifftop	up
comfyui	10.4.199.235:9103	bigscreen	up
nvidia	10.0.0.10:9104	moneta	up

Custom exporters

ollama-exporter — tails journald tok/s lines + polls /api/ps
comfy-exporter — polls ComfyUI /system_stats, /queue, /prompt
amd-gpu-exporter — consumer AMD via sysfs (:9101)
nvidia-gpu-exporter — Moneta Tesla T4s (:9104)

Dashboard provisioning

Two file providers reload every 30s with UI edits allowed. Helios folder → .../dashboards/json; Garage folder → .../dashboards/garage.

Folder	Title	UID	Panels	Datasource
Helios	Moneta Overview	moneta-overview	76	Prometheus
Helios	UniFi-Poller: Network Sites	9WaGWZaZk	6	Prometheus
Helios	UniFi-Poller: Client Insights	jMfvAjxWz	21	Prometheus
Helios	UniFi-Poller: UAP Insights	g5wFWqxZk	27	Prometheus
Helios	UniFi-Poller: USG Insights	4Yo8IZ-Wk	3	Prometheus
Helios	UniFi-Poller: USW Insights	FsfxpWaZz	10	Prometheus
Garage	Ioniq 5 · garagepi	ioniq5-garagepi	16	Telemetry Postgres

Moneta Overview rows

Moneta — CPU & memory
Storage pools
UniFi
Desktops — CPU / RAM / disk
AMD GPUs — Glifftop & Bigscreen
NVIDIA GPUs — Moneta 2× Tesla T4
Ollama — tokens / second
ComfyUI — queue & VRAM

Ioniq 5 · garagepi

HV SOC / SOH / Remaining / Pack V / 12V stats
Timeseries for pack, temps, CEC/CED, 12V
Recent garagepi events table

/d/ioniq5-garagepi

Garage telemetry → Grafana

Relational EV/garage data — Grafana is the visualization front door; writes go through the ingest API only.

:5432Postgres on moneta

:8120telemetry-api ingest

grafanaread-only DB role

/dockpostgres + telemetry-api

SQL views

View	Purpose
ioniq_events_for_grafana	arrival + obd_manual events with HV SOC
ioniq_readings_for_grafana	Mode 22 / Mode 01 readings joined to sessions
arrivals_for_grafana	arrival-only (legacy)

Contract boundaries

Writers: garagepi outbox, Android trips, Pi web UI — bearer-keyed POST /v1/…
Dashboard readers: Grafana → Postgres SELECT on views only
Do not point apps at Postgres credentials; Pi is not the system of record

Day-to-day operations

Start / reload

cd /dock/helios && docker compose up -d

Prom config: docker compose exec prometheus kill -HUP 1

Helpers: ./reload-grafana-dashboards.sh, ./apply-monitoring.sh

Persistent data

Grafana: ./data/grafana

Prometheus: ./data/prometheus

Homarr: ./data/homarr

Provisioning: ./grafana/provisioning + ./prometheus/prometheus.yml

Caddy ingress

helios.gliffy.tv → 127.0.0.1:7575

grafana.gliffy.tv → 127.0.0.1:3000

Snippet: /dock/helios/caddy/Caddyfile.snippet

Known sharp edges

Unpoller container healthchecks have flaked while scrapes still succeed. Desktop exporters need linger. Bigscreen may report ollama_up=0 if Ollama is stopped even when the exporter scrape is up.