How this dashboard works

Leadership requirements — and where each lives

Overview KPIs — formulas

count(issues) where LAST transition into a configured end status falls in range AND current status is still in end statuses AND project in scope

issuesDone(assignee ∈ active agents) / issuesDone × 100

median(businessHours(firstStart → completionAt) − pauseTime) in hours — the active variant

count(successful deployments) in range / weeksInRange

AI Impact rows — formulas

Each row is "agent median vs human median, with Δ%". Δ% is(agent − human) ÷ human × 100, colored green when better for the team (typically faster). All inside the chosen range and projects.

median(businessHours(firstStart → completionAt) − pauseTime) over agent-assigned vs human-assigned completed tickets

median(businessHours(createdAt → completionAt) − pauseTime) where completionAt = last in-range end transition, split by assignee, in hours

median(firstReadyTransition − createdAt) over agent vs human assignee, in hours

count(pullRequests) where status = completed AND author ∈ agents (or humans), in range

median(completedAt − firstReviewAt) for PRs author ∈ agents vs humans, in hours

AI Activity columns — formulas

count(completedIssueSet) where assignee ∈ agent in range

count(pullRequests) where author = agent.adoUserId AND createdAt in range

count(issues) where reporter ∈ agent AND createdAt in range

count(issue_transitions) where author = agent AND changedAt in range

count(issue_comments) where author = agent AND createdAt in range

Before vs After AI — comparing eras

A second lens on AI impact. Instead of agent-vs-human at the same moment, this splits each project's timeline at the date AI agents started doing meaningful work, then compares the same team's metrics before vs after.

All per-project metrics — quick reference

businessHours(firstTransitionTo(cycle_time.startStatuses) → completionAt) − pauseTime

businessHours(createdAt → completionAt) − pauseTime

firstTransitionTo(ready.readyStatuses) − createdAt

count(completedIssueSet) per ISO week

count(issues where issueType ∈ bugIssueTypes) created per week

unplanned / total × 100, where unplanned = issueType ∈ unplannedTypes OR label ∈ unplannedLabels

business time spent in each workflow status for issues with transitions in the selected range

count(PRs merged) per week, by author segment

businessHours(createdAt → completedAt)

businessHours(createdAt → firstReviewAt)

completedAt − firstReviewAt

completed / (completed + abandoned) × 100

businessHours(first transition into time_to_first_pr.startStatuses → first linked PR createdAt)

count(deployments) per week

failed / total deploys × 100

businessMsBetween(pr.completedAt, firstSuccessfulDeploy.finishedAt) in hours

Flow metrics (newer)

count(open issues where status ∈ activeStatuses - pauseStatuses)

for each ticket, sum hours between consecutive status transitions; aggregate per stage (median, p75, p90)

across projects in scope: median over per-project medianHours per stage; bulk-computed via work-waits-bulk.ts in one SQL pass. End statuses (Done · Closed · Released On Live · Released On Stage · Resolved) are EXCLUDED entirely. Intake/backlog statuses (New · Open · Backlog · To Do · Approved · Selected for Development · Approved for Sprint · Ready · Triaged) are routed to the sibling 'Backlog / Intake Aging' card.

same compute as Active Flow Bottlenecks, but filtered to intake-only statuses: New · Open · Backlog · To Do · Approved · Selected for Development · Approved for Sprint · Ready · Triaged

active tickets sorted by max(now − firstActiveTransition, now − createdAt) descending

Data flow & freshness

1. JIRA + ADO are the external sources of truth for issues, transitions, comments, PRs and deployments. Agent accounts, project metadata, repo/pipeline mappings, calendars, metric configs and exclusions are local app settings in SQLite.
2. Pressing Sync kicks off a worker pool that fetches external data via REST and writes to local SQLite. The page itself never talks to JIRA/ADO.
3. SQLite lives on a Kubernetes PersistentVolumeClaim — survives redeploys and pod restarts.
4. Every metric page is a pure read from SQLite. Metric range/assignee/aggregation filters are applied by the server while computing the metric; DataTable search, column filters, sorting, grouping, pagination and export are client-side over the already-loaded rows.

JIRA sync — what we actually do

• One paginated call per page-of-50. POST /rest/api/3/search/jql with expand=changelog and fields: [..., comment] — the response returns the issue, its full status changelog AND its comments INLINE for each ticket.
• Per-issue fallback only when needed. If JIRA reports the inline changelog or comments are paginated (total > what fit in one page), we call /issue/<key>/changelog or /issue/<key>/comment for that specific issue. Verified strictly — we'd rather make one extra HTTP call than silently drop data.
• Incremental. Each project records lastSyncedAt; subsequent runs only refetch issues whose updated >= lastSyncedAt.
• Resilience. Every JIRA request has a 30-second hard timeout (AbortController). On 429 / 502 / 503 / 504 we retry up to 5× with exponential backoff or the server's Retry-After header — so a transient throttle never kills the sync.
• Concurrency. Up to 3 projects in flight via a sliding-window worker pool (a slot frees the moment any project finishes; no batch barriers). Inside each project, up to 5 issue requests in parallel. Per-project hard timeout: 30 minutes (only ever needed for a truly hung response — IT's 11.8k-issue cold sync now finishes in ~1-2 min thanks to inline data).
• Audit trail. Every project sync attempt — success or error — is written to sync_runs. See /logs for the last-24h dashboard, the most-recent failure's full error message, and a sortable history of all runs.

ADO sync — what we actually do

• Repo mapping. A one-time discovery walks each ADO project and registers repos whose default branch references a known JIRA project key, storing the relation in ado_repos.
• Pipeline discovery. For every ADO project where a mapped repo lives, every classic Release Definition is registered as an ado_pipeline_configs row. If a project has no Release Definitions and the "Also discover YAML/build deploy pipelines" toggle is on, build pipelines whose name matches deploy / release / cd / prod / publish are added as a fallback.
• PR sync. Pulls merged + abandoned PRs into pull_requests; first-review touches and merge times come from the PR's thread data, so PR cycle / review / acceptance metrics work without extra wiring.
• Deployment sync. Each pipeline's recent runs go to deployments with a rolled-up environment result: inProgress / succeeded / failed / partiallySucceeded / canceled. Deployment Frequency and Change Failure Rate use the same classifier: succeeded counts as a successful deploy unless manualFailure=true, failed/partiallySucceeded/canceled enter the CFR denominator, manualFailure=true forces a change failure, manualFailure=false marks a non-success result as not-failure, and inProgress is excluded from the CFR denominator.

Where to see what happened

• /logs — last 500 sync runs with status, duration, issues fetched, full error message. Top-of-page KPIs show OK / failed counts for the last 24 h.
• On /sync and /projects: any project whose most recent sync_runs row is an error gets a red ⚠ failing chip. Hover for the reason; click to jump straight to /logs.
• While a sync is running, a small spinner appears next to the project key on both pages — the polling status comes from /api/sync/status every 2 s.
• The Sync page has a Clear synced data (keep config & agents) button if you need to force a full re-fetch — wipes synced issues / PRs / deployments + resets sync cursors but preserves all per-project configuration and the agent roster.

Filters & configuration

Top-bar (Overview)

• Range — 7 / 30 / 90 / 180 days. Same length immediately before = the "previous period" the Δ% chips compare against.
• Favorites only — restrict to projects you starred on /projects.
• Projects — explicit multiselect. Top of the dropdown: Select all · None (= all) · Invert · Favorites for bulk operations. Closes on outside click + Esc.
• Type — Task / Story / Bug / Epic / Spike. Filters every KPI, trend bucket, AI impact row, top-team count and longest-stuck list.
• Priority — Highest..Lowest. Same scope as Type.

Per-project (metric pages)

• Range — 7 / 14 / 30 / 90 days or custom dates; Assignee (All / AI agents / Humans) and the aggregation toggle (median / p75 / p90) where relevant.
• DataTable controls on every list — global search, per-column filters (toggle the "Filters" button in the toolbar), sortable headers, group-by (any column with a label), paginate (25 / 50 / 100 / 200 / All), export the visible view as CSV or JSON. All state is persisted via localStorage per-table, so reopening the page returns you to the same view.
• Exclude an outlier — open /exclusions. Issue exclusions write scope = "all" and apply across issue metrics plus linked-PR lanes; PR-only exclusions remove a specific PR from PR-level lanes.

Per-project configuration

• Project Configuration page (link from any project page) — edit the status / type lists for every metric. Defaults are audit-derived from the actual data of all 35 synced projects (see table above); override only the names your project uses differently.
• ADO repos & pipelines on the same page — see/add/remove the mapped repos and registered release/pipeline definitions for the project.
• Favorites (★ on any project tile) — pin to the top of /projects and the Overview Projects dropdown.
• Disable a project — hides it from Overview / Top Teams / per-project rollups but doesn't delete data. Toggle on /sync.
• Import / Export settings — the JSON block on /sync exports the entire config (project meta + metric configs + ADO repos / pipelines + calendars + exclusions + agents + optionally the synced issues / PRs / etc) and the same shape imports back. Granular checkboxes pick which parts.

What this does NOT do (yet)

• Track AI use outside JIRA/ADO — pure chat sessions, ad-hoc analyses without a ticket, off-platform planning. No data source.
• Track PR reviewers — only PR authors today (Veronica/Nikolaos as approvers not yet counted).
• Send digests — no email/Slack push; you open the page.
• Auto-sync — manual trigger only.
• Public access — internal AKS, VPN required.

Data Quality panel

Bottom of Overview. Seven honest checks against the data the rest of the page is computed from, rendered as a 4-column table (Check · Status · Current value · What it means). Hover the ⓘ on each column header for what that column shows; hover the ⓘ next to each check name for Formula · Example · When to worry · What to do. Amber = a caveat to mention when quoting the rollup; not necessarily a bug.

• Sync freshness — count of projects whose last sync (MAX(startedAt)) is older than 24h. Stale > 24h means quoted numbers may lag real-world JIRA state.
• Latest sync status — for each project, the status of the MOST RECENT sync_runs row. This is the current-health signal: green here means every project's data came from a successful pull, even if older runs in the 24h window failed.
• Historical errors (last 24h) — count of sync_runs.status = 'error' rows in the last 24h. Transient noise unless Latest sync status above is also red. Was previously merged with the row above; split out so a recovered project doesn't look broken.
• Metric configs — projects missing one or more of the 11 metric_configs rows. Compute falls back to DEFAULT_CONFIGS for missing rows, but a missing row often signals an unfinished onboarding.
• AI Period coverage — number of projects with ai_period_settings.mode != 'off'. Drives Before vs After AI. When this is 0/N, the Before vs After AI section auto-hides.
• Stage config coverage — semantic check: for each status currently in use (≥1 unresolved issue OR ≥50 transitions in 30d), is it listed in every observing project's time_in_status.stages? If not, Where Work Waits silently drops it. Cancellation statuses (Removed / Deprecated / Cancelled / Won't Do / Rejected / Duplicate) are whitelisted — they intentionally end tickets without delivering value, not workflow waypoints. Also checks that every wip config has pauseStatuses populated.
• PR ↔ JIRA linkage — per project, share of completed PRs whose branch name or PR title contains a JIRA issue key. <60% means PR/JIRA join metrics for that project are unreliable; quote PR-only metrics (PR Throughput, Review Duration) instead.

Migration when defaults change. When DEFAULT_CONFIGS in src/lib/config.tsgains new statuses, stored project configs don't auto-update — they only get upgraded on the next page visit via the LEGACY_DEFAULTS exact-match in ensureDefaultConfigs(). To proactively migrate all 35 projects at once, run npm run db:migrate-configs (calls ensureDefaultConfigsfor every project; prints before/after counts of wait-status presence). The cancellation whitelist excludes Removed/Deprecated/Cancelled/Won't Do/Rejected/Duplicate statuses.

Change Failure Rate weighting. The Before vs After AI rollup's CFR row is a weighted average across projects. The weight is totalDeploymentCount from the shared DORA classifier (successful deployments plus finished non-success deployments, with manual failure/not-failure overrides applied), NOT just successful deploys — otherwise a project with high failure rates gets under-weighted and the rollup looks optimistic. Throughput / Deploys-per-week n-counters still use successfulDeploymentCount (those measure delivery cadence, not exposure to failure).

AI Narrative Layer

The dashboard has a content-aware AI layer that writes the executive narrative on top of the deterministic metrics. The AI never calculates metrics; it explains pre-computed numbers from the evidence pack and cites evidence IDs for every claim. When AI is off or fails, the deterministic templated narrative fills in seamlessly.

How a request becomes a narrative

1. computeEngineeringFlowReport builds the deterministic JSON (KPIs as MetricNumber with current/previous/deltaPct/n/confidence, AI-owned + AI-touched cohorts, team health matrix, longestStuck, warnings).
2. buildEvidencePack flattens the report into evidence facts (kpi.*, ai.owned.*, ai.touched.*, top_team.*, team_health.*, active_wait.*, intake_aging.*, data_quality.*).
3. The variant-specific prompt + the evidence pack are sent to Azure OpenAI (gpt-5.4 via the v1 Responses API with strict Structured Outputs).
4. A deterministic verifier validates that every evidenceIds citation in the narrative exists in the pack. If invalid, it retries once with the errors appended to the prompt; if still invalid, the deterministic narrative fallback kicks in.
5. On success, the structured ReportNarrative JSON is wired into Page 1 of the PDF (with a small AI-narrated chip in the footer) and into the JSON response (narrative field).

Report variants

Same evidence pack, different writing instructions. All return the same ReportNarrative JSON shape so the PDF and Teams renderers can be variant-agnostic.

/overview AI brief banner

Sticky banner at the top of /overview rendering narrative.bottomLine + the top 4 keySignals for the current scope. Auto-refreshes when scope or range changes. Compact "AI brief unavailable" line + retry button when the AI service is off / down. Manual refresh button bypasses any cache.

Cohort and structural extensions

• AI-owned: strict attribution — agent is the assignee at completion. The historical AI Impact rows. Available as report.aiImpact.owned.
• AI-touched: broader cohort — agent is assignee or reporter or comment author or transition author on the ticket. Surfaces AI participation that isn't visible from assignee alone. Omitted when no AI activity in scope. Available as report.aiImpact.touched.
• Team health matrix: every top team now carries leadP50Hours, cycleP50Hours, timeToReadyP50Hours, activeWipNow, blockedWipNow, topBottleneckStage, topBottleneckHours, aiOwnedPct. Same canonical cohort as headline throughput.
• Longest stuck: top 8 active tickets sorted by max(now − firstActiveTransition, now − createdAt) with assignee + isAgent flag — drives narrative recommendations.
• Warnings: data-quality-derived findings (sync_freshness, stage_coverage, ai_period, sample_size, data_quality) — the narrative cites them in the dataQuality section.
• Period boundaries: period.start / end / previousStart / previousEnd / rangeDays on every report so the period is unambiguous.
• Versioning: reportVersion: "engineering-flow.v8" and metricDefinitionVersion: "2026-05-24.audit18.v1" for reproducibility.

API endpoints

POST /api/reports/engineering-flow            # JSON report (full body + optional narrative)
POST /api/reports/engineering-flow/pdf        # PDF render (narrative wired into Page 1)
POST /api/reports/narrative                   # narrative-only (faster — used by the /overview banner)
POST /api/reports/parse-intent                # natural language → strict ReportRequest JSON
GET  /api/reports/snapshots                   # list persisted snapshots
GET  /api/reports/snapshots/[id]              # fetch one snapshot

Request body for the first three:
{
  "scope": { "projectKeys": ["AN","EVO",...], "name": "Leadership scope" },
  "rangeDays": 30,
  "variant": "leadership-brief",       // or any of the 9 variants above
  "depth": "brief" | "standard" | "deep",
  "language": "en" | "ru",
  "includeNarrative": true,             // default true; set false to skip the LLM
  "persist": false,                     // set true to save the snapshot for follow-ups
  "filters": { "issueType": "...", "priority": "..." }
}

Trust + fallback behaviour

• AI never blocks reporting. When AI_REPORTS_ENABLED=false, API key missing, Azure 5xx, schema invalid, or verifier still failing after retry → narrative: null is returned and the deterministic templated narrative is used in the PDF.
• The verifier (cheap second pass) checks every evidenceIds citation against the evidence pack. Citations to unknown IDs reject the narrative. High-confidence recommendations with empty evidenceIds reject. Gated by AI_VERIFIER_ENABLED; can be skipped without disabling the rest.
• Snapshots persist the full report + narrative under a UUID. The intent parser, the banner, and future Teams bots can reference snapshot IDs to resolve evidence citations back to specific numbers.

Environment

AI_REPORTS_ENABLED=true
AZURE_OPENAI_ENDPOINT=https://<resource>.openai.azure.com
AZURE_OPENAI_API_KEY=<rotated>
AZURE_OPENAI_REPORT_MODEL=gpt-5.4
AZURE_OPENAI_API_VERSION=preview
AZURE_OPENAI_REASONING_EFFORT=low | medium | high
AZURE_OPENAI_VERBOSITY=low | medium | high
AZURE_OPENAI_MAX_OUTPUT_TOKENS=20000     # reasoning models burn many tokens internally
AZURE_OPENAI_TIMEOUT_MS=60000
AI_VERIFIER_ENABLED=true | false
AI_REPORT_CACHE_TTL_SEC=300              # safety net; primary cache is content-addressed
AI_REPORT_DEFAULT_VARIANT=leadership-brief
AI_REPORT_DEFAULT_LANGUAGE=en

Engineering Flow Report endpoints

Two POST endpoints that return the same report shape, one as JSON and one rendered to a 2-page A4 PDF (executive brief on page 1, detail tables on page 2). Both are deterministic from {scope, rangeDays, now}.

POST /api/reports/engineering-flow
POST /api/reports/engineering-flow/pdf

Body:
{
  "scope": { "projectKeys": ["AN","EVO","ADS"], "name": "Leadership scope" },
  "rangeDays": 30
}

Same compute as Overview — Issues done, AI Impact (with sample sizes), Top Teams, Where Work Waits, Agent Activity, AI Period rollup, Data Quality. No new metrics. Use the JSON endpoint when you want to embed the data into another system; use the PDF endpoint when you want a shareable artifact.

Questions? The repo lives at custom-agile-metrics-app. Specs and plans for ongoing work live under docs/superpowers/.