AI Infra Readiness Audit: What I Check (and What You Get)

Most “AI infra” conversations go straight to tooling. In practice, the failures are usually simpler:

• GPU spend climbs without a clear baseline.
• Production reliability is shaky (and telemetry doesn’t answer the first two questions you ask).
• Deploys are stressful because rollback paths aren’t real.

When those are true, you don’t need a new platform. You need a diagnostic that turns “we think” into “we know”, then gives you a sequence you can actually ship.

TL;DR

• Start with a fixed-scope readiness audit, not an open-ended retainer.
• Your first deliverable is a baseline (cost + reliability), not a redesign.
• The output should be a 90‑day roadmap with dependencies, owners, and quick wins.

If you want the packaged version of this, it’s here: /services/ai-infra-readiness-audit.

The checklist (high signal, no fluff)

1) Workloads and success metrics

I want to know:

• What are the top 3 workloads that matter (revenue, customers, internal dependency)?
• What does “success” look like for each (p95 latency, errors, throughput, cost per request)?
• What is the current bottleneck: GPU saturation, memory, IO, networking, queueing, model choice?

2) GPU cost baseline (and what moves it)

You can’t optimize what you can’t explain.

Baseline questions:

• What did GPU-related compute cost in the last 30/60/90 days?
• What’s the unit economics: cost per request, cost per minute, cost per token?
• What are the “cost multipliers” (always-on idling, over-provisioning, retries, large context windows)?

Output:

• A written baseline and a lightweight cost model with assumptions (so it’s debuggable).

3) Reliability and operational visibility

If you don’t have SLIs/SLOs, I look for “proxy signals”:

• Error rates (by endpoint/model)
• Latency percentiles
• Saturation (GPU/CPU/mem)
• Queue depth / backpressure
• Deployment frequency and rollback rate

Output:

• Candidate SLIs/SLOs and the minimum dashboards/alerts to operate the system.

4) Deployment, rollouts, and rollback paths

I’m looking for operational safety:

• Can you roll out gradually (canary, weighted routing, feature flags)?
• Can you roll back without guessing?
• Are model artifacts versioned and traceable to deploys?

Output:

• A practical “release hygiene” backlog (what to change first to make deploys boring).

5) Failure modes and a risk register

The audit should name the top ways the system fails and what you do about each:

• GPU node failures / driver drift
• Model crashes / OOMs
• Thundering herds and retry storms
• Bad deploys and broken compatibility
• Silent partial failures (slowdowns, queue growth)

Output:

• A risk register with mitigations and owners.

What you get after the audit

• A scorecard across cost, reliability, security, and operability
• A GPU cost baseline/model (with explicit assumptions)
• A risk register (top failure modes + mitigations)
• A 90‑day roadmap (sequenced, owned, executable)

If you want help implementing

If the roadmap is clear and you want experienced hands shipping it:

• Phase 2 is typically a build/stabilization project.
• If you mainly need senior guidance, a retainer can be a better fit.

Start here: AI Infra Readiness Audit.

---

Related reading

For deeper dives on specific aspects of the audit, see:

• GPU Cost Baseline: What to Measure, What Lies - Building a cost model that is actually useful
• SLOs for Inference: Latency, Errors, Saturation - Defining meaningful reliability targets
• Hybrid/On-Prem GPU: The Boring GitOps Path - When and how to run your own GPU infrastructure
• GPU Failure Modes: What Breaks and How to Debug It - Common failures and how to diagnose them

Or browse the full AI Infrastructure Readiness topic guide.