AI product launch checklist: 42 checks before real users touch it

How to use it

1. Go through it with the person who built it and the person who will support it. Each check needs a "yes, and here is how we know" — not "I think so".
2. Every Blocker must be a yes. For the rest, a conscious "not for launch, owner + date" is an acceptable answer.
3. Re-run the groups "Trust and errors" and "Analytics and evals" after every model or prompt change.

1. UX of AI output

• Anything over 2 seconds shows progressBlockerStream tokens or show steps ("Reading 3 documents…"). A static spinner for 15 seconds reads as broken.
• Output is editable, copyable and easy to act onThe user can fix the 10% the model got wrong without starting over.
• Regenerate and undo existRegenerate does not destroy the previous answer; accepted changes can be reverted.
• The answer shows what it was based onInputs, files or sources used are visible next to the output.
• Layout survives long, short, empty and malformed outputTested with a 5,000-word answer, a one-word answer, an empty string and broken Markdown/JSON.
• Tone and length are tested on 20 real inputsNot the 3 demo prompts — 20 messy inputs from real users or realistic samples.

2. Trust and errors

• Every AI error state is designedBlockerTimeout, provider down, rate limit (429), content refusal, empty retrieval, invalid structured output — each has its own message and next step.
• There is a fallback when the provider failsBlockerSecondary model, cached result, queued retry or a clear "try again in X" — never a dead button.
• Factual claims carry sourcesFor RAG and research features: citations link to the exact document or passage.
• Uncertainty is expressed, not hiddenLow-confidence answers say so; the product never presents a guess with the same weight as a fact.
• Irreversible actions need human confirmationBlockerSending, deleting, paying, publishing or writing to another system shows a preview and asks first.
• "Report a bad answer" is wired to a real queueThumbs-down saves input, output, prompt version and model to a place someone reviews weekly.
• The model cannot claim actions it did not takeBlocker"I've emailed the client" only appears if the tool call actually succeeded.

3. Cost and limits

• Cost per task is measured on real inputsBlockerKnow p50 and p95 cost per task, not an average from the demo.
• Per-user quotas and abuse caps are enforced server-sideBlockerDaily limit per user and per workspace; one script cannot burn the monthly budget overnight.
• Input size and context limits are enforcedMax file size, max pages, max tokens — with a friendly message, not a provider error.
• Provider spend alerts are onBlockerAlerts at 50%, 80% and 100% of the monthly budget, going to a person who reads them.
• Repeated work is cachedSame document, same question → cached embeddings or responses; prompt caching enabled where the provider supports it.
• Pricing covers your heaviest usersPlan price covers p95 usage with healthy margin; heavy usage is metered or capped, not absorbed.

4. Privacy and legal

• Model providers are listed as subprocessorsBlockerPrivacy policy names who processes user data (model API, vector DB, transcription, analytics).
• Training and retention settings are confirmed in writingBlockerAPI data not used for training; zero or minimal retention where available; DPA signed if you serve the EU.
• Personal data in prompts and logs is minimizedRedact or pseudonymize what the model doesn't need; logs don't keep full prompts forever.
• Users know they are dealing with AIAI-generated content and AI agents are labelled (also an EU AI Act transparency expectation).
• Terms cover AI outputWho owns output; no professional advice in regulated domains (medical, legal, financial) unless that is your licensed business.
• Deletion works end to endDeleting an account removes data from your DB, file storage, vector index and caches — tested, not assumed.
• Prompt injection is consideredBlockerContent from users, web pages or documents cannot make the model reveal system prompts, other users' data or trigger tools.

5. Analytics and evals

• A golden set existsBlocker30–100 real inputs with expected outputs or grading criteria, covering common, edge and adversarial cases.
• Evals run before every prompt or model changeScore compared with the last release; a regression blocks the change.
• AI events are trackedTask started, completed, failed, accepted, edited, regenerated, rated — per feature.
• Acceptance rate is your first quality metricShare of outputs used without edits (or with small edits). It beats "thumbs up" for signal.
• Prompt and model version are logged with every outputSo any bad answer can be reproduced and traced to a change.
• Latency and error rate are on a dashboardp95 latency and error rate per AI feature, with an alert threshold.

6. Onboarding

• First useful result in under 2 minutesBlockerTime it with a new user. Signup, connect, wait — all of it counts.
• The empty state is never blankSample data, a demo document or a pre-filled example the user can run immediately.
• Three example prompts or templatesReal tasks for your audience, not "Ask me anything".
• One sentence on what the AI can and cannot doSets expectations before the first wrong answer, not after.
• Connecting data can be deferredUsers can try the value before granting OAuth access to their mailbox or CRM.

7. Support and operations

• Each AI feature has a kill switchBlockerA feature flag turns it off (or to a fallback) without a deploy.
• Someone owns provider outagesNamed person, provider status pages subscribed, a runbook of what to switch.
• Users see degraded service, not mysteryIn-app banner or status page when AI is slow or down.
• Support has answers for "the AI got it wrong"Saved replies, a way to look up the exact output, and a path to escalate.
• Flagged outputs are reviewed weeklyA 30-minute weekly review of bad answers feeds the golden set and the prompt backlog.