The AI Founder's Technical Checklist Before Launch
Before you launch your AI product, run through this checklist. The founders who skip these steps are the ones who call us six months later with a product that can't scale.
Zeidex Editorial
zeidex.com
This is the checklist we run through with every AI product before launch. Not after. Not when something breaks. Before.
Model and AI layer
☑ Evaluation framework is in place. You have a test set of at least 100 representative queries with expected outputs. You have a way to run this evaluation automatically.
☑ Hallucination mitigations are implemented. You're using techniques like constrained generation, source citations, or confidence scoring to reduce hallucination risk in your specific use case.
☑ Prompt injection defences are in place. You've tested adversarial inputs. You have input sanitisation. You have output filtering for harmful content.
☑ Fallback behaviour is defined. When the model returns a bad answer, times out, or the API is down — what happens? Users should never see a raw error message from an LLM API.
☑ Token costs are understood and modelled. You know your average cost per conversation, the 90th percentile cost, and the maximum possible cost for a single interaction. You have alerts for cost anomalies.
Data and retrieval
☑ Knowledge base is fresh. Your RAG data is current. You have a pipeline that keeps it updated. You've verified the retrieval quality against your test set.
☑ Embedding model version is pinned. Embedding model updates change vector representations, which breaks your vector index. Pin the version and have a re-indexing procedure.
☑ Data access controls are correct. Users can only retrieve data they have permission to access. This is critical in multi-tenant applications.
Infrastructure
☑ Rate limit handling is implemented. You have request queuing and graceful degradation for API rate limits.
☑ Caching is in place. At minimum, exact-match caching. Consider semantic caching for high-volume applications.
☑ Timeouts are set correctly. LLM calls can hang. Every call should have a timeout with a fallback path.
☑ Logging is comprehensive. Every LLM call is logged with full prompt, response, latency, token count, and user context.
☑ Monitoring alerts are configured. You have alerts for latency spikes, error rate spikes, cost spikes, and evaluation score drops.
Security
☑ API keys are not in client-side code. All LLM calls go through your backend. Never expose provider API keys to users.
☑ User inputs are validated. You have length limits and content filtering on user inputs.
☑ Output is reviewed for PII and sensitive data. Your model is not inadvertently returning other users' private data.
Business
☑ Cost at scale is modelled. What does this product cost at 10x your launch volume? Is the business viable?
☑ Model provider risk is understood. What happens if OpenAI changes pricing, deprecates your model, or has an outage? Do you have a fallback?
☑ Terms of service compliance is confirmed. Your use case complies with your model provider's usage policies.
If you can check all of these before launch, you're ahead of 80% of AI products that shipped this year. The ones that skip this list are the ones that need an emergency rebuild at the worst possible time.