Build an LLM API With Python and FastAPI: OpenAI Responses API Tutorial is a hands-on guide for AI developers who need a dependable system rather than another chatbot demonstration. It presents a reproducible setup path, a realistic workflow, tests for failure, and the controls that matter when generated text meets real users or external tools. Choose a model and SDK version supported by your account and validate all behavior in a non-production environment.
Define a small API with predictable behavior
An internal team wants to summarize a support ticket, not build an autonomous general-purpose assistant. Start with a narrow endpoint that accepts text within a defined length and returns a summary plus a request ID. A useful API validates the caller, protects provider credentials, limits response size and handles model failures. Keep the language model behind a service function so you can test HTTP handling without spending on model requests. This example assumes Python 3, a virtual environment and legitimate API credentials from your provider account.
Install a minimal development environment
Create and activate a virtual environment, then run python -m pip install fastapi uvicorn openai. Store your API key in a protected environment variable named OPENAI_API_KEY; never hardcode it into the Python module or a public repository. Use a separate OPENAI_MODEL setting that names a model actually enabled for your account. The official SDK reads the key from its environment automatically. Check the current provider documentation for eligible models and pricing instead of hardcoding a model name that may change.
Send your first Responses API request
In a Python module, import OpenAI, instantiate client = OpenAI(), and call client.responses.create(model=os.environ["OPENAI_MODEL"], input="Summarize this support issue in one sentence: checkout times out"). Print response.output_text only in a secure development environment. The model response is untrusted output: validate content, handle errors and avoid placing it directly into privileged operations. A simple synchronous request is appropriate for learning; a real web service must consider worker concurrency, request timeouts and upstream rate limits.
Wrap the model in FastAPI
Create a Pydantic input model with a required text field and length constraints. Define a POST /summarize route that passes validated text to a separate service function and returns JSON. Keep API authentication in dependency middleware or another appropriate layer, not inside the prompt. Prefer an async-compatible SDK client or a suitable thread strategy rather than blocking the event loop with a synchronous network request inside a high-concurrency async route. Make errors actionable but do not return raw provider tracebacks, keys or confidential prompt content to clients.
Test normal and abnormal inputs
Use FastAPI’s generated schema during development or a test client to submit a short ticket, an empty input, a very long request and a malformed JSON body. A successful route returns a useful summary, while validation failures return appropriate client errors without calling the model. Mock upstream SDK calls for unit tests. Add a controlled integration test in staging using non-sensitive prompts to verify credentials and model availability. Do not interpret one fluent response as proof of factual accuracy; domain-dependent answers need their own evaluation set.
Plan reliable operations and cost controls
Set server-side request limits, concurrency controls, retry policies for transient errors and a strict cap on tool or model spending where supported. Record latency, request IDs, failure classes and aggregate token costs while minimizing retained user content. Never retry every 4xx client error, and use exponential backoff with jitter for retriable rate-limit or server conditions. Add an explicit health endpoint that checks process readiness without generating a paid model response on every probe. Document the behavior when the model provider becomes unavailable, such as returning a structured service error or safely queuing work.
Practical acceptance test: prove the application works
Build a small test fixture before allowing real customers to interact with the system. Include a typical request, a confusing request, an explicitly out-of-scope request and a deliberately malformed input. Define the acceptable answer shape and whether the system should abstain, return a validation error or request approval. Record the exact model identifier, prompt revision and configuration used. When the assistant can invoke a tool, mock that tool in unit tests and prove that authorization is checked even when the model proposes an unauthorized argument. Include at least one request with adversarial instructions inside user-provided data.
After local tests, use a staging environment with synthetic or sanitized data to check network, credentials, health routes, timeouts and monitoring. Avoid interpreting the ability to generate plausible sentences as a successful end-to-end task. A good release criterion measures task completion and safe failure behavior in the same suite.
Production engineering checklist
Protect provider credentials and tool tokens, scope permissions to authenticated identities, and establish limits for request length, concurrency and cost. Separate read-only information retrieval from state-changing operations. Add predictable timeouts, bounded retries for transient errors, monitoring of upstream status and a process for disabling compromised integrations. Avoid retaining sensitive prompts unless necessary, protected and documented. Keep deployment configuration in reviewable version control and use tests to catch changes in schema or model behavior. If the system influences payments, legal obligations, hiring or other significant decisions, include appropriate human oversight and clear boundaries.
Frequently asked questions
Can an API key be placed in a Docker image or source file?
No. Treat it as a secret and inject it at runtime through an approved secret-management mechanism. Rotate leaked keys and prevent their appearance in build logs, telemetry and repositories.
How can I avoid unsupported or unsafe model output?
Restrict the task, use typed responses where possible, verify claims or permissions against trusted systems, test refusals and anomalies, and require confirmation for consequential actions. A good prompt helps, but deterministic application controls remain essential.
Documentation and further learning
Follow the official API or tooling documentation for version-specific examples and supported features. Our guides to RAG evaluation and prompt injection defense provide complementary approaches for testing evidence quality and tool trust boundaries.
Minimal Python request example
For a compact end-to-end smoke test, define a model identifier from your environment and make a single Responses request. This keeps model selection out of source control and avoids hard-coding a particular SKU as a long-lived tutorial dependency:
import os
from openai import OpenAI
client = OpenAI()
response = client.responses.create(
model=os.environ["OPENAI_MODEL"],
input="Summarize: a checkout attempt timed out.",
)
print(response.output_text)Run the snippet only after installing the SDK and securely setting OPENAI_API_KEY and OPENAI_MODEL. Check the current provider documentation for your model’s feature support. For deployment into a container, use our FastAPI Docker operations guide.
