Chat completions API

You asked your agent to add a chat box to your app, it wrote a call to a model, and the first reply came back fine. Then the second message forgot the first one, or the call failed with an error nobody on the team could read.

A chat completions API is an address on a model provider's server that takes a conversation so far and sends back the next reply. Your app supplies the instructions, the messages already exchanged and the name of a model. The provider answers with one new message, and your app adds it to the conversation before the next turn.

Why this is harder than it sounds

The first call is the easy part. Most guides stop there, and the trouble starts on the second turn.

A chat completions API remembers nothing between calls. Each request has to carry the whole conversation again, because the model only sees what arrives in that one request. Your app rebuilds the chat your users see on every call.

Four things make that harder than the first demo suggests:

• The memory lives in your app. If your code sends only the latest message, the model answers as if the chat just started.
• The instructions travel with every call. The prompt that sets tone and rules is part of each request, so it usually ends up written into your code.
• The conversation keeps growing. Long chats get slower and cost more per call, and at some point they no longer fit what the model accepts.
• Each provider differs a little. Moving to another model often means changing names, keys and small details of the request in code.

None of this is unusual. A chat feature needs this work once real people use it, and the work falls on whoever owns the code.

What it actually looks like

For most small teams the chat feature arrives in one sitting. You describe what you want to Claude Code or Cursor, and the agent writes a function that calls a model with a prompt written as a string in a source file.

The chat feature works in testing. A few weeks later the requests start:

• the support lead wants the bot to stop promising refunds
• a founder wants to try a cheaper model on the same prompt
• a customer reports a reply that made no sense, and nobody can see what was sent
• the provider has a bad afternoon and the chat box shows an error

Every one of those means opening the code, editing a string or a model name, and shipping the app again. A one word change to the prompt becomes a deploy, which is the step where your new version goes live.

The pattern is common because the first integration is quick. The cost shows up later, spread across many small edits to a file that was never meant to be the place where prompts are managed.

If the bill is part of the worry, our guide to the cheapest LLM API covers how to compare model prices without guessing.

Chat completions API, step by step

Each call to a chat completions API follows the same shape, whichever provider you use. The exact field names and options belong to each provider's own reference, so check theirs before you write the request. The steps below are the parts every call has.

1. Get an API key and send it with each request

An API key is a long secret string that tells the provider the request is yours and which account pays for it. Keep it on your server or in an environment variable, never in code that runs in the browser. Our guide on how to use an API key walks through storing one safely.

Authentication means proving who you are on every call. The key goes in a header of the HTTPS request, which is a label sent alongside the body. A missing or mistyped key is the most common reason a first call fails.

2. Write the instructions

The instructions tell the model who it is and how to behave: the tone, what to refuse, what format to reply in. This is the prompt that changes most often, so think now about where it will live.

3. Add the conversation so far

Send every earlier message from the user and every earlier reply from the model, in order. Leave one out and the model loses that part of the chat. For long chats, decide what to trim before the request gets too big.

4. Choose the model and send

Name the model you want and send the request over HTTPS. A small curl command or a few lines in any language is enough, because the call is an ordinary web request.

5. Read the reply and store it

The response carries the model's next message. Show it to the user, then save it with the rest of the conversation so the next call can include it.

6. Handle the errors

The failures you will meet most often:

• A rejected key. The key is wrong, expired, or sent in the wrong header.
• Too many requests. The provider is limiting how fast you call it; wait and retry with a growing delay. Our post on API rate limit exceeded covers this case.
• A malformed request. A field is missing or the conversation is in the wrong shape.
• A conversation that is too long. The model cannot take everything you sent; trim older messages or summarize them.
• A provider outage. The call times out or fails; retry, or send it to a second model.

Log each call while you build. When a reply looks wrong, the first question is always what was sent.

The step people skip

Most teams treat the call as the whole job. The step people skip is deciding where the prompt, the model choice and the record of each call live once the feature is in production.

If those stay inside the code, each one turns into a deploy or a guess. If you move them outside it, a prompt edit becomes a change you can make and undo without touching the app.

What that looks like in practice:

• Prompts kept as versions. You can change the instructions, compare the result and roll back if the new wording is worse.
• The model chosen outside the code. Trying another model on the same prompt becomes a setting you change, and your app stays as it is.
• Every call recorded. When a user reports a bad reply, you can see the input and the output that produced it.

Supagen is the backend for AI features and agents: small teams and solo founders keep prompts, model choice and routing, and call logging out of their code and call every model, from text to image, video and speech, through one API.

Supagen calls are ordinary HTTPS requests, signed with an API key that belongs to your workspace, so they work from any language your app or your agent writes. Supagen also uses that one template and invoke API beyond text: structured JSON that follows a schema you define, pictures, video, sound, text to speech, speech to text, changing one voice into another, embeddings, reranking, and models that classify an input into a decision. That matters once your chat box needs a picture or a voice. Supagen has a USD 0 a month plan and a USD 29 a month plan, with no free trial on top.

When this is not the real problem

Sometimes the chat completions API is not what is going wrong.

• If one prompt calls one model and rarely changes, calling the provider directly is fine. Adding a layer in front of it buys you little.
• If replies are wrong because the model lacks your data, the fix is giving it that data, and no change to how you call it will help.
• If you need to run everything on your own servers, a hosted service is the wrong fit; look at tools you can self-host.
• If the error is a rate limit, the answer is retry logic or a higher limit with the provider.

Pick the fix that matches the failure you see in the logs.

Frequently asked questions

What is a chat completions API?

A chat completions API is a model provider's endpoint that receives a conversation, meaning the instructions plus the messages so far, and returns the model's next reply as a new message.

Does a chat completions API remember earlier messages?

No. A chat completions API keeps no memory between calls, so your app has to send the earlier messages again with every request.

How do I authenticate a chat completion API call?

Send your API key in a header of the HTTPS request. Keep the key on your server and out of any code that runs in the browser.

Why does my chat completions API call fail?

The usual causes are a wrong or missing key, too many requests in a short time, a malformed request, a conversation that is too long, or a provider outage.

Can I call it without an SDK?

Yes. Each call is an ordinary HTTPS request, so curl or the built-in web request tools of any language are enough.

You can try this with your own prompts today. Start building free.