Calling a Model API: Messages, Roles, and Response

What a Call to an LLM Really Does

Integrating a large language model (LLM) into an application amounts, technically, to sending an HTTP request to a service (a cloud provider or a server you host) and using the response. Most chat APIs share the same logic, even though the exact field names vary from one provider to another: you send a list of messages and parameters, and you receive a response message accompanied by metadata.

The architecture to remember is always the same:

browser  →  your API (server)  →  model service
         ←                      ←

The browser never calls the model directly. Otherwise, the API key would be visible in the page’s code, anyone could use it at your expense, and your prompts (your know-how) would be readable. The server adds the key, the instructions, the logging, and the checks.

Messages and Their Roles

A conversation is an ordered list of messages, each with a role:

messages = [
    {"role": "system", "content": "Tu es l'assistant du service client. Réponds en français, en trois phrases maximum."},
    {"role": "user", "content": "Ma commande n'est pas arrivée."},
    {"role": "assistant", "content": "Pouvez-vous me donner votre numéro de commande ?"},
    {"role": "user", "content": "C'est A100."},
]

system: your application’s instructions (tone, scope, format). The user does not write it.
user: what the user typed.
assistant: the model’s previous replies, which you send back to provide context.

The Model Has No Memory

Each call is independent. If the model “remembers” the conversation, it is because the application resends the history with every request. This history consumes tokens (and therefore time and cost) and must fit within the model’s context window. So you generally keep the system message, the latest exchanges, and the new question:

def build_messages(system, history, question, max_turns=3):
    messages = [{"role": "system", "content": system}]
    for user_text, assistant_text in history[-max_turns:]:
        messages.append({"role": "user", "content": user_text})
        messages.append({"role": "assistant", "content": assistant_text})
    messages.append({"role": "user", "content": question})
    return messages

For long conversations, you can also summarize older exchanges rather than dropping them.

Reading the Response

A response contains at least three useful pieces of information. In this course, we use a simplified, neutral format:

response = {
    "output": {"role": "assistant", "content": "Votre commande A100 est expédiée."},
    "stop_reason": "end",            # "end", "max_tokens" or "tool_call"
    "usage": {"input_tokens": 52, "output_tokens": 9},
}

The text of the response.
The stop reason: end means the model finished; max_tokens means the response was cut off by the limit you set. A truncated response must not be treated as complete (truncated JSON is invalid, a truncated sentence is misleading).
The usage in input and output tokens, essential for tracking costs (lesson 5).

Common Parameters

max_tokens: maximum length of the response. It is a safeguard for cost and latency.
temperature: low (0 to 0.3) for tasks that require consistency (classifying, extracting); higher for varied writing. Even at zero temperature, an identical output is not guaranteed by all services.
The model itself: a small, fast model is often enough to classify or summarize; reserve large models for hard tasks.

Simulating the Model for Development

In the exercises, the model is a local, deterministic Python function that returns a response in the same format. It is also good practice in industry: your automated tests should depend neither on the network nor on a random response. You test the application’s logic (building messages, reading the response, handling errors) with a fake model, and you evaluate the quality of the real model separately.

Key Takeaways

The server calls the model; the browser calls your server.
A request = system, user, assistant roles in order; the application resends the history.
Always check the stop reason and record token usage.

Exercises

Solutions and tests stay on the server. In the student space, your code is run and checked, and the tutor gives hints before the solution.

CodeDifficulty 1Python in the browser2 hints
Build the messages of a request
Write build_messages(system, history, question, max_turns=3):
- history is a list of (user_question, assistant_answer) tuples from oldest to most recent;
- the result starts with the system message, then the max_turns latest exchanges (one user message then one assistant message per exchange), then the new question as a user message;
- the question is stripped of leading and trailing whitespace; if it is empty, raise ValueError;
- with max_turns=0, no exchange is included.
```
def build_messages(system, history, question, max_turns=3):
    ...
```

CodeDifficulty 2Python in the browser2 hints

Read the response and detect truncation

The simulated model fake_model(messages, max_tokens) is provided. Write:

read_response(response) which returns a dictionary {"text": ..., "truncated": ..., "tokens": ...}: the text of output.content without surrounding whitespace, truncated true if stop_reason equals "max_tokens", and tokens = input tokens + output tokens;
ask(model, messages, max_tokens=300) which calls model(messages, max_tokens=max_tokens), reads the response with read_response, and raises RuntimeError if the text is empty.

def fake_model(messages, max_tokens=300):
    """Modèle simulé : répond en reprenant la dernière question (déterministe)."""
    question = messages[-1]["content"]
    words = ("Réponse à : " + question).split()
    truncated = len(words) > max_tokens
    words = words[:max_tokens]
    return {
        "output": {"role": "assistant", "content": " ".join(words)},
        "stop_reason": "max_tokens" if truncated else "end",
        "usage": {"input_tokens": sum(len(m["content"].split()) for m in messages),
                  "output_tokens": len(words)},
    }


def read_response(response):
    ...


def ask(model, messages, max_tokens=300):
    ...

Multiple choiceDifficulty 11 hint
Where should the API key go?
You are adding an assistant to a web application (Vue front end, back-end API). Where should the model service’s API key be?
- In a front-end environment variable, injected at build time
- In a server environment variable, used only by the API
- In the user's localStorage after login
- In the system prompt, so that the model authenticates itself

End-of-lesson quiz

Why is the conversation history resent with every call?
- So that the provider bills correctly
- Because the model keeps no memory between two calls
- To speed up the response
A response has stop_reason = "max_tokens". What can you deduce?
- The response is complete and short
- The response was cut off by the length limit
- The model wants to call a tool
Which role carries your application’s instructions (tone, scope, format)?
- user
- assistant
- system

The rest of the course

Structured Outputs: Getting and Validating JSON20 min
Tool Calls: Letting the Model Use Your Functions20 min
Streaming: Displaying the Response as It Is Generated15 min
Costs, Limits, and Logging20 min

Build my path See the full syllabus

Integrating an LLM into an Application · Intermediate · 1 h 30 min

Objectives