Table of Contents

Chat and prompting

Every request TTMSMCPCloudAI makes is built from the same three ingredients: which service to talk to, what to say, and how the model should generate its answer. The component keeps those separated. Service and APIKeys decide the destination, SystemRole / AssistantRole / Context supply the prompt, and Settings carries the per-service model names plus the generation parameters. Because every provider reads its own model property, you can configure all of them once and switch providers by changing a single enum value — which is what makes a service picker practical in a shipped application. This guide covers provider selection and key storage, the three prompt roles, the generation settings including the newer UseTemperature and Claude CacheControl options, local and aggregated endpoints, and how to read the answer and account for what it cost.

Selecting a service and a model

Start here because everything else depends on it: the model names, the generation parameters a service accepts, and even whether file or function support is available differ per provider.

procedure TForm1.UseProvider(AService: TTMSMCPCloudAIService);
begin
  AI.Service := AService;

  { Each service reads its own model property, so all of them can be
    configured up front and the Service property alone decides which is used. }
  AI.Settings.OpenAIModel := 'gpt-5.2';
  AI.Settings.ClaudeModel := 'claude-sonnet-4-5';
  AI.Settings.GeminiModel := 'gemini-2.5-pro';
  AI.Settings.GrokModel := 'grok-4';
  AI.Settings.MistralModel := 'mistral-large-latest';
  AI.Settings.DeepSeekModel := 'deepseek-chat';
  AI.Settings.PerplexityModel := 'sonar-pro';
  AI.Settings.OpenRouterModel := 'openai/gpt-5.2';

  { Data-residency variant of the OpenAI endpoint. }
  AI.Settings.OpenAIServers := oaEU;
end;

The TTMSMCPCloudAIService value and its matching key and model property line up like this:

Service Key Model property
aiOpenAI APIKeys.OpenAI Settings.OpenAIModel
aiClaude APIKeys.Claude Settings.ClaudeModel
aiGemini APIKeys.Gemini Settings.GeminiModel
aiGrok APIKeys.Grok Settings.GrokModel
aiMistral APIKeys.Mistral Settings.MistralModel
aiDeepSeek APIKeys.DeepSeek Settings.DeepSeekModel
aiPerplexity APIKeys.Perplexity Settings.PerplexityModel
aiOpenRouter APIKeys.OpenRouter Settings.OpenRouterModel
aiOllama — Settings.OllamaModel
aiLlamaCpp — Settings.LlamaCppModel

Settings.OpenAIServers switches the OpenAI endpoint between the default (oaDefault) and the EU-resident one (oaEU) — see TTMSMCPCloudOpenAIServers.

Keys are stored on TTMSMCPCloudAIAPIKeys, which encrypts them with SaveToFile(AFileName, Password) and reads them back with LoadFromFile. Keep them out of source and out of version control.

Building the prompt: three roles

A prompt is rarely just a question. Splitting it lets you set policy once and vary only the user's turn.

  • SystemRole — standing instructions, sent with every request. Tone, format rules, domain constraints, a schema description.
  • AssistantRole — a seeded assistant turn. Useful to demonstrate the reply shape you want without asking for it in prose.
  • Context — the user turn: the actual question or material.

All three are TStrings, so multi-line content and LoadFromFile work directly.

procedure TForm1.AskAboutInvoice(const AQuestion: string);
begin
  { Standing instructions - sent with every request, not part of the question. }
  AI.SystemRole.Clear;
  AI.SystemRole.Add('You are an accounting assistant for a Delphi application.');
  AI.SystemRole.Add('Answer with plain text. Never invent invoice numbers.');

  { An assistant turn seeded by the application steers the reply format
    without the user having to ask for it. }
  AI.AssistantRole.Clear;
  AI.AssistantRole.Add('I reply with one short paragraph followed by a bullet list.');

  { The user turn. }
  AI.Context.Clear;
  AI.Context.Add(AQuestion);

  AI.Execute;
end;

Generation settings

These control how the model answers rather than what it is asked. Getting them wrong is the most common cause of a request that is rejected outright.

procedure TForm1.ConfigureGeneration;
begin
  { Deterministic-leaning output for a data extraction task. }
  AI.Settings.Temperature := 0.2;

  { Newer reasoning models reject a temperature parameter. Setting
    UseTemperature to False omits it from the request body altogether,
    instead of sending a value the service will refuse. }
  AI.Settings.UseTemperature := False;

  { 0 leaves the limit to the service. }
  AI.Settings.MaxTokens := 2048;

  { Thinking budget on models that expose one. aiIgnore leaves the request
    untouched; aiOff asks the model not to spend reasoning tokens. }
  AI.Settings.Reasoning := aiHigh;

  { Let the service ground its answer in live search results. }
  AI.Settings.WebSearch := True;

  { Anything the component does not surface as a property can be merged into
    the request body as raw JSON. }
  AI.Settings.CustomOptions := '"top_p": 0.9';
end;
Setting Effect
Temperature Randomness. Lower is more deterministic.
UseTemperature When False, the temperature parameter is omitted from the request body entirely. Default True.
MaxTokens Upper bound on the answer length. 0 leaves it to the service.
Reasoning Thinking effort on models that expose one: aiLow, aiMedium, aiHigh, aiOff, or aiIgnore to send nothing.
WebSearch Lets the service ground its answer in live search results, where supported.
ParallelToolExecution Allows several tool calls in one turn — see Function calling.
CustomOptions Raw JSON fragment merged into the request body, for anything not surfaced as a property.

UseTemperature exists because several newer reasoning models reject a temperature parameter rather than ignoring it. Setting Temperature := 0 is not the same thing — the parameter is still sent. Set UseTemperature := False and the component leaves it out.

Reasoning defaults to aiIgnore, which sends no reasoning field at all; that is the safe value for a model or service that does not understand one. See TTMSMCPCloudAIReasoning.

Caching a long Claude prompt

A large system role — a style guide, a schema, a policy document — is re-sent and re-billed on every turn of a conversation. Claude can cache such a block and reuse it.

procedure TForm1.ConfigureClaudeWithCachedInstructions;
begin
  AI.Service := aiClaude;
  AI.Settings.ClaudeModel := 'claude-sonnet-4-5';

  { A large, stable system prompt - a style guide, a schema description, a
    policy document - is re-sent with every single turn of a conversation. }
  AI.SystemRole.LoadFromFile(InstructionsFileName);

  { ccEphemeral adds Claude's cache_control marker to the request, so the
    service can reuse its own cached copy of that block instead of
    reprocessing it. ccNone (the default) sends no marker. }
  AI.Settings.CacheControl := ccEphemeral;

  AI.Context.Text := memoPrompt.Lines.Text;
  AI.Execute;
end;

Settings.CacheControl takes a TTMSMCPCloudAICacheControl value: ccNone (the default) sends no cache marker, and ccEphemeral adds Claude's cache_control marker so the service may serve the block from its own cache. The setting only affects requests made with Service = aiClaude; other services ignore it. It pays off when the cached block is large and stable and the same conversation continues over several turns — it does nothing for a single one-shot request with a short system role.

Local servers and aggregators

Not every deployment can send data to a hosted service, and not every project wants one key per vendor.

procedure TForm1.UseLocalOllama;
begin
  AI.Service := aiOllama;
  AI.Settings.OllamaHost := 'localhost';
  AI.Settings.OllamaPort := 11434;
  AI.Settings.OllamaPath := '/api/chat';
  AI.Settings.OllamaModel := 'llama3.2';
  { A local server needs no API key. }
end;

procedure TForm1.UseLocalLlamaCpp;
begin
  AI.Service := aiLlamaCpp;
  AI.Settings.LlamaCppHost := 'localhost';
  AI.Settings.LlamaCppPort := 8080;
  AI.Settings.LlamaCppPath := '/v1/chat/completions';
  { llama.cpp ships without a model. Reference a downloaded GGUF model by its
    repository name, for example Qwen/Qwen3-4B-GGUF. }
  AI.Settings.LlamaCppModel := 'Qwen/Qwen3-4B-GGUF';
end;

procedure TForm1.UseOpenRouter;
begin
  AI.Service := aiOpenRouter;
  { One key, many upstream models - the model string selects the provider. }
  AI.Settings.OpenRouterModel := 'anthropic/claude-sonnet-4.5';
end;

aiOllama and aiLlamaCpp talk to a server you run, addressed by OllamaHost / OllamaPort / OllamaPath and LlamaCppHost / LlamaCppPort / LlamaCppPath. Neither uses an API key.

aiOpenRouter is an aggregator: one key, and Settings.OpenRouterModel names the upstream model in vendor/model form. It is the quickest way to reach a model whose vendor you have no direct account with.

Setting up a llama.cpp server

llama.cpp is the one local option that needs preparing before aiLlamaCpp can answer anything, because it is installed and stocked separately from the component. Three steps, once per machine:

  1. Install the server. Take the precompiled binaries for your platform from the llama.cpp releases page, or build it yourself by following the build guide.
  2. Download a model. The installation comes with none — a llama.cpp server with no model answers every request with an error. Pick a GGUF model from Hugging Face and download it.
  3. Name the model in LlamaCppModel. Use its repository form, username/model. For the Qwen3 4B GGUF model, that is Qwen/Qwen3-4B-GGUF.

Execute raises at the call site when Service = aiLlamaCpp and Settings.LlamaCppModel is empty, so an unnamed model surfaces immediately rather than as a service error. The default endpoint is localhost:8080 with path /v1/chat/completions; change LlamaCppHost, LlamaCppPort, and LlamaCppPath when the server runs elsewhere.

Ollama needs no equivalent step beyond installing it and pulling a model with ollama pull; it defaults to localhost:11434 with path /api/chat.

Reading the response and the usage

The answer is asynchronous. Execute returns straight away and OnExecuted fires when the exchange — including any tool round trips — has finished.

procedure TForm1.AIExecuted(Sender: TObject; AResponse: TTMSMCPCloudAIResponse;
  AHttpStatusCode: Integer; AHttpResult: string);
begin
  { AResponse is nil when the request itself failed. Check it before
    touching any of its properties. }
  if not Assigned(AResponse) then
  begin
    memoAnswer.Lines.Text :=
      Format('Request failed (%d): %s', [AHttpStatusCode, AHttpResult]);
    Exit;
  end;

  memoAnswer.Lines.Text := AResponse.Content.Text;

  { Per-request accounting, plus what the service actually used - handy when
    a provider silently substitutes a model. }
  lblRequest.Caption := Format('%s on %s - %d prompt / %d completion / %d total',
    [AResponse.ServiceId, AResponse.ServiceModel,
     AResponse.PromptTokens, AResponse.CompletionTokens, AResponse.TotalTokens]);

  { Usage accumulates across every request the component performs, including
    speech and transcription, until Reset is called. }
  lblSession.Caption := Format('Session: %d tokens, %d speech characters',
    [AI.Usage.TotalTokens, AI.Usage.AudioCharacters]);
end;

procedure TForm1.btnNewSessionClick(Sender: TObject);
begin
  AI.Usage.Reset;
end;

OnExecuted is a TTMSMCPCloudAIResultEvent. Its AResponse parameter is nil when the request failed, in which case AHttpStatusCode and AHttpResult carry the reason. ServiceModel reports the model the service actually used, which is worth logging — providers do substitute models.

Usage accumulates PromptTokens, CompletionTokens, TotalTokens, AudioDuration, and AudioCharacters across every request until Usage.Reset is called, so a per-session or per-document budget needs no bookkeeping of your own.

Turn on Logging and set LogFileName while developing: every REST call to the service is then written out in full, which is usually faster than guessing why a provider rejected a request.

Discovering services and models at run time

Hard-coding a model string ages badly, and a service with no key configured should not appear in a picker at all.

procedure TForm1.FillServiceList;
begin
  { GetActiveServices returns the component's own internal list - copy its
    contents, never take ownership of it and never free it. Assign copies
    both the strings and the TTMSMCPCloudAIService value in Objects[]. }
  cbService.Items.Assign(AI.GetActiveServices);

  if cbService.Items.Count > 0 then
    cbService.ItemIndex := 0;
end;

procedure TForm1.cbServiceChange(Sender: TObject);
begin
  AI.Service :=
    TTMSMCPCloudAIService(Integer(cbService.Items.Objects[cbService.ItemIndex]));

  { Ask the selected service which models it currently offers. The list
    arrives asynchronously, so read it in OnGetModels. }
  AI.OnGetModels := AIGetModels;
  AI.GetModels;
end;

procedure TForm1.AIGetModels(Sender: TObject; AResponse: TTMSMCPCloudAIResponse;
  AHttpStatusCode: Integer; AHttpResult: string);
begin
  if AHttpStatusCode div 100 <> 2 then
  begin
    memoAnswer.Lines.Add('Could not list models: ' + AHttpResult);
    Exit;
  end;

  cbModel.Items.Assign(AI.Models);
end;

GetServices(UseFunctionCalling, UseFiles, HasAPIKey) returns the service names that match the requested capabilities, with the matching TTMSMCPCloudAIService value in each string's Objects[] entry. GetActiveServices(UseFunctionCalling, UseFiles) is the same call with HasAPIKey already set to True, so it lists only services you can actually reach.

Both return the component's own internal list. Copy the contents with Assign — never free the result and never keep the reference past the next call. See Common mistakes.

GetModels asks the selected service what it currently offers; the list lands in Models and OnGetModels fires when it is ready.

Combining provider selection, roles, settings, and usage

A real request touches all of the above at once. This one picks a reachable provider, loads a long checklist as the system role, omits the temperature parameter, caches the checklist when the provider is Claude, and reports the result through the same OnExecuted handler:

procedure TForm1.RunReviewPass(const ASourceText: string);
var
  Services: TStringList;
begin
  { 1. Provider selection - prefer Claude when a key is present, and fall
       back to whatever else is configured. GetActiveServices hands back the
       component's internal list, so only its contents are copied. }
  Services := TStringList.Create;
  try
    Services.Assign(AI.GetActiveServices);
    if Services.IndexOf('Claude') >= 0 then
      AI.Service := aiClaude
    else if Services.Count > 0 then
      AI.Service := TTMSMCPCloudAIService(Integer(Services.Objects[0]))
    else
      raise Exception.Create('No AI service has an API key configured.');
  finally
    Services.Free;
  end;

  { 2. Roles - a long, stable review checklist as the system role. }
  AI.SystemRole.LoadFromFile(ReviewChecklistFileName);
  AI.AssistantRole.Text := 'I answer with a numbered list of findings.';
  AI.Context.Text := ASourceText;

  { 3. Generation settings - deterministic, bounded, and with the temperature
       parameter omitted for models that reject it. }
  AI.Settings.UseTemperature := False;
  AI.Settings.MaxTokens := 4096;
  AI.Settings.Reasoning := aiMedium;

  { Claude only: let the service cache the checklist between turns. }
  if AI.Service = aiClaude then
    AI.Settings.CacheControl := ccEphemeral;

  { 4. Response and usage are handled in OnExecuted. }
  AI.OnExecuted := AIExecuted;
  AI.Execute('review');
end;

The id passed to Execute comes back as AResponse.Id, so one OnExecuted handler can serve several call sites and still tell them apart.

Common mistakes

  • Freeing the list returned by GetServices or GetActiveServices. Both return the component's internal TStringList, not a copy made for you. Freeing it corrupts the component and the next call will fail. Copy the contents (cbService.Items.Assign(AI.GetActiveServices) or Local.Assign(...)) and leave the original alone. The same rule applies to Models, to Files[i], and to the SoundBuffer handed to OnSpeechAudio.
  • Reading AResponse.Content without checking Assigned(AResponse). A failed request calls OnExecuted with AResponse set to nil and the reason in AHttpStatusCode / AHttpResult.
  • Setting Temperature := 0 for a model that rejects the parameter. The parameter is still sent. Use Settings.UseTemperature := False to omit it.
  • Treating Execute as synchronous. It returns immediately and its Boolean result only says the request was started. Do your work in OnExecuted, and guard the entry point with Busy so a second request does not start on top of the first.
  • Expecting CacheControl to help every service. It emits Claude's cache_control marker; other services ignore it. It also only pays off for a large, stable block reused across turns.
  • Leaving stale attachments in Files. AddFile, AddText, and AddURL append. Call ClearFiles before building a new request or the previous turn's documents are sent again — see Files, assistants, speech, and transcription.

See also