Table of Contents

Files, assistants, speech, and transcription

A prompt does not have to be text you typed. TTMSMCPCloudAI can carry documents, spreadsheets, images, and URLs into the same request as the question, so the model answers about them rather than from memory. For material too large or too often reused to re-send every turn, files can instead be uploaded to the service and referenced by ID — which is what the assistants and threads API builds on. The same component also handles the audio direction in both senses: turning an answer into spoken MP3, and turning a recording into text or into translated English. This guide covers inline file context including the newer PDF support, uploaded file management, assistants and threads, speech generation, and transcription.

Sending a file with the prompt

Use this when the document belongs to this question — a contract to summarise, an invoice to check, a screenshot to describe. Nothing is stored at the service: the file is read, encoded, and sent inline with the request.

procedure TForm1.AskAboutContract(const APdfFileName, AQuestion: string);
begin
  { Files accumulate across calls - clear the previous turn's attachments
    before building a new request. }
  AI.ClearFiles;

  { AddFile reads the file and base64-encodes it into the request body.
    Nothing is uploaded to the service beforehand. }
  AI.AddFile(APdfFileName, aiftPDF);

  AI.SystemRole.Text := 'Answer strictly from the attached document.';
  AI.Context.Text := AQuestion;
  AI.Execute;
end;

procedure TForm1.SelectPdfCapableService;
var
  Services: TStringList;
begin
  { Pass UseFiles to get only the services that accept file content.
    The returned list belongs to the component - copy it, do not free it. }
  Services := TStringList.Create;
  try
    Services.Assign(AI.GetActiveServices(False, True));
    cbService.Items.Assign(Services);
  finally
    Services.Free;
  end;
end;

AddFile(AFileName, AType) reads the file and attaches it. AType is a TTMSMCPCloudAIFileType value:

Value Content
aiftPDF A PDF document.
aiftImage A PNG or JPEG image.
aiftText Plain text; the file is read as text rather than encoded.
aiftCSV Comma-separated data.
aiftExcel An .xlsx spreadsheet.
aiftWord A .docx document.
aiftAudio MP3 audio submitted as context.
aiftBinary Opaque binary content.
aiftUnknown Type not determined.

PDF context is accepted by Claude, Gemini, Mistral, and OpenAI, each in its own request format — the component handles the difference, so the same AddFile(..., aiftPDF) call works across all four. aiftExcel and aiftWord are submitted to OpenAI. Images are broadly supported; the remaining combinations vary by provider, and GetServices(False, True) is the reliable way to ask which services accept file content at all.

FileExtToFileType maps a file name to its type when the caller does not know it in advance.

Spreadsheets, documents, in-memory text, and URLs

The same Files collection accepts more than files on disk, which matters when the material is generated, already in memory, or public on the web.

procedure TForm1.BuildMixedContext;
begin
  AI.Service := aiOpenAI;
  AI.ClearFiles;

  { Office documents are sent inline as base64 to OpenAI. }
  AI.AddFile('Q4-figures.xlsx', aiftExcel);
  AI.AddFile('board-minutes.docx', aiftWord);

  { An image, read from disk. }
  AI.AddFile('chart.png', aiftImage);

  { Text that is already in memory needs no file on disk. }
  AI.AddText(memoNotes.Lines.Text, aiftText);

  { A URL is passed to the service as a reference - the service fetches it,
    the application does not. FileExtToFileType picks the type from the
    extension when it is not obvious. }
  AI.AddURL('https://example.com/policy.pdf', aiftPDF);

  AI.Context.Text := 'Summarise the attached material into one briefing note.';
  AI.Execute;
end;
Method Adds
AddFile(AFileName, AType) A file read from disk and attached inline.
AddText(AText, AType) Text already in memory, with no file on disk.
AddURL(AURL, AType) A reference the service fetches; your application does not download it.

Attachments accumulate. Call ClearFiles before assembling a new request, or the previous turn's documents are sent again — and paid for again.

Uploading and managing stored files

Upload instead of attaching when the same document is used across many requests, when it is too large to re-send, or when an assistant needs to search it.

procedure TForm1.btnUploadClick(Sender: TObject);
begin
  if not OpenDialog1.Execute then
    Exit;

  { UploadFile stores the file with the service and returns its ID through
    the callback. FileExtToFileType maps the extension to a file type. }
  AI.UploadFile(OpenDialog1.FileName,
    FileExtToFileType(OpenDialog1.FileName),
    procedure(const AID: string)
    begin
      memoLog.Lines.Add('Uploaded as ' + AID);
      AI.GetFiles;
    end);
end;

procedure TForm1.AIGetFiles(Sender: TObject; AResponse: TTMSMCPCloudAIResponse;
  AHttpStatusCode: Integer; AHttpResult: string);
var
  I: Integer;
begin
  lstFiles.Items.Clear;
  for I := 0 to AI.Files.Count - 1 do
    lstFiles.Items.Add(Format('%s  %s  %d bytes  %s',
      [AI.Files[I].ID, AI.Files[I].FileName,
       AI.Files[I].FileSize, AI.Files[I].MimeType]));
end;

procedure TForm1.btnDeleteFileClick(Sender: TObject);
begin
  if lstFiles.ItemIndex < 0 then
    Exit;

  { Delete removes the file at the service. OnFileDeleted reports the
    outcome. The collection item is owned by Files - do not free it. }
  AI.Files[lstFiles.ItemIndex].Delete;
end;

procedure TForm1.AIFileDeleted(Sender: TObject; HttpStatusCode: integer;
  HttpResult: string; Index: integer);
begin
  memoLog.Lines.Add(Format('File %d deleted (%d)', [Index, HttpStatusCode]));
  AI.GetFiles;
end;

UploadFile stores the file at the service and hands its ID to the callback. GetFiles refreshes the Files collection from the service and fires OnGetFiles; OnFileUpload and OnFileDeleted report the outcome of an upload and a deletion. Each TTMSMCPCloudAIFile exposes ID, FileName, FileSize, MimeType, and a Delete method that removes it at the service.

A file that carries an ID is referenced by that ID in the request instead of being re-encoded, so uploading once and reusing the ID is markedly cheaper than attaching the same document repeatedly.

Assistants and threads

An assistant is a stored configuration at the service — instructions plus tools such as file search — that answers over a conversation thread. Reach for it when you want retrieval across a set of uploaded documents rather than a single-shot question over one attachment. This is an OpenAI feature, so set Service := aiOpenAI first.

procedure TForm1.AskAssistant(const AQuestion: string);
begin
  AI.Service := aiOpenAI;

  { Assistants live at the service. Reuse one instead of creating a new one
    on every run. }
  AI.GetAssistants(
    procedure(AResponse: TTMSMCPCloudAIResponse; AHttpStatusCode: Integer;
      AHttpResult: string)
    begin
      if AHttpStatusCode div 100 <> 2 then
      begin
        memoLog.Lines.Add('Could not list assistants: ' + AHttpResult);
        Exit;
      end;

      if AI.Assistants.Count > 0 then
        RunAssistantThread(AI.Assistants[0].ID, AQuestion)
      else
        AI.CreateAssistant('Document assistant',
          'You answer questions about the attached documents.',
          [aitFileSearch],
          procedure(const AID: string)
          begin
            RunAssistantThread(AID, AQuestion);
          end);
    end);
end;

procedure TForm1.RunAssistantThread(const AAssistantID, AQuestion: string);
begin
  AI.CreateThread(
    procedure(const AThreadID: string)
    var
      FileIDs: TStringList;
      I: Integer;
    begin
      FileIDs := TStringList.Create;
      try
        for I := 0 to AI.Files.Count - 1 do
          FileIDs.Add(AI.Files[I].ID);

        AI.CreateMessage(AThreadID, 'user', AQuestion, FileIDs, aitFileSearch,
          procedure(const AMessageID: string)
          begin
            { RunThreadAndWait polls the run until it finishes and then
              calls back with the assistant's answer. }
            AI.RunThreadAndWait(AThreadID, AAssistantID,
              procedure(AResponse: TTMSMCPCloudAIResponse;
                AHttpStatusCode: Integer; AHttpResult: string)
              begin
                if Assigned(AResponse) then
                  memoAnswer.Lines.Text := AResponse.Content.Text
                else
                  memoAnswer.Lines.Text := AHttpResult;
              end);
          end,
          procedure(AResponse: TTMSMCPCloudAIResponse;
            AHttpStatusCode: Integer; AHttpResult: string)
          begin
            memoLog.Lines.Add('Message failed: ' + AHttpResult);
          end);
      finally
        FileIDs.Free;
      end;
    end);
end;

The flow is four steps, each asynchronous and each handing the next its ID:

  1. GetAssistants lists what already exists — the Assistants collection is filled and OnGetAssistants fires when the list arrives, in addition to the optional AComplete callback you can pass to the call. CreateAssistant(AName, AInstruction, ATool, ACreated) makes a new one instead. ATool is a TTMSMCPCloudAIAssistantTools set built from TTMSMCPCloudAIAssistantTool values — aitFileSearch, aitCodeInterpreter, aitFunction.
  2. CreateThread opens a conversation.
  3. CreateMessage(AThreadID, ARole, AContent, AFiles, ATool, ACreated, AFailed) adds a turn, attaching uploaded file IDs so the assistant can search them.
  4. RunThread starts the assistant and returns immediately, or RunThreadAndWait polls until the run completes and calls back with the answer. CheckStatus queries a run in progress — with or without an explicit thread ID.

Assistants are stored at the service and outlive your application. Reuse one rather than creating a fresh assistant on every run, or the account accumulates them. TTMSMCPCloudAIAssistant items expose ID, Name, Instructions, and Model, and a Delete method.

Generating speech

Use this when the answer should be heard rather than read — an accessibility path, a kiosk, a hands-free workflow.

procedure TForm1.SpeakAnswer(const AText: string);
begin
  AI.Service := aiOpenAI;
  AI.Settings.OpenAISoundModel := 'gpt-4o-mini-tts';
  AI.OnSpeechAudio := AISpeechAudio;

  { Speed 1 is normal pace. Voice names and the tone instruction are passed
    to the service as-is. }
  AI.Speak(AText, 1.0, 'alloy', 'calm and professional');
end;

procedure TForm1.AISpeechAudio(Sender: TObject; HttpStatusCode: integer;
  HttpResult: string; SoundBuffer: TMemoryStream);
begin
  if (HttpStatusCode div 100 <> 2) or not Assigned(SoundBuffer) then
  begin
    memoLog.Lines.Add('Speech failed: ' + HttpResult);
    Exit;
  end;

  { The stream belongs to the component - copy what is needed before the
    handler returns rather than keeping the reference. }
  SoundBuffer.Position := 0;
  SoundBuffer.SaveToFile(TPath.Combine(TPath.GetTempPath, 'answer.mp3'));

  memoLog.Lines.Add(Format('Spoken characters this session: %d',
    [AI.Usage.AudioCharacters]));
end;

Speak(AText, Speed, Voice, Tone) generates MP3 audio through OpenAI using the model named in Settings.OpenAISoundModel. Speed is a multiplier where 1 is normal pace; Voice and Tone are passed to the service as written. The audio arrives in OnSpeechAudio — a TTMSMCPCloudAISpeechEvent — as a TMemoryStream owned by the component. Save or copy it inside the handler; do not keep the reference and do not free it.

Usage.AudioCharacters accumulates the number of characters spoken, which is how these calls are normally billed.

Transcribing and translating audio

The other direction: an MP3 recording becomes text, optionally rendered in English.

procedure TForm1.TranscribeRecording(const AFileName: string);
begin
  AI.Service := aiOpenAI;
  AI.Settings.OpenAITranscribeModel := 'gpt-4o-transcribe';
  AI.OnTranscribeAudio := AITranscribeAudio;

  { An empty language lets the service detect it. Pass an ISO code such as
    'nl' or 'de' when the language is known. }
  AI.Transcribe(AFileName, '');
end;

procedure TForm1.TranscribeBuffer(ABuffer: TMemoryStream);
begin
  { The same call accepts an in-memory MP3 buffer, for audio that was just
    recorded and never written to disk. }
  AI.OnTranscribeAudio := AITranscribeAudio;
  AI.Transcribe(ABuffer, '');
end;

procedure TForm1.TranslateRecording(const AFileName: string);
begin
  { Translate transcribes and renders the result in English in one call. }
  AI.OnTranscribeAudio := AITranscribeAudio;
  AI.Translate(AFileName);
end;

procedure TForm1.AITranscribeAudio(Sender: TObject; HttpStatusCode: integer;
  HttpResult: string; Text: string);
begin
  if HttpStatusCode div 100 <> 2 then
  begin
    memoLog.Lines.Add('Transcription failed: ' + HttpResult);
    Exit;
  end;

  memoTranscript.Lines.Text := Text;
  memoLog.Lines.Add(Format('Audio seconds this session: %d',
    [AI.Usage.AudioDuration]));
end;

Transcribe accepts either a file name or an in-memory TMemoryStream, plus an optional language code — leave it empty to let the service detect the language. Translate takes the same two forms and renders the result in English. Transcription is available through OpenAI, Gemini, and Mistral, using Settings.OpenAITranscribeModel and Settings.MistralTranscribeModel respectively; translation is an OpenAI feature.

The text arrives in OnTranscribeAudio, a TTMSMCPCloudAITranscribeAudioEvent. Usage.AudioDuration accumulates the seconds processed.

Combining transcription, file context, and speech

The three halves of this guide chain naturally: a spoken question becomes the prompt, a PDF becomes the context, and the answer is read back aloud.

procedure TForm1.RunVoiceBriefing(const ARecordingFileName, APdfFileName: string);
begin
  FBriefingPdf := APdfFileName;

  AI.Service := aiOpenAI;
  AI.Settings.OpenAITranscribeModel := 'gpt-4o-transcribe';
  AI.Settings.OpenAISoundModel := 'gpt-4o-mini-tts';
  AI.Settings.UseTemperature := False;

  { 1. Transcription - the spoken question becomes the prompt. }
  AI.OnTranscribeAudio := BriefingTranscribed;
  AI.Transcribe(ARecordingFileName, '');
end;

procedure TForm1.BriefingTranscribed(Sender: TObject; HttpStatusCode: integer;
  HttpResult: string; Text: string);
begin
  if HttpStatusCode div 100 <> 2 then
  begin
    memoLog.Lines.Add('Transcription failed: ' + HttpResult);
    Exit;
  end;

  { 2. File context - answer the transcribed question over the PDF. }
  AI.ClearFiles;
  AI.AddFile(FBriefingPdf, aiftPDF);

  AI.SystemRole.Text := 'Answer from the attached document in two sentences.';
  AI.Context.Text := Text;
  AI.OnExecuted := BriefingExecuted;
  AI.Execute;
end;

procedure TForm1.BriefingExecuted(Sender: TObject;
  AResponse: TTMSMCPCloudAIResponse; AHttpStatusCode: Integer;
  AHttpResult: string);
begin
  if not Assigned(AResponse) then
  begin
    memoLog.Lines.Add('Request failed: ' + AHttpResult);
    Exit;
  end;

  memoAnswer.Lines.Text := AResponse.Content.Text;

  { 3. Speech - read the answer back. }
  AI.OnSpeechAudio := AISpeechAudio;
  AI.Speak(AResponse.Content.Text, 1.0, 'alloy', 'calm and professional');
end;

Each step's event handler starts the next, because every one of these calls is asynchronous. Keep the state each step needs — here the PDF file name — in a field rather than trying to close over it across events.

Common mistakes

  • Freeing the SoundBuffer from OnSpeechAudio, or keeping the reference. The stream belongs to the component. Save or copy what you need inside the handler and let it go. The same applies to the list returned by GetServices / GetActiveServices and to items in Files and Assistants — copy their contents, never take ownership.
  • Forgetting ClearFiles. AddFile, AddText, and AddURL append to the collection. Without a clear, the second question re-sends the first question's documents.
  • Assuming every service accepts every file type. PDF reaches Claude, Gemini, Mistral, and OpenAI; .xlsx and .docx reach OpenAI. Ask GetServices(False, True) for the services that take file content at all, rather than discovering it from an HTTP error.
  • Creating a new assistant on every run. Assistants persist at the service. Call GetAssistants and reuse one.
  • Treating the assistant flow as synchronous. CreateThread, CreateMessage, and RunThread each complete through a callback. Chain them from those callbacks; do not call them in sequence and expect the IDs to be ready.
  • Attaching a document inline when it is reused. Inline content is encoded and re-sent on every turn. Upload it once and reference the ID.

See also