Logging

The logs from the LLM containers is standardized across all of the different models. This has been done in order to be able to parse all of the relevant information from the logs and create monitoring on top of them.

In order to make use of these logs, which are written to the standard out of each container, you must collect the logs. If you do not have an established logging and monitoring process, you can use the simplified logging script provided below. Otherwise, if you do have an established way of collecting logs, ensure that you can export them to a folder that can be accessed from SAS Studio as a .log file (as that it is the input for the parsing utilities). The file has to hold the lines as the containers wrote them: the parser only reads a line that starts with <timestamp> - <logger> - <level> - , so a timestamp, a pod name or a JSON wrapper that your process adds to each line has to be removed again. If it is not, the parser reads no request and says so with a warning in the SAS log. The lines of each container also have to stay together in the file, as kubectl logs writes them: an export that orders the lines of all containers by their time is read with gaps, see Requests that run at the same time.

For more information on how to parse the log and load it into SAS please take a look at SAS-Viya-Integrations/Logging-Monitoring.

Three programs of that folder work on the collected log, and they run in this order:

Collect the log - the script below, or your own process.
Parse it: Log-Parser-Code.sas or the LLM - Log Parser custom step write one row per request to the table LLM_LOGS - see Running the Parser and The Columns of LLM_LOGS.
Pull what the Prompt Builder keeps in SAS Model Manager: Get-All-Prompts.sas or the LLM - Get All Prompts custom step write the experiments, the RAG setups and a catalog of models - see The Experiments of Every Kind.
Link the two: Link-Logs-To-Models.sas fills, for every row of LLM_LOGS, the SAS Model Manager id of the model that answered and of the prompt, decision template or RAG setup that sent the request - see Linking the Log to the Models.

Steps 2 to 4 of release 2.3.0 ran as one chain on the test system three times, on 2026-10-11 and twice on 2026-10-12, each time with the programs as they were then - the parse with its 24 further columns, the pull with its four further tables, the link, and the joins of the tables: see The Chain on SAS Viya. The first run found defects and the three programs were changed; the second run measured those changes and found further defects, and the programs were changed again; the third run measured those changes - version 2.4 of the LLM - Log Parser step, version 2.2 of the LLM - Get All Prompts step, Link-Logs-To-Models.sas and the ingestion job of a RAG setup. Of what was changed after the third run, the settings a program takes only where they are not set (Settings that stay yours), Load-Fact-Sheets.sas through the job Load-Accelerator-Tables, and Build-RAG-Cost-View.sas with one and with two run tables were measured later that day in test content roots. On 2026-10-13 the collection script as it was that day - with its line in front of the lines of every container, without escape_llm_log - collected the model pods of the test system, and the three code files of that day parsed, pulled and linked that file in a test content root; Build-RAG-Cost-View.sas as it is since it leaves at the first refusal ran there as well: see The Pass of 2026-10-13. Later that day the collection script with escape_llm_log collected the pods again, and version 2.7 of the parser as a code file, the pull and the link ran on that file: see The Second Pass of 2026-10-13. Not run in SAS, and the sections say which part that is: the two steps as steps in any version after 2.4 and 2.2 - the two passes of 2026-10-13 ran the code files -, the rules of version 2.7 for which the collected file holds no case, the collection script with sed in the C locale, and the counter of an optimizationId. The Python of the three programs runs in the tests of the repository and was run outside of SAS on the collected log of those runs. What Was Measured is about the parser as it was before the further columns.

What the containers log

The shipped deployment templates set SAS_SCR_LOG_LEVEL_SCR_IO=TRACE, and the parser depends on it: the Request: POST / Request Data: lines carry the inputs and the options of every call, and the lines of the score code the token counts, the run time and the answer. For an LLM that is the system prompt, the user prompt and the full response; for a decision model the state, the questions and every answer with its confidence; for an embedding model the embedded text. That is what makes the usage report possible, and it also means prompt text and responses land in your log pipeline. Of an embedding call the table keeps the hash of the text in place of the text, and it also keeps the options of every call without their keys and the first error of a request that failed. Decide who may read the collected log file and the LLM_LOGS table accordingly, or lower the level and forgo the report.

The containers also log secrets

At this level a container writes more than prompts. The options of a request are logged as they arrive, and for every model that is called with a provider key they hold it as API_KEY:<the key>. The request headers are logged too, and a call from the browser (the Prompt Builder) carries the user's Cookie header. Both must not reach a file that SAS Studio users can open: the collection script below replaces them with REDACTED before a line is written, and an established logging process needs the same step. The redact_llm_log function of the script expects the lines as the container wrote them, which is how kubectl logs prints them without --timestamps or --prefix: its three header expressions match a header at the start of a line, and only the API_KEY expression finds its text anywhere in a line. A process that puts a timestamp or a pod name in front of every line, or wraps the line (the log files of the container runtime, JSON), has to redact before that happens or adapt the three header expressions.

The redaction only covers the files and tables the accelerator writes: kubectl logs and a cluster log pipeline still read the key and the cookie. A model whose key the container holds (mdb add --auth env) is called without API_KEY, so its key does not reach the log unless a caller still sends one; its route then has no authentication of its own and needs an allowlist or a NetworkPolicy (Container Deployment).

A log file collected without this redaction - with the script of release 2.2.0 or earlier, for example - may contain provider keys and session cookies. Delete those files, including copies and backups, and rotate every provider key that was in use while they were collected. The LLM_LOGS table never held either and does not need to be rebuilt for this: the parser never copied the key or a header into it. The parser of this release keeps the options of a request and replaces the keys in them itself, and of the headers it keeps the X-Request-Id and the kind of client.

Simplified Logging via a Script

Only use this script if you do not have a more robust process in place.

Create a script—for example collect_llm_logs.sh—and ensure that it is executable. Use the code below and change both the namespace and the log path. The namespace should be the namespace into which you have deployed the models, and the path needs to be reachable from inside of SAS Studio. The path below is an example and site specific - see Getting the File to the SAS Compute Server for what it assumes and for a way without it. You can also change the logging interval; note that any changes you make here will have to be reflected in the entry for the crontab. The default here also replaces the log file, as the assumption is that you pick up the new entries using the parsing script and append them to the table (a full replacement is also supported). The parser appends only where it is told to: set %let _log_append_table = 1; in front of it, see Appending. With %let _log_append_table = 0; it replaces the table by the requests of the one file, and without the setting it stops at a table that holds rows.

# Set the script variables for your environment
llm_namespace="llm"
llm_log_path="/viya-share/pvs/sasdata/data/llm/llms.log"
llm_logging_interval="1h"
# Selects the model pods: the deployment templates label each of them app=<model name>
llm_pod_selector="app"

# Replaces the secrets in the container log with REDACTED: the provider key in the
# options of a request and the values of the Cookie, Authorization and X-CSRF-TOKEN
# request headers (in any spelling of upper and lower case, which needs GNU sed).
# sed runs in the C locale, where every byte is a character: in a UTF-8 locale a byte
# that is no UTF-8 ends a match, and what stands behind it would stay as it is
redact_llm_log() {
  LC_ALL=C sed -E \
    -e "s/(API_KEY:)[^,}'\"]*/\1REDACTED/g" \
    -e 's/^((.* Request Headers: )?Cookie:).*/\1 REDACTED/I' \
    -e 's/^((.* Request Headers: )?Authorization:).*/\1 REDACTED/I' \
    -e 's/^((.* Request Headers: )?X-CSRF-TOKEN:).*/\1 REDACTED/I'
}

# Keeps the lines of a container apart from the line this script writes in front of them: a
# line of a container that starts like that line - at its start or behind a carriage return,
# with or without backslashes in front - gets one more backslash in front, which the parser
# takes off again. The last line of a container is ended where it is not (needs GNU sed)
escape_llm_log() {
  sed -E \
    -e 's/(^|\r)(\\*#=# llm-log-collection )/\1\\\2/g' \
    -e '$a\'
}

# The time of this collection, in UTC: the same in the line in front of every container
llm_collected_at="$(date -u +%Y-%m-%dT%H:%M:%SZ)"

# Remove the next line to append to the log file instead
rm -f "$llm_log_path"
for pod in $(kubectl get pods -n "$llm_namespace" -l "$llm_pod_selector" -o name); do
  for container in $(kubectl get -n "$llm_namespace" "$pod" -o jsonpath='{.spec.containers[*].name}'); do
    # One line in front of the lines of each container: the parser ends a part of the file there
    {
      printf '#=# llm-log-collection pod=%s container=%s collected=%s #=#\n' "${pod#pod/}" "$container" "$llm_collected_at"
      kubectl logs -n "$llm_namespace" "$pod" -c "$container" --since="$llm_logging_interval" | escape_llm_log
    } | redact_llm_log >> "$llm_log_path"
  done
done

The script reads the model pods only. -l app selects every pod that carries a label named app, which the shipped deployment templates set to the name of the model. The pod of a stage-weights job does not carry it and stays out of the file, and so does every other workload that is labelled differently. If something else in your namespace is labelled app as well, give the model deployments a label of their own and set llm_pod_selector to it (for example llm_pod_selector="accelerator=model").

The script reads every container of every model pod on its own, and writes one line in front of its lines: #=# llm-log-collection pod=<pod> container=<container> collected=<time of the collection, UTC> #=#. All lines of one run carry the same time. A container can write a line of that shape - a model writes what it is asked for, and its response is logged line by line -, so the script keeps the two apart: escape_llm_log puts a backslash in front of every line of a container that starts like the line of the script, at the start of a line and behind a carriage return, with or without backslashes in front, and it ends the last line of a container where kubectl logs did not. In a file this script wrote, a line of that shape is therefore a line of the script. The redaction leaves the line of the script as it is and makes no other line a line of that shape. The parser ends a part of the file at each such line and nowhere else, whatever time of collection the line carries, and takes the backslash off again, so a response holds the line as the model wrote it: a request that was running when its container was collected ends there without a response and without an error - failed is missing, and a NOTE counts it - and no line of one container is taken for a line of another. A file without such lines - collected with the script of release 2.2.0 or earlier, or by a process of your own - is read by the time of its lines, as before, and so is what stands in front of the first such line in a file that is appended to across the upgrade. In those lines nothing was put in front of a look-alike: a line of that shape in a response there is read as the line of the script. It ends the response, its request has no status and failed is missing - the NOTE counts it -, and the lines behind it are read as one part up to the next line of the shape. No code can tell it from the line of the script: a file with such a line in a response and a file in which the first collection of this release begins in the middle of that response are the same line for line, and a test of the repository holds the two. So start a new file with the script of this release instead of appending to a file of an earlier one. A process of your own can write the same line in front of the lines of each container it collects; it then has to put the backslash in front as the script does. The pod is no column of LLM_LOGS. On 2026-10-13 the script as it was that day - without escape_llm_log - collected the nine model pods of the test system and wrote nine such lines, all with one time of collection, and Log-Parser-Code.sas of that day read the file (The Pass of 2026-10-13). The script with escape_llm_log, and version 2.7 of the parser, which takes the backslash off, ran in the tests of the repository only - the script with a kubectl of the test that answers from files and with GNU sed 4.9 of Git for Windows, which drops a carriage return in front of a line feed, so the lines that end with both were held by their rows and not by their bytes. They have not run in a live pass and are not run in SAS.

The redaction happens in the pipe, so an unredacted line is never written to disk, and the parser reads a redacted log exactly like an unredacted one. redact_llm_log runs sed in the C locale (LC_ALL=C), where every byte is a character. In a UTF-8 locale GNU sed matches a byte that is no UTF-8 with neither . nor a bracket expression, and the function as it was first written for this release - without the locale - redacted the value of a header or of API_KEY only up to such a byte. Measured on 2026-10-13 with GNU sed 4.9 of Git for Windows and the expressions of the script: in the locale en_US.UTF-8 and in C.UTF-8 the line Cookie: abc<FF>secret-value (<FF> being the byte 0xFF) became Cookie: REDACTED<FF>secret-value, and x API_KEY:sk-abc<FF>secret, y became x API_KEY:REDACTED<FF>secret, y; in the C locale they became Cookie: REDACTED and x API_KEY:REDACTED, y. A shell that a cron job starts often runs in the C locale already, an interactive one usually does not. The function with the locale is not run on Linux or against a cluster; the tests of the repository run it with GNU sed 4.9 of Git for Windows. The four sed expressions cover what the containers are known to log; if your requests carry a secret in another header, add a line for it.

An hour without a single request leaves a file without requests. The parser reports that with a note and leaves the LLM_LOGS table as it is.

Next add the following entry to your crontab (crontab -e), adjusting your path to the script:

0 * * * * /viya-share/pvs/sasdata/data/llm/collect_llm_logs.sh

This entry will run the log collection script at the top of every hour.

Getting the File to the SAS Compute Server

The paths of this guide are examples, and both are site specific. The script writes to /viya-share/pvs/sasdata/data/llm/llms.log and the parser reads the path that %let _log_path = sasserver:/export/pvs/sasdata/data/llm/llms.log; names where _log_path is not set - the statement a site sets its own path with, in front of the program (Settings that stay yours). The two name the same file only on a system where the host that runs the script and the SAS compute server mount the same storage under these two paths. Set llm_log_path to the file as the collecting host sees it and _log_path to the same file as a compute session sees it, with sasserver: in front: the parser reads the file from the file system of the compute server, stops with an error for any other location and does not read from SAS Content.

Whether a compute session sees a directory can be asked in that session, exists=1 means it does:

data _null_;
    exists = fileexist('/export/pvs/sasdata/data/llm');
    put exists=;
run;

Without a Shared Mount

On the test system (SAS Viya 2026.09, measured on 2026-10-11) there is no such mount: in a compute session neither /export nor /viya-share existed, and the workstation that collected the logs shares no directory with the compute server. The file went through SAS Content there:

The script of this guide wrote the redacted file to a local directory of the collecting host (llm_log_path).
The file was uploaded to a folder in SAS Content with the Files service: POST /files/files with the folder as parentFolderUri, the file as the body and its name in the Content-Disposition header.
The job that runs the parser first copied the file from SAS Content into the compute session, with a FILESRVC file reference and fcopy, and the parser then read the copy:

/* Copy the collected file from SAS Content to the SAS compute server */
filename llmsrc filesrvc folderpath='/Users/<user>/My Folder/<folder>' filename='llms.log' recfm=n lrecl=32767;
filename llmdst '/tmp/llms.log' recfm=n lrecl=32767;
data _null_;
    length msg $300;
    rc = fcopy('llmsrc', 'llmdst');
    msg = sysmsg();
    put 'Copy of llms.log: ' rc= msg=;
run;
filename llmsrc clear;
filename llmdst clear;

/* Set the path in front of Log-Parser-Code.sas: the program takes _log_path where it is set */
%let _log_path = sasserver:/tmp/llms.log;

From release 2.3.0 on the last line stands in front of the program - in a program of your own that then includes the shipped file, see Settings that stay yours: Log-Parser-Code.sas takes _log_path only where it is not set. That is how the program ran in The Second Pass of 2026-10-13, with its settings in front of the shipped file. Up to then the program set _log_path itself, in a %let line among its first lines, so a %let that only stood in front of it had no effect and the line of the program had to be changed. Both were measured on 2026-10-11, with the program of that day. With the block in front and the program's own line changed to sasserver:/tmp/llms.log, the parser wrote the 174 rows of the file. With the block in front of the unchanged program, the copy went through and the program then read its default path: FileNotFoundError: [Errno 2] No such file or directory: '/export/pvs/sasdata/data/llm/llms.log', then ERROR: The log could not be parsed, ..., and the job ended as failed. The custom step has no such line: there the copy is the file the step is given as its log.

These are the statements that ran. On 2026-10-11 they ran as they are printed here, with only the folder filled in - one below the My Folder of the user who ran the job: the log said Copy of llms.log: rc=0 msg=, and the copy had the size and the MD5 digest of the uploaded file (558,176 bytes). In the first runs, earlier that day, each collection had a file name of its own; fcopy returned 0 there as well, and a file of 0 bytes arrived as a file of 0 bytes. /tmp is a directory that existed in the compute session of that system - the fileexist step above gave exists=1 for it and exists=0 for /export/pvs/sasdata/data/llm; take one your compute server can write to. The three steps were started one after the other, not from a crontab entry.

The file in SAS Content holds the prompts and the responses like the log itself, so whoever may read that folder reads them (see the note at the top of this page).

Running the Parser

Log-Parser-Code.sas and the LLM - Log Parser custom step hold the same program. The README of SAS-Viya-Integrations/Logging-Monitoring describes the table it writes, what a row holds for each kind of model and what happens to a table that already exists. The two differ in what they bring along:

Log-Parser-Code.sas and the LLM - Log Parser custom step reach a site with the transfer package, in the folder Logging and Monitoring of the content folder (Install or upgrade with the transfer package); from a checkout, mdb content-deploy brings both. A registered step is updated in place - PUT /dataFlows/steps/{id} with the file and its id, answered 200 with the id kept when it was measured on 2026-10-11 and on 2026-10-12 -, which is what mdb content-deploy does: a step that is deleted and uploaded again gets a new id, and saved flows that use it then have to be given the step again. The import and the deploy bring the step to its path of this release, Logging and Monitoring/LLM - Log Parser.step. A step your flows use from another folder - the starter package of earlier releases had it in Custom Steps - is not touched by either: update it in place from a checkout with mdb content-deploy --only "Logging and Monitoring/LLM - Log Parser.step" --to "<path or id of your step>", see One part, somewhere else. Measured on 2026-10-13 in two test content roots: with the path of a registered step of that name the line said equal and named the step and its folder; with the id of a step of another name nothing was written and the line asked for --force, and with --force the step was updated with its id kept. A flow or a job that runs the step more than once in one SAS session needs the step of release 2.3.0. That step announces 36 columns to a flow; a flow that holds an older step announces twelve. A copy of the package SAS Agentic AI Accelerator.json of earlier releases must not be imported for the parser: that starter package holds the step and the code file of May 2025 - twelve columns, LLM calls only, and the session check that fails in a second run.

Log-Parser-Code.sas starts a CAS session itself (cas mysess;) when neither the session the SAS option SESSREF names nor a session mysess is connected - a connected mysess is used, and the option is set to it -, assigns the librefs public and casuser and takes each of its settings where the macro variable is not set when it starts (Settings that stay yours). The session and the two librefs stay when the program ends. _log_target_table_lib names a libref: the program looks up the caslib behind it and, when no libref of that name is assigned, hands the value to CAS as the name of the caslib.
The custom step is the same program without those first lines, so it takes the following from the SAS session it runs in.

Run the chain in a compute context that runs as you

The parser and the link program hand their rows to CAS from Python, in the CAS session of the SAS session they run in. In a compute context whose servers run as a service account - on the test system the SAS Job Execution compute context, which runs as sas-be-sa - that fails: Session reconnect failed: Users can only reconnect to their own sessions, then ERROR: The log could not be parsed, the table Public.LLM_LOGS was left as it is., and Get-All-Prompts.sas cannot read SAS Model Manager either. Measured on 2026-10-22; in the SAS Studio compute context, which runs as the user, the same code ran. For a scheduled chain pick such a context (_contextName of the job, or the context of the compute session you submit to).

Settings that stay yours

The five programs of the folder Logging and Monitoring - Log-Parser-Code.sas, Get-All-Prompts.sas, Link-Logs-To-Models.sas, Load-Fact-Sheets.sas and Build-RAG-Cost-View.sas - take a setting only where its macro variable is not set when the program starts. Up to release 2.2 the settings were %let lines at the top of each file, so a newer file, or an import of the content folder, replaced the values of a site with the defaults. Keep your values in a program of your own that sets them and includes the shipped file from the content folder; the shipped file can then be replaced without a value of yours in it:

%let _log_path = sasserver:/data/llm/llms.log;
filename shipped filesrvc folderPath='/SAS Agentic AI Accelerator/Logging and Monitoring' filename='Log-Parser-Code.sas';
%include shipped;

Program	Settings, and what the program takes where one is not set
`Log-Parser-Code.sas`	`_log_path` (`sasserver:/export/pvs/sasdata/data/llm/llms.log`), `_log_target_table_engine` (`CAS`), `_log_target_table_lib` (`Public`), `_log_target_table_name` (`LLM_LOGS`), `_log_append_table` (no default: `1` appends, `0` replaces, and without a value the program stops at a table that holds rows - see Appending), `_log_promote_table` (`1`), `_log_save_table` (`1`)
`Get-All-Prompts.sas`	The eleven settings under Settings of the pull
`Link-Logs-To-Models.sas`	The twelve settings under Settings of the Link
`Load-Fact-Sheets.sas`	`_lfs_path` (`/SAS Agentic AI Accelerator/Data`), `_lfs_caslib` (`Public`)
`Build-RAG-Cost-View.sas`	`_rcv_fact_caslib` (`Public`), `_rcv_runs_caslib` (`Public`), `_rcv_runs_tables` (`RAG_HR_RUNS`; several tables separated by commas, with or without blanks around them), `_rcv_out_caslib` (`Public`), `_rcv_out_table` (`RAG_RUN_COST`)

Set means that the macro variable exists. A variable that is set and empty stays empty. For _gap_llm_project, _gap_embedding_project and _gap_decision_project an empty value leaves the models of that kind out of the catalog.
Each program deletes its settings at its end, with its other macro variables, as before. A program of yours therefore sets them every time, in front of the include. For the parser that includes _log_append_table: a second include in one SAS session without its %let line is a run without the setting, which stops at a table that holds rows. Measured on 2026-10-13: the first include, with 1, appended; the second, with the log and the table set again and _log_append_table not, wrote its ERROR line and left the 4,654 rows and the saved file as they were.
A macro variable of that name that something else left in the SAS session counts as set - a run of the program that was cancelled before its end, for one. Of the parser such a run leaves _log_path without its sasserver:, which the next run refuses. Start a new SAS session, or delete the variable with %symdel.
The two custom steps are not affected. A step gets every setting from one of its controls, and the program inside a step has none of these lines.
Before the first run on a table of an earlier release - a site that upgrades from 2.2.x and has an LLM_LOGS -, read Appending: the parser of this release stops at that table until _log_append_table is set.

The defaults are the values the %let lines had, but for _log_append_table, whose line held 0 and which has no default any more: the other 36 settings of the five programs were compared with them in the code. Measured on the test system on 2026-10-12 with the parser, the pull and the link, each from a program like the one above - the settings, then the include of the shipped file from a test content root:

The parser with _log_path, _log_target_table_lib and _log_target_table_name set wrote its table under that name into that CAS library: 1040 rows and 36 columns, promoted and saved, so the four settings that were not set had their defaults. Behind the include none of the seven _log_ variables was left, and Public.LLM_LOGS had the rows and the modified time it had before.
The pull with the CAS library and the five table names set wrote its five tables there - 318 rows and 61 columns, 27 and 32, 17 and 26, 107 and 40, 421 and 21 -, the tables of those names in Public were as before, and the eleven _gap_ variables were gone. With %let _gap_decision_project = ; the catalog had 416 rows instead of 421; that the five rows are the decision models was not read.
The link wrote 1040 of 1040 rows ... linked against 421 catalog rows, and the twelve _lnk_ variables were gone.
Build-RAG-Cost-View.sas ran the same way with one run table, see The Third Run. Load-Fact-Sheets.sas took _lfs_path and _lfs_caslib from the job Load-Accelerator-Tables, which sets both in front of its include (The tables of the reports).

Not run: a program with its defaults, which writes Public; and whether the control variables of a step stay behind it in a SAS Studio session, where a code file run next would take them for settings.

What the Custom Step Takes from the Session

A CAS session. The step asks the SESSFOUND function whether the session that the SAS option SESSREF names is connected - that is the session its CAS actions run in. When it is connected, the step starts no session, runs in that one and does not end it. When it is not and a session llmLogMySess is connected, the step sets the option to that session (options sessref=llmLogMySess;), runs in it and does not end it either. Otherwise the step starts a session of its own (cas llmLogMySess;) and ends it as its last action. A table that is not promoted belongs to the session that created it, so of a run with a session of its own only what Promote the table to global scope or Save the table kept is left. The option goes on naming a session that was ended: on the test system _CASNAME_, _SESSREF_ and the option SESSREF still named LLMLOGMYSESS after such a run, while cas _all_ list; listed no session, and SAS wrote NOTE: The default CAS session LLMLOGMYSESS identified by SAS option SESSREF= was terminated. Use the OPTIONS statement to set the SESSREF= option to an active session. That is why the step does not go by a name, and why statements of your own that follow the step in the same SAS session and need CAS have to start a session or set SESSREF to one.
The libref of the output table. SAS Studio hands the step the output table as a libref and a table name. The step looks the libref up in dictionary.libnames and writes to the caslib it finds there. When the libref is not assigned in the session, nothing is found and the name of the libref is handed to CAS as the name of the caslib - which is the caslib you chose only where the two have the same name, as with CASUSER and PUBLIC. Without the libref the lines SAS Studio adds behind the step also report the output table as not existing: in the job that had a CAS session and no libref, the log held the line _FLW_ACTION_TABLE_|CASUSER|<table>|0, where the job with the libref had |1. The table was there - promoted, saved and read from a new CAS session with its 174 rows. What SAS Studio does with that line in a flow was not looked at; assign the libref in front of the flow to have it report the table.
The caslib CASUSER of the user who runs the step. The parsed rows are uploaded there as _TEMP_LLM_LOGS, and the staging table is dropped at the end. In replace mode they are copied from there to <table>_NEW in the caslib of the output table; in append mode a DATA step reads them from there. Two more tables of the program can stand there for the time of a run: _TEMP_LLM_LOGS_SEEN, the transaction IDs of the table a run appends to, and _TEMP_LLM_LOGS_SAVED, the saved file of a table that is not in memory, loaded to count its rows by a run that replaces and by a run of Log-Parser-Code.sas without the setting.
Two librefs and a work table of its own, in append mode. To append to a table that exists, the program assigns _llmtgt to the caslib of the output table and _llmsrc to CASUSER, asks CAS for the columns of the existing table (kept in work._log_target_columns while it is read, then deleted), writes the existing rows and the new ones to _TEMP_LLM_LOGS_ALL in the caslib of the output table with a DATA step, and clears both librefs. The DATA step reads the output table through _llmtgt, so the name of the table has to be one a libref can address (32 characters at most). Log-Parser-Code.sas does the same. The append ran on the test system on 2026-10-11, see Appending.
PROC PYTHON with the pandas and swat packages, and what Python needs to reach CAS: the SAS options CASHOST and CASPORT, the certificate file named in the environment variable SSLCALISTLOC and the token in SAS_SERVICES_TOKEN. The Python code connects to the CAS session of the step with them.

The code SAS Studio generates for a flow provides neither a CAS session nor the libref: the code generated on the test system for a flow of the step and its output table held no cas and no libname statement. The measurements ran that code unchanged as SAS Job Execution jobs, with the output table in CASUSER, in three kinds of session. The first had these two lines in front (the name of the session is free):

cas mysess;
libname casuser cas caslib='casuser';

The second had the cas statement alone in front, and the third neither of the two, so that the step started its own session. What Was Measured has what each of them did. These runs were made with the step as it was earlier on 2026-10-11, which started its own session where the macro variable _CASNAME_ did not exist. The step now reaches the same two branches through SESSFOUND, and with that check it was run again later that day: see The Session Check with SESSFOUND.

Reading the Job Log

The parser says in the SAS log what it did with the output table:

NOTE: The log contains no requests, the table <caslib>.<table> was left as it is. - an empty log, or one that holds no scoring request.
WARNING: Python-Subprocess - No request was read: 'Request: POST' was found in <n> line(s) that do not start as a log line ..., followed by the same note - the lines of the log carry something in front, see the top of this page.
ERROR: The log could not be parsed, the table <caslib>.<table> was left as it is. - the Python code did not finish, for example because the file is missing. Measured for a file that is missing: the line stood below ERROR: Unhandled Python exception. and a traceback that ends in FileNotFoundError: [Errno 2] No such file or directory: '<path>', SYSCC was 1012 and the job ended as failed.
An error that says The saved table <caslib>.<table> could not be loaded, nothing was appended and the saved file was left as it is. (append), The saved table <caslib>.<table> could not be loaded, nothing was replaced and the saved file was left as it is. (replace: the rows of a table that is only its saved file are counted before anything is written, and where the file can not be loaded nothing is written) or The new table could not be created, the table <caslib>.<table> was left as it is. (replace and append). In append mode that line goes on with what the DATA steps returned: The DATA step that appends ended with the return code <n> and wrote <r> rows, the table held <e> and the log <u>, and <x> rows of the table were taken out for <p> rows of the log (-1 stands for a table that was not found). The DATA steps that leave out the requests the table holds already ended with the return code <c>. Measured on 2026-10-13 with version 2.8, whose line did not name the rows that were taken out, and with a table whose name has 33 characters, which no libref addresses: both return codes 1012, -1 rows written, the table with its 1,024 rows as before, SYSCC 1012 and a failed job. In append mode, a table that the run loaded from its saved file for the append is dropped again behind that line, so only its saved file is left, as before the run; a table that was in memory before the run stays.
ERROR: The table <caslib>.<table> holds <n> rows and _log_append_table is not set, so nothing was read and nothing was written - not the table and not its saved file. Set it in front of the program: %let _log_append_table = 1; adds the requests of the log to the table, %let _log_append_table = 0; replaces the table with them. - Log-Parser-Code.sas was run without a value of _log_append_table and the table holds rows, in memory or only as its saved file. It is the one line of such a run, written in front of the log. Set 1 or 0; see Appending. The custom step never writes it: its checkbox sets one of the two.
ERROR: The rows of the table <caslib>.<table> could not be counted and _log_append_table is not set, so nothing was read and nothing was written - not the table and not its saved file. Set it in front of the program: %let _log_append_table = 1; adds the requests of the log to the table, %let _log_append_table = 0; replaces the table with them. - Log-Parser-Code.sas was run without a value of _log_append_table, table.tableExists found the table in memory, and the listing of its rows (table.tableInfo) named no table. It is the one line of such a run, written in front of the log. Set 1 or 0; see Appending. A run with 0 replaces such a table and writes no NOTE of the rows it held. The custom step never writes it.
ERROR: CAS refused or did not run an action that the count of the rows of the table <caslib>.<table> needs and _log_append_table is not set, so nothing was read and nothing was written - not the table and not its saved file. - Log-Parser-Code.sas ran without a value of _log_append_table, and CAS refused table.tableExists of the table or table.fileInfo, the listing of the saved files of a table that is not in memory; the ERROR lines of CAS stand above it. Nothing was read or written, and the job ends as failed. New in version 2.10 of the step; the words or did not run since version 2.11.
ERROR: CAS refused or did not run an action that the run needs for the table <caslib>.<table>, nothing was appended and neither the table nor its saved file was written. (append), or the same line with nothing was replaced (replace) - CAS refused one of the actions the run goes by: table.tableExists of the table (in front of the load of its saved file or behind it), table.fileInfo, table.columnInfo of the table, or the listing of CASUSER that counts what is left of the rows of the log; its ERROR lines stand above. The table and its saved file are as before, and a table the run loaded from its saved file for the append is dropped again. Up to version 2.9 of the step a refused table.fileInfo was taken for a library without the saved file, and the run saved a new table over that file.
A PROC CAS step of the count or of the output macro that halts - its CAS session ended, say - runs nothing after the action it stopped at. The run takes it for an action CAS did not run: it ends as a failed job, with SYSCC 1012, and a halt in front of the step that writes the table leaves the table and its saved file as they were - a table that was only its saved file may then be a table of the session that holds what the file holds. Where the question whether the table is there halts in a run that appends, the line is the second one above. New in version 2.11 of the step; not run in SAS.
ERROR: _log_append_table is <value>, which is no value of this setting: 1 adds the requests of the log to the output table, 0 replaces the table with them, and without a value the program only starts a table that is not there yet. No log was read and nothing was written. - the setting holds something else than 1, 0 or nothing.
NOTE: The table <caslib>.<table> held <n> row(s). They were replaced by the <m> request(s) of the log, as _log_append_table is 0. - a run with 0 replaced a table that held rows, in memory or only as its saved file.
NOTE: <r> request(s) were read from the log. <m> were written to the table <caslib>.<table> and <l> were left out, as another row of the log holds their request. - a run with 0, or a run without the setting that started a table, read a log that holds a request more than once: one row is written for it.
NOTE: <r> request(s) were read from the log. <a> were appended to the table <caslib>.<table> and <p> took the place of <x> row(s) of it whose failed was missing: the table held <o> row(s) and holds <n> now. <s> were left out, as another row holds their request: <t> a row of the table, <l> a row of the log. - a run with 1. r is a + p + s, s is t + l, and n is o + a + p - x; p rows of the log each took the place of the rows of the table of their request, which held neither a response nor an error. Where there was no table the line reads ... <m> were written to the table <caslib>.<table>, which was not there before. <l> were left out, as another row of the log holds their request.
NOTE: <r> request(s) were read from the log and none of them was written: the table <caslib>.<table> keeps its rows, and neither the table nor its saved file was written. All were left out, as another row holds their request: <t> a row of the table, <l> a row of the log. - a run with 1 found every request of the log in the table: no table was built, and the saved file was not written.
ERROR: The table <lib>.<name> is not a CAS table. For this step to run the target table needs to be in CAS, no log was read. - _log_target_table_engine is not CAS, or the output table of the step is no CAS table. It is the one line of such a run: no log is read and no table is touched. The same holds for a _log_path that does not start with sasserver: and its line ERROR: For this step to run please ensure the the selected file is located on the SAS Server. Measured on 2026-10-12 with the program of that day, which ended such a run with an ABORT statement: ERROR: The table casuser.<table> is not a CAS table., ERROR: In order for this step to run your target table needs to be in CAS and ERROR: Execution terminated by an ABORT statement; SAS went on in syntax check mode, and a fourth line said ERROR: The log could not be parsed, the table casuser.<table> was left as it is. about a log that had not been read. No table was written, SYSCC was 20000 and the job ended as failed. Since then the two checks set a macro variable in place of the ABORT, the Python reads no log and the macro of the output table returns before it says or drops anything. Measured later that day in the third run with the engine V9: that one ERROR line and no other line of the program, no table, SYSCC 1012 and a failed job; and with a _log_path of /tmp/x.log its one line, the same way.
NOTE: <n> request(s) were running at the same time as another request - of their container, or of another container whose lines are mixed with theirs in the file. ... <m> request(s) can have missing values, and failed is missing for <k> request(s). A request that was running or waiting where the lines of its container end is counted in the NOTE of those requests, and not in the last number. - see Requests that run at the same time. The last sentence is new in version 2.7 of the step and not run in SAS. Not seen in a collected log so far; seen on 2026-10-12 on files put together by hand from blocks of the collected log.
NOTE: <n> request(s) were running where the lines of their container end - at the end of the file, or at the end of a part of it: where the collection script wrote its line in front of the lines of the next container, or, in a file without such lines, where the time of the lines goes back. The log holds neither a response nor an error for them, and failed is missing for them. - see the same section. The NOTE is new in version 2.4 of the step and was seen on 2026-10-12 in the third run, on files put together by hand, with the words of that version (... at the end of a part of it, where the time of the lines goes back. ...); the words about the line of the collection script are version 2.6, whose code file read a file with such lines on 2026-10-13 and wrote no NOTE (The Pass of 2026-10-13), so they have not been seen in a SAS log.
NOTE: <n> request(s) have no response: a line that reads as a request line stands in their response, in front of the next line of their transaction, and nothing behind it in its part says whether it is a line of the response or a request that came in while the response was written and that the end of the part cut off. It was read as a line of the response. - see Requests that run at the same time, the paragraph about a response that holds lines of a log. New in version 2.6 of the step and not seen in a SAS log: the file its code file read on 2026-10-13 gave no NOTE.
NOTE: <n> line(s) that read as a request line stand in the response of another request, in front of the next line of its transaction, and got no line of their own - no transaction, no error, no status and no response - while a request that came later in their part got its transaction. They were read as lines of that response, and no row was written for them. A request whose container wrote nothing but its first lines there reads the same. - see the same paragraph. New in version 2.7 of the step, not run in SAS.
NOTE: <n> request(s) hold a value that could not be read, for example a 'Request Data:' line that was cut off or a request body that is nested deeper than Python reads. The rows were kept with what was read. Options that are nested deeper than Python decodes are no such value: they are read as names and values, the way options that are no JSON are, the row has them as they were sent, and no NOTE counts it. Measured on 2026-10-12 with options of 100000 opening brackets: the row was in the table with its options of 100000 characters and status 200, and this NOTE was not written. The NOTE named such options as its example until then. As it is here it was written later that day for a Request Data: line that was nested 100000 deep: the row had its status 200 and its response, no prompts and no kind.
NOTE: <n> request(s) hold a character that has no UTF-8 bytes in a text column - half of a surrogate pair in a request body. It was replaced by a question mark, the way the hash of a text counts it. Such a character stopped the upload of the whole file before. Measured on 2026-10-12: a request whose user prompt held half of a surrogate pair was in the table with a question mark in its place, the upload did not fail, and the NOTE counted 1 request.
In append mode the log holds the notes of a DATA step that writes _TEMP_LLM_LOGS_ALL. Seen on the test system on 2026-10-11, with the program as it was that day: the step ran in Cloud Analytic Services, in several threads, and for an existing table whose temperature was text it wrote ERROR: Variable temperature has been defined as both character and numeric.; SAS went on in syntax check mode (OBS=0), the program wrote ERROR: The new table could not be created, the table <caslib>.<table> was left as it is., the table was unchanged and the job ended as failed. Since then the program also asks for the return code of that step and compares the rows it wrote with the rows of the two tables it read, before the new table takes the place of the existing one. On 2026-10-12 that check ran for a table of the same kind: the ERROR of the DATA step, two WARNINGs about _TEMP_LLM_LOGS_ALL, and then ERROR: The new table could not be created, the table <caslib>.<table> was left as it is. The DATA step that appends ended with the return code 1012 and wrote -1 rows, the table held 174 and the log 1040 (-1 stands for a table that was not found). SAS was in syntax check mode behind the DATA step, and the return code and the three row counts were read all the same. The table was unchanged, no staging table was left, SYSCC was 1012 and the job ended as failed.

In the SAS log a single quote of these messages shows as a backtick. Since version 2.4 of the step every line the Python of the parser hands to SAS is made one text first, by the function the pull and the link program carry - What the Program Does to the Table lists what it replaces. On 2026-10-12, with the parser of that day, which handed its lines over as they were, the NOTEs arrived whole with their quotes and parentheses, over two log lines and without an ERROR 180-322. With the function, later that day, the NOTEs of the parser arrived whole as well, with Response: in backticks and without an ERROR 180-322.

The state of the job. A run that wrote an ERROR line and no table sets the condition code of the SAS session (SYSCC) to 1012 as its last statement in front of the macro variables it deletes. These runs are a run that is refused - a _log_path that is not on the SAS server, a table that is no CAS table, a _log_append_table that is no value of the setting, and a run without that setting at a table that holds rows, whose rows can not be counted, whose saved file can not be loaded, or for which CAS refused an action of the count -, a log that could not be parsed, a saved table that could not be loaded - by a run that appends or by a run that replaces -, an action that CAS refused and that a run which appends or replaces goes by, and a new table that could not be created. A WARNING, a NOTE and a log without requests leave the condition code as it was. Such a run behaves like any SAS error. Measured on 2026-10-12 in the third run, in SAS Job Execution jobs - an output table that is no CAS table, a path without sasserver:, a file that is missing, a refused append: the job ended as failed, with the condition code 1012 in the error the service reports, and the first DATA step behind the program in the same SAS session wrote NOTE: Due to ERROR(s) above, SAS set option OBS=0, enabling syntax check mode. This prevents execution of subsequent data modification statements. and did not run, while %put and PROC CAS behind the program ran. With %let syscc = 0; behind the program that DATA step ran and the job ended as completed - so a job that runs a second program behind a refused one has to reset SYSCC, and then no longer ends as failed. In the code SAS Studio generates for a flow, the first DATA step behind a step is the end action of the flow itself; that was measured with the step LLM - Get All Prompts, see Messages and Limits of the Pull. A log of one line feed gave NOTE: The log contains no requests, ..., SYSCC 0 and a completed job. What SAS Studio shows for the node of a refused step was not looked at, and no schedule was run. Measured in the second run, before the statement existed: the refused append ended with SYSCC 1012 and a failed job, as the DATA step had set it, and the run with an engine other than CAS with 20000 after its ABORT.

When you search a job log for ERROR, start at the first line of the program: lines before the first source line are not the parser's. On the test system every job log of the SAS Studio compute context began with two such messages - ERROR: Failed to connect to host ... and ERROR: Unable to connect to Cloud Analytic Services ... - before source line 1, also in a job that held no parser code. They stand on three log lines, all three of the type error, because the second message is longer than the line and goes on with parameters and retry.: a check that counts the lines of the type error finds three, one that counts the lines that start with ERROR finds two. The jobs in which nothing else failed ended as completed, SYSCC was 0 after the parser had run, and from source line 1 on their logs held no ERROR and no WARNING line.

The Columns of LLM_LOGS

One row per scoring request, whatever kind of model answered it. The first twelve columns are the ones the table had before release 2.3.0 - names, order, types and meanings unchanged, the LLM Usage Report reads them - and 24 follow behind them.

Column	What it holds
`timestamp`, `model`	When the request arrived, and the model it was sent to: the name of its container
`system_prompt`, `user_prompt`, `response`	For an LLM the two prompts and the response; for a decision model the questions as the JSON that was sent, the state and the answers as JSON; missing for an embedding model
`temperature`, `top_p`, `top_k`, `max_tokens`	From the options of the request. `max_tokens` is read from `max_tokens`, then `max_completion_tokens`, then `max_length`
`prompt_length`, `output_length`, `runtime`	What the score code logged: input tokens, output tokens, seconds. For an embedding model `prompt_length` holds its tokens
`kind`	`llm`, `embedding` or `decision`, by the names of the inputs of the request (`document`; `questions` / `state`; `systemPrompt` / `userPrompt`). Missing when the `Request Data:` line can not be read
`status_code`	The status of the `Response:` line. Missing when the log holds no such line for the request
`failed`	1 when the log holds an ERROR line for the request, no `Response:` line although its container went on with another request, or a status of 400 and above; otherwise 0. Missing where the log does not decide it: for a request that was still running where the lines of its container end - at the end of the file or of a part of it -, and for a request that ran at the same time as another one where no line of the log says which of them a response belongs to. See Requests that run at the same time
`error_text`	The first ERROR line of the request, cut after 500 characters; for the line the container writes for an exception, with the last line of the traceback added. What has the shape of a key is replaced by `REDACTED`
`transaction_id`	The id the container gave the request (the `transaction_<id>` logger, the `transaction_id` of `Response Data:`)
`request_id`	The `X-Request-Id` header
`user_agent`	The kind of client by the `User-Agent` header: `python-requests` (the score code of another model, a job, a script), `browser`, `SAS` or `other`. No version, no address and no Referer is kept, and no other header is read
`embedding_mode`	The `Embedding_Mode` option of a call to an embedding model, as sent. Missing when the call sent none; the model then used its default, which the log does not say
`project`	The `project` input of a call to an embedding model (the RAG runtime sends `rag`)
`input_hash`	The SHA-256 of the text an embedding model was given. The text itself is not kept
`answer`, `answer_confidence`	The headline of a decision: the answer to the first question and its confidence. `response` holds all answers
`options`	The options as they were sent, with the value of `API_KEY` and of every other option that is a secret by its name (key, secret, password, passwd, pwd, credential, authorization, bearer, cookie, a token that is no number) or by its shape (`sk-...`, `Bearer ...`, a JWT, 20 and more letters and digits in a row that hold both) replaced by `REDACTED`. A name also says so with `pass` or `pw` as one of its words (`my_pass`, `userPass`, `pw` - not `bypass`) and with `passphrase`. Inside a value the user and the password of an address (`scheme://user:password@host`), a header that carries a secret with everything behind its name, and what follows `bearer` or `basic` are replaced as well. The parser does this itself, whether or not the collection redacted the log
`rag_collection`, `rag_query_hash`	For an LLM whose user prompt holds the results of a retrieval: the collection and the SHA-256 of the question. The results are the JSON object the RAG runtime writes, with all six of its keys (`query`, `hits`, `graph_context`, `retrieval_mode`, `collection`, `ingestion_run_id`); of several such objects in one prompt the last one counts. The search tries 500 opening braces of a prompt and decodes 2,000,000 characters at most, so a prompt with more braces than that in front of its retrieval gets no collection
`model_id`, `caller_model_id`, `caller_project_id`, `caller_kind`, `caller_name`, `experiment_run_id`, `rag_model_id`, `match_method`, `match_candidates`	Written empty by the parser. No line of a log names the model in SAS Model Manager or the prompt, decision template or RAG setup that sent a request; `Link-Logs-To-Models.sas` fills them

A hash is the SHA-256, in lower-case hex, of the UTF-8 bytes of the text with every run of white space made one blank. The three programs hash a text the same way. That is how the question of a retrieval is found again: its rag_query_hash is the input_hash of the call that embedded it in query mode. timestamp is a SAS datetime; temperature, top_p, top_k, max_tokens, prompt_length, output_length, runtime, status_code, failed, answer_confidence and match_candidates are numbers; every other column is text (VARCHAR).

A request that failed

A request whose score code raises writes no Response: line: the container logs the traceback and Error occurred in the model scoring function. Such a request ends at that line and is a row with failed = 1, no status_code and its error in error_text. Before this release it was a row without numbers and response that nothing told apart from a request still running.

Appending

The rows of the existing table and the rows of the log are written to one new table, which takes the place of the existing one as in replace mode; a table that was promoted is promoted again, whatever the option of the run says. The 36 columns stand in front in their order. A column the existing table lacks - a table of release 2.2 has the first twelve - is added and is missing in its rows; a column the parser does not know stays behind the others, with its type and length, and is missing in the new rows; what the link wrote into existing rows stays. A text column that the existing table holds with a fixed length (CHAR) is read under another name and its values are handed to the VARCHAR column without their trailing blanks, so nothing of the new rows is cut to the old length. A column that is a number in the table and text in the parser, or the other way round, stops the DATA step: the table is left as it is and the log has The new table could not be created, the table <caslib>.<table> was left as it is. The saved copy is replaced by the wider table when saving is on, and left as it was when it is off. Appending rewrites the whole table on every run, and other sessions do not see it between the drop and the promote. The parser goes back over a row of the table in one case only - a row that holds neither a response nor an error, see below -, and the new columns of rows an earlier release wrote can not be filled by parsing their log files again: those rows have no transaction_id, so with _log_append_table = 1 their requests are appended a second time (the last entry of the list of 2026-10-13 below), and with 0 the table is replaced by the requests of the one file.

A request is written once, in every state of the setting. A request is known by three columns together: its transaction_id, its model and its timestamp, the time of its request line. The container writes the transaction ID into the lines of the request, so the three are the same in every file and every run that holds the request; the X-Request-Id header is set outside the container and is not used for it. The ID alone is not taken for the request: the container gives a request its ID unless the client sends one, and a client can send the same ID with every request. Of a request the log itself holds more than once - the lines of a container that were collected twice into one file - one row is written, with 1, with 0 and without the setting: the first that is complete, and the first of all where none is. A row is incomplete when its failed is missing: the lines it was read from hold neither a response nor an error for the request, as for a request that was still running where a collection ended. A run with 1 leaves out a request of the log that the table holds already, with one exception: a complete row of the log takes the place of the rows of the table of its request when each of them is incomplete - a request that was cut off where an earlier collection ended and that a later one holds whole. A complete row of the table is never replaced, and no row is replaced in a table without a numeric column failed. The row that takes the place is the row of the log: what the link wrote into the row it replaces goes with that row, and the link fills the new one at its next run. A table that lacks one of the three columns is compared with nothing. A request without a transaction_id is never taken for another one: it is appended every time it is parsed, and no rule of the program can decide that case - in a file without the lines of the collection script, such a request and another request with the same lines are the same file line for line, and onto a table that holds the row the first has to add nothing and the second a row. In a file of the collection script the line of the script names the pod and the container, but the table keeps neither of a row: the row of such a request of one container and the row of the same request of another container are the same row, so the same file onto the same table has to add nothing in the one case and a row in the other. A column of LLM_LOGS for the pod and the container would tell them apart; the table has none. And a request read again and another request of the same container, written in the same second with the same lines, are the same file line for line with the line of the script as well: no column of the table can tell those two apart. The run says its numbers in one line: NOTE: <r> request(s) were read from the log. <a> were appended to the table <caslib>.<table> and <p> took the place of <x> row(s) of it whose failed was missing: the table held <o> row(s) and holds <n> now. <s> were left out, as another row holds their request: <t> a row of the table, <l> a row of the log. A run that appends nothing writes nothing: where every request of the log is in the table already and none takes the place of a row, no table is built and the saved file is not written - NOTE: <r> request(s) were read from the log and none of them was written: the table <caslib>.<table> keeps its rows, and neither the table nor its saved file was written. All were left out, as another row holds their request: <t> a row of the table, <l> a row of the log. A table that is only its saved file is still loaded for such a run, and promoted where the run promotes. Version 2.9 ran on 2026-10-14 (the last lists below). The ERROR line for a table whose rows can not be counted and the drop of a table an append loaded for itself were added after that pass; the drop ran in a second pass that day, and the ERROR line and what version 2.10 changed have not run in SAS: they are followed in the tests of the repository against a CAS library that is made up (log_parser_table_harness.py).

The new table only takes the place of the existing one when the DATA steps that wrote it ended with a return code of at most 4, when its rows and the rows that were taken out of it are as many as the existing table held and the log added, and when the two DATA steps that leave out the requests the table holds already ended with 0 or 4. Otherwise the program writes one ERROR line with the return codes and the row counts, drops what the DATA step left and leaves the existing table as it is. The new table gets the label of the existing table, and Extracted LLM Log Data - the label of a table the parser writes fresh - where that one had none. Both were added after the runs of 2026-10-11 and ran on 2026-10-12, see the second list below.

An LLM_LOGS of release 2.2 is kept by appending to it. Run the parser of this release with %let _log_append_table = 1; in front of it, in a new SAS session:

%let _log_append_table = 1;
filename shipped filesrvc folderPath='/SAS Agentic AI Accelerator/Logging and Monitoring' filename='Log-Parser-Code.sas';
%include shipped;

The file of release 2.2 held the setting as a %let line of its own, with the value 0: it replaced the table on every run unless a site had changed that line to 1. It read the setting in %if &_log_append_table. %then, where every whole number other than 0 is true: a site that had set 2, 01 or another such value for append has to set 1 now, as this release refuses every value but 1 and 0. The import replaced the file, and the file of this release has no such line and no default for the setting. Without it the program writes only where there is no table or a table without rows; at a table that holds rows it stops in front of the log with ERROR: The table <caslib>.<table> holds <n> rows and _log_append_table is not set, so nothing was read and nothing was written - not the table and not its saved file. Set it in front of the program: %let _log_append_table = 1; adds the requests of the log to the table, %let _log_append_table = 0; replaces the table with them. The condition code is 1012 and the job ends as failed. A job or a schedule that ran the file of 2.2 therefore fails from its first run on, with nothing written, until its program sets 1 - the rows stay and the new requests are added - or 0, which is what the file of 2.2 did as it was shipped: the table holds the requests of the one log file afterwards. The rows of 2.2 keep the values and the labels of their twelve columns, and the 24 further columns are missing in them.

Make a copy of the table before the first run - two CAS actions, here into CASUSER. The copy is a table of the CAS session, and its saved file LLM_LOGS_BEFORE_2_3.sashdat stays when the session ends:

proc cas;
    table.copyTable / table={caslib='Public', name='LLM_LOGS'} casOut={caslib='casuser', name='LLM_LOGS_BEFORE_2_3'};
    table.save / table={caslib='casuser', name='LLM_LOGS_BEFORE_2_3'} caslib='casuser' name='LLM_LOGS_BEFORE_2_3' replace=true;
quit;

Before the first append to a table of an earlier release, try it on a copy of the table. The table has to be reachable through a libref (a name of at most 32 characters), and the user who runs the parser has to be allowed to drop and promote it, as in replace mode. And if the CAS server was restarted since you started collecting, compare the first timestamp in LLM_LOGS with the day you started: the parser of release 2.2.0 and earlier began a new table after a restart and saved it over the file with the rows collected until then (see Fixed in the changelog of 2.3.0). Those rows can not be parsed again where the collection script replaces the log file every hour.

Measured on the test system on 2026-10-11, with the program as it was that day (version 2.2 of the step), on a table of twelve columns and 174 rows that the parser had written before it had the further columns (version 2.1 of the step):

Into the table of twelve columns. Appending the log of 1040 requests gave 1214 rows and 36 columns in their order, promoted again and saved again. The 174 earlier rows had missing values in kind, failed and the link columns, and the first twelve columns of the 1214 rows were those of the 174 earlier rows and of the 1040 parsed ones. The DATA step ran in Cloud Analytic Services, in several threads, read 174 + 1040 rows and wrote 1214. Afterwards no _TEMP_LLM_LOGS_ALL, no libref _llmtgt or _llmsrc and no work._log_target_columns was left, and the log held no ERROR and no WARNING. 138 requests are in both collections and are in the table twice.
Into a table with CHAR text columns (model, system_prompt and user_prompt as CHAR 64, 300 and 100): no error, 1214 rows, every text column VARCHAR, and the longest values of the three columns were those of a fresh parse - 1315, 22559 and 2481 characters -, so nothing was cut. The 174 earlier rows were unchanged.
Into a table with two columns the parser does not know: 38 columns, the 36 in front and the two behind them, filled in the 174 earlier rows and missing in the 1040 new ones.
Into a table whose temperature is text: the two ERROR lines quoted under Reading the Job Log. The table kept its 174 rows, its twelve columns and its modified time, no file was saved and no staging table was left.
The label of the table was lost. The table the parser had written fresh had the label Extracted LLM Log Data; after the append it had none. That is what the label of the paragraph above was added for.

Measured on the test system on 2026-10-12, with the program as it was that day (version 2.3 of the step), again on tables of twelve columns and 174 rows:

With the check of the rows and the label. Appending the log of 1040 requests gave 1214 rows and 36 columns in their order, promoted again and saved again, without an ERROR or a WARNING, in 4.7 seconds. The DATA step ran in Cloud Analytic Services in several threads, read 174 + 1040 rows and wrote 1214, and the two listings of the tables and the action that sets the label ran without a message. The table had the label Extracted LLM Log Data afterwards, which it had before the append.
A label of the site. On a copy with the label ZZ label of the site the append gave the same 1214 rows and 36 columns, and the table kept that label.
Into a table whose temperature is text: the lines quoted under Reading the Job Log, with the return code 1012 and -1 rows written. The table kept its 174 rows, its twelve columns and its modified time, no file was saved and no _TEMP_LLM_LOGS_ALL was left.

Measured on the test system on 2026-10-13 (SAS Viya 2026.09), with version 2.8 of the step in SAS Job Execution jobs - the program of this release is version 2.10, and the entries say where it does something else -, on copies in CASUSER of a table of release 2.2 - twelve columns, 3,630 rows of four models - and with a collected log of 1,024 requests:

Without the setting. The ERROR line above with 3630 rows, and no other line of the program. The copy and its saved file were as before: the same rows by a checksum over the twelve columns, the file with its size and its time. SYSCC was 1012, and a job that ended at the program was reported as failed, with that line whole as its error. For a copy that was only its saved file the same: the file was loaded into CASUSER for the count and dropped again, and no table was in memory afterwards.
With 1. 4,654 rows and 36 columns, promoted and saved, SYSCC 0. The 3,630 earlier rows had the values of their twelve columns - the checksum they had before - and missing values in the 24 further columns. The twelve columns kept their labels, also system_prompt, user_prompt and response, whose labels are longer in a table this release starts; the 24 got theirs. The log had NOTE: 1024 request(s) were read from the log. 1024 were appended to the table casuser.<table>, which held 3630 row(s) and holds 4654 now. 0 were left out, as a request with their transaction_id is there already: 0 in the table, 0 earlier in the log. The same for a copy that was only its saved file.
The same log once more, with 1. ... 0 were appended ..., which held 4654 row(s) and holds 4654 now. 1024 were left out ...: 1024 in the table, 0 earlier in the log. The table held the same rows and was written and saved again - a new creation time of the table, a new time of its file. Version 2.9 writes neither in such a run.
A file that holds every request twice - the log twice, one behind the other -, with 1: 2,048 requests read, 1,024 appended and 1,024 left out as earlier in the log; onto the table that held them already, 0 appended, 1,024 in the table and 1,024 earlier in the log. With 0 the same file gave a table of 2,048 rows; version 2.9 writes 1,024.
With 0. The copy held the 1,024 requests in 36 columns afterwards, in memory and in its saved file, and the log had NOTE: The table casuser.<table> held 3630 row(s). They were replaced by the 1024 request(s) of the log, as _log_append_table is 0. The same for a copy that was only its saved file.
No table, and a table without rows. Without the setting, with 1 and with 0, a table that was not there was written with the 1,024 rows. Without the setting a copy with twelve columns and no rows - in memory, and as a saved file only - was replaced by them, without the ERROR line.
A value that is none. 2, yes, 01 and TRUE each gave the ERROR line of Reading the Job Log and SYSCC 1012; no table was written where there was none, and the copy kept its 3,630 rows and its file.
Requests without a transaction ID - a file made by hand, three requests with one and three whose lines name none: the first append wrote 6 rows, a second append of the file added the 3 without an ID again and left out the 3 with one, and a file that held the six twice added 6 and left out 6, 3 in the table and 3 earlier in the log.
A request that was cut off - two files made by hand, in the first the lines end while its second request runs, the second holds both whole: after the first file the row of that request had no status, no failed and no token counts; the second file appended nothing, and the row was as before. In version 2.9 the row of the second file takes its place.
The custom step, as its program with the macro variables of its controls set in front of it: not checked, the copy was replaced with the NOTE of 0; checked, another copy had 4,654 rows afterwards; checked with the log the table held already, 0 were appended and 1,024 left out.
A table the parser of release 2.2.0 wrote, and its log once more. The file of release 2.2.0 wrote 1,024 rows of twelve columns from the log. The parser of this release, with 1 and the same log, appended 1,024 rows: the table held every request twice, once without and once with a transaction_id.

A saved file that can not be loaded was measured that day with a copy of the program of 2.8 whose count asked for a file that was not there, on copies that were only their saved file. Without the setting: the ERROR lines of the action, ERROR: The saved table <caslib>.<table> could not be loaded and _log_append_table is not set, so nothing was read and nothing was written - the saved file was left as it is., SYSCC 1012, and the saved file as before. With 0: the ERROR lines of the action and SYSCC 1012 - and the table and its saved file replaced by the 1,024 requests of the log all the same. Version 2.9 ends that run with ERROR: The saved table <caslib>.<table> could not be loaded, nothing was replaced and the saved file was left as it is. and writes nothing.

Measured on the test system on 2026-10-14 with version 2.9 - the code file, and the program of the step with the macro variables of its controls in front of it - in SAS Job Execution jobs, on copies in CASUSER of the same table of release 2.2 (3,630 rows) and with a collected log of 1,024 requests. The ERROR line for a table whose rows can not be counted and the drop of a table an append loaded for itself were added after that pass: the drop ran in a second pass that day (the list after this one), the ERROR line has not run in SAS.

The three states. At no table, without the setting, with 1 and with 0: 1,024 rows. At the copy: without the setting the ERROR line with 3630 rows, SYSCC 1012 and the copy as before; with 1 ... 1024 were appended ..., ... held 3630 ..., ... holds 4654 now ..., and the earlier rows with the checksum they had; with 0 the NOTE that the 3630 rows were replaced by the 1024. The same three answers for a copy that was only its saved file, and 1,024 rows in each state for a table without rows. 2, yes, 01 and TRUE: the ERROR line, SYSCC 1012 and the copy as before.
The same log once more, with 1: NOTE: 1024 request(s) were read from the log and none of them was written: ...; the table kept its creation and its modified time, and its file its size and its time. With the table dropped from memory and only its file left: the same NOTE, the table loaded and global, and the time of the file as before.
A file that holds every request twice: at no table, without the setting and with 1, 2,048 requests read, 1,024 written and 1,024 left out; with 0 at the copy 1,024 rows; with 1 at the copy 1,024 appended and 1,024 left out.
A request that was cut off - the two files made by hand of 2026-10-13, with 1: the second gave ... 0 were appended ... and 1 took the place of 1 row(s) of it whose failed was missing: the table held 2 row(s) and holds 2 now. ..., and both rows had the status 200 and failed 0.
Requests without a transaction ID, parsed twice: 6 rows, then 3 appended again and 9 rows. Three requests with one transaction ID: 3 rows, and a second run with 1 wrote none of them, with the times of the table and of its file as before.
The step, as its program with the macro variables of its controls set: with 1 the file that was cut off and the whole one gave the same two NOTEs, and the log onto a copy appended 1,024 and the second time nothing; with 0 each run replaced the table.
A saved file that can not be loaded could be staged only with two changed copies of the program: CAS refused to save a CSV under a .sashdat name (ERROR: Save exportoptions file type and name extension do not match.), and that caslib took no encrypted file. Without the setting, with 0 and with 1 each run wrote its ERROR line - ... could not be loaded and _log_append_table is not set ..., ... nothing was replaced and the saved file was left as it is., ... nothing was appended ... -, SYSCC 1012, and the file kept its 3,630 rows, its size and its time.
The DATA steps. In the appends that were looked at, each DATA step on a CAS table ran in Cloud Analytic Services, no WARNING was written, SYSCC was 0, and no _TEMP_LLM_LOGS table or file was left.

In a second pass on 2026-10-14, with the code file after those two changes, in SAS Job Execution jobs with test table names in CASUSER:

An append that can not build its table, at a table of 2 rows that was only its saved file, whose saved copy held temperature as text: ERROR: Variable temperature has been defined as both character and numeric., and then ERROR: The new table could not be created, ... with the return code 1012, -1 rows written, 2 rows of the table and 1,024 of the log, and 0 rows taken out. SYSCC was 1012, table.tableExists answered 0 afterwards - the table the run had loaded was dropped again -, and the file had the size and the time it had before.
The same with a valid table that was only its saved file: loaded from the file, 1,024 rows appended - 2, then 1,026 -, promoted and saved.
The three states, on copies of the table of release 2.2 (3,630 rows): without the setting at a promoted copy and at a copy that was only its saved file, the ERROR line with 3630 rows, SYSCC 1012, and the copy and its file as before; without the setting and no table, 1,024 rows, promoted and saved; with 1 4,654 rows; with 0 the NOTE that the 3630 rows were replaced by the 1024.

Not measured: the could not be loaded branches with the program as it ships, as no pass could stage a saved file that can not be loaded; a table without a label, which gets Extracted LLM Log Data; and on 2026-10-12 the appends into a table with CHAR text columns and into one with further columns, which ran on 2026-10-11.

Requests that run at the same time

Only the lines of a transaction (the transaction_<id> logger) and the Response Data: line name the request they belong to. Every other line is given to a request where the lines around it leave one request it can be the line of, and to none otherwise:

The first lines of a request - the headers, the request data, the two Randomly generated lines - are the lines of the request whose request line stands in front of them, as long as no other request line stands in between. A first line that no request is waiting for shows that requests wrote their first lines at the same time: every request since the one whose first lines broke off keeps its request line only.
The first line of a transaction belongs to the one request of its model that has all its first lines and no transaction yet. Where two such requests are waiting, the transaction is given to neither: both rows keep their prompts, options, request_id, kind and user_agent and have no transaction_id, status_code, token counts, run time, response or error_text.
The lines of the score code (prompt_length:, output_length:, run_time:, response:, answer:, answers:, its ERROR line) and the Response: line are the lines of the one transaction that is running. A request is taken as still running when a later line of the log names its transaction. While a second one is running, such a line is given to neither.
failed is missing for a request whose response could not be given to it, and for a request that was running where the lines of its container end. It is not 1 for either reason, within the limits at the end of this section.
A request that waits. A request whose first lines are complete and that never gets a transaction stays waiting until its container's part of the file ends. The requests of that model that follow it in the part are then left open, and the NOTE counts them.
First lines that broke off where the lines end. Where the first lines of a request broke off at the request line of another one and no request since has a status or an error when the lines of the container end, what stands behind the break - first lines, a transaction, lines of the score code - can be of the request that broke off as well as of a request since: the requests since keep their request line only, and all are counted in the NOTE. A transaction that starts while no request of its model has all its first lines, behind such a break, shows that the request went on, as a first line does that no request is waiting for. Where a request since has ended, nothing is taken back and the request that broke off is read as failed; a file in which it went on while each request since waited for the next one is the same line for line.
One line of each kind. The score code writes each of its lines once for a request. A second line of a kind for the one request that is taken as running was written by another request: the request keeps none of the values of the score code and is counted. A Response: line without its Response Data: line behind it is the line of the one running transaction only behind the line of that transaction that follows the score code, or with a status of 400 and above.

The SAS log then has NOTE: <n> request(s) were running at the same time as another request - of their container, or of another container whose lines are mixed with theirs in the file. ..., with the number of requests that can have missing values and of those whose failed is missing.

Where a part of the file ends. The collected file holds the lines of one container after the other, appended pod after pod and collection after collection, and a part can begin and end in the middle of a request. In a file the collection script of this release wrote, a part ends at the line the script writes in front of the lines of each container and nowhere else: a request that is running or waiting there ends without a response and without an error, nothing that was read goes on in the next part, and a part that begins with a later line of a transaction follows that transaction to its end and gives it to no request. The time of the lines decides nothing in such a file: a line that is older is a line of its container that was written late, and a transaction that has not ended runs until it ends or its part does, so a line of the score code is given to no request while another transaction runs beside it. The time of collection of such a line decides nothing: every line of that shape ends a part, and in a file the script wrote every line of that shape is a line of the script - it puts a backslash in front of a line of a container that starts like its own, which the parser takes off again behind the first line of the script. What stands in front of that first line - lines an earlier release collected, in a file that is appended to - is read as a file without such lines, and ends at it. In a file without such lines - collected before this release, or by a process of your own - the time of the lines goes back where one part ends and the next begins, and that is the one sign of it: a line that is more than one second older than a request that is running or waiting starts the next part. Either way a request that is running where its part or the file ends has neither a response nor an error in the log: the leftover lines of the next part are not added to it, its failed is missing, and a NOTE of its own counts such requests - NOTE: <n> request(s) were running where the lines of their container end - at the end of the file, or at the end of a part of it: where the collection script wrote its line in front of the lines of the next container, or, in a file without such lines, where the time of the lines goes back. The log holds neither a response nor an error for them, and failed is missing for them. Every request whose failed is missing is counted once: in this NOTE where the lines of its container end on it, also where another request ran beside it, and in the NOTE of the requests that ran at the same time where a response could not be given to it. A request whose first lines broke off at the request line of such a request is counted with it. So is, in a file that holds no line of a transaction, a request that had all its first lines and ended at that request line. A request without a Response: line whose container went on with another request is failed = 1. In a file without the lines of the script a part boundary at which the time does not go back by more than one second is not seen; the end of this section has the further limits of such a file. The line of the script is read since version 2.6 of the step, whose code file read a file with nine such lines on 2026-10-13 (The Pass of 2026-10-13). Of what version 2.7 changed here, the backslash and the lines in front of the first line of the script ran in SAS in The Second Pass of 2026-10-13, on staged lines and on a copy of the collected file; the time of collection that decides nothing and which NOTE counts a request are not run in SAS.

A line that is one second older. Inside a part a line can be one second older than the line in front of it - it is stamped when it is made and written a moment later - and ends no request. It is the line of its transaction when it names one of this part; a line of the score code or a Response: line that is older by a second is given to no request, and the requests that are running then are counted in the NOTE of the requests that ran at the same time. A transaction is over at a request line only when every later line that names it is more than a second older than that request line. That a container writes a line at most one second late is read from the line format and not from a case: a line that is stamped two or more seconds before a request line in front of it is taken for the start of the next part, and the rows around it lose values and are counted in a NOTE. Behind the first line of the collection script none of this depends on how much older a line is: a line that is older is a late line of its container, whatever its time.

The lines of each container have to stay together in the file, as kubectl logs writes them and the collection script appends them. In a file that mixes the lines of several containers - a time-ordered export of a log store - the score lines of a request that ran while a request of another container was running are given to neither, so its token counts, its run time and its response are missing, and the NOTE counts it. A request line for another model does not end a request that is running.

A response that holds lines of a log. While a request is the only one that is running, the lines of its response up to the next line of its transaction are kept as text, also where they start like log lines; a line there that names another transaction of the part - the Response Data: line of a request whose Response: line came earlier - ends the response. A request line there, with a Request Headers: line directly behind it or with no further line that starts like a log line behind it up to the next line of the transaction, is read as a request first and decided at the end of the file. It is a request where it gets a transaction, an error, a status or a response of its own, or where a transaction of its model starts behind it that is given to no request. A request line that gets none of them while a request that came later in its part got its transaction - which a container gives a request at once - is read as a line of the response: the file is read once more with it as text, no row is written for it, the response is whole, and the request keeps its status whatever time the quoted lines carry. A request whose container wrote nothing but its first lines there - no transaction, no error, no status - is the same file line for line, so a NOTE counts the request lines that were read that way: NOTE: <n> line(s) that read as a request line stand in the response of another request, in front of the next line of its transaction, and got no line of their own - no transaction, no error, no status and no response - while a request that came later in their part got its transaction. They were read as lines of that response, and no row was written for them. A request whose container wrote nothing but its first lines there reads the same. Where nothing came later in its part, the line can also be a request that came in while the response was written and that the end of the file, or of its part, cut off before its transaction began - two such files are the same line for line, and a test of the repository holds two that are. It is read as a line of the response, no row is written for it, the response is missing, and a NOTE counts such requests: NOTE: <n> request(s) have no response: a line that reads as a request line stands in their response, in front of the next line of their transaction, and nothing behind it in its part says whether it is a line of the response or a request that came in while the response was written and that the end of the part cut off. It was read as a line of the response. A request line that scores nothing (GET) gets no transaction either way and is read the same way. With one more line that starts like a log line behind the quoted request line, and that is no Request Headers: line, the request line is text of the response. A prompt is logged on one line and is always kept as text. This is version 2.6 of the step, and the NOTE that counts the lines read that way version 2.7; neither has run in SAS on a file that is known to hold such a line. Up to version 2.5 one more row was written for a quoted request line, the response ended in front of it, and the runs below were made with that reading.

In the six collected logs of the test system (1247 requests) no request started before the one in front of it had ended, so no collected log shows these rules at work. On 2026-10-12 the parser of that day - version 2.3 of the step - ran on the test system on files put together by hand from blocks of the collected log:

Two requests of one model - the five first lines of the first, the five of the second, the rest of the first, the rest of the second, with the times in file order: 2 rows with prompts and options, without a transaction_id, status_code and failed missing, and NOTE: 2 request(s) were running at the same time as another request ... 2 request(s) can have missing values, and failed is missing for 2 request(s).
Two requests of two models, split behind their third transaction line: both rows with status 200, failed = 0 and their own transaction ids, the first without numbers and response, and the NOTE ending failed is missing for 0 request(s).
Split inside the header lines: 2 rows with the request line only - model and timestamp -, failed missing, and the NOTE ending failed is missing for 2 request(s).
A response that holds lines of a log. A response with a prompt_length: line and a request line in its middle and a further line that starts like a log line behind them: one row, status 200, failed = 0, and a response of 254 characters that holds both lines. A response of four lines whose third is a request line with one line of text behind it: 2 rows - the request with status 200 and its response cut in front of the request line, 106 characters, and a row with the model of the quoted line and everything else missing - and the NOTE 2 request(s) ... 0 request(s) can have missing values, and failed is missing for 1 request(s). This is the second of the two cases above, which this guide did not name until then.
Files whose times were not in file order - the first files made for this run, in which a later block was older than the one in front of it, which is not what a container writes. Two requests of one model gave 2 rows with the request line only, failed = 1 for both and no NOTE. Two requests of two models gave one row with failed = 1 and without numbers, and one with status 200 and the token count, the run time and the response of the other request - 1029 characters that held log lines of the other transaction -, and no NOTE.

The rules for the end of a part and for a line that is one second older were written after that run, for the last point and for a line that is stamped a second before the request line in front of it. They are version 2.4 of the step, which ran on the test system later that day (the third run) on these files and on three new ones put together from two real blocks. Of the two files whose times were not in file order, the one with one model gave 2 rows without a transaction and with failed missing and the NOTE of the requests that were running where the lines of their container end; the one with two models gave the first request its transaction id, no status and failed missing, and the second status 200, failed = 0, its own numbers and a response of 87 characters, with two NOTEs. Of the new files, one gave both rows status 200 with their own numbers and no NOTE; one gave a complete row and a row with its prompt length and its response but without a status, with NOTE: 2 request(s) were running at the same time ... 2 request(s) can have missing values, and failed is missing for 1 request(s).; and one a row with failed missing and NOTE: 1 request(s) were running where the lines of their container end .... Outside of SAS, the parser as it is now gives 2 rows with failed missing and the NOTE of the requests that were running where the lines of their container end for the first of those two files, and for the second each request its own values, the one without a status with failed missing. Outside of SAS as well, one line made one, two, five and thirty seconds older at each of the 31 positions of two requests that overlap gave no answered request with failed = 1 and no value of the other request, and 24,000 made-up logs whose requests were interleaved at random - with one model, and with a model per request - gave no row with a value of another request, no answered request with failed = 1 and no missing value without a NOTE. The parser as it ran on 2026-10-11 had 430 such faults on 2,000 of the logs with one model.

Limits a reading of the parser found after the third run, with logs that were made up for them. None is a line of a collected log. Versions 2.6 and 2.7 of the step were changed for them; the numbers that follow each are of the Python of the parser outside of SAS, on the generators of that reading, in files with and without the lines of the collection script:

A line that is two or more seconds older, between the first lines of two requests of one model. With the five first lines of a request A, the five of a request B, the rest of A and the rest of B, and one of A's first lines - or B's request line - 2, 5 or 30 seconds older than the request line in front of it, row B holds the transaction id, the token counts, the run time, the response and the status 200 of A, and the only NOTE counts row A. Of 62 such logs for each of the three, 5 gave that row where all lines carry one second, and 1, 2 and 5 where two seconds lie between the lines; with a line one second older none did. The sentence above about the 31 positions holds for two requests that are split behind their third transaction line, as the test of the repository splits them. The same row is written, without a NOTE and without a time that goes back, where a file begins in the middle of the first lines of a request while a second request of that model starts. With the lines of the collection script a line that is older is a late line of its part and starts no part, whatever its time; the tests of the repository read such files with a line 2, 5 and 30 seconds older inside a part. Without them this limit stands as written.
A part that ends on a running request, in front of a part that starts later than that request did. The end of a part is seen where a line is more than a second older than the request line of a request that is running or waiting, and not where the time goes back against the line in front of it. A request that started at :00 and wrote its last line at :22, followed by the part of another container that starts at :10, is failed = 1 without a NOTE. And where two requests were running when a part ends, the earlier one is ended at the request line of a later request of its model and is failed = 1, although the lines of its container end on it. So failed can be 1 for a request that was cut off. Both cases need a second container that started seconds earlier, or requests that ran at the same time, which no collected log has shown. With the lines of the collection script the part ends at the line, the request is cut off with failed missing and counted in the NOTE. Measured with the Python of version 2.7 outside of SAS, five seeds a generator: the second generator of that reading (24,000 files) gave no cut-off request with failed = 1 and no row with a value of another request, and the numbers of the two NOTEs are the rows they count in every file that has a row for each request line; 29 files have one row fewer, the request line in a response that nothing behind it decides. Its first generator (the same seeds, 18,000 files) gave no row with a value of another request and no answered request with failed = 1; 13 files have one row fewer. A third generator, of a second reading (2,000 files), gave every request its row, no value of another request, no cut-off request with failed = 1 unless its part holds an ERROR line of it, and NOTE numbers that are the rows in every file. Version 2.6 gave 13 of the 42,000 files of the first two a row with a value of another request and one a cut-off request with failed = 1. Without them this limit stands, for failed in four ways. Every failed that is not what the part of its request says was put to its kind file by file, in the generated files of all three generators read without the lines - the 12,000 files of the four modes of the second generator that end a part while a request runs and the 18,000 of the first, five seeds each, and 2,000 of the third: 3,400 of the 32,000 files have such a value, 27 more have a row fewer than request lines and were left out, and no value was left over. No code decides them: for each kind a test of the repository holds two files that are the same byte for byte and need different rows (test_log_parser.py), and test_log_parser_generated.py sorts the values of its own files in every run and fails on a value of no kind. (1) failed = 1 for a request the lines of its container end on (3,954 rows): a request without a Response: line and without an error that is followed by another request is the same bytes whether its container was collected while it ran and the lines of a later collection or of the next container follow (failed missing), or the request ended without a line and its container went on (failed = 1). The parser reads the second wherever the time of the lines does not go back by more than a second against a request that is running or waiting, and writes no NOTE for it. Of the 2,060 such rows of the second generator, 1,453 have a request line of their own part behind them and 607 the lines of the next part; in 425 of those the next part begins more than a second before the last line in front of it and later than the request, which is no end of a part for the parser, because a file whose containers are mixed holds such lines with the container going on behind them. (2) failed = 1 for a request that was answered (47 rows), and (3) failed = 0 for a request the lines end on (none in these files, one row in 1,200 files of a sixth seed): a Response: line names no request - the Response Data: line behind it does -, and the lines end between the two. The line is read as the line of the one transaction that is running where that one has the line that follows its score code, and is given to none otherwise. (4) failed = 1 with the text of an error for a request that wrote none (235 rows, all of the third generator, the only one whose requests fail): the ERROR line of the score code names no request, like its other lines. Of the 12,000 files of the four modes of the second generator that end a part while a request runs (with and without lines a second late, a next part that starts seconds before the last line of the part, and two collections of two containers appended; the same five seeds), 319 give a row a value of another request - 62 of the 2,400 of one seed, for which version 2.6 gave 90. They are of three classes, and for each a test of the repository holds two files that are the same byte for byte and need different rows: lines of the score code behind a request whose transaction no later line names - the lines of the next container, or of that request, which was still running (313 files); such a line inside the response of the request that is running - or a response that quotes it (9 files, 4 of them also in the first class); and first lines that broke off in front of requests that ended - or a request that went on while each request since waited for the next one (1 file).
What the parser makes of a part could depend on the parts in front of it. Of 600 made-up files of three parts, without a cut and without a late line, 10 gave a request the values of another request of its model. It needs three or four requests whose first lines interleave. The cause: the time of the newest request that could not be told apart from another one was kept after the last such request had been counted, so in a part that began while nothing ran a line read as older by more than a second, a part began that was none, and the request that was waiting took the next transaction. Since version 2.6 that time counts only while such a request is counted, and the 600 files - and 600 more with lines a second late - give no such row, with and without the lines of the collection script.

The parser of this release on SAS Viya

On 2026-10-11 Log-Parser-Code.sas as it was that day - the program with the 24 further columns, version 2.2 of the step - ran on the test system as a SAS Job Execution job, on a redacted collection of all model containers with 1040 requests to seven models, and wrote to a table of its own in CASUSER:

No ERROR, no WARNING and no NOTE about requests that ran at the same time. 1040 rows and 36 columns in their order, 24 of them VARCHAR and 12 numbers, promoted and saved. The program took 4.8 seconds, the job 21.
437 embedding, 355 decision and 248 LLM rows.
787 rows with status 200 and 253 failed rows - 170 of a decision model, 83 of an LLM. They are exactly the requests without a Response: line: 251 with HTTP 401 of the score code and 2 with a KeyError.
290 embedding rows in query mode and 147 in document mode; 185 decisions with an answer.
238 LLM rows with a collection - 193 of one and 45 of another -, each with a question hash that is the input_hash of an embedding row in query mode.
867 requests of python-requests and 173 of a browser; 1040 distinct transaction ids and request ids.
options in every row, 426 of them with API_KEY:REDACTED; max_tokens in 235 rows.
The nine link columns empty in every row.
The first twelve columns against the parser as it was before the further columns (version 2.1 of the step), which was run on the same file into a second table: 1035 rows were equal, and 5 differed in max_tokens only - missing before, and now 4000 (three times), 64 and 120, read from max_completion_tokens and max_length.
The step in a flow. The code SAS Studio generated for a flow of the step (version 2.2) held the program of the step character for character and one %let line for each of the 36 column controls. Run as a job it wrote the same 1040 rows with the same 36 columns, types and labels, without an ERROR or a WARNING, and its log held _FLW_ACTION_TABLE_|CASUSER|<table>|1. The step had been put on the server with PUT /dataFlows/steps/{id} and had kept its id.
The three appends and the refused one are under Appending.

The parser was changed after these runs: how a line is given to a request (Requests that run at the same time), which retrieval of a prompt counts, further secrets in options, a request whose content can not be read, and what the append checks before it swaps. That parser - version 2.3 of the step - ran on the test system on 2026-10-12, on the same collection:

No ERROR, no WARNING and no NOTE of the parser. 1040 rows and 36 columns in their order, 24 of them VARCHAR and 12 numbers, promoted and saved, with the label Extracted LLM Log Data. The program took 5.5 seconds, the job 21.
The same counts as on 2026-10-11: 437 embedding, 355 decision and 248 LLM rows; 787 rows with status 200 and 253 failed rows; 238 rows with a question hash, 44 of them distinct, each the input_hash of an embedding row in query mode.
Against the parser of 2026-10-11, which was run on the same file into a second table and joined by the transaction id: 1040 rows and 1040 distinct transaction ids each. Of the 26 columns that were compared - 11 numbers, and 15 texts by their digest - 25 were equal in every row. rag_query_hash differed in 41 rows and was filled in both: those prompts hold two or three retrievals with different questions, and the hash is now the one of the last.
The step in a flow (version 2.3, as the code SAS Studio generated for a flow): the same 1040 rows in the same order and the same 36 columns with their names, order, types and labels, the 27 columns the parser fills equal in 1040 of 1040 rows, no ERROR or WARNING, and _FLW_ACTION_TABLE_|CASUSER|<table>|1 in its log. The step had been put on the server with PUT /dataFlows/steps/{id} and had kept its id.
In Public. In one chain of that day the parser wrote its table to Public, under a name of its own, with the same 1040 rows and 36 columns; the table and its file were dropped again.
The files made by hand are under Requests that run at the same time, the appends under Appending, and the lines of a run that is refused under Reading the Job Log.

The parser was changed again after that run - version 2.4 of the step: where a part of the file ends and what a line that is one second older does, the one line of a run that is refused, the condition code of a run without a table, the function every message goes through, and the wording of one NOTE. That parser ran on the test system later that day, in the third run: 1040 rows and 36 columns for the same log, without a message, and joined by transaction_id no row differed from a parse with the parser of the second run in any of the 26 compared columns. Version 2.5 of the step differs from 2.4 in the text of its About tab and in one comment, and the code file in how it takes its settings (Settings that stay yours); neither is run in SAS. Version 2.6 reads the line the collection script of this release writes in front of every container, decides a request line in a response at the end of the file, and keeps the time of the newest request that could not be told apart only while such a request is counted (Requests that run at the same time); its code file ran on 2026-10-13 (The Pass of 2026-10-13). Outside of SAS, its Python gives for the collected log of 1040 requests the same 1040 rows, the same values in all 36 columns and the same messages as the Python of version 2.5. Version 2.7 ends a part at every line of the collection script whatever its time of collection, takes off the backslash the script puts in front of a line of a container that starts like its own, reads what stands in front of the first line of the script by the time of its lines, counts every request whose failed is missing in one of the two NOTEs, counts the request lines it read as lines of a response, and gives fewer rows a value of another request. Its code file ran in SAS in The Second Pass of 2026-10-13, on a collected file that holds no case for the two NOTEs and for the rules about a value of another request; those are not run in SAS. Outside of SAS, its Python gives for three collected logs - the 1040 requests, and the file of 2026-10-13 with and without the lines of the script, 1024 requests each - the same rows, the same values in all 36 columns and the same messages as the Python of version 2.6. Version 2.8 has the three states of _log_append_table and appends a request once; it ran on 2026-10-13 in SAS Job Execution jobs. Version 2.9 writes a request once in every state of the setting, lets a row that was cut off give way to the whole one and writes nothing where it has nothing to add; it ran on 2026-10-14 (Appending, The Pass of 2026-10-14). Version 2.10 takes an action CAS refused for no answer and then writes nothing (Reading the Job Log); its code file ran once, late on 2026-10-14, in an append with every action of CAS answered (The Pass of 2026-10-14), and a refusal has not run in SAS. Version 2.11 takes a PROC CAS step that halts for an action CAS did not run. Its code file ran early on 2026-10-15, in the ninth pass, in the three states of _log_append_table, and the step ran with Append new rows to existing table ticked and unticked; every action of CAS answered (The Pass of 2026-10-14), and a halt has not run in SAS. Version 2.12 and the code file use a connected session of their own name where the session SESSREF names is not connected; that came after the pass and has not run in SAS.

What Was Measured

The collection script of this guide, Log-Parser-Code.sas and the LLM - Log Parser step were run on the test system (SAS Viya 2026.09) on 2026-10-11, the parser as SAS Job Execution jobs that wrote to a table in CASUSER. All of it was run with the parser as it was before the 24 further columns - version 2.1 of the step, a table of twelve columns, and an append that added rows to the existing table with table.append:

Collection and redaction. The script, with only llm_log_path and llm_logging_interval changed, read the pods its selector picked - all of them model pods - and wrote 5,773 lines with 174 requests. An unredacted copy of the same logs held one provider key and seven Cookie headers; the written file held API_KEY:REDACTED once and Cookie: REDACTED seven times, neither the key nor one of the cookie values, and the same number of lines.
Every kind of model that had requests. The 174 requests gave 174 rows with the twelve columns: 57 of three LLMs with prompts, response, token counts and run time; 50 of a decision model with the questions and the answers as JSON, the state, token counts and run time; 67 of an embedding model with its tokens in prompt_length and its run time, and no prompt, response or output_length. No cell held API_KEY or a Cookie header.
An empty log and a log without requests. A file of 0 bytes and a file with start-up lines only each ended with the note, in replace and in append mode. The table kept its 174 rows and its saved file was not written.
Append. Two newer collections with 21 and 7 requests took the table from 174 to 195 to 202 rows, and no earlier row was lost. Before the second one the table was dropped from memory while its .sashdat stayed: the parser loaded the saved table first, appended, promoted and saved it again. The three files held different requests; with the program of that day a log that was parsed twice was in the table twice.
The warning for unread requests. A log collected with kubectl logs --timestamps (858 lines, 26 of them with Request: POST) gave the warning with that count and the note, and the table was left as it is.
A request that is still open. A request that was still running when its container's part of the file ended gave one row with its prompts and without token counts, run time and response. The lines of another container's finished request followed it in the file, and their numbers were not added to it.
Failed requests. Two requests to a decision model that failed in the container - a traceback in the log, no score lines, no Response: Status Code line - gave one row each with the questions and the state, without numbers and response.
The custom step. Run as the code SAS Studio generated for a flow, in a session with a CAS session and the libref, the step wrote the same 174 rows as the program in replace mode and the same 195 after appending the second collection.
The custom step in a session without a CAS session. With no cas and no libname statement in front of the generated code, the macro variable _CASNAME_ did not exist; in a job of the same context that held no other code, cas _all_ list; listed no session and the option SESSREF named CASAUTO. The step ran cas llmLogMySess; - NOTE: The session LLMLOGMYSESS connected successfully to Cloud Analytic Services ..., NOTE: The SAS option SESSREF was updated with the value LLMLOGMYSESS. -, wrote the table and ended the session: NOTE: Deletion of the session LLMLOGMYSESS was successful., the NOTE about the option SESSREF= quoted above, and NOTE: Request to TERMINATE completed for session LLMLOGMYSESS. Promoted and saved, the table was read from a new CAS session, global, with the 174 rows. Saved and not promoted, no table was in memory and the .sashdat held the 174 rows. Neither promoted nor saved, nothing was left. SYSCC was 0 and the jobs ended as completed.
The custom step without the libref. Behind cas mysess; alone the step ran in that session, started none of its own and wrote the promoted and saved table to CASUSER, with the same 174 rows. The log of that job held the line _FLW_ACTION_TABLE_|CASUSER|<table>|0; with the libref it was |1.
The step twice in one SAS session. The code SAS Studio generated for two flows of the step, one behind the other in one SAS Job Execution job whose session had no CAS session: the first run started LLMLOGMYSESS and ended it. Between the runs _CASNAME_ existed, the option SESSREF named LLMLOGMYSESS and cas _all_ list; listed nothing. The second run started no session: ERROR: There is no server connection to execute the action 'session.sessionId'., ERROR: Unhandled Python exception., ERROR: The log could not be parsed, the table CASUSER.<table> was left as it is., SYSCC 1012, no table, and a job that ended as failed. This is what the session check was changed for: the step now asks SESSFOUND, and the same job was run again with that check (below).
The step put on the server in place. PUT /dataFlows/steps/{id} with the step file plus its id was answered 200 and the id was kept; the stored program and the texts of the About tab equalled the file.
The copy from SAS Content, as it is printed in this guide - see Without a Shared Mount: the copy with the size and the digest of the upload, and the 174 rows from Log-Parser-Code.sas behind it.
A log file that is missing, in replace mode: the ERROR described under Reading the Job Log, no table, no <table>_NEW and no saved file. No table of that name existed before the run, so that an existing table is left as it is was not seen there.

The runs from The custom step in a session without a CAS session on were made later on 2026-10-11, with the step and with Log-Parser-Code.sas as they were at that time of the day: their session check went by the macro variable _CASNAME_. Every run of Log-Parser-Code.sas in the list above was made before that line was changed as well; the program holds the same check behind its own cas mysess;.

The Session Check with SESSFOUND

After the check was changed, the step was put on the server again - POST /dataFlows/steps, then PUT /dataFlows/steps/{id}, answered 200 with the id kept, the stored program and ui equal to the file - and run the same day in seven more SAS Job Execution jobs of the SAS Studio compute context, as the code SAS Studio generated for flows of the step, with the 174 requests of the first collection. It was the step as it was at that point of the release: version 2.1, with the twelve columns.

SESSFOUND on that system. No log of the seven jobs held an ERROR or a WARNING at a line that calls the function. It returned 0 in a session without a CAS session, where the option SESSREF named CASAUTO; 0 for a session that had been ended (LLMLOGMYSESS, MYSESS); and 1 for a connected MYSESS.
Twice and three times in one SAS session without a CAS session. In a job with two flows of the step one behind the other, both runs started and ended LLMLOGMYSESS, with two session ids. Between the runs the option SESSREF named LLMLOGMYSESS, SESSFOUND returned 0, _CASNAME_ existed and cas _all_ list; wrote no line. The second run wrote, promoted and saved its table, and none of the lines of the failed second run described above occurred. In a job with three runs the third did the same and replaced the global table of the second.
A CAS session of the user, used and left alone. Behind cas mysess;, the libref and a session table of the user, the step ran in MYSESS: outside the echoed source no line of the log named LLMLOGMYSESS, and no Deletion of the session and no Request to TERMINATE line was written before the job read the table back. After the step the session had the same id, SESSFOUND returned 1 for it, cas mysess list; named MYSESS as ACTIVE and the session table of the user was still there. The same held after the step had run twice behind that session.
A session that was ended before the step. Behind cas mysess; cas mysess terminate; the option SESSREF named MYSESS and SESSFOUND returned 0: the step started LLMLOGMYSESS, wrote and saved its table and ended LLMLOGMYSESS.
A CAS session without the libref. The step used MYSESS, wrote the promoted and saved table and left the session active.
Log-Parser-Code.sas with the same check, three of its %let lines changed: the program ran in the mysess it starts itself, no line outside the source named LLMLOGMYSESS, and it wrote the 174 rows.
All seven jobs ended as completed with SYSCC 0 and without a line of the type error or warning from source line 1 on. Each of the seven tables, read from a new CAS session, was global and had the 174 rows and the twelve columns, the rows equal to those of the earlier runs.
_FLW_ACTION_TABLE_. The line ended in |0 in every run in which the step ran in a session of its own or without the libref - four of the jobs - although the table existed, and in |1 in the two jobs with the libref.
An observation, not caused by the check. After the step, or Log-Parser-Code.sas, had run in a CAS session of the user, the action session.sessionId returned that session under the name py-session-1 where it had returned MYSESS before, with the same id. SAS went on listing the session as MYSESS, ACTIVE, and SESSFOUND returned 1 for mysess. The Python part of the program is the one of the runs before the check changed; whether the step of those runs did the same was not looked at.

The step and the program of this release hold that check line unchanged. What this release changes beside it is not part of these runs: for version 2.2 of the step, with the 24 further columns, see The parser of this release on SAS Viya.

Not Measured

The redaction of the Authorization and X-CSRF-TOKEN headers: none occurred in the logs, so only the API_KEY and the Cookie expression were exercised.
The shared mount of the example paths and the hourly crontab entry.
The models that had no request in the collected day. The rows come from three LLMs - one of them, with a single row, is called with a provider key -, one embedding model and one decision model. No embedding model that is called with a provider key had a request, and a second decision model has failed requests only.
The error branches: an append whose saved .sashdat can not be loaded, a replace that can not build the <table>_NEW table, and a log file that is missing while the output table exists.
Append without promoting, a saved table whose name differs from the table name in upper and lower case, and a target caslib other than CASUSER - nothing was written to Public.
The step clicked in the SAS Studio UI, and how SAS Studio shows its About tab: the step was run as the code SAS Studio generates for a flow, in SAS Job Execution jobs. Whether an interactive SAS Studio session of the test system starts without a CAS session was not looked at either.
The step's own session in append mode, a CAS session of the user in append mode, and a run without the libref where the name of the libref is not the name of a caslib.
The condition code SAS Job Execution keeps for a job: the job API returned none for the seven jobs of the session check, their SYSCC of 0 was written by the jobs themselves.
A log with bytes that are not valid UTF-8, and two requests that overlap in one container. Both are covered by the tests of the repository, which run the parser's Python outside of SAS.
Of the parser with the 36 columns: the output port of the step on the flow canvas of SAS Studio and its About tab as SAS Studio shows it - the stored step, its 36 column controls and the generated code were looked at -, the default table names - the runs of 2026-10-11 wrote to CASUSER, and on 2026-10-12 one chain wrote to Public under names of its own -, a collected log with requests that ran at the same time, and the LLM Usage Report opened on a table of this release: the twelve columns it reads of LLM_LOGS were compared with the report package and are all there. The package joins LLM_LOGS with LLM_FACT_SHEET; whether the new column model_id of LLM_LOGS touches that join was not looked at.
What the three programs do since the chain ran a second time on 2026-10-12: version 2.4 of the LLM - Log Parser step, version 2.2 of the LLM - Get All Prompts step and Link-Logs-To-Models.sas as they are now. The Chain on SAS Viya lists what was changed after each run.

The Experiments of Every Kind

SAS-Viya-Integrations/Logging-Monitoring/Get-All-Prompts.sas reads what the Prompt Builder keeps in SAS Model Manager and writes it into five CAS tables. Every row of every table carries modelID and projectID, the SAS Model Manager ids of the model and of its project - the model being the prompt template, the decision template or the RAG setup itself, where its trackers and its manifested score code are stored. The LLM - Get All Prompts custom step runs the same program.

The program ran on the test system on 2026-10-11 as it was that day - version 2.0 of the step - and on 2026-10-12 with what that run had led to - version 2.1 of the step: the refusal of a table name that no libref can address, the runs taken from a ledger, the further columns and roles of the catalog, the setup without a collection and the session check of the code file. See The Pull on SAS Viya. What was changed after the second run - version 2.2 of the step: the condition code of a run that wrote an ERROR line, the function its messages go through, the counter of an optimizationId that a tracker holds twice - ran there later that day (the third run). Version 2.3 of the step - the counter that passes over the id of another entry - and the settings the code file takes only where they are not set are not run in SAS. The Python of the program runs in the test suite of the repository against a made-up SAS Model Manager, and the SAS code around it is held by the tests as text.

What It Reads

The projects with the tag Prompt-Engineering - projects of decision templates carry that tag as well. Of every model: Prompt-Experiment-Tracker.json, Prompt-Optimization-Tracker.json and the manifested score code (the .py file with the role score).
The projects with the tag RAG-Engineering. A setup saved by the Prompt Builder is read from its rag-setup.json. A model registered by the step RAG - Register Setup (named RAG Retrieval - <setup>) has no such file: it is read from the pipeline.yaml the step stores with it, from its properties collection, pipelineVersion, configuration and ingestionRunId, and from its training table, which the step sets to <CAS server>/<caslib>/<prefix>_LEDGER. Any other model of such a project is left out with a NOTE.
The projects that hold the definitions of the models that answer a request. The Prompt Builder finds them by id (its options llmProjectID and embeddingProjectID) and the decision models by the project name Decision Model Project; the program finds all three by name - LLM Model Project, Embedding Model Project, Decision Model Project, the names mdb setup gives them - so that it needs no id.

Nothing is written to SAS Model Manager, and every listing is read to its last page.

Settings

The program takes each of these settings where its macro variable is not set when it starts (Settings that stay yours). In the custom step the output table gives the CAS library and the table of the experiments, and the others are fields of the step.

Macro variable	Default	Meaning
`_gap_tag`	`Prompt-Engineering`	Tag of the projects with prompt templates and decision templates
`_gap_rag_tag`	`RAG-Engineering`	Tag of the projects with RAG setups; empty reads none (in the step an empty field takes the default)
`_gap_llm_project`	`LLM Model Project`	Project of the LLM definitions: a name, a project id or `tag:<tag>`, several separated by `\|`; empty leaves the LLMs out of the catalog
`_gap_embedding_project`	`Embedding Model Project`	The same for the embedding models
`_gap_decision_project`	`Decision Model Project`	The same for the decision models
`_gap_out_caslib`	`Public`	CAS library of all five tables
`_gap_out_experiments`	`PROMPT_EXPERIMENTS`	Table names, one per output
`_gap_out_optimizations`	`PROMPT_OPTIMIZATIONS`
`_gap_out_rag_setups`	`RAG_SETUPS`
`_gap_out_rag_runs`	`RAG_RUNS`
`_gap_out_catalog`	`MODEL_CATALOG`

Two outputs can not have the same name, and an output needs a name: the program says so and reads nothing. The five table names are SAS names of 32 characters at most - letters, digits and underscores, no digit in front. CAS takes a longer name, but no CAS libref can address such a table: the program ends with one ERROR line before it reads SAS Model Manager. The program needs PROC PYTHON with the package requests.

Get-All-Prompts.sas runs in the CAS session the option SESSREF names when that session is connected, and otherwise in a connected session mySess, to which it sets the option; only where neither is connected does it start one (cas mySess;), and it ends only a session it started. The step does the same with a session called gapMySess.

The Tables

Each table is dropped, loaded, promoted and saved, as PROMPT_EXPERIMENTS was. A table without rows is written as well, with all its columns: the DATA step that builds it declares the columns and reads nothing. A long text column is as wide as its longest value, at most 32,767 bytes; a longer text is cut between two characters.

Table	One row per	Key
`PROMPT_EXPERIMENTS`	experiment run and model that answered, of a prompt template or decision template; one row without a run for a model that has none	`modelID`, `runID`, `model`
`PROMPT_OPTIMIZATIONS`	entry of a `Prompt-Optimization-Tracker.json`: an optimization or a comparison of target models	`modelID`, `optimizationId`
`RAG_SETUPS`	RAG setup, saved by the Prompt Builder or registered by the step RAG - Register Setup	`modelID`
`RAG_RUNS`	ingestion run of a setup: the rows of its run table when that is in memory, otherwise the runs its ledger still names	`modelID`, `run_id`
`MODEL_CATALOG`	model and pattern	`modelID`, `role`, `runID`, `optimization_id`, `target_model`

PROMPT_EXPERIMENTS. The 49 columns the table had, unchanged in name, order and type: unique_id (the row number, no key), name, description, createdBy, modifiedBy, function, runID, model, systemPrompt, userPrompt (the templates; for a decision run the questions and the state template), response, run_time, prompt_length, output_length, cost, cheapest_prompt, best_prompt, fastest_prompt, fewest_tokens_prompt, judge_rank, judge_best, judge_cost, judge_model, judge_confidence, judge_mode, judge_panel, judge_agreement, judge_chairman_model, variables_count, integrated_llm_call, output_variables_count, creationTime, modifiedTime, temperature, top_p, top_k, max_tokens, thinking_budget, max_completion_tokens, seed, frequency_penalty, presence_penalty, dimensions, api_key_required, reasoning_effort, input_type, normalize, projectLink, modelLink. Behind them: modelID, projectID, projectName, kind (prompt or decision), expected (the expected answers of a decision run, JSON), options (as stored; the value of an option that is a secret by its name, by the names the log parser goes by, is replaced by REDACTED), variables (JSON), resolved_system_prompt, resolved_user_prompt (the prompts as they were sent), resolved_system_hash, resolved_user_hash, judge_reasoning (at most 2,000 bytes). runID is a number.

PROMPT_OPTIMIZATIONS. modelID, projectID, projectName, name, optimizationId, startedAt, finishedAt (SAS datetimes, UTC), status, targetModel, optimizer, metric, judgeModel, datasetSource, datasetRef, sampleCount, trainSize, validationSize, metricBefore, metricAfter, targetCalls, targetPromptTokens, targetOutputTokens, targetRunTime, judgeCalls, judgePromptTokens, judgeOutputTokens, judgeRunTime, error, type (comparison for a comparison of target models, else empty), skippedReason, jobId, computeContext. The prompts, the few-shot examples and the evaluations of an optimization stay in the tracker.

RAG_SETUPS. modelID, projectID, projectName, name, description, creationTime, modifiedTime, source (builder or step), collection, backend, embedding_model, embedding_dims, chunker, input_token_limit, overlap_tokens, tables_prefix, tables_caslib, source_path, pipeline_version, artifacts_folder, job_definition_uri (both only for a setup of the Builder), ingestion_run_id, configuration (both only for a model of the step), runs_table (<tables_prefix>_RUNS), createdBy, modifiedBy. A setup that was saved in the Builder and registered with the step is two models and two rows with the same collection.

RAG_RUNS. The rows of every setup's run table <tables_caslib>.<tables_prefix>_RUNS, stacked (run_source = run table). Both ingestion paths publish that table: the step RAG - Load Vector Store, and since release 2.3.0 the ingestion job of a setup of the Prompt Builder (Ingest-Documents.sas), which names it by its ledger table without the ending _LEDGER - the table prefix of the setup. Only a table that is in memory is read, and no table of a setup is loaded or changed. Columns: modelID, projectID, then the columns rag_core writes - as text run_id, rag_project, collection, backend, started_at, finished_at, status, pipeline_version, config_id, embed_model, chunker, enrich_model, error_text; as numbers embed_dims, input_token_limit, overlap_tokens, docs_new, docs_changed, docs_unchanged, docs_deleted, docs_skipped, docs_excluded, docs_failed, docs_ingested, chunks_written, chunks_retired, collection_chunks, embed_calls, embed_tokens, embed_seconds, enrich_calls, enrich_input_tokens, enrich_output_tokens, enrich_seconds, enrich_failed - then source_caslib, source_table and run_source: 40 columns. A column a run table of an earlier release lacks is empty, a count that was published as text is a number here, and started_at and finished_at are the text the run table holds (UTC; on the test system YYYY-MM-DD HH:MM:SS+00:00). Only the runs of the setup's own collection are taken. A run table two setups share gives its rows once per setup.

A setup that the job ingested before has no run table until its job definition holds the program that publishes it - Manifest setup writes the program of the release into the job definition - and the job ran once with the run history switched on. That run publishes every run the vector store holds for the collection (the newest 500), so after the next pull, and a link with _lnk_redo_all = 1, the document embeddings of its earlier ingestions can be given to their runs, as far as the store recorded those runs. The publication by the job was measured on 2026-10-12 with a setup of its own, and the pull and the link on its run table - see the third run. Outside of SAS, the file the job writes for the table gives the same rows of RAG_RUNS as the table the step stages - by a reading of the two macros that stack the tables, which no test runs.

For a setup whose run table is not in memory - its job definition holds the program from before, or its run history is switched off - the pull writes the runs that the ledger <tables_prefix>_LEDGER still names, if that table is in memory (run_source = ledger): one row per run id, started_at the second the id of the run holds (run-<seconds since 1970>-<digits>), finished_at the latest updated_at of its rows, collection and embed_model those of the setup, source_table the ledger, and every count missing. Every run writes its id into the row of every document of the folder, so a ledger names the newest run only. The ledger is read only where the run table is not in memory, so a run that both name is one row. A setup with neither table in memory is left out with a NOTE. The rows taken from a ledger ran on the test system on 2026-10-12, see The Pull on SAS Viya.

MODEL_CATALOG. modelID, projectID, projectName, name, role, runID (text), target_model, system_template, user_template, resolved_system_prompt, resolved_user_prompt, collection, template_source, resolved_system_hash, resolved_user_hash, and behind these fifteen the six that were added after the run of 2026-10-11: run_time, prompt_length, output_length (numbers), window_start, window_end (text, YYYY-MM-DDTHH:MM:SSZ in UTC) and optimization_id (text). The link program reads every column but projectName, prompt_length and output_length.

`role`	Rows	What the row holds
`prompt`, `decision`	one per template (`runID` empty)	The templates with their `{{variables}}`; for a decision template the questions and the state template. `template_source` says where they are from: `score code` (what the manifested score code sends, with the model it calls in `target_model`), `best run` (the newest run marked as the best prompt) or `newest run`. A model without a run and without such score code has no row
`prompt`, `decision`	one per experiment run and model (`runID` set)	The templates of the run, the prompts as they were sent, their hashes, the model they were sent to, and what the tracker holds of that call: `run_time` in seconds, `prompt_length` and `output_length` in tokens. The run time is what tells the request of this run from another request with the same prompts
`rag`	one per RAG setup that names a collection	`collection`, the embedding model in `target_model`, and in `template_source` who wrote the model: `builder` or `step`. A setup without a collection - its properties or its setup file could not be read - keeps its row of `RAG_SETUPS` and has none here, as no request could be linked to it
`judge`	one per run that was judged and model that judged it	`modelID`, `projectID` and `name` are the prompt's, `runID` the run, `target_model` the judge - the judge, each member of a panel, the chairman. `resolved_system_prompt` and `resolved_user_prompt` are the prompts of the run as they were sent, which the Prompt Builder shows a judge in the user prompt of its call. No run time and no window: the tracker holds none for the call of a judge
`optimization`	one per entry of the optimization tracker and model it called	`runID` is empty and `optimization_id` the id of the entry; `target_model` is a target model or the judge of the optimization. `window_start` and `window_end` are the start and the end of the job, the start rounded down and the end up to the second, empty where the tracker has none. `resolved_system_prompt` is the system prompt the optimization started from, as it is stored, and the two templates are the optimized prompt where the tracker holds one. An `optimizationId` that a tracker holds on more than one entry is counted, so that no two rows of the catalog have one key: the rows of the second entry carry it as `<id>#2`, of the third as `<id>#3`, and a WARNING names the model. `PROMPT_OPTIMIZATIONS` holds such entries as the tracker has them, so its key (`modelID`, `optimizationId`) is not unique for that model. Measured on 2026-10-12 in the third run with a copy of a tracker that held its entry twice under the id 1: the catalog rows `1` and `1#2`, a key that was unique, and two rows with the id 1 in `PROMPT_OPTIMIZATIONS`. Since version 2.3 of the step a counter that would make the id of another entry is passed over - the ids 1, 1 and `1#2` give 1, `1#3` and `1#2` -, an id longer than the 16 bytes of the column makes room for its counter, and an entry without an id carries the counter alone (`#2`); that is not run in SAS
`llm`, `embedding`, `decision_model`	one per model definition	`name` is the name of the container, which is the model a log row names

The Prompts as They Were Sent

The Prompt Builder stores the templates and the variables of a run and sends the prompts with the values filled in. The program fills them in the same way: the variables one after the other in the order they are stored, {{name}} with blanks or line ends inside the braces, the value as it is - not trimmed, whatever the type of the variable - and a {{name}} of no variable left as typed. A decision run sends its questions as they are stored and gets the variables filled into its state only. The hash is the one of the log table, taken of the whole text before it is cut to the length of its column.

The templates of a manifested model are read from its score code, which is parsed and never run. A manifested model strips every value before it puts it into the prompt, the Prompt Builder does not.

Measured outside of SAS on files saved from the test system: the templates were read from nine manifested score codes, one of them a decision template manifested with 2.2.0 whose score code does not compile and is read statement by statement; the questions and the state template read from the score code of one decision template fit 41 of the 105 requests to its decision model in the collected log of 1040 requests, and the questions of its tracker equalled the questions of those requests character by character. Not compared: a prompt as an experiment run sent it with a logged request.

Messages and Limits of the Pull

A model whose file can not be read (not JSON, a failed request), and a project or model that can not be listed, is a WARNING with its name; the model keeps its row without runs and the pull goes on.
When the projects themselves can not be listed, the program ends with an ERROR and leaves the output tables as they are.
A definition project that is not found is a WARNING; an empty setting asks for nothing.
The run ends with one NOTE per output table: <caslib>.<table> was written with <n> rows and <m> columns.
A run table whose name is no SAS name of at most 32 characters, or whose caslib holds a quote, &, % or ;, is left out with a WARNING.
An output table name of more than 32 characters, or one that is no SAS name: ERROR: The table name <name> of the output <output> has <n> characters. A SAS libref addresses a CAS table by a name of 32 characters at most, nothing was read or written., or ... is no SAS name: letters, digits and underscores, and no digit in front. Nothing was read or written. Nothing is read and no table is touched.
A RAG setup without a collection: WARNING: The properties of the model <name> (<id>) could not be read: <reason>. The setup has no collection without them and is left out of the model catalog, where no request could be linked to it. Its row of the RAG setups stays., or WARNING: The RAG setup <name> (<id>) names no collection and is left out of the model catalog, ...
A setup whose runs are taken from its ledger: NOTE: The run table <caslib>.<prefix>_RUNS is not in memory. <n> run(s) were taken from the ledger <caslib>.<prefix>_LEDGER - the runs its rows still name, each from the start its id holds to the latest of its rows.
A setup whose CAS library is not available in the session is left out, and so is a setup where CAS refused the question for its run table, the question for its ledger where the run table is not in memory, or - for a CAS library other than CASUSER - the listing of the CAS libraries, and a setup whose PROC CAS step halted before it set what it found: NOTE: The CAS library <caslib> of the run table <table> is not available in this session, or CAS did not say whether it, the table or its ledger table is there, so its runs are not in <RAG_RUNS>. The setup has no rows in RAG_RUNS. Measured in the eighth pass, late on 2026-10-14, with a CAS session that had ended: up to version 2.5 a halted step wrote NOTE: The run table <caslib>.<table> is not in memory, its runs are not in <RAG_RUNS>., also for a run table of 5 rows in memory (The Pass of 2026-10-14). The halt as version 2.6 handles it ran in the ninth pass, on 2026-10-15: the program and the step wrote the NOTE above for each run table whose step halted, and their five tables. A refusal as version 2.6 handles it is not run in SAS.
An optimizationId that a tracker holds on more than one entry: WARNING: The file Prompt-Optimization-Tracker.json of the model <name> (<id>) holds the optimizationId <ids> on more than one entry. In the model catalog the rows of a later such entry carry the id with a counter, as in 1#2 - for an entry without an id the counter alone -, so that no two rows of the catalog have the same key. The table of the optimizations holds the entries as the tracker has them. Where entries without an optimizationId repeat, the message says ... holds <n> entries without an optimizationId. in that place, or both joined by and. The WARNING of version 2.2 of the step was seen on 2026-10-12 in the third run; its wording as it is here, of version 2.3, is not run in SAS.
In the step, an output table that is no CAS table: ERROR: The selected output library does not point to a CASlib, nothing was read or written. - one line, where the step ended with an ABORT statement up to version 2.1. Not run in SAS.
A run that wrote an ERROR line and left an output table unwritten sets the condition code of the SAS session (SYSCC) to 1012 as its last statement in front of the macro variables it deletes: a table name that is refused, projects that could not be listed or another reason for which SAS Model Manager could not be read, a table that was not written, and in the step an output library that is no CAS library. A WARNING - a model that could not be read, a setup that is left out - leaves the condition code as it was. Measured on 2026-10-12 in the third run: the refusal of a table name of 34 characters ended with SYSCC 1012 and a failed job. And the step in the code SAS Studio generates for a flow, with its output table in WORK: ERROR: The selected output library does not point to a CASlib, nothing was read or written., no ABORT line, no request to SAS Model Manager, SYSCC 1012 and a failed job. The first DATA step behind the step is the flow's own %_flw_action_end; it wrote NOTE: Due to ERROR(s) above, SAS set option OBS=0, enabling syntax check mode. and did not run, so the log held _FLW_ACTION_START_ and neither _FLW_ACTION_TABLE_ nor _FLW_ACTION_END_ for the node, and obs was 0 behind the flow code. A refusal behaves like any SAS error: later DATA steps of the session do not run until SYSCC is reset. What SAS Studio shows for such a node was not looked at. A table name that is refused still gives two ERROR lines: its own, and the line that says that the experiments could not be read, which is the line that ends such a run.
Every message the Python writes is made one text before it is handed to SAS, by the function the parser and the link program carry: in a name from SAS Model Manager a quote shows as a backtick, % as pct, & as +, a semicolon as a comma, a comment mark is split by a blank, a tab or a line break is a blank, and parentheses that do not pair up show as brackets - then the ones around the id of the model as well. Up to version 2.1 of the step the function let comment marks, tabs and a parenthesis without its partner through. Measured on 2026-10-12 in the third run with models named /* open (x) setup, /* open setup and a name with quotes, a semicolon, a percent sign and %put: the WARNINGs arrived whole - / * open (x) setup (<id>), with the parentheses that pair up left as they are, and `q` `d` , 50pct pct put x, setup (<id>) -, the closing NOTEs followed, SYSCC was 0 and RAG_SETUPS held the names unchanged. SAS Model Manager refused a model name with a parenthesis without its partner (HTTP 400), so that such a parenthesis shows as a bracket was not measured.
The requests are made one after the other: one per project and one or more per model.

The Pull on SAS Viya

Measured on the test system on 2026-10-11 with Get-All-Prompts.sas as it was that day (version 2.0 of the step), in SAS Job Execution jobs that wrote to tables of their own in CASUSER. Where a point goes on with On 2026-10-12, the program was changed after the run, and the changed program - version 2.1 of the step - ran on the test system that day. Where it ends in Since then, the program was changed once more - version 2.2 of the step -, and that change ran in the third run later that day.

What it read. 41 projects with the tag Prompt-Engineering and 92 models in them, 10 projects with the tag RAG-Engineering and 17 RAG setups, in 17 to 20 seconds per run - the Python step 16.6 of the 19.5 seconds of the first.
The five tables. PROMPT_EXPERIMENTS with 318 rows and 61 columns, PROMPT_OPTIMIZATIONS with 27 and 32, RAG_SETUPS with 17 and 26, RAG_RUNS with 105 and 39 and MODEL_CATALOG with 365 and 15, each promoted and saved. The keys were unique: 318 of 318 rows of the experiments and 365 of 365 of the catalog. The JSON libname with its map, the loop over the CAS libraries and the look at each written table through the libref ran as they are written. On 2026-10-12: PROMPT_EXPERIMENTS with 318 rows and 61 columns, PROMPT_OPTIMIZATIONS with 27 and 32, RAG_SETUPS with 17 and 26, RAG_RUNS with 107 and 40 and MODEL_CATALOG with 421 and 21, in 18.6 seconds (the job 34). All four keys were unique: 318 of 318, 27 of 27, 107 of 107, and 421 of 421 rows of the catalog - 392 without optimization_id in the key. The catalog held 285 prompt, 47 decision, 24 judge, 35 optimization, 14 rag, 8 llm, 3 embedding and 5 decision_model rows, and run_time, prompt_length and output_length were numbers in CAS.
Three runs with one content. The program alone, the program behind the parser in one SAS session, and the step (version 2.0) as the code SAS Studio generated for a flow wrote the same five tables - compared by the ids, counts, lengths and hashes of every row, not by the texts. On 2026-10-12: the step (version 2.1) as the code SAS Studio generated for a flow wrote the five tables of the program of that day, with the same columns in name, order, type, length and label and the same ids, counts, lengths and hashes in every row, and the same three WARNINGs, in 16.1 seconds.
A decision template, a prompt template and a RAG setup of the test system, looked at row by row. The decision template had 41 rows of 21 runs with two decision models and kind = decision, and in the catalog its template (score code) and 41 run rows with both hashes. The prompt template had 32 rows of 14 runs with four LLMs and one optimization, and in the catalog its template and 32 run rows. The setup had source = builder with its collection, vector database, embedding model and table prefix, and in the catalog one rag row marked builder. The catalog held 8 llm, 3 embedding and 5 decision_model rows. On 2026-10-12, in the catalog with its further columns and roles: the prompt template with 32 run rows, each with a run_time; 13 judge rows for its runs 2 to 14, with the judge in target_model and without a run time; and one optimization row with the model the optimization called, the id 1, the window 2026-10-11T17:34:10Z to 2026-10-11T17:35:11Z and the system prompt it started from, 817 characters. The decision template had its template row and 41 run rows.
The columns the LLM Usage Report reads - 38 of PROMPT_EXPERIMENTS, read from the report package - were all in the table under their names. The report was not opened on it.
A table name of 34 characters. For an output with such a name the last line of the run was ERROR: The table <caslib>.<name> was not written. - for a table that had been written, promoted and saved with its 27 rows and 32 columns. The closing check looks at the table through a CAS libref, and the exist function returned 0 for the name of 34 characters and 1 for one of 32. On 2026-10-12: a name of more than 32 characters, or one that is no SAS name, ends the program before anything is read. With a name of 34 characters for the optimizations the run wrote ERROR: The table name <name> of the output optimizations has 34 characters. A SAS libref addresses a CAS table by a name of 32 characters at most, nothing was read or written. and then ERROR: The experiments could not be read from SAS Model Manager, the output tables in <caslib> were left as they are., in 3.6 seconds, and no table of that job existed afterwards. SYSCC was 0 and the job ended as completed. Since then a run that wrote an ERROR line and left an output table unwritten sets SYSCC to 1012, see Messages and Limits of the Pull.
A model whose properties the service does not return. GET /modelRepository/models/<id> of one model registered by the step was answered with HTTP 500, by the program and outside of it. The program wrote one WARNING and kept the setup, with an empty collection and embedding model, in RAG_SETUPS and in the catalog. On 2026-10-12: such a setup keeps its row of RAG_SETUPS and is left out of the catalog, and the WARNING says so. Each of the six runs that read SAS Model Manager that day wrote three such WARNINGs - for that model, which the service still answered with HTTP 500 (... The setup has no collection without them and is left out of the model catalog, ... Its row of the RAG setups stays.), and for two setups that name no collection (... names no collection and is left out of the model catalog, ...). RAG_SETUPS had its 17 rows, and the catalog 14 rag rows where there had been 17, none of them without a collection.
The run table of the Builder's setup does not exist. The pull wrote NOTE: The run table Public.<prefix>_RUNS is not in memory, .... The caslib held the ledger of the setup (<prefix>_LEDGER, 7 rows, and its .sashdat) and no run table, in memory or as a file. The 105 rows of RAG_RUNS came from four run tables of setups that the step path had ingested; three of those tables are shared by two or three models the step registered and are in RAG_RUNS once per model. 15 of the 105 rows hold a finished_at about two hours before their started_at. On 2026-10-12: for a setup without a run table in memory the pull writes the runs its ledger still names. For that setup it wrote NOTE: The run table Public.<prefix>_RUNS is not in memory. 1 run(s) were taken from the ledger Public.<prefix>_LEDGER - the runs its rows still name, each from the start its id holds to the latest of its rows. The row had the run id of the ledger, started_at 2026-10-11 17:49:41, finished_at 2026-10-11 17:49:47, the collection and the embedding model of the setup, run_source ledger, and no counts and no status. A second setup gave one row from its ledger as well: 2 of the 107 rows of RAG_RUNS were from ledgers and 105 from the four run tables. The ledger was unchanged - 7 rows, the same modified time, the same file -, and the steps that read it wrote no note about a variable that is not initialized or a value that was converted. Since then the ingestion job publishes the run table of its setup, see RAG_RUNS above; in the third run the pull read four runs of a setup from the table its job had published.
Behind the parser in one SAS session the cas mySess; of the program wrote WARNING: A session with the name MYSESS already exists. - SAS Job Execution keeps such a warning in the record of a job: another job of that day with the same warning ended as completed and carried the error code 5710 with that message -, and the program ended the session the parser had started and its librefs were bound to. On 2026-10-12: both code files ask SESSFOUND before they start a session, and Get-All-Prompts.sas ends only a session it started. Alone in a fresh SAS session the program started MYSESS and ended it. In one job with the parser, the pull, the parser, the pull and the link twice, no line said already exists, the session and the librefs public and casuser were there after every program, and both pulls wrote their five tables with the row counts above. Behind a CAS session of the user no program started one of its own.
Not measured: an output without rows - every output had rows -, a pull that takes longer than two minutes (51 projects and 109 models took 17 to 20 seconds), the default table names - on 2026-10-12 one pull wrote its five tables to Public under names of its own, with the same rows and columns -, the step run from its page in SAS Studio, and of the runs taken from a ledger a ledger without the columns run_id and updated_at and a CAS library that is not available. What the program does since the third run - version 2.3 of the step, and the settings it takes only where they are not set - is not run in SAS.

Linking the Log to the Models

SAS-Viya-Integrations/Logging-Monitoring/Link-Logs-To-Models.sas fills the nine link columns of the log table. It reads the log table, MODEL_CATALOG and, when it is there, RAG_RUNS, links the rows without a match_method and, where it linked a row, puts the table back, promoted and saved. A run that links no row leaves the table as it is.

The program ran on the test system on 2026-10-11 and on 2026-10-12, each time as it was that day, see The Chain on SAS Viya. The first run linked the 1040 rows of the collected log, and its summary broke the SAS log. The program was changed - its messages, the rule for an experiment run, the window of a retrieval query, the calls of optimizations and judges, what it does to a table it links no row of -, and the second run measured the rules as this section has them. What was changed after the second run - the condition code of a run that wrote an ERROR line and no table, and what the program does with a log table that it loaded from its saved file and linked no row of - ran in the third run later that day. The link program has no custom step. The parser and the link program must not run on one table at the same time. Since the third run the program takes its settings only where they are not set (Settings that stay yours), which is not run in SAS.

Settings of the Link

Macro variable	Default	Meaning
`_lnk_log_caslib`, `_lnk_log_table`	`Public`, `LLM_LOGS`	The log table. Its link columns are filled in place
`_lnk_catalog_caslib`, `_lnk_catalog_table`	`Public`, `MODEL_CATALOG`	The catalog of Get-All-Prompts.sas
`_lnk_runs_caslib`, `_lnk_runs_table`	`Public`, `RAG_RUNS`	The ingestion runs. Without the table no embedding row is linked to an ingestion run; the program says so in a NOTE
`_lnk_redo_all`	`0`	`0` links the rows without a `match_method` and leaves every other row as it is. `1` links all rows again
`_lnk_min_literal`	`20`	The characters of fixed text (whitespace not counted) a template needs to take part, an experiment run in its prompts, and a call of an optimization or of a judge in the texts the catalog holds of it
`_lnk_utc_offset`	`0`	The hours the clock of the containers is ahead of UTC. The ingestion runs and the windows of the optimizations are recorded in UTC
`_lnk_query_window`	`300`	The seconds within which the prompt that carries a retrieval follows the embedding of its query. A query embedding without such a prompt is not linked
`_lnk_promote_table`, `_lnk_save_table`	`1`, `1`	They decide for a table that is not promoted and for one that has no saved file. A table that is promoted is promoted again and a table that has a saved file is saved again, whatever the two say - as the parser does it when it appends

A table that was saved but is not in memory (after a restart of the CAS server) is loaded from its .sashdat file first; this holds for all three tables. A log table that was loaded and of which the run then links no row is promoted as _lnk_promote_table says, and its saved file is left as it is (see What the Program Does to the Table); the catalog and the runs table that were loaded stay tables of the session of the program. The program needs PROC PYTHON with the pandas and swat packages and the caslib CASUSER, as the parser does. A setting that has to be a number and is none gives a WARNING, and its default is taken. The program runs in the CAS session the option SESSREF names when that session is connected, and otherwise in a connected session lnkMySess, to which it sets the option; only where neither is connected does it start one (cas lnkMySess;), and it ends only that one.

The Rules, in Their Order

No request, header or log line names the caller. Every link is therefore made by content, and match_method says which rule decided.

The model that answered (model_id): the catalog row with the role llm, embedding or decision_model whose name is the name of the container (model). When two models of that role have the name, model_id stays empty and a NOTE names the two.
experiment run (LLM and decision rows): the system and the user prompt of the row are the prompts of an experiment run as the Builder sent them (for a decision: the questions and the state), the run was sent to the model of the row, the request was not sent by the score code or a SAS session (user_agent is not python-requests or SAS) - the Builder page sends its runs from a browser -, and the runtime of the row is the run_time the catalog holds for that run and model, compared with three decimals. The prompts are compared by the hashes the catalog holds of them, so a prompt that is longer than a column of the catalog is still found. A run is given to one row: the earliest request gets it, and a run that a row of an earlier link holds is taken. A request with the prompts of a run whose run time is the one of no run is another request with the same prompts: the rules below say who sent it, and it gets no experiment_run_id. Where the catalog holds no run_time for the runs with these prompts - a catalog written before the column existed -, the row gets the newest of them, match_candidates is their number, and a NOTE counts such rows.
questions and state, questions (decision rows): the questions of the row are the questions of a decision template. Questions are compared as JSON, so the order of the keys and the whitespace do not matter; a text that is no JSON is compared as text. When the state of the row fits a state template that has fixed text, the method is questions and state, otherwise questions.
template (LLM rows): the system prompt and the user prompt of the row fit the templates of a prompt. Every {{variable}} stands for any text, also for none and for several lines; the fixed text has to be there in its order, from the first character to the last, whitespace squeezed. The templates of earlier experiment runs of a prompt count as its templates.
optimization, then judge (LLM rows): the request is a call the Prompt Builder made for a prompt and not with it - a call of a prompt optimization, or the call a judge gets to rank the answers of a run. A catalog row of that role links a request when three things hold: the request holds every text of the row (resolved_system_prompt, resolved_user_prompt) in its system or in its user prompt, at least one of them as a part of the prompt and not as the whole of it; it went to the model in target_model; and, where the row has window_start and window_end, it was sent inside that window (whole seconds, _lnk_utc_offset applies). The texts need _lnk_min_literal characters together. caller_model_id is the prompt the call was made for and caller_kind what the catalog says that caller is. A judge call gets the experiment_run_id of the run that was judged when exactly one run remains; two judged runs with the same prompts leave it empty. An optimization is asked first: a call of a judge that was made while an optimization of the same prompt was running, to a model that optimization called and with the system prompt it started from, is an optimization row.
The RAG setup of a prompt (LLM rows, column rag_model_id, does not change match_method): the prompt carries the results of a retrieval, and the setup is the one the catalog has for its collection. So a call of a prompt that is fed by a retrieval is linked to its prompt and to the setup. Of several retrievals in one prompt the last one counts, as for the parser.
retrieval query (embedding rows in query mode): the hash of the embedded text is the hash of the query of a retrieval that an LLM row of the table carries, and that LLM row follows the embedding within _lnk_query_window seconds. The nearest such row decides: the embedding row gets the setup of the collection it names. The same question is embedded many times and for more than one collection, so a prompt that came before the embedding, or long after it, says nothing about this request. Two prompts of two collections in the same second are ambiguous. A setup of which the catalog names another embedding model than the model of the row is not taken.
ingestion run (embedding rows in document mode whose project input is rag, as the RAG runtime sends it, or empty): the request lies between started_at and finished_at of exactly one run of RAG_RUNS whose embed_model is the model of the row. A run without a start or an end, or whose end lies before its start, takes no part, and a NOTE counts such runs.

Between several callers that fit, the one with the most fixed text wins and match_candidates is their number. A caller of which the catalog only names other target models is taken when no other caller fits. Two callers with the same amount of fixed text, two collections for one query or two overlapping ingestion runs give match_method = ambiguous and empty caller columns. A row that fits nothing gets none and match_candidates = 0; its model_id is filled all the same.

Where two models hold one collection - a setup that was saved in the Prompt Builder and registered with the step RAG - Register Setup - the model the Builder saved is taken, for rag_model_id, for a retrieval query and for an ingestion run, when the other model is its registration: the catalog marks it step, the two name no different embedding models, and it is named RAG Retrieval - <name of the setup> or holds an ingestion run of RAG_RUNS that the setup holds as well. Another model with the same collection name, two setups of the Builder with one collection, or two models of the step stay open: rag_model_id is empty and the embedding rows are ambiguous, unless the retrieval names an ingestion run that RAG_RUNS holds for one of them only.

A row whose content can not be compared - questions that are nested deeper than Python reads JSON - gets none, a catalog row that can not be read takes no part, and a WARNING names both. The rows around them are linked. Measured on 2026-10-12: a decision row whose questions were 30000 opening brackets gave WARNING: 1 row(s) could not be compared with the catalog - RecursionError - and got match_method none., and every other row was linked as before. With 2000 opening brackets the Python of that system (3.12.12) raised no such error: the row got none without the WARNING, as questions that fit no template do.

The limits of the rules optimization and judge. Both link by containment: the texts of the catalog row have to stand in the prompts of the request, with _lnk_min_literal characters together as the only limit of size, without a word boundary, and for a judge row without a window of time. A request that is no call of the Prompt Builder is therefore linked when it holds those texts and no rule above fits it. Outside of SAS, on made-up rows: a request to the target model inside the window of an optimization that asks about the prompt (Is this a good system prompt? <the system prompt> Tell me.) is an optimization row, and so is a variant of the prompt - the system prompt and one more sentence - that is sent inside the window; a request to the judge model that quotes both prompts of a judged run is a judge row of that run, months later as well; and Claim: hail is found inside Claim: hailstorm. On the collected log the 49 optimization and 13 judge rows of 2026-10-12 were the calls of the Prompt Builder and none more: checked outside of SAS, the whole text of the catalog row stood in the system prompt of all 49, inside the window, and both prompts of the judged run in the user prompt of all 13, and no other caller and no other run held them. Such a row carries a rag_model_id when the prompt it quotes held the results of a retrieval, although the call made no retrieval itself: on that log 62 of the 193 LLM rows with the setup of one collection are such calls, and the summary line and the second statement of the example under How the Tables Join count them among the prompts of the setup.

The window of a retrieval query on the collected log. Of the 290 query embeddings of the log of 1040 requests, 124 were followed by the prompt with their question within one second, 13 more within 33 seconds and one after 178 seconds. 83 were followed only after 6.8 to 23.8 minutes, 18 only had such prompts before them, and 51 had none. The 83 and the one of 178 seconds are 84 query embeddings of one collection that were sent before the first prompt of that collection - retrievals that were tested before a prompt used them. The default of 300 seconds links the 138, the one of 178 seconds among them; with _lnk_query_window = 1800 it is 221. Both numbers are what the program linked on the test system on 2026-10-12.

What Cannot Be Linked, and Why

No caller id is logged. A request carries its inputs and nothing about who built them. A direct call of a model (a script, a health check, the test of a deployment) has no caller in SAS Model Manager at all.
Two templates with the same text. A copy of a prompt, or two decision templates with the same questions and no state that tells them apart, fit the same requests: ambiguous.
A template that is one variable. {{input}} fits everything, so it says nothing. A template with less than 20 characters of fixed text takes no part; the program counts such callers in a NOTE.
The Builder's test calls against a template that was changed afterwards. The catalog holds the saved template and the templates of the runs that were kept. A test call that was not kept as a run and was made with a text that is no longer the template fits nothing.
The health check of an optimization, and a call whose instructions the optimizer rewrote. The calls of a prompt optimization and of a judge are linked through a text of the prompt they hold. The health check an optimization sends first (You are a health check.) holds none, and neither does a call of an optimizer that has replaced the instructions by a proposal of its own.
A further request with the prompts of an experiment run - the run sent again from the Builder page without being kept - is linked to the prompt by its template or its questions, and to no run.
A retrieval that never reached an LLM prompt (the test of a retrieval, a flow that only retrieves), or whose prompt does not follow within _lnk_query_window seconds: its query embedding has no LLM row to be linked through.
Embedding calls outside of an ingestion run, a run without finished_at, a setup with neither a run table nor a ledger in memory when RAG_RUNS was written, and the earlier ingestions of a setup whose run table is not in memory - a setup of the Prompt Builder whose job definition holds the program from before release 2.3.0, or whose run history is switched off: its ledger names the newest run only.
The LLM calls of an enrichment during an ingestion: their prompt is in no catalog.
A published model that sends the prompts of an experiment run again is linked to the prompt, not to the run.
A template longer than 32,767 bytes is cut in the catalog and fits no request.
Rows of an earlier parser have no kind, no hashes and no collection. The kind is taken from the role of the model that answered, where the catalog holds a model of that name; a row whose model the catalog does not hold is counted in a NOTE of its own, <n> rows of no known kind. Measured on 2026-10-13 on a copy of a table of release 2.2 after the append of a collected log: of its 3,630 earlier rows none got a caller, 3,434 got no model_id - their model names are not in the catalog - and all 3,630 carried the match_method none afterwards, so a later link leaves them out unless _lnk_redo_all is 1.

How the Tables Join

From	To	Meaning
`LLM_LOGS.model_id`	`MODEL_CATALOG.modelID` (roles `llm`, `embedding`, `decision_model`)	The model that answered
`LLM_LOGS.caller_model_id`	`PROMPT_EXPERIMENTS.modelID`	The prompt or decision template (`caller_kind` = `prompt` or `decision`)
`LLM_LOGS.caller_model_id`, `experiment_run_id`, `model`	`PROMPT_EXPERIMENTS.modelID`, `runID`, `model`	The row of the experiment run (`match_method` = `experiment run`). `experiment_run_id` is text and `runID` a number, and a run id counts inside one model only. A judge call (`match_method` = `judge`) carries the id of the run it judged, and its `model` is the judge
`LLM_LOGS.caller_model_id`	`PROMPT_OPTIMIZATIONS.modelID`	The optimizations of that prompt. A call of an optimization (`match_method` = `optimization`) carries no id of the optimization: it is the one with `startedAt <= timestamp <= finishedAt`
`LLM_LOGS.caller_model_id`	`RAG_SETUPS.modelID`	The setup of an embedding row (`caller_kind` = `rag`)
`LLM_LOGS.rag_model_id`	`RAG_SETUPS.modelID`	The setup whose retrieval an LLM prompt carried
`RAG_SETUPS.modelID`	`RAG_RUNS.modelID`	The ingestion runs of a setup
`LLM_LOGS.model_id`, `caller_model_id`, `rag_model_id`	`MODEL_CATALOG.modelID`	The name and the project of any of the three

MODEL_CATALOG has one row per model and pattern, so a model is there several times: join to it through a distinct list of modelID. MODEL_CATALOG.target_model, PROMPT_EXPERIMENTS.model and RAG_SETUPS.embedding_model are names of containers and equal LLM_LOGS.model.

experiment_run_id is the place of the run in the tracker of the prompt, not a lasting id: when a run is deleted in the Prompt Builder, the runs behind it are numbered again, so after a new pull a row that was linked earlier can name another run. The link program counts such rows in a WARNING - rows whose run id the catalog no longer has, and rows whose run has another run time than the request - and %let _lnk_redo_all = 1; links all rows again.

An example - what each prompt, decision template and RAG setup asked of the models, and the calls of the Builder's experiment runs beside their rows of PROMPT_EXPERIMENTS. The three statements ran on the test system on 2026-10-11 and on 2026-10-12 on the tables of the chain, with only the table names changed, without an ERROR or a WARNING and in 0.3 seconds; input(l.experiment_run_id, best32.) read the VARCHAR column:

libname public cas caslib='Public';

proc sql;
    /* Requests, failures, tokens and seconds per caller and answering model */
    create table work.usage_by_caller as
    select l.caller_kind, l.caller_name, l.caller_model_id, l.model,
           count(*) as requests,
           sum(l.failed) as failed_requests,
           sum(l.prompt_length) as input_tokens,
           sum(l.output_length) as output_tokens,
           sum(l.runtime) as seconds
    from public.llm_logs as l
    where l.caller_model_id is not missing
    group by l.caller_kind, l.caller_name, l.caller_model_id, l.model;

    /* The prompts that were fed by a retrieval, per RAG setup */
    create table work.prompts_by_rag_setup as
    select s.name as rag_setup, s.collection, l.caller_name as prompt, count(*) as requests
    from public.llm_logs as l
         inner join public.rag_setups as s on s.modelID = l.rag_model_id
    group by s.name, s.collection, l.caller_name;

    /* The logged request of an experiment run beside what the tracker kept of it */
    create table work.experiment_run_requests as
    select e.name, e.runID, e.model, e.best_prompt, e.run_time as tracked_seconds,
           l.timestamp, l.runtime as logged_seconds, l.status_code
    from public.llm_logs as l
         inner join public.prompt_experiments as e
            on  e.modelID = l.caller_model_id
            and e.runID   = input(l.experiment_run_id, best32.)
            and e.model   = l.model
    where l.match_method = 'experiment run';
quit;

What the Program Does to the Table

The log table has no key, and its prompts and every column a site added must not be rewritten. The program therefore numbers the rows in a copy in the caslib CASUSER, reads what it compares into Python (the prompts only of the rows it links), uploads the nine link columns with the row number, merges them back by that number into <table>_NEW beside the log table, checks that this table has as many rows as the log table, restores the order of the columns, gives the new table the label of the log table, and replaces the log table the way the parser does: dropped, renamed, promoted, saved. While it runs it needs room for two more copies of the table. A table that grew while it was linked - the parser appended to it - is left as it is, with an ERROR line; run the two one after the other.

A run that links no row - every row has a match_method - builds no new table: the log table, its promotion and its saved file stay as they are, and the SAS log has NOTE: 0 of <n> rows of <caslib>.<table> were linked, as every row has a match_method. The table was left as it is. With _lnk_redo_all = 1 all rows are linked again.

A log table that was not in memory, and that the program loaded from its saved file, is put in place by such a run as it was loaded: it is promoted where _lnk_promote_table is 1, the saved file is not written again, and one line says so - NOTE: The table <caslib>.<table> was not in memory. It was loaded from its saved file and promoted as it was loaded, the saved file was left as it is. With _lnk_promote_table = 0 the line is NOTE: ... It was loaded from its saved file and not promoted, as _lnk_promote_table is not 1, so it is a table of this CAS session and gone when the session ends. The saved file was left as it is., and where the promotion does not take, WARNING: ... could not be promoted, so it is a table of this CAS session and gone when the session ends. ... Where CAS refuses to say whether the promotion took, or the step that asks halts, the line is WARNING: ... It was loaded from its saved file, and CAS did not say whether it was promoted then: where it was not, it is a table of this CAS session and gone when the session ends. The saved file was left as it is. (not run in SAS). Before, the loaded table was a table of the session of the program and ended with it. Measured on 2026-10-12 in the third run, with the table dropped from memory and its saved file left: with _lnk_promote_table 1 the first NOTE, and the next job found the table in memory, promoted, with its 1040 rows and 36 columns and the saved file as it was; with 0 the second NOTE, and the next job found no table and the file as it was. The WARNING was not provoked.

Every message of the program is made one text before it is handed to SAS, where a line for the log is a statement on its way: in a message a quote shows as a backtick, % as pct, & as + and a semicolon as a comma, a comment mark is split by a blank, a line break or a tab is a blank, parentheses that do not pair up show as brackets, and a message of more than 900 characters ends in three dots. Measured on 2026-10-12 with a catalog in which two models carried one name with a quote, a semicolon and %put ERROR: x;: the NOTE about the two models showed the quote as a backtick and the % as pct, and no ERROR followed. The parser and the pull carry the same function since version 2.4 and 2.2 of their steps; with it their messages arrived as text in the third run.

A log table without the link columns was written by an earlier parser and is refused with one ERROR line that names the parser. Without the catalog there is one ERROR line that names Get-All-Prompts.sas. In both cases the table is left as it is. Where CAS refuses to say whether a table the program reads is in memory, or to list the saved files of its CAS library - for a table of the link that is not in memory, or for the saved file of the log table where _lnk_save_table is not 1 -, or where the PROC CAS step that asks halts, the run links nothing and writes nothing: ERROR: CAS did not say whether a table the program reads is in memory or did not list the saved files of its CAS library, so it is not known whether that table is there or whether the log table has a saved file. Nothing was linked and the tables were left as they are. Up to this release such a listing was taken for a library without the file, and the question whether a table is in memory was read without asking whether CAS had refused it. A new table that CAS does not say is there takes the place of nothing; where CAS refused the question the line is ERROR: CAS refused to say whether the new table <caslib>.<table>_NEW is there, so it did not take the place of the table <caslib>.<table>, which was left as it is., and a step that puts the table in place and halts ends the run with SYSCC 1012. The Python of the program takes the answers of table.fetch, table.columnInfo, simple.numRows and table.tableInfo only with the severity 0: a warning of one of the first three ends the link with ERROR: The rows could not be linked, ..., and a label that could not be read gives NOTE: The label of the table <caslib>.<table> could not be read, so the linked table has none. Not run in SAS. A run that wrote an ERROR line and no table - a log table or a catalog that was not found, a question or a listing of saved files that CAS refused or did not answer, a table the program refused, rows that could not be linked, a new table that could not be put in place - sets the condition code of the SAS session (SYSCC) to 1012 as its last statement; a WARNING, a NOTE and a run with nothing to link leave the condition code as it was. In the second run, before the statement existed, the refusal of a table of twelve columns ended with SYSCC 0 and a completed job. In the third run the same refusal and the run without a catalog ended with SYSCC 1012 and a failed job, and a second run with nothing to link and a run with a _lnk_query_window that is no number - a WARNING - as completed with SYSCC 0. As for the parser, the first DATA step behind a refused run does not run until SYSCC is reset (Reading the Job Log).

The SAS log ends with one NOTE per kind of row, for the whole table, and with a fourth one where the table holds rows whose kind is not known - rows of an earlier parser whose model is not in the catalog: NOTE: 3434 rows of no known kind: 0 linked to a model, caller by ambiguous 0, none 3434. (2026-10-13). The three lines of a table without such rows:

NOTE: 248 llm rows: 248 linked to a model, caller by experiment run 32, template 144, optimization 49, judge 13, ambiguous 0, none 10, and 238 linked to a RAG setup.
NOTE: 355 decision rows: 355 linked to a model, caller by experiment run 41, questions and state 310, questions 0, ambiguous 0, none 4.
NOTE: 437 embedding rows: 437 linked to a model, caller by retrieval query 138, ingestion run 3, ambiguous 72, none 224.

The program wrote these three lines on the test system on 2026-10-12 - each with Python-Subprocess - behind NOTE: , as every line the Python hands to the SAS log - for the 1040 rows its parser had made of the collected log, the catalog of 421 rows and the 107 ingestion runs its pull had written that day. The matching of the 1040 rows took 0.09 seconds. The Chain on SAS Viya has the numbers of both runs, and what changed between the two.

The Chain on SAS Viya

The three programs ran one after the other on the test system three times, each time as they were then: on 2026-10-11, on 2026-10-12 with the changes the first run had led to, and later that day with the changes the second run had led to. This section has the first run, then for each defect of the first run what the changed program did in the second, then the second run as a whole with what was changed after it, and then the third run, which measured those changes. What was changed after the third run is the part that is not run in SAS.

On 2026-10-11 the chain was 23 SAS Job Execution jobs of the SAS Studio compute context, all of them on tables of their own in CASUSER, on a redacted collection of 1040 requests. The CAS server of that system runs without workers, and its PROC PYTHON is Python 3.12 with pandas 2.3.3, swat 1.18.1 and requests 2.34.2. The job logs hold the build stamp of the system (V.04.00M0P09082026), not the name of its release. What the parse did is under The parser of this release on SAS Viya and Appending, what the pull did under The Pull on SAS Viya. The link and the joins:

What held

The link. NOTE: 1040 of 1040 rows of <caslib>.<table> were linked against 365 catalog rows in 0.09 seconds. Every row had a model_id. The table kept its 36 columns in their order with their types and labels - only the lengths of the eight text link columns changed -, its rows in their order and the other 27 columns unchanged in every row. It was promoted and saved, and _LNK_STAGE, _LNK_LINKS and <table>_NEW were gone. The program took 4.9 seconds.
Per kind and method, counted in the table: LLM rows - experiment run 32, template 144, none 72; decision rows - experiment run 50, questions and state 301, none 4; embedding rows - retrieval query 239, ingestion run 3, ambiguous 72, none 123.
The callers were right. For all 769 linked rows the caller was compared with the content of the row afterwards, outside of SAS: the questions or the system prompt of the row are those of the caller's template or of a version kept in its runs, the state or the user prompt fits its user template, and no other caller of the catalog fits. 310 decision rows went to one decision template - 168 of them failed requests, linked like the others -, 131 LLM rows to one prompt template, and all 193 LLM rows with the collection of one RAG setup carried that setup in rag_model_id, whatever their match_method.
A second run linked nothing: NOTE: 0 of 1040 rows ... were linked against 365 catalog rows in 0.06 seconds, 1040 rows that have a match_method were left as they are. All nine link columns were unchanged in every row. With _lnk_redo_all = 1 the program linked the 1040 rows again and wrote the same links.
The refusals. A table of twelve columns as the log table: one ERROR message that names the missing columns and the parser, and the table unchanged. A catalog that does not exist: ERROR: The catalog table <caslib>.<table> was not found. Run Get-All-Prompts.sas first, ..., and the log table unchanged. Both jobs ended as completed.
More rows than one read. A table of 3120 rows - the 1040 three times - takes two reads of 2000 rows, with prompts of up to 22559 characters: 3120 of 3120 rows ... in 0.17 seconds, the rows in the same order before and after, and every row with the links its request had in the table of 1040 rows.
Tables that are saved and not in memory. The log table, the catalog and the runs table were loaded from their .sashdat files. The log table was promoted again, the two others stayed tables of the session. That run linked 0 rows, and the program of that day built, promoted and saved the table on every run - which is what promoted it. A run that links no row leaves the table as it is since (below), so a table that was loaded and not linked is now promoted by statements of their own, see What the Program Does to the Table. They ran in the third run.
The joins. The three statements of How the Tables Join ran. Counted: 527 rows join to PROMPT_EXPERIMENTS by caller_model_id, which are the rows with caller_kind prompt or decision; 242 to RAG_SETUPS by caller_model_id, the rows with caller_kind rag; 238 by rag_model_id; all 1040 to the answering rows of the catalog by model_id; and 82 to the row of their experiment run.
In one SAS session - parse, pull, link, link again - the chain gave the same links as the separate jobs, in 47 seconds.
What the programs do in CAS, on this server: the DATA step of the append ran in CAS in several threads; the rows that were numbered with single='yes' and merged back by that number came out in their order, for 1040 and for 3120 rows; and the order of the columns was restored. A CAS server with workers was not tried.

What did not hold on 2026-10-11, and what the changed program did on 2026-10-12. Each point names the change that was made after the first run, and then what the second run measured of it.

The summary broke the SAS log. The three summary lines held semicolons. A line is handed to the SAS log as a statement, so the statement ended at the first semicolon and the rest ran as SAS code: four ERROR 180-322: Statement is not valid or it is used out of proper order. in each of the seven runs that reached the summary, PROC PYTHON stopped, and the job ended as failed although the table had been written, promoted and saved correctly. The NOTE lines ended behind linked to a model; the counts per method stood in the echoed source lines of the log only. The change: every message takes one way on which it is made one text, the summary is worded without a semicolon, and a macro variable is only ever given a whole number. On 2026-10-12: no log of the 35 jobs held ERROR 180-322, the job of the link ended as completed with SYSCC 0, and the three summary lines arrived in full.
An experiment run was given to more than one request. The 82 rows with an experiment_run_id belonged to 70 runs and models: twelve were a second row. Three requests of the first run of a prompt carried the id of its second run, which had sent the same prompts - their logged run times are the ones the tracker holds for the first run -, and nine decision requests that had been sent again carried the id of a run whose tracked run time is not theirs. The third statement of the example therefore showed two requests for that second run and none for the first. The change: a row gets the run whose run_time is its runtime, and a run is given to one row. On 2026-10-12: 73 rows with an experiment_run_id - 32 LLM and 41 decision rows - for 73 runs and models, none of them on two rows. Joined to PROMPT_EXPERIMENTS, the logged run time equalled the run time of the tracker to three decimals in all 73. The three requests of the first run carried the first run, and the nine decision requests that had been sent again were questions and state without a run id. With a copy of the catalog without run_time the program linked 82 rows again, twelve runs and models on two rows, and wrote NOTE: 82 row(s) were linked to an experiment run without a run time to compare, as the catalog holds none for the runs with their prompts. ...
A query embedding was linked through any prompt with its question. 18 query embeddings were linked to a setup whose last prompt with their question had been sent 20 minutes to two hours before them, and 84 embeddings of another setup were made before the first prompt of their collection was sent, up to 20 minutes before it. The rule had no time window. The change: the prompt has to follow within _lnk_query_window seconds. On 2026-10-12: 138 of the 290 query embeddings were linked - 100 to one setup and 38 to the other - where 239 had been. The 18 were none, with match_candidates 0. Of the 84, 83 were none; one, sent 178 seconds before a run of the Prompt Builder with the same text, is still linked (see the second run). With _lnk_query_window = 1800 and _lnk_redo_all = 1 221 were linked, 183 and 38, and with _lnk_query_window = soon the program wrote WARNING: The setting _lnk_query_window is soon, which is no number. 300 is taken in its place. and linked the 138.
The calls of a prompt optimization and of a judge stayed none: 49 and 13 of the 72 LLM rows without a caller. The change: they are linked to their prompt with the methods optimization and judge. On 2026-10-12: 49 and 13, all to that prompt, the judge calls with the ids of the runs 2 to 14 of the prompt, one each. 10 LLM rows stayed none: the health check of the optimization, three browser requests with a system prompt that no kept run holds, and six test calls.
A second run rewrote the table. With 0 rows linked the program still built <table>_NEW, dropped, renamed, promoted and saved: the table had a new modified time and its .sashdat was written again, with the same content. The change: such a run leaves the table as it is. On 2026-10-12: one message, NOTE: 0 of 1040 rows of <caslib>.<table> were linked, as every row has a match_method. The table was left as it is. With _lnk_redo_all = 1 all rows are linked again. The modified time of the table and the size and the time of its .sashdat were the same before and after, and so were the nine link columns of every row.
The label of the table was lost by the link, as it was by the append. The change: the new table gets the label of the log table. On 2026-10-12: the table had the label Extracted LLM Log Data after the link, as before it. Two copies that had no label before the link had none after it, and the link was not run on a table with a label of a site's own - that such a label stays was seen for the append only.
A second start of mysess. In the chain of one session the second run of the link wrote WARNING: A session with the name MYSESS already exists., as the pull behind the parser did. The change: the program asks SESSFOUND and starts lnkMySess only when no session is connected. On 2026-10-12: alone in a fresh SAS session the program started LNKMYSESS and ended it. In one job - parser, pull, parser, pull, link, link - no line said already exists, the job ended as completed without an error, one session with one id was connected after every program and at the end, and the links equalled those of the separate job in 1040 of 1040 rows. Behind a CAS session of the user - parser, pull, link, link - no program started a session of its own, and that session was the only one at the end.
No ingestion link for the setup of the Builder. Its run table does not exist (see The Pull on SAS Viya). Of the 147 document embeddings 3 were ingestion run - they lie inside the one run of a setup that holds its run table alone -, 72 were ambiguous, lying inside runs of run tables that two or three models registered by the step share, and 72 were none. An embedding row carries no collection, so the log does not say which setup those 72 belong to; 60 of them lie in four bursts of that afternoon. The change: the pull writes the runs a ledger still names, which is the newest run of a setup only. On 2026-10-12: RAG_RUNS held one run of that setup, of six seconds, and the log holds no document embedding in those seconds, so no link changed: 3 ingestion run, 72 ambiguous and 72 none as before. Since then the ingestion job of a setup publishes its run table with every run the vector store holds for the collection (see RAG_RUNS under The Tables). That was measured in the third run with a setup of its own, whose four document embeddings were linked to its runs. Whether the store of the setup of this point holds the runs of that afternoon is not known: its ingestion was not run again.

The Second Run, 2026-10-12

The chain of that day - version 2.3 of the LLM - Log Parser step, version 2.1 of the LLM - Get All Prompts step and the link program as it was then - ran as 35 jobs. 31 ended as completed and 4 as failed: the refused append and the run with an engine other than CAS, as they should, one job the compute service started no session for, and one with an error of the test harness; the last two were run again. One of the chains wrote to Public, all others to CASUSER, and every table and file was dropped at the end. The parse is under The parser of this release on SAS Viya, Appending and Requests that run at the same time, the pull under The Pull on SAS Viya, and what each change after the first run did is in the list above. The link as a whole:

What held

The link. NOTE: 1040 of 1040 rows of <caslib>.<table> were linked against 421 catalog rows in 0.09 seconds., the three summary lines quoted under What the Program Does to the Table, and three more NOTEs: 4 prompt(s) or decision template(s) of the catalog have a template with less than 20 characters of fixed text. ..., 8 catalog row(s) of an optimization or a judge name no model, hold less than 20 characters of text or have a window that can not be read. ... and 15 ingestion run(s) have no start or no end and take no part. - the 15 are runs whose finished_at lies about two hours before their started_at. No ERROR and no WARNING, and the job ended as completed with SYSCC 0. The table kept its 36 columns - only the lengths of the eight text link columns changed -, its rows in their order and the 27 columns of the parser unchanged in 1040 of 1040 rows. The program took 5.6 seconds, the job 23.
Per kind and method, counted in the table: LLM rows - experiment run 32, template 144, optimization 49, judge 13, none 10; decision rows - experiment run 41, questions and state 310, none 4; embedding rows - retrieval query 138, ingestion run 3, ambiguous 72, none 224. Against the run of 2026-10-11, by transaction id: 9 decision rows went from experiment run to questions and state, 101 embedding rows from retrieval query to none, 13 LLM rows from none to judge and 49 to optimization; no row had another model_id or rag_model_id.
The links were checked afterwards, outside of SAS. All 730 linked rows were compared with the texts, run times, windows and runs of the catalog by rules written for the check, and none was wrong; a run of the link function outside of SAS on a parse of the same file gave the same nine link columns in 1040 of 1040 rows. All 238 rows without a caller were classed, and by the rules none of them should have been linked. One link is doubtful: a query embedding of 17:02:37, one of a batch of 40 of which the 39 others are none, is linked to a setup through a run of the Prompt Builder that was sent 178 seconds later with the same text. The setup is probably the right one, but what links the row is not a prompt of that request.
The refusal of a table of twelve columns: one ERROR message of about 430 characters, over two log lines, and the table unchanged.
The two table options. With _lnk_promote_table = 0, _lnk_save_table = 0 and _lnk_redo_all = 1 a table that was promoted and saved was still promoted afterwards and its .sashdat was written again. A table of a CAS session of the user, without a file, stayed a table of that session without a file, with its 1040 rows linked, and the session was still connected.
Names that SAS would read as code. With a catalog in which two models carried one name with a quote, a semicolon and %put ERROR: x;, the NOTE about two models of one name showed the quote as a backtick and % as pct, no ERROR was written, the job ended as completed, and model_id was empty in the 6 rows that those two models could have answered.
Run ids that no longer fit. With two linked rows set to the run id 99 and a run without redo: the NOTE of a run that links nothing and WARNING: 2 row(s) that were linked to an experiment run earlier no longer name that run: for 2 the catalog has no run with their id, for 0 the run with their id has another run time than the request. ..., and the table untouched.
In Public. One chain wrote its six tables to Public under names of its own: the rows and columns of the parse and of the pull as in CASUSER, the same three summary lines of the link, all six tables promoted and saved, and dropped again afterwards.
The joins. The three statements of How the Tables Join ran without an ERROR or a WARNING, in 0.3 seconds. Counted: 589 rows join to PROMPT_EXPERIMENTS by caller_model_id, the rows with caller_kind prompt or decision; 141 to RAG_SETUPS by caller_model_id and 238 by rag_model_id; all 1040 to the answering rows of the catalog by model_id; 73 to the row of their experiment run, one request for each run and model; the 13 judge rows to the run they judged; and the 49 optimization rows to the one optimization whose startedAt and finishedAt hold their time.
ERROR and WARNING lines. From the first source line on, the logs of the 33 jobs that count held 41 such lines, all of them lines this page names: the three WARNINGs of the pull in each of its six runs that read and its two ERROR lines for the name of 34 characters, the lines of the refused append and of the engine other than CAS, one WARNING of the link each for a window that is no number, for the run ids that no longer fit and for questions of 30000 brackets, and its ERROR message for the table of twelve columns. Twelve runs of the link wrote none, and no log held 180-322 or already exists.

What did not hold in the second run, and what the programs do since. The second half of each point is a change made after the run; the third run measured it later that day.

A run that was refused left its job completed. The pull's refusal of a table name of 34 characters, with two ERROR lines, and the link's refusal of the table of twelve columns ended with SYSCC 0, and SAS Job Execution reported both jobs as completed, so a schedule saw no failure. The refused append ended with SYSCC 1012 and a failed job, as its DATA step had set it. Since then each of the three programs and both steps sets SYSCC to 1012 as its last statement in front of the macro variables it deletes, for every run that wrote an ERROR line and no table - the value and the place Ingest-Documents.sas has for a failed run. Reading the Job Log, Messages and Limits of the Pull and What the Program Does to the Table name the runs of each program. In the third run eight refusals ended their jobs as failed, and in the code of a flow the end action of the flow behind a refused step did not run.
An engine other than CAS gave four ERROR lines, the fourth about a log that was never read (see Reading the Job Log). Since then such a run ends with its one line and reads no log.
A response that holds a request line gave a second row where one line of text followed the request line: see Requests that run at the same time. This page named only the case with a Request Headers: line. Since then it names both, and a request whose response quotes lines of an earlier or a later time has status_code and failed missing, where it had failed = 1. The second row is still written: the lines do not tell these two cases from a request.
Files whose times were not in file order gave failed = 1 for requests that had been answered, without a NOTE, and in one file a request the numbers and the response of another (same section). Since then the next part of a file begins where the time goes back by more than one second, a line that is older by one second ends no request, and a request the lines of its container end on has failed missing and is counted in a NOTE.
Questions of 2000 opening brackets gave no WARNING. A decision row with such questions got none without a message - the Python of that system (3.12.12) raised no RecursionError at that depth -, and with 30000 brackets the WARNING of The Rules, in Their Order came. The row is none either way, and the program was not changed for it.
Still no ingestion link for the setup of the Builder, see the last point of the list above.

Changed after the second run for what a reading of the programs found - none of it a line of a job log of that run. The third run ran each of them but Load-Fact-Sheets.sas:

The function of the pull that makes a message one text let comment marks, tabs and a parenthesis without its partner through, and three messages of the parser were handed to SAS as they were. They arrived whole on 2026-10-12. The three programs and both steps now carry one function, and each has one place that writes to the SAS log and one that sets a macro variable.
A log table that the link program loaded from its saved file and linked no row of stayed a table of the session of the program. It is now promoted as _lnk_promote_table says, see What the Program Does to the Table.
Load-Fact-Sheets.sas started and ended mySess whatever the SAS session held - behind Log-Parser-Code.sas in one SAS session that is the WARNING of a session that exists, and the end of the session the librefs of the parser are bound to. It now runs in the CAS session that is connected, starts one of its own only when there is none, and ends only that one. Build-RAG-Cost-View.sas does the same with its _rcvSess.
An optimizationId that two entries of one tracker hold gave two rows of the catalog with one key; the later entry now carries a counter. The NOTE for a value that could not be read named options that it does not count.

Not measured in either run: the default table names; a CAS server with workers; a message near 900 characters - the longest was about 430, and SAS wraps a message at about 250 characters onto a second log line -; the NOTE for a label that could not be given, as CAS took the label every time; the steps run from their page in SAS Studio and their output port on the canvas - both ran as the code SAS Studio generates for a flow -; _lnk_utc_offset other than 0 - the timestamps of the log met the windows of the runs that have one -; _lnk_min_literal other than 20; a log table that grows while it is linked; the methods questions and ambiguous for a prompt or a decision, which no row got; the texts of the pulled columns between the program and the step, of which the ids, counts, lengths and hashes were compared; and the LLM Usage Report opened on these tables. Not repeated on 2026-10-12: the link on tables that are saved and not in memory, the table of 3120 rows and the refusal without a catalog. Of the optimization and judge rows ten of each were read on 2026-10-12, with 60 characters of each prompt; the others were checked by rule.

The Third Run, 2026-10-12

The programs as they were after the second run - version 2.4 of the LLM - Log Parser step, version 2.2 of the LLM - Get All Prompts step, Link-Logs-To-Models.sas, and Ingest-Documents.sas with the run history it publishes - ran on the test system later that day in 46 SAS Job Execution jobs: six ingestion runs and 40 jobs of the chain, all on tables of their own in CASUSER. The compute context is reported by the service for the six ingestion jobs and was the argument of the request for the 40 others.

The ingestion job publishes its run tables. Six runs of a setup of its own: see Run history for each of them. After the third the run table held 3 rows, and after both tables had been dropped and a further run 4 - every run the store held.
The link finds those runs. The log of the model containers of 24 hours was collected with the script of this page - 1027 requests, 426 values API_KEY:REDACTED and no other, every header value redacted - and parsed into 1027 rows: 440 embedding, 345 decision and 242 LLM rows, 253 of them failed. In the minutes of the ingestion runs the log holds ten requests to the embedding model: four in document mode and six in query mode, one per run of the job. The setup was made known to the pull by a project and a model in SAS Model Manager that were created through the REST API, not by the Prompt Builder. The pull wrote 111 rows of RAG_RUNS where it had written 107: four for the setup, all with run_source = run table and counts that are numbers, and the key of the table was unique. The link wrote 440 embedding rows: ... retrieval query 138, ingestion run 4, ambiguous 72, none 226, and counted in the table exactly four rows carried ingestion run: the three document embeddings of the first run and the one of the run with a changed document - the chunks_written of the two runs -, each with one candidate, caller_kind rag and the setup as its caller. The six rows in query mode were none.
With the ledger only. The run table was dropped from memory and the ledger kept. The pull wrote NOTE: The run table casuser.<prefix>_RUNS is not in memory. 1 run(s) were taken from the ledger ... and one row with run_source = ledger, the newest run, without counts. A link with _lnk_redo_all = 1 then wrote ingestion run 0, ambiguous 72, none 230: the four document rows were none, with no candidate.
A run that is refused ends its job as failed. Eight refusals - the parser with a table that is no CAS table, a path without sasserver:, a file that is missing and a refused append; the pull with a table name of 34 characters and its step with an output table in WORK; the link with a table of twelve columns and with a catalog that is missing - each ended with SYSCC 1012 and a failed job. A run with a WARNING only, a log of one line feed and a run with nothing to link ended as completed with SYSCC 0. What a refusal does to the statements behind it is under Reading the Job Log and Messages and Limits of the Pull.
The numbers of the second run held. The collected log of 1040 requests: 1040 rows and 36 columns, no message, 437 embedding, 355 decision and 248 LLM rows, 253 failed, status 200 for 787; joined by transaction_id no row differed from a parse with the parser of the second run in any of the 26 compared columns. The pull: 318 rows and 61 columns, 27 and 32, 17 and 26, 107 and 40, 421 and 21, with three WARNINGs and four unique keys. The link: experiment run 32, template 144, optimization 49, judge 13 and none 10 for the LLM rows; 41, questions and state 310 and none 4 for the decision rows; retrieval query 138, ingestion run 3, ambiguous 72 and none 224 for the embedding rows; a second run linked 0 of the 1040 rows.
One job, one CAS session. Parser, pull, parser, pull, link and link in one job gave the same numbers, no line already exists, one session to the end and SYSCC 0 six times.
The steps. Both steps were registered and then updated with PUT /dataFlows/steps/{id}: answered 200 with the id kept, and the stored program equal to the file. As the code SAS Studio generates for a flow, the parser step wrote 1040 rows and 36 columns with _FLW_ACTION_TABLE_ and _FLW_ACTION_END_ in the log, and the pull step its five tables.
A table that was loaded and not linked was promoted with _lnk_promote_table 1 and gone with the session with 0, each with its NOTE - see What the Program Does to the Table. Names that would have been code - a model named with the start of a comment, with quotes, a semicolon, a percent sign and %put - arrived as text in the WARNINGs of the pull and in the NOTE of the link about two models of one name; SAS Model Manager refused a model name with a parenthesis without its partner (HTTP 400), so that case was not measured. A tracker with the id 1 on two entries gave the WARNING and the catalog rows 1 and 1#2.
Build-RAG-Cost-View.sas ran on the run table the job had published: behind a connected CAS session it started none and left that one connected, alone it started and ended one of its own, and it wrote its table of 4 rows and 30 columns both times. Its closing NOTE said RAG_RUN_COST built although the output table had another name; it names the table it wrote since. Measured later that day with one run table and _rcv_out_table set to another name: NOTE: <that name> built in casuser., a table of 15 rows and 30 columns, SYSCC 0, and no RAG_RUN_COST in Public before or after.
With two run tables the program failed, measured that same day: _rcv_runs_tables set to two tables separated by a comma gave ERROR: A character operand was found in the %EVAL function ..., ERROR: Argument 2 to macro function %SCAN is not a number. and ERROR: The macro _RCV_STACK_RUNS will stop executing., and the run still ended with the NOTE that the output table was built in casuser, for a table that did not exist. The change: the list is handed to the macro functions through %superq, the output table is dropped and loaded only behind run tables that were stacked without an error, and the last line is the NOTE only when CAS holds the table with the rows of the run - otherwise a line that starts with ERROR: Build-RAG-Cost-View: and SYSCC 1012. Measured later that day in a second test content root, see Changed after the third run below; since then the program also leaves at the first refusal with one ERROR line, measured on 2026-10-13 (The Pass of 2026-10-13).

Not run in the third run: Load-Fact-Sheets.sas, which had no setting for the names of its tables, so that a run would have replaced the fact sheets of the system. Not provoked: an upload of the ingestion job that fails, a row count that differs, a promote or a save that is refused. Not looked at: what SAS Studio shows for a node of a flow whose step was refused, and a schedule.

A check of the saved logs of this run against its report carried every measurement above and corrected four statements of the report: the number of pods the log was collected from is in no saved output - the collected log falls into seven parts, one per model with requests -; the log holds ten requests to the embedding model in the minutes of the runs, not nine; the tables of the ingestion runs were read back in a compute session and not in jobs; and the stored program of the ingestion job was compared with the file for the first run - for the five others the source lines of the job log are the program.

Changed after the third run. Measured later on 2026-10-12, in a test content root: the settings each program takes only where they are not set (Settings that stay yours); the closing NOTE of Build-RAG-Cost-View.sas, with one run table; and Load-Fact-Sheets.sas, which replaces a table only with a sheet that was read - through the job Load-Accelerator-Tables, with its three table names replaced by test names (The tables of the reports). Build-RAG-Cost-View.sas as it was after its failure with two run tables ran in a second test content root later that day: with one run table it wrote its table of 15 rows and 30 columns, with two its table of 34 rows (19 and 15), also with blanks around the comma, and with a run table that was not there it ended with its ERROR line and SYSCC 1012 and left the table of the run before as it was. Not run in SAS: versions 2.5 and 2.3 of the two steps, which say in their About tabs what a refusal does as it was measured here - they were registered in the test content root, and what the service held equalled the files -, and versions 2.6 and 2.4 that followed them, whose code files ran on 2026-10-13; the counter of an optimizationId that passes over the id of another entry (The Tables); and the reason of a run history that could not be read in the ingestion job.

The Pass of 2026-10-13

On 2026-10-13 the collection and the programs ran once more, in a test content root and with test table names in casuser, each as it was that day: the collection script with its line in front of the lines of every container and without escape_llm_log, and the code files of the folder, which hold the Python of version 2.6 of the LLM - Log Parser step and of version 2.4 of the LLM - Get All Prompts step.

The collection. The script of this page, with only its path and its interval (48 hours) changed, read nine model pods with one container each and wrote nine lines of its own, one in front of the lines of each container, all with one time of collection. The file had 36,307 lines. Its 426 API_KEY: values and its 173 Cookie headers were all REDACTED, and a search of the file found no provider key and no token.
The parse, with and without the lines of the script. Log-Parser-Code.sas read that file and a copy without the nine lines into two tables: 1,024 rows and 36 columns each, SYSCC 0, and no NOTE and no WARNING of the parser. Paired by model, transaction_id, request_id and timestamp, no row differed in any of the 36 columns. failed was 0 for 771 rows and 1 for 253 and missing for none, and status_code was 200 for 771 rows and missing for 253. That file does not show the two readings apart: it gave no NOTE of a request that was running where the lines of its container end.
The pull and the link. Get-All-Prompts.sas wrote its five tables with three WARNINGs - two RAG setups that name no collection, and a model whose properties SAS Model Manager answered with HTTP 500 -, and Link-Logs-To-Models.sas linked 1,024 of the 1,024 rows against 421 catalog rows. The link ran once, and its numbers were read from its NOTE lines.
Build-RAG-Cost-View.sas, as it is since it leaves at the first refusal: with an empty _rcv_runs_tables exactly one ERROR line of the program, no WARNING, no table and SYSCC 1012; with a run table that is not there the line of SAS that says so, one WARNING and exactly one ERROR: Build-RAG-Cost-View: <table> was not built ..., SYSCC 1012; with one run table the NOTE and a table of 15 rows and 30 columns, and with two a table of 34 rows, SYSCC 0.
Load-Fact-Sheets.sas, from a copy with test table names: 30, 16 and 5 rows, each table global and saved, no WARNING, one load per sheet - into <table>_NEW -, then the drop, the promote and the save; a second run gave the same rows (The tables of the reports).

Not run in this pass: version 2.7 of the LLM - Log Parser step and of its code file and the collection script with escape_llm_log, which were written after it; the two steps as steps in any version after 2.4 and 2.2; the counter of an optimizationId; and of Load-Fact-Sheets.sas a drop or a promote that is refused and the request it sends for its folder since then.

The Second Pass of 2026-10-13

Later on 2026-10-13 the collection and the programs ran again, in a test content root and with test table names in casuser, each as it was then: the collection script with escape_llm_log and with sed in the locale of the shell, and the code files of the folder, which hold the Python of version 2.7 of the LLM - Log Parser step.

The collection. The script of this page as it was then, with only its path and its interval (48 hours) changed, read nine model pods with one container each and wrote nine lines of its own, one in front of the lines of each container, all with one time of collection. The file had 30,395 lines. Its 441 API_KEY: values and its 45 Cookie headers were all REDACTED.
The escape. In a copy of that file, four lines of the shape of the script's line were staged inside one response - without a backslash, with one, with two, and behind a carriage return - and sent through the pipe of the script: each had one backslash more afterwards, and the nine lines of the script were as before.
The parse, with and without the lines of the script. Log-Parser-Code.sas read the file and a copy without the nine lines into two tables: 890 rows and 36 columns each, SYSCC 0, and no NOTE and no WARNING of the parser. Paired by model, transaction_id, request_id and timestamp, no row differed in any of the 36 columns. failed was 0 for 559 rows and 1 for 331 and missing for none, and status_code was 200 for 559 rows and missing for 331: the 331 are requests of two models that hold the text of an error and no response. This file does not show the two readings apart either, and it holds no request whose failed is missing: the numbers of the two NOTEs are not measured in SAS.
The staged lines, read back. The copy with the staged lines gave the same 890 rows, different in one value: the response of the request the lines were staged in held the four staged texts, each without the backslash the script had added.
Lines without the script's line in front of lines with it. The copy without the nine lines, with the file that has them appended behind it, gave 1,780 rows - compared as whole rows, the 890 of the one and the 890 of the other. A file that an earlier script wrote was not at hand.
The pull and the link. Get-All-Prompts.sas wrote its five tables with three WARNINGs - two RAG setups that name no collection, and a model whose properties SAS Model Manager answered with HTTP 500 -, and Link-Logs-To-Models.sas linked 890 of the 890 rows against 404 catalog rows.
Load-Fact-Sheets.sas, from a copy with test table names and with the request for its folder: pointed at a folder that is not there, NOTE: 404 Not Found, one WARNING per sheet, no ERROR line and SYSCC 0; with the folder Data of a test root, 30, 16 and 5 rows (The tables of the reports).

Not run in SAS: the step LLM - Log Parser 2.7 through the code SAS Studio generates, and the step LLM - Get All Prompts in any version after 2.2; the counter of an optimizationId; of Load-Fact-Sheets.sas a drop or a promote that is refused, and the program as it ships, whose table names are not settings. The collection script with sed in the C locale and the question Load-Fact-Sheets.sas asks before it loads under <table>_NEW, both changed after this pass, ran in the third pass of that day. No model was asked for a response that holds a line of the shape of the script's line: the lines were staged.

The Third Pass of 2026-10-13

Once more that day, in two new test content roots and with test table names in casuser, what had been changed after the second pass ran: the collection script with sed in the C locale, and the question Load-Fact-Sheets.sas asks before it loads under <table>_NEW. The parser of that pass was the code file of version 2.7.

The collection. The script of this page, with only its path and its interval (48 hours) changed, ran from Git Bash with GNU sed 4.9 in a shell whose locale was en_US.UTF-8: nine model pods with one container each, nine lines of the script with one time of collection, 5,767 lines. Its 82 API_KEY: values were all REDACTED. The file held no Cookie, Authorization or X-CSRF-TOKEN header and no byte that is no UTF-8, so the collection did not exercise those.
The redaction of a byte that is no UTF-8, through the function redact_llm_log taken out of the script: Cookie: abc<FF>secret came out as Cookie: REDACTED and x API_KEY:sk-abc<FF>secret, y as x API_KEY:REDACTED, y (<FF> being the byte 0xFF), also with LC_ALL=en_US.UTF-8 set around the function. The same expressions without LC_ALL=C left the byte and what stands behind it in place, in en_US.UTF-8 and in C.UTF-8.
The parse, with and without the lines of the script. Log-Parser-Code.sas read the file and a copy without the nine lines into two tables: 172 rows and 36 columns each, SYSCC 0 and no WARNING. Paired by model, transaction_id, request_id and timestamp, no row differed in any column. failed was 0 for 90 rows and 1 for 82, and missing for none.
Load-Fact-Sheets.sas, from a copy with three test table names: as it is, 30, 16 and 5 rows, each table global and saved, no WARNING and SYSCC 0. With a global table <table>_NEW made first for one of the three: WARNING: CAS did not answer that there is no table <caslib>.<table>_NEW - tableExists said 2, and 0 is no table. ..., that sheet not loaded and its table and the _NEW table with the rows and the created time of before, the other two sheets loaded again, and SYSCC 0 (The tables of the reports).

Not run in SAS in that pass: the two custom steps of the folder as steps, the pull and the link, the escape of the collection script on staged lines, and the redaction of a Cookie, Authorization or X-CSRF-TOKEN header on collected lines. The collection script is not run on Linux. Version 2.8 of the parser ran in jobs of its own that day, and version 2.9 in the pass of 2026-10-14.

The Pass of 2026-10-14

On 2026-10-14 version 2.9 of the parser ran as its code file and as the program of its step, in SAS Job Execution jobs with test table names in casuser, on copies of a table of release 2.2 and a collected log of 1,024 requests: Appending has what each state wrote.

The pull and the link, once, behind the appends: Get-All-Prompts.sas wrote its five tables with three WARNINGs of the data on the system - two RAG setups without a collection, and a model whose properties SAS Model Manager answered with HTTP 500 -, and Link-Logs-To-Models.sas linked 4,654 of 4,654 rows - the 3,630 of release 2.2 and the 1,024 the parser appended - against 404 catalog rows. The 3,630 earlier rows had the checksum they had before.
Load-Accelerator-Tables with a table _LAT_SAVED of the session: see The tables of the reports.

Not run in SAS in that pass: the collection script, and the two changes of the parser made after it - the ERROR line for a table whose rows can not be counted, and the drop of a table an append loaded for itself. The drop ran in a second pass that day, with the three states once more (Appending); the ERROR line and what version 2.10 changed have not run in SAS.

Late that day, in the eighth pass of the release, the code files as they were then ran once more from a test content root, with test tables in casuser and one CAS session connected throughout: the parser (version 2.10) appended the collected log of 1,024 requests to a copy of the table of release 2.2 (3,630 rows, 4,654 after); Get-All-Prompts.sas wrote its five tables - 312 experiments, 27 optimizations, 8 RAG setups, 5 RAG runs and 404 catalog rows - and the link linked 4,654 of 4,654 rows against that catalog, each with SYSCC 0; a copy of Load-Fact-Sheets.sas with test table names loaded 30, 16 and 5 rows without a WARNING; and Build-RAG-Cost-View.sas built its table of 3 rows from a made-up run table, with the WARNING for the one embedding model without a price that the run table named on purpose. CAS answered every action, so what the refusals of version 2.10 do did not come up. The run before it, with the CAS session that SESSREF named ended while the session of the parser was still connected - so that the program's own cas mySess only warned that the session exists -, showed what a PROC CAS step that halts did to version 2.5 of the pull: each such step stopped with ERROR: There is no server connection to execute the action 'table.caslibInfo'. and ERROR: Execution halted, the program wrote NOTE: The run table <caslib>.<table> is not in memory, its runs are not in <RAG_RUNS>. for each setup - also for a run table of 5 rows in memory -, RAG_RUNS got 0 rows, and at its end the pull ended the parser's session MYSESS, which it had not started. Version 2.6 of the pull and of its step take such a step for no answer, and in that state the parser, the pull, the link, Load-Fact-Sheets.sas, Build-RAG-Cost-View.sas, Load-Accelerator-Tables.sas and the steps LLM - Log Parser 2.12 and LLM - Get All Prompts 2.7 now set SESSREF to their own connected session instead of starting it again; Create-Optimization-Dataset.sas starts _optbuild only where it is not connected, and Optimize-Prompt-DSPy.sas, Ingest-Documents.sas and the RAG steps start a session of their own name as before.

Early on 2026-10-15, in the ninth pass, the code files as they were before that last change ran from a test content root, with test tables in casuser. The parser (version 2.11), each time on a new table: without _log_append_table and with no table it wrote a table of the 6 requests of the test log; at a copy of the table of release 2.2 (3,630 rows) in memory, and at a table of that many rows that was only saved, it wrote ERROR: The table casuser.<table> holds 3630 rows and _log_append_table is not set, so nothing was read and nothing was written - not the table and not its saved file. ..., set SYSCC 1012 and left the table as it was - the saved one not loaded; a table of 0 rows it replaced; _log_append_table = 2 gave the ERROR that 2 is no value of the setting; 1 appended the 6 requests (3,636 rows) and 0 replaced the table with them; and the step did the same with Append new rows to existing table ticked and unticked. Then the state of the run before: the session of the program's own name started first, another session started and the option set to it, and that session ended (cas mySess; cas zzs; options sessref=zzs; cas zzs terminate;). In that state Get-All-Prompts.sas, and the step LLM - Get All Prompts with gapMySess, each wrote the WARNING that the session exists; each PROC CAS step for a run table stopped with ERROR: There is no server connection to execute the action 'table.caslibInfo'. and ERROR: Execution halted and was followed by NOTE: The CAS library <caslib> of the run table <table> is not available in this session, or CAS did not say whether it, the table or its ledger table is there, so its runs are not in <RAG_RUNS>.; the five tables were written, RAG_RUNS with 0 rows. Staged the same way, the link (lnkMySess) stopped at session.sessionId and table.tableExists, wrote ERROR: CAS did not say whether a table the program reads is in memory or did not list the saved files of its CAS library, ..., linked nothing and set SYSCC 1012; and Load-Accelerator-Tables (_latSess) wrote ERROR: Load-Accelerator-Tables: CAS did not say whether there is a table casuser.<table>_NEW, ..., uploaded nothing, named every table in its end check and set SYSCC 1012. Each of the four ended, at its end, the session of its own name that the staging had started. The use of that session instead (options sessref=<own>;) ran in the eleventh pass; a halt of the parser and of Load-Fact-Sheets.sas was not staged.

On 2026-10-16, in the eleventh pass, the code files with that change ran from a test content root, with test tables in casuser - Load-Fact-Sheets.sas and Create-Optimization-Dataset.sas as copies with test table names, the steps LLM - Log Parser 2.12 and LLM - Get All Prompts 2.7 through their programs with their settings as macro variables -, each in three states: no session, with the option on CASAUTO as in a job; the session of the program's own name connected and the option on a session that was ended (cas <own>; cas zzs; cas zzs terminate;); and both connected with the option on the other (cas <own>; cas zzs;). Where the option named the ended session, the parser, Get-All-Prompts.sas, the link, Load-Fact-Sheets.sas, Build-RAG-Cost-View.sas, Load-Accelerator-Tables and the two steps set the option to their own session, wrote no WARNING that the session exists, had no PROC CAS step halted, and left that session connected. In that state the parser appended the collected log of 1,024 requests to a copy of the table of release 2.2 (3,630 rows, 4,654 after); Get-All-Prompts.sas wrote its five tables - 312 experiments, 27 optimizations, 8 RAG setups, 404 catalog rows, and RAG_RUNS with 0 rows, as no run table of the test system was in memory; the link ended with SYSCC 0; and the copy of Load-Fact-Sheets.sas loaded 30, 16 and 5 rows. Where the option named a connected session it stayed on it (the steps did not run in that state), and with no session each program but the parser, which keeps its session, started its own and ended it. Build-RAG-Cost-View.sas ended in each state with its ERROR that a step that prices the runs ended with an error, as the fact sheet tables of Public were saved and not in memory on the test system, and Load-Accelerator-Tables with the ERROR of its end check for the three fact sheet tables the test root lacked. Create-Optimization-Dataset.sas ran clean with no session and where the option named a connected session, and failed where the option named a session that was ended while _optbuild was connected: its proc casutil session=_optbuild; steps wrote ERROR: No active Cloud Analytic Services session., the DATA step could not promote over the table that was not dropped, and the run ended with SYSCC 1012. In that state the program now sets the option to _optbuild; that change has not run in SAS.

Making the Links Exact

The links are made by content because nothing else is logged. What would make them exact is a caller id that the manifested models send themselves: a key in options - the SAS Model Manager id of the prompt or decision template - that the generated score code of the answering model ignores and the container logs with the request, and, for a retrieval, the id of the setup or its collection in the project input of the embedding call, which the RAG runtime sends as the constant rag today. The parser would read both from the Request Data: line and no rule above would be needed for such a request. It is not built: it needs a change of the Prompt Builder and of the RAG runtime, and a new manifest of every model - a prompt, a decision template or a retrieval model that was manifested before sends no id.

The LLM Usage Report

The LLM Usage Report is a ready-made SAS Visual Analytics report. It is part of the transfer package of the content folder and sits in SAS Content > SAS Agentic AI Accelerator > Logging and Monitoring - see Install or upgrade with the transfer package. It is the recommended template for monitoring LLM usage and prompt experimentation in your environment — import it and use it as-is, or as the starting point for your own dashboards. Up to release 2.2 the report was a package of its own, LLM Usage Report.json; that package carried a share and an authorization rule that named two users, and the one package carries neither.

Prerequisites — the tables it reads

The report builds on seven CAS tables, all expected in the Public caslib by default. Load them (promoted and saved) before or after importing so the report has data to bind to:

Table	Produced by
`LLM_LOGS`	Parsing the collected container logs — `Log-Parser-Code.sas` or the LLM - Log Parser SAS Studio custom step, then `Link-Logs-To-Models.sas`
`LLM_FACT_SHEET` / `EMBEDDING_FACT_SHEET` / `DECISION_FACT_SHEET`	`mdb load-facts` (or `Load-Fact-Sheets.sas`) — see Model Definition Builder
`PROMPT_EXPERIMENTS` / `RAG_SETUPS` / `RAG_RUNS`	`Get-All-Prompts.sas` or the LLM - Get All Prompts SAS Studio custom step - see The Experiments of Every Kind

Every metric is a calculated item of the report; the joins and aggregations are data sources of the report too, so no table is written for it:

Model Calls (all kinds) - LLM_LOGS left-joined to the three fact sheets on the model. Cost per call is the input tokens times the token price (plus the output tokens times the output price), or the run time times the price per second for a model priced by the second. Behind the Overview, Decision Models and RAG pages.
LLM Calls - LLM_LOGS left-joined to LLM_FACT_SHEET; every object on it counts the calls of LLMs only (kind is llm, or missing in rows of a release before 2.3.0). Behind the LLM pages.
Templates (experiments and calls) and RAG Setups (ingestion and calls) - the templates of PROMPT_EXPERIMENTS and the setups of RAG_SETUPS, each left-joined to its calls in LLM_LOGS (by the caller the link program found, and by the setup whose collection a prompt carried), the setups also to their ingestion runs in RAG_RUNS. A template or setup without calls is listed as Never called.

The pages:

Page	What it shows
Overview	What LLMs, decision models and RAG are; calls, estimated cost, failed calls and response time; calls per month and cost per 1,000 calls by kind of model
Prompting Overview, LLM Usage Overview, LLM Usage Details, LLM Details	Prompt experiments and LLM calls as before; LLM Usage Details also lists every prompt template with its experiment runs and calls
Decision Models	Calls, confidence, answers below 0.7, failed calls, time and cost; calls by template and model; answers by confidence band; the experiments of each template and model with the accuracy of their first answer against the expected one; every decision template with its calls; the calls
RAG	Grounded LLM calls, collections, chunks and questions embedded, tokens embedded, cost; per setup the ingestion summary (runs, last run, documents, chunks, embedding tokens, failed documents) and its calls; the grounded calls

RAG_RUNS holds the runs of a setup only where its run table was in memory when Get-All-Prompts.sas ran; load the saved <prefix>_RUNS tables first, or a setup shows No ingestion run recorded. The first-answer accuracy compares the answer to the first question of a decision template with the first expected answer.

The report reads the twelve columns LLM_LOGS had before release 2.3.0 by their names, and the link columns of release 2.3.0. A table that a parser of an earlier release wrote, and that was not appended to with the parser of release 2.3.0, lacks them: run the parser of this release and the link program once.

Import the report

mdb builders-import imports the content folder, and the report with it:

mdb builders-import                              # the tables of the report stay bound to Public
mdb builders-import --report-caslib MyLib        # ... or are pointed at the tables of the same name in MyLib

See First install and upgrade for what the command does, and Importing without mdb for SAS Environment Manager and the sas-viya CLI. After the import the report is under SAS Content > SAS Agentic AI Accelerator > Logging and Monitoring > LLM Usage Report. The three fact sheets are loaded by mdb load-facts, or by the job Data/Load-Accelerator-Tables of the content folder (The tables of the reports).

Change the CAS library (default: `Public`)

The report binds each of its seven tables to server=cas-shared-default;library=Public. If your tables live in a different CAS library, repoint them one of two ways:

mdb builders-import --report-caslib <library> points every table of the report at the table of the same name in that library, through the import mapping of the transfer service. The file is not edited. The release table of the Prompt Builder report follows --caslib / SAS_CAS_LIBRARY. The option has not run against SAS Viya.
SAS Environment Manager / SAS Visual Analytics — import the package as-is, then open LLM Usage Report in SAS Visual Analytics and, in the Data pane, use Edit → Change data source on each of the seven tables to point to the same table in your library.

note

The package of earlier releases carried the report under the identifier /reports/reports/1f08db8d-d6c6-4ed1-9684-e7d11c4ec50c — the URI shown as the example SAS_LLM_MODEL_CARD_REPORT_URI and the Prompt Builder Model card report URI. Which identifier the report has on your site after an import of this release — the one of the package, or another where your site had a report of that name — has not been measured. Read it from the report in the folder Logging and Monitoring of the content folder after the import, and use that URI where the report doubles as the model-card custom chart for your registered models and manifested prompts.

Simplified Logging via a Script​

Getting the File to the SAS Compute Server​

Without a Shared Mount​

Running the Parser​

Settings that stay yours​

What the Custom Step Takes from the Session​

Reading the Job Log​

The Columns of LLM_LOGS​

A request that failed​

Appending​

Requests that run at the same time​

The parser of this release on SAS Viya​

What Was Measured​

The Session Check with SESSFOUND​

Not Measured​

The Experiments of Every Kind

What It Reads​

Settings​

The Tables​

The Prompts as They Were Sent​

Messages and Limits of the Pull​

The Pull on SAS Viya​

Linking the Log to the Models

Settings of the Link​

The Rules, in Their Order​

What Cannot Be Linked, and Why​

How the Tables Join​

What the Program Does to the Table​

The Chain on SAS Viya​

The Second Run, 2026-10-12​

The Third Run, 2026-10-12​

The Pass of 2026-10-13​

The Second Pass of 2026-10-13​

The Third Pass of 2026-10-13​

The Pass of 2026-10-14​

Making the Links Exact​