Model Definition Builder

The Model Definition Builder (mdb) generates complete LLM model definitions from a single manifest file per model, replacing the manual 10-step copy-and-edit process described in LLM Definitions. You pick a provider and a model, answer a few questions, and every asset the framework needs is generated consistently: the score script, inputVar.json/outputVar.json, modelConfiguration.json, options.json, requirements.json, the documentation files and the fact-sheet row.

Generated definitions work with the established registration flow — continue with Register & Publish LLMs exactly as before.

Installation

cd Model-Definition-Builder/cli
python -m venv .venv
.venv/Scripts/activate        # Windows; on Linux/macOS: source .venv/bin/activate
pip install -e ".[viya]"     # keep the quotes; the [viya] extra adds sasctl for the Viya commands

Requires Python 3.10 or newer. Without the [viya] extra mdb still adds, generates, validates and tests definitions; setup, register, publish, ship, load-facts, options-* and credentials-* need it. For air-gapped environments, download the dependency wheels on a connected machine and install with pip install --no-index --find-links <wheel-dir> sas-mdb.

Adding a model

New to mdb? The Model Definition Builder user guide is a step-by-step walkthrough of creating, checking, iterating on and registering a model (including embeddings and self-hosted Ollama/vLLM servers). This section is the reference summary.

Run mdb add from anywhere inside the repository for the interactive wizard, or use the non-interactive form in scripts and pipelines:

# Interactive: pick provider and model from a live (or bundled) catalog
mdb add

# Non-interactive examples
mdb add openrouter deepseek/deepseek-v3.1 --yes
mdb add anthropic claude-sonnet-4-5-20250929 --id claude_sonnet_4_5 --yes
mdb add azure-foundry --resource myres --deployment my-gpt41 --id gpt_41_az --yes
mdb add hf-selfhosted --repo Qwen/Qwen2.5-0.5B-Instruct --id qwen_25_05b --params-billions 0.5 --yes

Supported providers: OpenRouter, OpenAI, Azure AI Foundry (any host flavor; GA v1 or legacy api-version endpoint, key auth), Mistral, Anthropic, AWS Bedrock (Converse API - Bedrock API key by default, --auth-variant sigv4 for boto3/IAM shops), Google Gemini, Voyage AI, self-hosted Hugging Face models (transformers for LLMs, sentence-transformers for embeddings) and self-hosted OpenAI-compatible servers (Ollama, vLLM).

Before anything is written, the wizard shows the catalog-derived values — option defaults, metadata (description, context length, dates) and token pricing — for you to confirm or adjust: these steer scoring behavior and cost monitoring, so they should be accepted consciously. Adjusting also lets you rename or drop options (max_tokens=max_completion_tokens renames, -top_p drops) — useful when a model's provider contract differs from the classic parameter names; where the catalog's supported_parameters reveal the right token-limit name, the wizard picks it automatically. --accept-defaults skips the review (and --yes implies it); unknown token pricing is always asked about (or warned about in non-interactive runs).

Embedding definitions work exactly like LLM definitions: the wizard picks the kind from the model you select (or the runtime you choose for self-hosted models), the folder lands in Embedding-Definitions/, the row goes to embedding_fact_sheet.csv, and registration continues with mdb register, exactly like an LLM. The generated embedding scorers return all three declared outputs (embedding, run_time, tokens) and embed the full vector — two long-standing bugs in several hand-written definitions that the templates fix centrally. The bundled open-source embedding models run on CPU: all_minilm_l6_v2, the BGE family, embedding_gemma_300m and the RTEB-leaderboard IBM Granite models granite_embedding_small_r2 (47M, 384-dim) and granite_embedding_r2 (149M, 768-dim), all Apache-2.0/MIT ModernBERT or MiniLM bi-encoders.

Self-hosted Ollama and vLLM

mdb add ollama and mdb add vllm create definitions that call a self-hosted OpenAI-compatible server, for either chat (--kind llm) or embeddings (--kind embedding). Unlike the hf-selfhosted path — which downloads weights into the SCR image — these keep the weights on the inference server and the container only carries the thin api-wrapper requirements. The server base URL is environment-neutral, resolving per call as option → OLLAMA_BASE_URL / VLLM_BASE_URL container environment variable → a localhost default, and an optional bearer token is read from OLLAMA_API_KEY / VLLM_API_KEY (leave blank for an open server). So one published image can point at a dev, test or production inference server without a rebuild.

# chat model served by a local Ollama, addressed by its Ollama tag
mdb add ollama llama3.1:8b --id llama31_8b_ollama --kind llm --yes
# embedding model served by vLLM, with the server URL baked as the default
mdb add vllm intfloat/e5-mistral-7b-instruct --id e5_mistral_vllm --kind embedding \
  --base-url http://vllm-svc.ml.svc:8000/v1 --yes

AWS Bedrock regions work like Azure resources: the region resolves per call as option → AWS_BEDROCK_REGION container environment variable → baked default, so one image can serve multiple regions.

Azure across subscriptions and projects

Azure definitions are environment-neutral by default: the resource host you enter during mdb add is used for smoke tests, but is not baked into the definition unless you confirm it (or pass --commit-resource). A deployed container resolves its target as AZURE_OPENAI_RESOURCE container environment variable → the baked default — where a container sends its requests is a property of the deployment, never a scoring option, so nothing a caller passes can redirect it (an optional AZURE_OPENAI_ENDPOINT replaces the built URL for a gateway in front of Azure). mdb deploy renders the Azure Deployment YAML, with that env block, for every Azure definition. Set AZURE_OPENAI_RESOURCE in each deployment's YAML to serve dev, test and production (or different customer subscriptions) from the same published image — and set it in your .env so the wizard and smoke tests use your environment automatically. mdb validate reminds you when a definition carries a baked resource.

Host flavors and API styles. Azure hands out the same OpenAI-compatible data plane under three host suffixes, and mdb accepts any of them verbatim (a bare name still expands to the classic *.openai.azure.com):

Host	Resource type
<res>.openai.azure.com	classic Azure OpenAI resource
<res>.cognitiveservices.azure.com	Azure AI Services / Foundry resource
<res>.services.ai.azure.com	Azure AI Foundry endpoint (sometimes region-qualified)

By default mdb calls the GA v1 endpoint (/openai/v1/chat/completions, deployment name in the body — Microsoft's recommended surface). If your resource or org policy still requires the legacy deployment-scoped route — recognizable by URLs like …/openai/deployments/<name>/chat/completions?api-version=2025-01-01-preview — answer the wizard's API version question (or set the AZURE_OPENAI_API_VERSION container environment variable) and the calls switch to that route. The reverse works too: a definition that bakes a version is forced back onto the GA route by setting AZURE_OPENAI_API_VERSION to an empty value in the container — a v1-only resource answers the legacy route with a bare 401 and no body, which is the symptom to look for. Embedding deployments work the same way: mdb add azure-foundry --kind embedding --deployment <name> builds on the emb_azure_openai_v1 template (/openai/v1/embeddings or the legacy route, api-key header, dimensions supported).

Not just OpenAI models. The whole Foundry Models catalog is served through this same OpenAI-compatible surface — a DeepSeek, Llama, Mistral, Phi or Grok deployment on a Foundry resource works exactly like a GPT one (mdb addresses the deployment name, not the vendor); Microsoft's older separate Model Inference API (/models/chat/completions) is deprecated in favor of it. The boundaries are API-surface ones: the deployment must serve chat completions — embedding, audio/image/realtime, and Responses-API-only models (/openai/v1/responses, a different request/response shape, e.g. computer-use previews) are outside the chat-completions score contract — and some non-OpenAI models reject specific OpenAI options (e.g. temperature on certain reasoning models), which you control per definition anyway.

API keys are read from environment variables or the .env file at the repository root (for example ANTHROPIC_API_KEY, AZURE_OPENAI_API_KEY). The wizard tells you which variable it used; keys are never written into any generated file.

Keeping your definitions in your own repository

By default mdb reads and writes definitions in the accelerator clone's LLM-Definitions/ and Embedding-Definitions/ folders. If you would rather commit your definitions to your own git repository (instead of carrying them in a fork of the accelerator), point MDB_DEFINITIONS at your repo in its .env:

# .env in your own repository
MDB_REPO=/path/to/sas-agentic-ai-accelerator   # supplies the definition-core templates
MDB_DEFINITIONS=/path/to/your-repo             # mdb keeps its layout under this root

mdb loads the .env from your current working directory (and its parents), so running it from inside your repo picks these up automatically. Under that root, mdb creates the familiar layout as needed — LLM-Definitions/, Embedding-Definitions/, and the mdb retire archive _archive/ (add that one to your .gitignore) — and every command (add, apply, generate, validate, register, list, retire, …) operates there, including the fact sheets (llm_fact_sheet.csv / embedding_fact_sheet.csv) inside the definition folders. The accelerator clone is still required for the templates, which is what MDB_REPO points at.

After adding a model, work through the pipeline in order — each step catches a different class of problem before the next one can be blamed for it:

mdb validate <model_id> --live     # coherence rules + one real provider call through the adapter
mdb test <model_id>                # run the GENERATED scoreModel() locally - exactly what SCR will execute
mdb register <model_id>            # register in SAS Model Manager
mdb publish <model_id>             # publish to the SCR destination (or mdb ship for register + publish)

mdb validate --live proves the provider, endpoint and key work; mdb test additionally proves the generated score code works — options parsing, request body, response extraction — so a template or option problem surfaces on your machine, not in a published container.

Register, update and publish from the CLI

mdb now owns the full Viya lifecycle for managed definitions (install the extra: pip install -e "Model-Definition-Builder/cli[viya]"):

mdb setup                            # create the LLM Repository + LLM/Embedding
                                     # projects if missing (idempotent; register
                                     # runs this automatically for its kind)
mdb register <model_id>              # create in SAS Model Manager (skips if it exists)
mdb register <model_id> --update     # replace a registered model IN PLACE: new minor
                                     # version + content replacement + refreshed attributes
mdb publish <model_id> --wait        # publish to SCR and poll until the image build finishes
mdb ship <model_id>                  # validate --live -> register --update -> publish --wait
mdb unregister <model_id>            # delete a registered model from Model Manager
mdb endpoints --json                 # SCR endpoint manifest for CI and testing
mdb options-save                     # save this deployment's builder options to a file
mdb options-restore                  # write them back after importing a report package
mdb builders-import [pkg...]         # import the Builder packages: release table, host,
                                     # data-source binding and saved options in one run
mdb package-export --folder <path>   # export a Builder folder into the repository, host removed

On a fresh environment, mdb setup creates the LLM Repository and the LLM/Embedding Model Projects (idempotent — existing objects are left untouched), and mdb register performs the same check automatically for the kind it registers, so you do not have to run setup explicitly. mdb setup also writes the authorization-group rules (sas-viya-cli-commands.txt) and the llm-prompt-builder.json / rag-builder.json builder seed files — it is the single entry point for bootstrapping the environment. Once the builders are configured, mdb options-save supersedes those seed files: it starts from the same discovered values and overlays what the live reports actually hold, so what you keep is your deployment as configured rather than as bootstrapped — see Preserving builder options across a report import.

--update removes the old delete-and-re-register workaround: after mdb generate, one command refreshes the registered model while keeping its ID, history (a new model version is created) and project placement. Both kinds use one implementation — embedding models register into the Embedding Model Project with the same content roles and fact-sheet enrichment. Each registered model also stores its definition.yaml as model content, so the source of truth travels with the model.

Importing and exporting the Builder packages

mdb builders-import imports the Prompt Builder and RAG Builder transfer packages (the two shipped under SAS-Viya-Integrations/, or the files you name) into the deployment in .env, doing in one run what an import otherwise leaves to hand work afterwards:

1. loads the release table (mdb load-releases; --no-releases skips it),
2. uploads each package and rewrites its import mapping before the job starts: the report's data-source connector is retargeted from the shipped Public.ACCELERATOR_RELEASES to SAS_CAS_LIBRARY.SAS_RELEASES_TABLE on your CAS server, and the placeholder host in the Data-Driven Content URL becomes SAS_VIYA_URL - nothing is edited after the fact,
3. imports and waits for the job, naming the failing task if there is one,
4. configures the imported reports: with --options builder-options.json the site's saved values are written back (see Preserving builder options across a report import); without it the values mdb setup discovers - repository and project ids, SAS_SCR_ENDPOINT, SAS_DEPLOYMENT_TYPE - are; and either way every placeholder host left in a report (the Data-Driven Content URL, the viyaHost and SCREndpoint defaults) becomes SAS_VIYA_URL. A first install needs no edits in Visual Analytics beyond the optional settings.

--dry-run lists the objects and the mapping changes and imports nothing. Other data sources a package binds are left as exported and reported: retargeting a report at a table with different columns breaks its data items.

mdb package-export --folder "/SAS Agentic AI Accelerator/Prompt Builder" is the other direction, for maintainers: it exports the folder (dependencies and rules included, as the shipped packages are), downloads the package, rewrites the exporting environment's hostname to the placeholder inside the compressed report content, checks the result and writes SAS-Viya-Integrations/SAS-Agentic-AI-Accelerator-Prompt-Builder.json. --name, --out, --keep (leave the package on the server) and --exclude <object name> (leave out something that lives in the folder but must not ship, such as a demo report) adjust that. It replaces the three manual steps - export in SAS Environment Manager, copy the file, mdb package-check --fix. (In Git Bash on Windows prefix the command with MSYS_NO_PATHCONV=1, or run it from PowerShell: the shell otherwise rewrites the /SAS Agentic AI Accelerator/... argument into a Windows path before mdb sees it.)

Deployment YAML and CI pipelines

mdb deploy <model_id> --registry myregistry.azurecr.io renders ready-to-apply Kubernetes YAML from the SCR-LLM-Deployment-YAML templates with every placeholder filled (resource names, image, ingress host and path) — the persistent-volume variant is selected automatically for self-hosted models. Model-Definition-Builder/ci-recipes/ contains thin GitHub Actions pipelines (a workflow_dispatch model-lifecycle job and a weekly deprecation radar) where every step is just an mdb verb.

Watching for provider retirements

mdb radar --all checks every managed model against its provider's live catalog; --probe sends one 1-token call per model for ground truth (catalog listing is not an availability guarantee — retired models can stay listed). mdb retire <model_id> tags a definition deprecated (hiding it from the Prompt Builder) and regenerates it.

Keeping definitions consistent

definition.yaml inside the model folder is the only file you edit. After changing it:

mdb generate <model_id>       # re-render the generated files
mdb sync <model_id>           # update the fact-sheet row
mdb validate <model_id>       # cross-file coherence rules with fix-it hints

The *_fact_sheet.csv files (LLM-Definitions/llm_fact_sheet.csv, Embedding-Definitions/embedding_fact_sheet.csv) are a generated artifact — you never edit them by hand. mdb sync <model_id> refreshes a single row; mdb sync --rebuild regenerates each sheet in full from every managed definition (sorted by model_id), creating the file if it does not exist yet:

mdb sync --rebuild            # rebuild both sheets from the definitions
mdb sync --rebuild --prune    # also drop rows for models with no definition folder

Rebuilding is idempotent (an unchanged fleet produces a byte-identical sheet) and preserves any hand-maintained rows that have no definition folder unless you pass --prune. This is the quickest way to bring the fact sheets in line after a bulk migration instead of syncing each model one by one.

To make the sheets available to the SAS Visual Analytics monitoring report, mdb load-facts uploads them to CAS — the Python equivalent of Load-Fact-Sheets.sas. Each sheet is loaded with global scope (promoted, so every session sees it) and saved to the caslib's data source on disk (so it survives a CAS restart), as the tables LLM_FACT_SHEET and EMBEDDING_FACT_SHEET. An existing table of the same name is unloaded first and its saved copy replaced:

mdb load-facts                       # load into the Public library (default)
mdb load-facts --caslib MyLib        # a different CAS library (env: SAS_CAS_LIBRARY)
mdb load-facts --rebuild             # regenerate the sheets from the definitions, then load

The CAS server defaults to cas-shared-default (auto-detected; override with --server or SAS_CAS_SERVER). This uses the casManagement REST API over the same session the register/publish commands use — no separate CAS connection. The save step writes a .sashdat, so target a path-based caslib (like Public); a database-backed caslib would reject the save.

The release table

mdb load-releases turns the repository's CHANGELOG.md into the CAS table ACCELERATOR_RELEASES (same caslib and server options as load-facts; the name comes from --table or SAS_RELEASES_TABLE) and mdb setup loads it by default (--no-releases skips it). It has one row per release - the release intro, change_type = Release - and one per changelog entry, with:

Column	Content
version, release_date	from the ## [x.y.z] - date heading; an Unreleased section is listed but never current
is_current	1 for the newest dated release
release_rank, item_rank	1 = newest release; the entry's position within it, so a report can order without dates
section, change_type	the heading as written and its normalised kind: Added, Changed, Fixed, Removed, Breaking, Release
component	Prompt Builder, RAG Builder, RAG runtime, Model Definition Builder, Credentials, Definitions, Documentation or General - from the section heading when it names one, else from the entry's wording
summary, detail	the entry's bold lead (or first sentence) and the rest, as plain text

It exists because a SAS Visual Analytics Data-Driven Content object needs a data assignment before it renders: the Prompt Builder and RAG Builder objects are assigned this table, and a report author can filter it by component to show what changed for the Builder next to it. mdb load-releases --csv out.csv writes the table without contacting SAS Viya, for a look at the rows.

mdb generate --all --check verifies that every generated file matches its manifest and is intended as a CI gate. Files you edited by hand are never overwritten silently — the command tells you to either fold the change into the manifest, declare the file as hand-maintained under generation.overrides, or pass --force.

Adopting existing definitions

Existing hand-written folders keep working unchanged and are never touched. To migrate one onto a manifest:

mdb import <model_id>             # writes definition.yaml, reports what would change
mdb import <model_id> --apply     # converges the folder onto the generated files

The import reports every intended normalization (canonical options parser, provider usage-based token counting, requirements standardization) so you can review the diff — re-test and re-publish affected models afterwards.

Restricted networks

• --offline (or MDB_OFFLINE=true): no outbound calls; model catalogs come from the bundled snapshots and manual entry
• HTTPS_PROXY / NO_PROXY and REQUESTS_CA_BUNDLE are honored for all provider calls
• --no-verify-ssl / MDB_VERIFY_SSL=false corresponds to the -k option of the existing Python scripts

Custom options

You can declare options that are not part of the standardized vocabulary — useful for provider-specific parameters. Give them an inline type (and a description) in definition.yaml and they are passed to the provider as-is under their own name. mdb generate and mdb validate warn you (rule V010) about what such an option gives up: UIs show it with its raw name and your description instead of a standardized label, and it gets no cross-provider value translation. That is often perfectly fine — the warning just makes it a conscious choice. To standardize an option instead, add it to definition-core/static/option-vocabulary.json with a label, type and per-family mapping. Custom options with informational: true appear in options.json for documentation but are never sent to the provider.

Scoring-time options

Options are defined once in the manifest and flow into the score script defaults, options.json and the fact sheet together. Beyond temperature, top_p, top_k and max_tokens, the typed option vocabulary covers seed, frequency/presence penalties, reasoning_effort for reasoning models (which reject temperature/top_p — the generator handles this), max_completion_tokens and thinking_budget for extended-thinking models. The full vocabulary lives in Model-Definition-Builder/definition-core/static/option-vocabulary.json.