API Documentation¶

Interactive Docs¶

Through the gateway:

/docs
/redoc
/openapi.json
/openapi.yaml

/openapi.json and /openapi.yaml are generated by the gateway from the backend schema plus gateway-owned routes.

Auth¶

When gateway auth is enabled, send one of:

x-flexserv-secret: <token>

or

Authorization: Bearer <token>

Public routes stay open:

/
/healthz/*
/docs
/redoc
/openapi.json
/openapi.yaml

Admin Secret¶

Some gateway administrative behavior uses a separate header:

x-flexserv-admin-secret: <admin-secret>

Current admin-secret protected or admin-augmented APIs:

POST /v1/flexserv/config Requires x-flexserv-admin-secret.
GET /v1/flexserv/config Works without the admin secret, but returns admin-only fields only when x-flexserv-admin-secret is present and valid.

Admin secret source:

If FLEXSERV_ADMIN_SECRET is set, that exact value is used.
If FLEXSERV_ADMIN_SECRET is not set, the gateway generates a fallback random UUID and derives the actual admin token as a base64url string.

Logging behavior:

When FLEXSERV_ADMIN_SECRET is set, the secret value is not printed to logs.
When the fallback is used, the gateway logs the fallback UUID and the actual base64url admin token.

To extract the fallback admin token from container logs:

docker logs <container> 2>&1 | awk -F': ' '/FlexServ admin secret token \(base64url\)/{print $2}' | tail -n1

When FLEXSERV_ADMIN_SECRET is explicitly set, you already know the value, so there is no need to use a base64 tool or parse logs.

Health¶

`GET /healthz/live`¶

Gateway process liveness.

Response body:

OK

`GET /healthz/startup`¶

Reports whether gateway startup work has completed.

Response body:

STARTING
STARTED

`GET /healthz/ready`¶

Reports whether the gateway is ready and the backend probe succeeds.

Response body:

READY
NOT_READY

Route Inventory¶

Public Gateway Routes¶

These routes are served directly by the gateway or deliberately left public through the gateway.

Method	Path	Notes
`GET`	`/`	Gateway landing page
`GET`	`/healthz`	Gateway liveness shortcut
`GET`	`/healthz/live`	Gateway process liveness
`GET`	`/healthz/startup`	Gateway startup status
`GET`	`/healthz/ready`	Gateway readiness status
`GET`	`/openapi.json`	Gateway-generated merged OpenAPI
`GET`	`/openapi.yaml`	Gateway-generated merged OpenAPI
`GET`	`/docs`	Backend Swagger UI exposed through the gateway
`GET`	`/redoc`	Backend ReDoc exposed through the gateway
`GET`	`/health`	Backend health route exposed publicly through the gateway

Authenticated Gateway Routes¶

These routes currently require normal gateway auth by either Authorization: Bearer <token> or x-flexserv-secret: <token>.

Method	Path	Notes
`GET`	`/v1/models`	OpenAI-style model list, optionally enriched by the gateway
`GET`	`/v1/flexserv/owner`	Returns resolved runtime owner
`GET`	`/v1/flexserv/config`	Returns runtime configuration; admin header unlocks extra fields
`POST`	`/v1/flexserv/models/fetch`	Starts model download/import workflow
`POST`	`/v1/flexserv/models/delete`	Starts model deletion workflow
`POST`	`/v1/flexserv/models/metadata/update`	Updates gateway-managed metadata for a private model
`POST`	`/v1/flexserv/models/metadata/fix`	Repairs metadata from Hugging Face for a private model
`POST`	`/v1/flexserv/models/metadata/refresh`	Refreshes stored size/metadata for a private model
`GET`	`/v1/flexserv/models/delete/{req_id}`	Returns delete status for a previous request
`POST`	`/v1/flexserv/models/copy`	Starts copy workflow between managed repositories
`POST`	`/v1/flexserv/models/index/rebuild`	Rebuilds managed model indexes
`POST`	`/v1/flexserv/models/unpack`	Unpacks discovered model archives
`GET`	`/v1/flexserv/models/unpack/status`	Returns unpack status

Admin-Secret Routes¶

These routes use x-flexserv-admin-secret in addition to normal gateway auth.

Method	Path	Requirement
`POST`	`/v1/flexserv/config`	Requires normal gateway auth and `x-flexserv-admin-secret`
`GET`	`/v1/flexserv/config`	Admin secret is optional, but required to see admin-only fields

Transformers Backend Routes¶

These routes are implemented by the in-repo transformers backend. Any route not owned by the gateway is proxied through to the backend.

Method	Path	Notes
`POST`	`/v1/chat/completions`	OpenAI-style chat completions
`POST`	`/v1/completions`	Text completions
`POST`	`/v1/embeddings`	Embeddings
`POST`	`/v1/responses`	OpenAI-style responses API
`POST`	`/v1/audio/transcriptions`	Audio transcription
`GET`	`/v1/flexserv/health`	Backend health endpoint used by the gateway
`GET`	`/v1/flexserv/info`	Backend info summary
`GET`	`/v1/flexserv/resources`	Backend resource and VRAM snapshot
`GET`	`/v1/flexserv/models/loaded`	Gateway-enriched backend runtime inventory of currently loaded models
`POST`	`/v1/flexserv/models/unload`	Unloads models from backend memory
`GET`	`/health`	Backend health route
`GET`	`/docs`	Backend Swagger UI
`GET`	`/redoc`	Backend ReDoc
`GET`	`/openapi.json`	Backend OpenAPI JSON
`GET`	`/openapi.yaml`	Backend OpenAPI YAML

Current Behavior Notes¶

GET /v1/flexserv/models/fetch/{req_id} is currently registered by the gateway but does not call the normal gateway auth hook.
GET /v1/flexserv/models/copy/{req_id} is currently registered by the gateway but does not call the normal gateway auth hook.
The direct transformers backend auth middleware exists in code, but the current backend app wiring does not register it, so the gateway remains the primary auth boundary.

Inference Endpoints¶

These are served by the backend and proxied by the gateway:

POST /v1/chat/completions
POST /v1/completions
POST /v1/embeddings
POST /v1/responses
POST /v1/audio/transcriptions
GET /v1/models

The in-repo transformers backend also exposes direct backend-only utility routes. Through the gateway, /v1/flexserv/models/loaded is intercepted and enriched before being returned:

GET /v1/flexserv/health
GET /v1/flexserv/info
GET /v1/flexserv/resources
GET /v1/flexserv/models/loaded

`GET /v1/models`¶

Default output preserves the standard OpenAI-compatible list shape.

Useful query flags:

complete=true: include gateway metadata
rebuild_index=true: force a synchronous index rebuild before returning

Gateway-added fields in complete mode may include:

dir_name
source
size_on_disk
total_size
pipeline_tag
model_info
modified
is_shared
labeled_task
storage_tier
fs_model_id
is_loaded
estimated_vram_gb_by_dtype

Example:

curl "http://127.0.0.1:8000/v1/models?complete=true&rebuild_index=true"

fs_model_id is the gateway-resolvable identifier for the model, such as FLEX:PRI:Qwen/Qwen3-0.6B, FLEX:PUB:yolo/yolo26n, or HF:Qwen/Qwen3-0.6B. is_loaded is computed by merging the gateway model catalog with the backend loaded-model inventory.

`GET /v1/flexserv/models/loaded`¶

The backend owns the raw runtime cache, but the gateway owns this public endpoint and enriches backend loaded entries with FlexServ identifiers:

{
  "models": [
    {
      "model_id": "Qwen/Qwen3-0.6B",
      "fs_model_id": "FLEX:PRI:Qwen/Qwen3-0.6B",
      "raw_model_id": "/app/models/private/Qwen--Qwen3-0.6B",
      "resolved_model_id": "/app/models/private/Qwen--Qwen3-0.6B",
      "storage_tier": "PRI_MODEL_REPO",
      "kind": "serve"
    }
  ]
}

The backend still reports its exact loaded cache key as raw_model_id; the gateway maps that key back to the friendly model_id and FlexServ fs_model_id whenever the loaded model is from a managed pool.

FlexServ Gateway Model IDs¶

The landing page displays friendly model IDs, but model-bearing gateway requests may use an explicit resolver prefix:

HF:author/model@revision forwards author/model@revision to the backend.
FLEX:PRI:author/model@revision resolves to PRI_MODEL_REPO/author--model--revision.
FLEX:PUB:author/model@revision resolves to PUB_MODEL_REPO/author--model--revision.
Bare author/model@revision checks the private pool first; if missing, it is forwarded unchanged.

Revision is optional. If omitted, the --revision suffix is omitted from the resolved path.

YOLO models use the yolo/ namespace and always resolve to a model.pt weight file:

FLEX:PRI:yolo/yolo26n -> PRI_MODEL_REPO/yolo--yolo26n/model.pt
FLEX:PRI:yolo/yolo26n-seg -> PRI_MODEL_REPO/yolo--yolo26n-seg/model.pt
FLEX:PUB:yolo/yolo26n-obb@main -> PUB_MODEL_REPO/yolo--yolo26n-obb--main/model.pt

The transformers backend does not perform FlexServ path resolution. It consumes the exact model or model_id string forwarded by the gateway.

Model Management Endpoints¶

`POST /v1/flexserv/models/fetch`¶

Downloads one or more models or archives into PUB_MODEL_REPO or PRI_MODEL_REPO.

Supported sources:

HF
YOLO
HTTP
HTTPS
TAPIS

Optional passthrough headers:

X-HF-TOKEN
X-TAPIS-TOKEN
Authorization is also forwarded by default

Example:

curl -X POST http://127.0.0.1:8000/v1/flexserv/models/fetch \
  -H "Authorization: Bearer my-secret" \
  -H "Content-Type: application/json" \
  -H "X-HF-TOKEN: hf_xxx" \
  -d '{
    "requests": [
      {
        "model_id": "hf-internal-testing/tiny-random-gpt2",
        "src": "HF",
        "is_shared": false,
        "labeled_task": "text-generation"
      }
    ]
  }'

YOLO download example:

curl -X POST http://127.0.0.1:8000/v1/flexserv/models/fetch \
  -H "Authorization: Bearer my-secret" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      {
        "model_id": "yolo/yolo26n",
        "src": "YOLO",
        "is_shared": false,
        "labeled_task": "yolo"
      }
    ]
  }'

This stores PRI_MODEL_REPO/yolo--yolo26n/model.pt and writes YOLO metadata with source: "Yolo", labeled_task: "yolo", and pipeline_tag: "object-detection".

Preparing A YOLO Model Package¶

Use this format when you want to bring a YOLO model into a FlexServ model pool from an archive instead of downloading one of the built-in Ultralytics assets.

The archive may be a .zip, .tar.gz, or .tgz. Its root should contain:

author--model/
  model.pt
  model.onnx        # optional
author--model.metadata

The directory name must use the FlexServ model-pool directory convention:

model id: author/model
directory name: author--model
metadata sidecar: author--model.metadata

model.pt is required for YOLO inference. model.onnx is optional and may be included for tooling or future runtime support.

The metadata file is JSON. Example:

{
  "model_id": "author/model",
  "dir_name": "author--model",
  "family": "yolo26",
  "task": "yolo",
  "labeled_task": "yolo",
  "pipeline_tag": "classification",
  "total_size": 28535928,
  "fetched_at": 1778708560,
  "asset_release": "v8.4.0",
  "asset_url": "https://github.com/ultralytics/assets/releases/download/v8.4.0/yolo26l-cls.pt",
  "source": "Yolo"
}

Required fields for a good catalog entry:

model_id: UI-facing model id, usually author/model.
dir_name: on-disk directory name, usually author--model.
task and labeled_task: logical task label. For YOLO inference packages, use yolo.
pipeline_tag: model capability tag used by the gateway to classify the model.
total_size: size in bytes. This is used by pool summaries and estimates.
fetched_at: Unix timestamp.
source: source label shown in the model pool. This may be HF, Yolo, or another brand/source string.

Optional fields:

asset_release
asset_url
family
model.onnx in the model directory

When a metadata sidecar exists, the gateway primarily uses pipeline_tag to classify the model during index rebuild. Keep labeled_task consistent with pipeline_tag; for YOLO packages, labeled_task should normally be yolo.

Accepted task labels:

`task` / `labeled_task`	Typical use
`unknown`	Unknown model type. Allowed for Hugging Face fetches while metadata is incomplete.
`multi-modal-chat`	Vision/audio/video-to-text chat models.
`text-generation`	Chat, completions, text generation, summarization, translation, conversational models.
`embeddings`	Embedding and feature-extraction models.
`ASR`	Automatic speech recognition models.
`yolo`	YOLO object detection, segmentation, pose, oriented boxes, and classification models.

Pipeline tag to task mapping:

`pipeline_tag` values	Classified task
`Any-to-Any`, `Image-Text-to-Text`, `Audio-Text-to-Text`, `Video-Text-to-Text`	`multi-modal-chat`
`Text-Generation`, `Text2Text-Generation`, `Summarization`, `Translation`, `Conversational`	`text-generation`
`Feature-Extraction`, `Sentence-Similarity`, `Image-Feature-Extraction`, `Audio-Classification`	`embeddings`
`Automatic-Speech-Recognition`	`ASR`
`object-detection`, `segmentation`, `pose-detection`, `oriented-boxes`, `classification`	`yolo`

The gateway normalizes case and separators for pipeline tags, so object_detection, Object Detection, and object-detection resolve to the same key. Prefer the exact strings above in package metadata for readability.

Recommended YOLO pipeline tags:

YOLO model kind	`pipeline_tag`
Detect / bounding boxes	`object-detection`
Segmentation	`segmentation`
Pose	`pose-detection`
Oriented boxes	`oriented-boxes`
Classification	`classification`

After placing the archive in a model repository, call:

curl -X POST http://127.0.0.1:8000/v1/flexserv/models/unpack \
  -H "Authorization: Bearer my-secret"

Then rebuild or refresh the model pool if needed:

curl -X POST http://127.0.0.1:8000/v1/flexserv/models/index/rebuild \
  -H "Authorization: Bearer my-secret"

You can also download this package from Tapis Files through the model fetch API by using src: "TAPIS". The request must include model_id, url, tapis_token, and a non-unknown labeled_task such as yolo. For archive packages, the URL should point to the .zip, .tar.gz, or .tgz object in Tapis Files; after fetch completes, run the unpack endpoint above so the archive is expanded into the model pool and then rebuild the index.

curl -X POST http://127.0.0.1:8000/v1/flexserv/models/fetch \
  -H "Authorization: Bearer my-secret" \
  -H "Content-Type: application/json" \
  -d '{
    "requests": [
      {
        "model_id": "author/model",
        "src": "TAPIS",
        "url": "https://{TAPIS_BASE_URL}/v3/files/content/{TAPIS_SYSTEM_ID}/{PATH}/author--model.zip",
        "tapis_token": "tapis_xxx",
        "is_shared": false,
        "labeled_task": "yolo"
      }
    ]
  }'

`GET /v1/flexserv/models/fetch/{req_id}`¶

Returns the persisted fetch status for a previous request.

`POST /v1/flexserv/models/copy`¶

Copies models between PRI_MODEL_REPO and PUB_MODEL_REPO.

Input fields:

source_repo
target_repo
model_ids
overwrite

`GET /v1/flexserv/models/copy/{req_id}`¶

Returns the persisted copy status for a previous request.

`POST /v1/flexserv/models/index/rebuild`¶

Rebuilds model index files for the managed repositories.

`POST /v1/flexserv/models/metadata/update`¶

Updates gateway-managed metadata for a private model.

Behavior:

when a metadata sidecar exists, the request updates the sidecar pipeline tag
when the sidecar is missing, the request updates the fallback task metadata in the model index
task values must stay within the gateway-supported task labels

`POST /v1/flexserv/models/metadata/fix`¶

Attempts to fetch Hugging Face metadata for a private model and write a sidecar metadata file beside the model directory.

This is intended for models whose model_id can be resolved through Hugging Face Hub.

`POST /v1/flexserv/models/metadata/refresh`¶

Refreshes model size metadata for a private model.

Behavior:

if the sidecar already has total_size, that value stays authoritative
if the sidecar is missing total_size, the gateway backfills it from a local directory scan
if no sidecar exists, the gateway updates the index fallback size metadata

`POST /v1/flexserv/models/unpack`¶

Scans managed repositories for archive payloads and unpacks them.

`GET /v1/flexserv/models/unpack/status`¶

Returns the latest unpack status summary.

`GET /v1/flexserv/owner`¶

Returns the resolved FlexServ owner string used by the runtime.

Landing Page¶

GET / returns:

HTML when called by a browser-like client
JSON with endpoint links otherwise

Current model-pool behavior on the landing page:

inference dropdowns are populated from private models only
private model stickers show loaded state with a star badge
private stickers expose single-model actions such as edit, fix metadata, refresh, unload, and delete
the “loaded private model size distribution” bar is metadata-based size visualization, not live RAM/VRAM telemetry

API Documentation¶

Interactive Docs¶

Auth¶

Admin Secret¶

Health¶

GET /healthz/live¶

GET /healthz/startup¶

GET /healthz/ready¶

Route Inventory¶

Public Gateway Routes¶

Authenticated Gateway Routes¶

Admin-Secret Routes¶

Transformers Backend Routes¶

Current Behavior Notes¶

Inference Endpoints¶

GET /v1/models¶

GET /v1/flexserv/models/loaded¶

FlexServ Gateway Model IDs¶

Model Management Endpoints¶

POST /v1/flexserv/models/fetch¶

Preparing A YOLO Model Package¶

GET /v1/flexserv/models/fetch/{req_id}¶

POST /v1/flexserv/models/copy¶

GET /v1/flexserv/models/copy/{req_id}¶

POST /v1/flexserv/models/index/rebuild¶

POST /v1/flexserv/models/metadata/update¶

POST /v1/flexserv/models/metadata/fix¶

POST /v1/flexserv/models/metadata/refresh¶

POST /v1/flexserv/models/unpack¶

GET /v1/flexserv/models/unpack/status¶

GET /v1/flexserv/owner¶

Landing Page¶

`GET /healthz/live`¶

`GET /healthz/startup`¶

`GET /healthz/ready`¶

`GET /v1/models`¶

`GET /v1/flexserv/models/loaded`¶

`POST /v1/flexserv/models/fetch`¶

`GET /v1/flexserv/models/fetch/{req_id}`¶

`POST /v1/flexserv/models/copy`¶

`GET /v1/flexserv/models/copy/{req_id}`¶

`POST /v1/flexserv/models/index/rebuild`¶

`POST /v1/flexserv/models/metadata/update`¶

`POST /v1/flexserv/models/metadata/fix`¶

`POST /v1/flexserv/models/metadata/refresh`¶

`POST /v1/flexserv/models/unpack`¶

`GET /v1/flexserv/models/unpack/status`¶

`GET /v1/flexserv/owner`¶