Skip to content

HPC Deployment Notes

This page documents the current HPC-related assets in the repository.

There are two different HPC tracks in the repo:

  • the Tapis app/job definitions in deploy/tapis/1.4.0
  • the archive-style runner script in deploy/hpc_archive/app/run_flexserv.sh

There is also older deployer code under deploy/src that still contains older image and entrypoint assumptions. Treat the files documented here as the better source for the current HPC runtime story.

Main Files

Tapis app definition

  • deploy/tapis/1.4.0/app.json

Defines:

  • Tapis app metadata
  • app arguments shown to users
  • scheduler option slots
  • environment variables supplied to the job
  • ZIP runtime bundle URL

Example job payloads

  • deploy/tapis/1.4.0/job.json
  • deploy/tapis/1.4.0/job_with_reservation.json

These are example job submissions that fill in:

  • app args
  • env vars
  • scheduler options

HPC runner script

  • deploy/hpc_archive/app/run_flexserv.sh

This is the most useful script-level reference for what the HPC runtime actually does after a job starts.

app.json Argument Surface

Current user-facing app arguments in deploy/tapis/1.4.0/app.json include:

App arg Purpose
--flexserv-port Requested compute-node service port.
--secret Gateway auth token. If omitted, TAP token flow can provide one.
--model-name Default model name/path passed into the runtime.
--login-port Login-node forwarded port.
--enable-https Turns on HTTPS mode.
--device Backend device hint such as auto, cpu, or cuda-like behavior.
--dtype Backend dtype selection.
--attn-implementation Backend attention backend hint.
--model-timeout Backend inactivity timeout.
--quantization Quantization mode.
--is-distributed Multi-node/distributed mode flag.

Important implementation note:

  • the example app metadata describes the runtime well at a high level
  • the exact behavior still depends on the runner script and image contents

app.json Environment Variables

Current notable env vars in deploy/tapis/1.4.0/app.json:

Variable Purpose
APPTAINER_CACHEDIR Apptainer cache location.
HUGGINGFACE_TOKEN Token for gated Hugging Face downloads.
FLEXSERV_TOKEN Optional fixed gateway token.
FLEXSERV_BACKEND_TYPE Backend type, currently defaulted to transformers.
FLEXSERV_VENV Python venv path in the image.
ENABLE_GATEWAY Gateway-mode enablement.
GATEWAY_BACKEND_PORT Internal backend port.
GATEWAY_PORT Public gateway port.
PRI_MODEL_HOST Host-side private model path.
PUB_MODEL_HOST Host-side public/shared model path.

job.json And job_with_reservation.json

These sample jobs are concrete payloads for the app definition.

Typical knobs you will edit:

  • execSystemId
  • execSystemLogicalQueue
  • maxMinutes
  • scheduler allocation
  • reservation name in the reservation variant
  • APPTAINER_IMAGE
  • model/storage env values

job_with_reservation.json is the same basic flow with an extra scheduler reservation example.

run_flexserv.sh Runtime Flow

Source: deploy/hpc_archive/app/run_flexserv.sh

The script currently:

  1. parses named or legacy positional arguments
  2. detects available GPUs
  3. verifies or rewrites the requested service port if occupied
  4. loads TACC TAP functions
  5. obtains a TAP token and login-node port
  6. prepares model and HF cache directories
  7. configures reverse port forwarding
  8. launches the Apptainer image

In single-node mode it runs:

/app/boot_loader.sh --default-model <MODEL_NAME> --host 0.0.0.0 --port <PORT> --flexserv-token <SECRET>

That means the single-node HPC path uses the same boot-loader based gateway/backend startup model as Docker.

Distributed Mode Status

run_flexserv.sh contains distributed-launch scaffolding, but it currently forces:

IS_DISTRIBUTED=0

before launch.

So in the checked-in script, distributed mode is effectively disabled even if the flag is supplied. Documented distributed options should therefore be treated as scaffolding rather than a currently active feature.

HPC Patching Mechanisms

There are a few ways to patch HPC behavior today.

1. Point the job at a different image

In the sample job payloads, change:

  • APPTAINER_IMAGE

This is the cleanest way to ship a patched runtime when you control the image build.

2. Patch the ZIP app payload

app.json points at a ZIP runtime bundle through containerImage.

If you need to patch launch scripts or auxiliary files shipped with the Tapis app bundle, update that ZIP payload and publish a new bundle.

3. Patch run_flexserv.sh

For archive-style HPC launches, deploy/hpc_archive/app/run_flexserv.sh is the right place to patch:

  • port forwarding behavior
  • Apptainer launch options
  • cache and model mount layout
  • HTTPS/gateway defaults

Important APPLY_PATCH Note

The sample job files include:

APPLY_PATCH=0

However, there is no consumer for APPLY_PATCH anywhere else in the repository at the moment.

So the current implementation state is:

  • APPLY_PATCH exists in sample job payloads
  • it is not wired into run_flexserv.sh
  • it is not wired into the app bundle launch logic in this repo

Treat it as a reserved or future-facing field, not as an active patch hook.

Older HPC Script Caveat

There is another script at:

  • deploy/src/tapisapp/app/flexserv.sh

It still uses the older /app/bin/flexserv entrypoint and --model-hash style arguments. That file is not aligned with the current gateway-plus-boot-loader runtime and should not be used as the source of truth for new deployments.