# Evals with the CLI

> Create Datasets and Scorers, operate suites and baselines, and automate metered SFT or RL jobs.

The CLI exposes task-oriented commands over the same Platform API used by the Console. JSON input may be inline, read from `@FILE`, or streamed from `@-`. Use files for pinned references and training recipes so code review captures every semantic input.

## Dataset workflow {#dataset-workflow}

```sh
constal evals datasets create support-regression \
  --body '{"displayName":"Support regression","description":"Representative support behavior"}'

constal evals datasets add-case support-regression --body @case.json
constal evals datasets cases support-regression --limit 100
constal evals datasets publish support-regression --body '{}'
constal evals datasets list
```

Draft case writes are idempotent by `caseId`. Publishing creates an immutable version and keeps the draft available for the next version.

## Scorer and Suite workflow {#suite-workflow}

```sh
constal evals scorers create --body @scorer.json
constal evals scorers list

constal evals suites start --body @suite.json
constal evals suites get support-v17
constal evals suites list --limit 50
```

Cancel a job only when its remaining work is no longer useful:

```sh
constal evals suites cancel support-v17
```

Use Suite result hashes—not display IDs—for comparison and baseline operations:

```sh
constal evals compare --body @comparison.json
constal evals baseline --body @baseline.json
```

## Continuous schedules {#continuous-schedules}

```sh
constal evals schedules apply daily-support --body @schedule.json
constal evals schedules get daily-support
constal evals schedules list
constal evals schedules run daily-support --event-id release-2026-08-23
```

Updating or deleting a Schedule uses its current definition hash as `expectedHash`. A manual run's event ID is caller-stable; reuse it after an uncertain response so the platform resolves the same Suite identity.

## Training workflow {#training-workflow}

```sh
constal evals training providers
constal evals training models \
  --provider "$TRAINING_PROVIDER_CRN" \
  --hash "$TRAINING_PROVIDER_HASH"
constal evals training start --body @training.json
constal evals training get support-rl-v2
constal evals training list --limit 50
```

The start response is a durable job reference, not a promise that training has already completed. Poll `get` for authoritative phase, step, usage, cost, retry state, error, checkpoints, and final result. Cancellation is also idempotent:

```sh
constal evals training cancel support-rl-v2
```

A successful job exposes content-addressed checkpoints. Ask the API for exact compatible Gateway revisions, then submit a separate Model creation body:

```sh
constal evals training checkpoint-gateways support-rl-v2 \
  --step 400 \
  --ref "$CHECKPOINT_REF"

constal evals training create-model support-rl-v2 \
  --body @candidate-model.json
```

The Model import is durable and idempotent. It creates a normal Model Resource with training provenance; it does not update an Agent binding.

## Automation rules {#automation-rules}

Choose caller-stable Suite and training IDs, commit request JSON, and retry transport failures with the same inputs. Never generate a new job ID merely because a request timed out. Set explicit concurrency, token ceilings, and `budgetMicroUsd` in every automated workflow. Read paged lists using the returned continuation cursor rather than assuming the most recent page is complete.

Run `constal help evals training start` or [the complete CLI reference](/docs/cli/reference.md) for global authentication, namespace, endpoint, output, and JSON options.