# Continuous AI Monitoring (/docs/concepts/continuous-monitoring)



[AI output evaluation](/docs/concepts/ai-output-evaluation) rates a single run on demand. Continuous monitoring is the always-on version of the same idea: you define a schedule once, and Crowdee periodically samples a recent run, dispatches a crowd evaluation for it, and tracks how the resulting transparency score moves over time — flagging a run when it deviates meaningfully from your schedule's own trailing average.

<Callout type="info">
  A monitoring schedule doesn't introduce a new rating mechanism — every tick dispatches a normal AI output evaluation batch under the hood. Monitoring is the scheduling and drift-tracking layer on top.
</Callout>

## Creating a Schedule [#creating-a-schedule]

```bash
curl -X POST https://api.crowdee.ai/v2/monitoring-schedules \
  -H "X-API-Key: crw_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "sourceType": "verification_pipeline_run",
    "cadence": "daily",
    "sampleCount": 1
  }'
```

| Field                | Required                                    | Description                                                                           |
| -------------------- | ------------------------------------------- | ------------------------------------------------------------------------------------- |
| `sourceType`         | **Required**                                | `"verification_pipeline_run"` or `"lt_pipeline_run"`.                                 |
| `projectId`          | Optional                                    | Scope monitoring to a single project. Omit to monitor across your whole organisation. |
| `cadence`            | **Required**                                | `"hourly"`, `"daily"`, or `"weekly"`.                                                 |
| `sampleStrategy`     | Optional (default `fixed_count_per_period`) | `"all"`, `"random_percent"`, or `"fixed_count_per_period"`.                           |
| `samplePercent`      | Optional                                    | Used with `random_percent`.                                                           |
| `sampleCount`        | Optional (default `1`)                      | Used with `fixed_count_per_period`.                                                   |
| `evaluationCriteria` | Optional                                    | Freeform JSON passed through to each dispatched evaluation batch.                     |

<Callout type="warn">
  **Current limitation**: each tick samples exactly one run — the most recent completed run in scope that this schedule hasn't already evaluated. `sampleStrategy`, `samplePercent`, and `sampleCount` are accepted and stored, but not yet enforced beyond selecting the sampling pool. Sampling more than one run per tick is planned; until then, use a shorter cadence (e.g. `hourly`) if you need tighter coverage.
</Callout>

## What Happens on Each Tick [#what-happens-on-each-tick]

On its configured cadence, a schedule:

1. Looks at completed runs in scope (organisation- or project-wide, matching `sourceType`) from the preceding period that haven't already been sampled by this schedule.
2. If none are found, records a `monitoring_runs` row with `sampledCount: 0` and `status: "completed"` — an empty period is not an error.
3. Otherwise, dispatches an AI output evaluation batch for the sampled run exactly as `POST /v2/projects/{projectId}/ai-output-evaluations` would, blocking credits from your organisation balance the same way.
4. Once the crowd panel reaches quorum, the batch's aggregate transparency score is written back to the monitoring run, along with a `driftFlag` comparing it against the schedule's trailing average of prior completed runs.

```bash
GET https://api.crowdee.ai/v2/monitoring-schedules/{scheduleId}/runs
X-API-Key: crw_YOUR_API_KEY
```

```json
[
  {
    "id": "mrun_01j9x...",
    "periodStart": "2026-07-02T00:00:00.000Z",
    "periodEnd": "2026-07-03T00:00:00.000Z",
    "sampledCount": 1,
    "batchId": "batch_01j9x...",
    "aggregateScore": "62.00",
    "driftFlag": true,
    "status": "completed"
  }
]
```

| Status           | Meaning                                                                                                              |
| ---------------- | -------------------------------------------------------------------------------------------------------------------- |
| `scheduled`      | Tick recorded but not yet processed (transient).                                                                     |
| `sampling`       | A run has been selected, evaluation batch is being dispatched (transient).                                           |
| `awaiting_crowd` | Evaluation batch is live, awaiting crowd responses.                                                                  |
| `completed`      | Scored (or found nothing to sample this period).                                                                     |
| `failed`         | Dispatching the evaluation batch failed — see `error` for details (most commonly insufficient organisation credits). |

## Cost [#cost]

A tick that finds nothing to sample is free. A tick that dispatches an evaluation batch costs the same as a manual [AI Output Evaluation](/docs/concepts/ai-output-evaluation) — **45 credits by default** (15 credits per crowd response × 3 responses) — blocked from your organisation balance at dispatch time. There is no separate monitoring-specific fee.

## Drift Detection [#drift-detection]

`driftFlag` is set when a run's aggregate transparency score deviates from the schedule's own trailing average of prior completed runs by more than a fixed threshold. There's no baseline to compare against on a schedule's first sampled run, so `driftFlag` is always `false` until at least one prior run has completed — drift detection is inherently a relative, self-baselining signal per schedule, not an absolute quality bar.

Use `PATCH` to pause a schedule (`isActive: false`) without losing its history, adjust its cadence, or change what it samples:

```bash
curl -X PATCH https://api.crowdee.ai/v2/monitoring-schedules/{scheduleId} \
  -H "X-API-Key: crw_YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"cadence": "hourly"}'
```

Deleting a schedule (`DELETE /v2/monitoring-schedules/{scheduleId}`) stops future ticks; past monitoring runs and their evaluation batches are unaffected.