> ## Documentation Index
> Fetch the complete documentation index at: https://emartai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# CircleCI

> Run the evalflow quality gate in CircleCI and block merges when quality regresses.

Add the following to your `.circleci/config.yml` file. It runs evalflow in a dedicated job that fails the workflow when quality regresses.

```yaml theme={null}
version: 2.1

jobs:
  evalflow:
    docker:
      - image: cimg/python:3.11
    steps:
      - checkout
      - run: pip install evalflow
      - run: evalflow doctor --no-provider-check
      - run: evalflow eval

workflows:
  version: 2
  quality-gate:
    jobs:
      - evalflow
```

The `evalflow` job runs in CircleCI's Python convenience image. Adjust the image tag if you need a different Python version.

## Storing API keys

<Warning>
  Never put API keys directly in `config.yml`. Use CircleCI project environment variables to keep secrets out of your repository.
</Warning>

Store your provider key as a project environment variable:

1. Go to your project in CircleCI.
2. Click **Project Settings** → **Environment Variables**.
3. Click **Add environment variable**.
4. Set the name (for example `OPENAI_API_KEY`) and paste your provider key as the value.

CircleCI automatically injects project environment variables into every job. evalflow reads `OPENAI_API_KEY` (or whichever key your `evalflow.yaml` is configured to use) from the environment at runtime.

## How the workflow fails

evalflow exits with a non-zero code when a run does not pass:

```text theme={null}
0  — all evals passed
1  — quality regression detected
2  — setup or provider error
```

CircleCI marks a job as failed when any step returns a non-zero exit code. When evalflow exits with `1` or `2`, the `evalflow` job fails and the `quality-gate` workflow fails.

<Tip>
  Run `evalflow doctor` locally before pushing to confirm your setup is valid. This keeps CI failures focused on model quality rather than configuration problems.
</Tip>