> ## Documentation Index
> Fetch the complete documentation index at: https://emartai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# GitLab CI

> Run the evalflow quality gate in GitLab CI and fail the pipeline on regressions.

Add the following job to your `.gitlab-ci.yml` file. It installs evalflow, validates your setup, and runs the quality gate on every pipeline.

```yaml theme={null}
stages:
  - eval

evalflow:
  stage: eval
  image: python:3.11
  script:
    - pip install evalflow
    - evalflow doctor --no-provider-check
    - evalflow eval
  variables:
    OPENAI_API_KEY: $OPENAI_API_KEY
```

The `variables` block maps a GitLab CI/CD variable to the environment variable evalflow reads at runtime. If you use a different provider, replace `OPENAI_API_KEY` with the variable name that matches your `evalflow.yaml` configuration — for example `GROQ_API_KEY` or `ANTHROPIC_API_KEY`.

## Storing API keys

<Warning>
  Never commit API keys to `.gitlab-ci.yml`. GitLab CI/CD variables keep secrets out of your repository.
</Warning>

Store your provider key as a CI/CD variable:

1. Go to your project in GitLab.
2. Click **Settings** → **CI/CD** → **Variables**.
3. Click **Add variable**.
4. Set the key (for example `OPENAI_API_KEY`) and paste your provider key as the value.
5. Check **Mask variable** to prevent the value from appearing in job logs.

## How the pipeline fails

evalflow exits with a non-zero code when a run does not pass:

```text theme={null}
0  — all evals passed
1  — quality regression detected
2  — setup or provider error
```

GitLab CI marks a job as failed when its script exits with a non-zero code. When evalflow exits with `1` or `2`, the `evalflow` job fails and the pipeline fails — blocking the merge request from merging if you have pipeline success required for merging.

<Tip>
  Run `evalflow doctor` locally before pushing to confirm your setup is valid. This keeps CI failures focused on model quality rather than configuration problems.
</Tip>