> ## Documentation Index
> Fetch the complete documentation index at: https://emartai.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# OpenAI

> Configure evalflow to run evals against OpenAI models.

Use OpenAI when you want the simplest setup and broad model coverage. It is a good default if your application already calls OpenAI in production, since your evals will run against the same models your users interact with.

## Configure evalflow\.yaml

Add an `openai` block under `providers` and set `default_provider` so evalflow knows which provider to use when you run `evalflow eval` without a `--provider` flag.

```yaml theme={null}
providers:
  openai:
    api_key_env: "OPENAI_API_KEY"
    default_model: "gpt-4o-mini"

eval:
  default_provider: "openai"
```

`api_key_env` is the name of the environment variable that holds your key — evalflow reads the variable at runtime and never stores the key itself.

## Set your API key

```bash theme={null}
export OPENAI_API_KEY="your-key-here"
```

<Tip>Add this line to your shell profile (`~/.bashrc`, `~/.zshrc`, etc.) or a `.env` file so you do not have to re-export it each session. Never commit your `.env` file to version control.</Tip>

## Verify the connection

Run `evalflow doctor` to confirm evalflow can see the key before running any evals:

```bash theme={null}
evalflow doctor
```

```text theme={null}
✓ OPENAI_API_KEY set
```

## Run evals

```bash theme={null}
evalflow eval --provider openai
```

```text theme={null}
Running test cases against gpt-4o-mini...
Quality Gate: PASS
```

If `eval.default_provider` is already set to `openai` in your `evalflow.yaml`, you can omit the `--provider` flag:

```bash theme={null}
evalflow eval
```

## Provider notes

* **Default model:** `gpt-4o-mini`. You can override it by setting `default_model` to any model name your account has access to (for example, `gpt-4o` or `o1-mini`).
* **API key required:** OpenAI requires a paid account or active free-tier credits. Requests without a valid key will fail immediately.
* **Judge model:** By default, evalflow uses Groq as the LLM judge. If you want OpenAI to serve as both the model under test and the judge, update the `judge` block in `evalflow.yaml`:

```yaml theme={null}
judge:
  provider: "openai"
  model: "gpt-4o-mini"
```