Use Gemini when you want Google’s latest models with a free tier and generous rate limits during development. It is a natural fit for teams already on Google Cloud, and the free tier is sufficient for routine CI eval runs.
Add a gemini block under providers and set default_provider to gemini:
providers:
gemini:
api_key_env: "GEMINI_API_KEY"
default_model: "gemini-1.5-flash"
eval:
default_provider: "gemini"
api_key_env is the name of the environment variable that holds your key — evalflow reads the variable at runtime and never stores the key itself.
Set your API key
Get a free key at aistudio.google.com, then export it:
export GEMINI_API_KEY="your-key-here"
Add this line to your shell profile (~/.bashrc, ~/.zshrc, etc.) or a .env file so you do not have to re-export it each session. Never commit your .env file to version control.
Verify the connection
Run evalflow doctor to confirm evalflow can see the key before running any evals:
Run evals
evalflow eval --provider gemini
Running test cases against gemini-1.5-flash...
Quality Gate: PASS
eval.default_provider is already set to gemini in your evalflow.yaml, you can omit the --provider flag:
Available models
Set default_model in your evalflow.yaml to any of these model names:
gemini-1.5-flash — fast, low cost, good for CI
gemini-1.5-pro — higher quality, slower
gemini-2.0-flash — latest generation
Provider notes
- Default model:
gemini-1.5-flash. This model is optimized for speed and is well-suited for high-volume CI runs.
- Free tier: Google AI Studio provides a free tier with rate limits that cover most routine eval pipelines.
- Judge model: By default, evalflow uses Groq as the LLM judge. If you want Gemini to serve as both the model under test and the judge, update the
judge block in evalflow.yaml:
judge:
provider: "gemini"
model: "gemini-1.5-flash"
Best for
- Teams already using Google Cloud infrastructure
- High-volume eval runs on the free tier
- Experimenting with Google’s latest model generations