BankBench 20-Scenario Eval

Gemma 2 9B IT — Safety Results

Per-Scenario Detail

All 20 Scenarios — Turns, Responses & Mech Interp

Click any scenario to expand full turn-by-turn responses and per-scenario mechanistic interpretability.

Mechanistic Interpretability

Internal Representations — All Scenarios

Aggregate analysis of hidden state norms, safety direction probes, and activation patterns.

Eval Awareness Detection

Does Gemma 2 Know It's Being Tested?

Comparing internal representations when evaluation cues are present vs absent.

Anomaly Detection

Flagged Anomalies & Findings

Automatically detected patterns that warrant investigation.

Capabilities Eval

Banking Tool Use — Gemma 2 vs Mistral 7B

Correct tool selection, argument accuracy, and clarification quality across 20 customer interaction scenarios.

Per-Scenario Results