STAT
STAT / Experiments / SessionBench: eight-turn sessions

SessionBench: eight-turn sessions

Four analysis sessions of eight turns, with a standing instruction before the first turn and a closing instruction after the last. BC-1 and BC-2 use two Xenium sections of one breast cancer block; CRC-1 and CRC-2 a Visium HD and a Xenium section of one colorectal cancer patient. Regions were drawn through STAT's viewer.

Four runs per session: Run 1 with Claude Sonnet 4.6, Runs 2–4 with DeepSeek V4.1 Flash (seeds 10, 20, 30). A Claude Opus 5.5 rater rated each of the eight core turns 1–5 and the session as a whole 1–5.

16 runs
ConditionRunRating
BC-1DCIS versus invasive breast cancer
STATRun 1 · Claude Sonnet 4.6turns 4.38 · session 4/5Replay →
STATRun 2 · DeepSeek V4.1 Flash · seed 10turns 4.75 · session 5/5Replay →
STATRun 3 · DeepSeek V4.1 Flash · seed 20turns 4.75 · session 5/5Replay →
STATRun 4 · DeepSeek V4.1 Flash · seed 30turns 4.88 · session 5/5Replay →
BC-2Immune exclusion and T-cell exhaustion
STATRun 1 · Claude Sonnet 4.6turns 4.88 · session 5/5Replay →
STATRun 2 · DeepSeek V4.1 Flash · seed 10turns 5.00 · session 5/5Replay →
STATRun 3 · DeepSeek V4.1 Flash · seed 20turns 4.88 · session 5/5Replay →
STATRun 4 · DeepSeek V4.1 Flash · seed 30turns 4.88 · session 5/5Replay →
CRC-1Goblet-cell changes near colorectal tumours
STATRun 1 · Claude Sonnet 4.6turns 4.75 · session 5/5Replay →
STATRun 2 · DeepSeek V4.1 Flash · seed 10turns 5.00 · session 5/5Replay →
STATRun 3 · DeepSeek V4.1 Flash · seed 20turns 5.00 · session 5/5Replay →
STATRun 4 · DeepSeek V4.1 Flash · seed 30turns 5.00 · session 5/5Replay →
CRC-2Tumour-associated macrophages
STATRun 1 · Claude Sonnet 4.6turns 5.00 · session 5/5Replay →
STATRun 2 · DeepSeek V4.1 Flash · seed 10turns 4.50 · session 4/5Replay →
STATRun 3 · DeepSeek V4.1 Flash · seed 20turns 5.00 · session 5/5Replay →
STATRun 4 · DeepSeek V4.1 Flash · seed 30turns 4.88 · session 5/5Replay →