# MITI 4.2.1 benchmark — HELD-OUT TEST split.
#
# Do NOT tune skills against this file. It exists to produce the numbers
# published on /benchmarks, and it is only meaningful while it stays unseen by
# the iteration loop (see dev.yaml for that). If you find yourself editing an
# instance here because the agent got it wrong, stop: either the ground truth is
# genuinely wrong (fix it and say so in the results notes) or you are tuning on
# the test set.
#
# Coverage: all nine behaviour counts appear at least once, every difficulty tier
# is represented, and roughly a quarter of instances are negatives so precision is
# measured rather than assumed.
meta:
  scale: miti
  split: test
  authored: 2026-08-03

instances:
  # --- Giving Information / Persuade boundary -------------------------------
  - id: miti-test-001
    difficulty: easy
    kind: pure
    turns:
      - speaker: subject
        content: "How long does the withdrawal usually last?"
      - speaker: interviewer
        content: "It usually peaks around day three and eases over the week after."
    expected:
      - phenomenon: GI
        turn: 1
        quote: "It usually peaks around day three and eases over the week after."
        severity: 3

  - id: miti-test-002
    difficulty: moderate
    kind: pure
    turns:
      - speaker: subject
        content: "I'm not sure the programme is for me."
      - speaker: interviewer
        content: "Everyone who finishes it does better, so it clearly makes sense for you to enrol."
    expected:
      - phenomenon: PER
        turn: 1
        quote: "Everyone who finishes it does better, so it clearly makes sense for you to enrol."
        severity: 3

  - id: miti-test-003
    difficulty: hard
    kind: pure
    turns:
      - speaker: subject
        content: "My wife thinks I should stop."
      - speaker: interviewer
        content: "There are three options that fit your schedule — an evening group, weekly one-to-one, or an online course."
    expected:
      - phenomenon: GI
        turn: 1
        quote: "There are three options that fit your schedule"
        severity: 2
    note: Options laid out neutrally. GI even though a decision is in the air.

  # --- Simple vs complex reflection ----------------------------------------
  - id: miti-test-004
    difficulty: easy
    kind: pure
    turns:
      - speaker: subject
        content: "Money is really tight at the moment."
      - speaker: interviewer
        content: "There's not much left after the rent."
    expected:
      - phenomenon: SR
        turn: 1
        quote: "There's not much left after the rent."
        severity: 2

  - id: miti-test-005
    difficulty: hard
    kind: pure
    turns:
      - speaker: subject
        content: "I've been handling all of this on my own since March."
      - speaker: interviewer
        content: "It's like carrying a bag you're not allowed to put down."
    expected:
      - phenomenon: CR
        turn: 1
        quote: "It's like carrying a bag you're not allowed to put down."
        severity: 3
    note: Metaphor adds meaning — CR.

  - id: miti-test-006
    difficulty: adversarial
    kind: pure
    turns:
      - speaker: subject
        content: "Honestly I don't see what the fuss is about."
      - speaker: interviewer
        content: "So as far as you're concerned there's no problem here at all."
    expected:
      - phenomenon: CR
        turn: 1
        quote: "So as far as you're concerned there's no problem here at all."
        severity: 3
    note: >
      Amplified reflection — deliberately overstated so the client can push back.
      Must NOT be coded CON; nothing is being disputed with the client.

  # --- Questions -----------------------------------------------------------
  - id: miti-test-007
    difficulty: easy
    kind: pure
    turns:
      - speaker: interviewer
        content: "How many days last week did you drink?"
      - speaker: subject
        content: "Four, I think."
    expected:
      - phenomenon: QUE
        turn: 0
        quote: "How many days last week did you drink?"
        severity: 3

  - id: miti-test-008
    difficulty: moderate
    kind: pure
    turns:
      - speaker: subject
        content: "Things have been all over the place."
      - speaker: interviewer
        content: "I wonder what's kept you going through it."
    expected:
      - phenomenon: QUE
        turn: 1
        quote: "I wonder what's kept you going through it."
        severity: 2
    note: Softly worded but requests information not yet given — QUE.

  # --- MI-adherent ---------------------------------------------------------
  - id: miti-test-009
    difficulty: moderate
    kind: pure
    turns:
      - speaker: subject
        content: "I came in even though I nearly cancelled twice."
      - speaker: interviewer
        content: "Coming in after a week like that took something."
    expected:
      - phenomenon: AF
        turn: 1
        quote: "Coming in after a week like that took something."
        severity: 3

  - id: miti-test-010
    difficulty: hard
    kind: pure
    turns:
      - speaker: subject
        content: "The guideline says one thing but my body says another."
      - speaker: interviewer
        content: "You know your body better than any guideline does — how does it land for you?"
    expected:
      - phenomenon: SC
        turn: 1
        quote: "You know your body better than any guideline does"
        severity: 3
    note: Explicitly defers to client expertise.

  - id: miti-test-011
    difficulty: moderate
    kind: pure
    turns:
      - speaker: subject
        content: "I might leave the medication as it is for now."
      - speaker: interviewer
        content: "That's completely your call."
    expected:
      - phenomenon: EA
        turn: 1
        quote: "That's completely your call."
        severity: 3

  # --- MI-non-adherent -----------------------------------------------------
  - id: miti-test-012
    difficulty: moderate
    kind: pure
    turns:
      - speaker: subject
        content: "I've got it under control."
      - speaker: interviewer
        content: "You're in denial about how bad this has got."
    expected:
      - phenomenon: CON
        turn: 1
        quote: "You're in denial about how bad this has got."
        severity: 3
    note: Labels the client — CON.

  - id: miti-test-013
    difficulty: hard
    kind: pure
    turns:
      - speaker: subject
        content: "I'll sort it out in my own time."
      - speaker: interviewer
        content: "What you need to do is call the clinic tomorrow."
    expected:
      - phenomenon: PER
        turn: 1
        quote: "What you need to do is call the clinic tomorrow."
        severity: 3
    note: Unsolicited directive pressing a decision — PER, not CON (no reproach).

  # --- Multi-code turns ----------------------------------------------------
  - id: miti-test-014
    difficulty: hard
    kind: multi
    turns:
      - speaker: subject
        content: "I did the whole week without a drink."
      - speaker: interviewer
        content: "A full week. That took real effort. What helped most?"
    expected:
      - phenomenon: SR
        turn: 1
        quote: "A full week."
        severity: 1
      - phenomenon: AF
        turn: 1
        quote: "That took real effort."
        severity: 3
      - phenomenon: QUE
        turn: 1
        quote: "What helped most?"
        severity: 3

  - id: miti-test-015
    difficulty: adversarial
    kind: distractor
    turns:
      - speaker: subject
        content: "I keep going round in circles with this."
      - speaker: interviewer
        content: "Going round in circles. It's your decision entirely, and if it helps I can tell you what the evidence says. What would be most useful?"
    expected:
      - phenomenon: SR
        turn: 1
        quote: "Going round in circles."
        severity: 1
      - phenomenon: EA
        turn: 1
        quote: "It's your decision entirely"
        severity: 3
      - phenomenon: QUE
        turn: 1
        quote: "What would be most useful?"
        severity: 2
    note: >
      Four candidate codes in one turn; the offer of information is conditional and
      permission-framed rather than delivered, so GI should NOT fire here.

  # --- Negatives -----------------------------------------------------------
  - id: miti-test-016
    difficulty: easy
    kind: negative
    turns:
      - speaker: interviewer
        content: "Mm-hm."
      - speaker: subject
        content: "Yeah."
    expected: []

  - id: miti-test-017
    difficulty: moderate
    kind: negative
    turns:
      - speaker: subject
        content: "I want to stop before it costs me my job, and I could probably manage a week."
      - speaker: interviewer
        content: "Mm."
    expected: []
    note: >
      Rich CLIENT language (change talk) with nothing codable from the clinician.
      MITI must stay on the clinician's side; MISC is where this content belongs.

  - id: miti-test-018
    difficulty: hard
    kind: negative
    turns:
      - speaker: interviewer
        content: "Let's start at the beginning. I'm going to take some notes as we go, if that's alright — just so I don't lose track."
      - speaker: subject
        content: "Fine by me."
    expected: []
    note: >
      Procedural framing. Tempting to code SC for "if that's alright", but the
      permission concerns note-taking logistics rather than sharing control of the
      conversation's direction or content.

  - id: miti-test-019
    difficulty: adversarial
    kind: negative
    turns:
      - speaker: subject
        content: "My last counsellor told me I was wasting everyone's time."
      - speaker: interviewer
        content: "Mm."
    expected: []
    note: >
      A confrontation is REPORTED by the client, not performed by this clinician.
      Probes whether the skill codes speech acts or mere topic mentions.

  - id: miti-test-020
    difficulty: adversarial
    kind: negative
    turns:
      - speaker: interviewer
        content: "The weather's been miserable, hasn't it? Did you find parking alright?"
      - speaker: subject
        content: "Took a while, but yes."
    expected: []
    note: >
      Social talk in question form. MITI's QUE is intended for questions about the
      client's situation; small talk should not inflate the question count.
