Rating ScalesAuditable Clinical Ratings: Score From the Transcript
Build an auditable clinical rating by tying every scale item to a quoted line, not a global impression. See how span-grounded scoring works.
Read
The Tagaroo Blog
Annotation, done right.
Field notes on annotation: inter-rater reliability, clinical rating scales, qualitative coding, image labeling, and running annotation campaigns that hold up.
98 articles7 topicsClinical scales · inter-rater reliability · qualitative coding
0 matching articles
Highlighted
Fresh from the annotation deskRating Scales
Validated clinician- and self-report instruments—items, anchors, cutoffs, and how to code them from a transcript.
Rating Scales
Rating ScalesBPRS vs PANSS: The Two Standard Psychosis Rating Scales
BPRS vs PANSS compared: item overlap, the positive-negative-general structure, trial usage, and score linking. See which psychosis scale to use.
Read
Rating ScalesDepression Rating Scales Compared: HAM-D, MADRS, PHQ-9
Depression rating scales compared: HAM-D, MADRS, PHQ-9, and content analysis by rater, item count, and change-sensitivity. See which fits your study.
Read
Rating ScalesGAD-7 vs HAM-A: Self-Report vs Clinician-Rated Anxiety
GAD-7 vs HAM-A compared: the 7-item self-report anxiety screen against the 14-item clinician-rated severity endpoint. See which anxiety scale to use.
Read
Rating ScalesTLC vs SAPS Thought Disorder: Coding Disorganized Speech
TLC vs SAPS thought disorder, compared: the 18-item TLC glossary vs the 8-item SAPS positive-FTD subscale, with an overlap grid and when to use each.
Read
Rating ScalesMADRS vs HAM-D: Which Depression Scale Should You Use?
MADRS vs HAM-D compared: items, somatic weighting, sensitivity to change, score conversion, and remission cutoffs. See which depression scale to use.
Read
Rating ScalesThe Brief Psychiatric Rating Scale (BPRS), Explained
The Brief Psychiatric Rating Scale (BPRS) rates psychopathology on 7-point items. See the scoring, the factor structure, BPRS vs PANSS, and how to code it.
Read
Rating ScalesHamilton Anxiety Rating Scale: Psychic vs Somatic HAM-A
The Hamilton Anxiety Rating Scale (HAM-A) is a 14-item clinician measure. See the psychic and somatic items, the scoring, the cutoffs, and how to code it.
Read
Rating ScalesThe GAD-7 Anxiety Scale: Items, Cutoffs & Transcript Coding
The GAD-7 is a seven-item anxiety screen. See the 5/10/15 severity bands, the ≥10 cutoff, and how to code it from an interview transcript.
Read
Rating ScalesHamilton Depression Rating Scale (HAM-D): A Coding Guide
How the Hamilton Depression Rating Scale (HAM-D) is scored and coded: the 17 items, the 0–52 range, remission and response cutoffs, and its known flaws.
Read
Rating ScalesMADRS Explained: 10 Items, Scoring Bands & Remission
MADRS scoring explained: the 10 clinician-rated items, the 0-6 anchors, severity bands, and the response and remission cutoffs. See how it's coded.
Read
Rating ScalesPHQ-9 Scoring Explained: Items, Cutoffs, and Coding
PHQ-9 scoring made clear: what the nine items measure, the 0-3 anchors, the severity bands, the ≥10 cutoff, and how to code a PHQ-9 from an interview.
Read
Rating ScalesYoung Mania Rating Scale (YMRS): Rating Mania from Speech
The Young Mania Rating Scale (YMRS) explained: its 11 items, the four double-weighted items, severity bands, and how to rate mania from interview speech.
Read
Clinical Phenomena
What the constructs actually look like in speech—and how to annotate them consistently.
Clinical PhenomenaEkman vs Plutchik: Which Emotion Model to Label With?
Ekman vs Plutchik for annotation: 6 discrete emotions or 8 with intensities and dyads, and why more categories lower agreement. See which to pick.
Read
Clinical PhenomenaCognitive Distortions List: 10 CBT Patterns, Coded
The cognitive distortions list, explained: all 10 CBT thinking errors with definitions and examples, where the list came from, and why they resist coding.
Read
Clinical PhenomenaEkman's Basic Emotions: The Six, and How to Label Them
Ekman's basic emotions are six universal states: anger, disgust, fear, happiness, sadness, surprise. See why they anchor most emotion datasets.
Read
Clinical PhenomenaPlutchik's Wheel of Emotions: 8 Primaries, Explained
Plutchik's wheel of emotions maps eight primaries into four opposing pairs, with intensity rings and dyads. See how to label emotion in text.
Read
Clinical PhenomenaPoverty of Speech and Alogia: Coding the SANS Subscale
Poverty of speech is the core of alogia. See the four SANS alogia items, how they differ from poverty of content, and how to code each from a transcript.
Read
Clinical PhenomenaPositive Formal Thought Disorder: The SAPS Subscale
Positive formal thought disorder is disorganized speech you can see. Meet the SAPS FTD items, how they differ from the TLC and SANS, and how to code them.
Read
Clinical PhenomenaThe Thought Language and Communication Scale, Explained
The Thought Language and Communication scale (TLC) is Andreasen's glossary of disordered speech. See the items, the reliability data, and how to code them.
Read
Methods & Frameworks
Coding schemes, taxonomies, and protocols for turning language into structured data.
Methods & FrameworksVR-CoDES Coding System: Cues, Concerns, and Responses
How the VR-CoDES coding system codes patient emotional cues versus concerns and the provider response, using the providing-space vs reducing-space axis.
Read
Methods & FrameworksLocalize a Rating Scale: Annotating in a Second Language
Localize a rating scale for coding in a second language: what back-translation validates, what it misses, and how to re-anchor the coding examples.
Read
Methods & FrameworksAnnotation Cost Calculator: Budget and Timeline Math
An annotation cost calculator: turn items, labels per item, time, wage, redundancy, and QA overhead into a labeling budget and timeline. See the math.
Read
Methods & FrameworksHow to Write Annotation Guidelines That Reduce Disagreement
Write annotation guidelines that reduce annotator disagreement: operational definitions, positive and negative examples, and an edge-case decision log.
Read
Methods & FrameworksClinician-Rated vs Self-Report Scales: A Methods Trade-off
Clinician-rated vs self-report scales compared: who rates, the bias each carries, reliability and cost trade-offs, and transcript coding as a third path.
Read
Methods & FrameworksDSM Criteria Annotation: From Diagnosis to Taggable Spans
Turn DSM-5 and ICD-11 criteria into observable, taggable phenomena: a worked MDD-to-PHQ-9 mapping and why you annotate symptoms, not a diagnosis.
Read
Methods & FrameworksChange Talk vs Sustain Talk: Coding Client Language
Change talk vs sustain talk, coded: how the MISC classifies client language, why the balance predicts outcomes, and how to segment it reliably.
Read
Methods & FrameworksMISC vs MITI: Coding the Client vs the Clinician in MI
MISC vs MITI, disambiguated: the MITI codes the clinician's MI fidelity; the MISC codes the client's change and sustain talk. Compare purpose and use.
Read
Methods & FrameworksMITI Motivational Interviewing: A Fidelity Coding Guide
How MITI motivational interviewing coding works: behavior counts, four global scores, and the R:Q ratio and %complex reflections that grade MI fidelity.
Read
Methods & FrameworksHow to Run a Pilot Annotation Round Before You Scale
Run a pilot annotation round before you scale: scope, sample size, double-coding, measuring agreement, and revising the codebook. See the checklist.
Read
Methods & FrameworksRating Scale to Codebook: Operationalize Any Instrument
A step-by-step method to go from rating scale to codebook: turn each item into a code, anchors, examples, and decision rules coders apply consistently.
Read
Methods & FrameworksThe Empathic Communication Coding System, Explained
The Empathic Communication Coding System codes empathy in clinical talk as a two-step sequence: patient opportunities, then a 0–6 provider response ladder.
Read
Methods & FrameworksLabov and Waletzky Narrative Analysis: The Six Parts
Labov and Waletzky narrative analysis breaks an oral story into six parts. See each part, why evaluation is the heart, and how reliably it can be coded.
Read
Methods & FrameworksThe NICHD Protocol: Coding Interviewer Prompt Types
The NICHD protocol classifies a forensic interviewer's prompts from open to suggestive. See the five types, why open prompts win, and how to code them.
Read
Methods & FrameworksThe OPTION Scale: Measuring Shared Decision-Making
The OPTION scale rates how far a clinician involves the patient in a decision. See the behaviors it scores, why real scores are low, and how to code it.
Read
Methods & FrameworksSearle's Speech Acts: The Five Illocutionary Types
Searle's speech acts sort every utterance into five illocutionary types. See the classes, direction of fit, examples, and the tie to dialogue-act tagging.
Read
Methods & FrameworksThe Toulmin Model of Argumentation: The Six Elements
The Toulmin model of argumentation breaks an argument into six functional parts. See the elements, a worked example, and why warrants are hardest to code.
Read
Methods & FrameworksGottschalk-Gleser Content Analysis: Coding Affect in Speech
How Gottschalk-Gleser content analysis scores affect in speech, clause by clause—the seven depression subscales, the weighted coding, and its NLP legacy.
Read
Reliability & Agreement
Inter-rater reliability done honestly—kappa, alpha, ICC, and the study design behind them.
Reliability & AgreementConfidence Interval for Kappa, Alpha, and ICC, Done Right
Why a bare agreement number misleads, and how to build a confidence interval for kappa, alpha, and the ICC. See analytic vs bootstrap methods.
Read
Reliability & AgreementPercent Agreement, Reconsidered: When It's Actually Fine
Raw percent agreement isn't useless. When to chance-correct, when kappa misleads, and when agreement plus a confidence interval is the honest report.
Read
Reliability & AgreementSpan-Level Agreement: Why Kappa Fails, Use F1 and IoU
Cohen's kappa needs a fixed item set that span and NER annotation never has. Measure span-level agreement with pairwise F1 and IoU instead—see how.
Read
Reliability & AgreementWhich ICC to Use: Choosing Model, Type, and Definition
Which ICC to use for continuous ratings: choose the model (1, 2, or 3), single vs average measures, and absolute agreement vs consistency, from one table.
Read
Reliability & AgreementGwet's AC1 and AC2: A Fix for Kappa's Prevalence Paradox
Why Cohen's kappa collapses under skewed prevalence, and how Gwet's AC1 and AC2 chance-correction fixes it—with the formula and a worked example.
Read
Reliability & AgreementKrippendorff's Alpha: Nominal, Ordinal & Missing Data
How to compute Krippendorff's alpha: the observed-over-expected disagreement formula for nominal, ordinal, and missing data, with a worked example.
Read
Reliability & AgreementSample Size for Inter-Rater Reliability: Subjects and Raters
Set the sample size for inter-rater reliability: how many subjects and raters pin kappa or an ICC to a target CI width. See the planning tables.
Read
Reliability & AgreementHow to Report Inter-Rater Reliability: A Methods Template
A fill-in-the-blanks methods template for how to report inter-rater reliability: coefficient, raters, unit of analysis, CI, and software, per GRRAS.
Read
Reliability & AgreementWhich Inter-Rater Reliability Coefficient Should You Use?
Which inter-rater reliability coefficient to use, by data type, rater count, and missing data—a decision guide to kappa, alpha, AC1, and ICC.
Read
Reliability & AgreementCohen's Kappa: How to Compute and Interpret Agreement
How to compute Cohen's kappa and weighted kappa by hand, read the result honestly, and know when to use Krippendorff's alpha, ICC, or Gwet's AC1 instead.
Read
Reliability & AgreementWelcome to the Tagaroo blog: annotation, done right
Field notes on annotation—inter-rater reliability, clinical rating scales, qualitative coding, and running labeling projects that actually hold up.
Read
Tools & Comparisons
Annotation and qualitative-analysis platforms compared, without the marketing gloss.
Tools & ComparisonsRLHF Data Annotation: A Playbook for Preference & Eval
An RLHF data annotation playbook: pairwise preference vs Likert rubrics, reward-model data, eval sets, and rater agreement. See the decision guide.
Read
Tools & ComparisonsRLHF Annotation Tools: Match the Task to the Right Tool
Compare RLHF annotation tools by task—SFT demos, preference ranking, red-teaming, eval rubrics—and find which tool fits each. See the matrix.
Read
Tools & ComparisonsBest Data Annotation Tools in 2026: An Honest Guide
Compare the best data annotation tools of 2026 by open-source status, pricing, modalities, and workforce—one master table, honest picks by use case.
Read
Tools & ComparisonsClinical Transcript Annotation Tools: A Buyer's Shortlist
Which clinical transcript annotation tool fits psychiatric, therapy, or SLP research? Compare options on HIPAA, speaker roles, severity scales, and IRR.
Read
Tools & ComparisonsCVAT vs Labelbox vs V7: Computer-Vision Tools (2026)
CVAT vs Labelbox vs V7 for computer-vision annotation: open-source vs enterprise, auto-labeling, DICOM, and pricing models, compared. See which fits.
Read
Tools & ComparisonsLabel Studio vs Prodigy vs doccano (2026 Comparison)
Label Studio vs Prodigy vs doccano for text annotation: open-source status, active learning, team QA, and pricing model, compared. See which one fits.
Read
Tools & ComparisonsThe Best NVivo Alternatives for Qualitative Research 2026
Looking for an NVivo alternative? Compare the best free, modern, and AI-guided qualitative coding tools by price, platform, and clinical fit.
Read
Tools & ComparisonsOpen-Source vs Commercial Annotation Tools: 2026 Guide
Open-source vs commercial annotation tools: the real trade-offs in total cost, support, compliance, and lock-in, plus a decision framework for your team.
Read
Tools & ComparisonsQualitative Data Analysis Software Comparison (2026)
A fair, four-way qualitative data analysis software comparison of NVivo, ATLAS.ti, MAXQDA and Dedoose: pricing, AI coding and built-in IRR, side by side.
Read
Image & Multimodal Annotation
Labeling beyond text—images, audio, documents, and video, with the quality metrics that fit each modality.
Image & Multimodal AnnotationText Annotation Use Cases: Support, Legal & Survey Text
How teams apply text annotation use cases beyond the clinic: support-ticket intent, legal clause tagging, and survey coding, with span evidence and IRR.
Read
Image & Multimodal AnnotationGround Truth for Medical AI: Reference Standards & the FDA
How ground truth for medical AI is built for FDA review: reference standards, expert panels, MRMC reader studies, and the SegAgree agreement idea.
Read
Image & Multimodal AnnotationAudio Annotation: From Transcription to Labeled Events
Audio annotation turns raw speech into labeled data: transcription, speaker diarization, timestamped events, and prosody tags. Compare the types.
Read
Image & Multimodal AnnotationChest X-ray Annotation: A Worked Example for AI Datasets
A finding-by-finding chest X-ray annotation example: build a CheXpert-style schema, handle uncertainty labels, and pick image-level vs region labels.
Read
Image & Multimodal AnnotationDice vs IoU vs Hausdorff: Segmentation Metrics Explained
Dice vs IoU, the Jaccard index, and Hausdorff distance compared: formulas, the Dice-IoU identity, and when overlap metrics mislead. See which to report.
Read
Image & Multimodal AnnotationDICOM Annotation: De-Identify and Prepare Data for AI
A DICOM annotation and data-prep guide: strip header and burned-in pixel PHI, window images for display, and store labels as DICOM-SEG, SR, or JSON.
Read
Image & Multimodal AnnotationDocument Annotation: Layout, Entities & Key-Value Fields
How document annotation works for ML: layout regions, document entities, key-value pairs, and table structure, plus the datasets. See the tasks.
Read
Image & Multimodal AnnotationGeospatial Image Annotation: Labeling Beyond the Everyday
Geospatial image annotation covers satellite, aerial, and microscopy imagery—where huge rasters, rare classes, and expert-only labels change the rules.
Read
Image & Multimodal AnnotationHow Many Annotators Medical Images Need: A Consensus Guide
How many annotators medical images need depends on clinical risk, from spot-checking low-risk labels to blind multi-reader reads with adjudication.
Read
Image & Multimodal AnnotationImage Annotation Types: Cost vs Precision, Explained
Bounding box, polygon, semantic vs instance vs panoptic masks, or keypoints? Compare image annotation types by cost, precision, and downstream task.
Read
Image & Multimodal AnnotationImage Labeling Guidelines Annotators Can Actually Apply
Write image labeling guidelines annotators actually apply: mutually exclusive vs overlapping classes, visual examples, and clear when-in-doubt rules.
Read
Image & Multimodal AnnotationImage Segmentation Agreement: Dice, IoU, and STAPLE
Why Cohen's kappa fails on pixels, and how Dice, IoU, and the STAPLE algorithm measure image segmentation agreement across annotators. See the math.
Read
Image & Multimodal AnnotationMedical Image Annotation: A Complete Guide for AI Teams
Medical image annotation, end to end: annotation types, who should label, consensus and adjudication, DICOM de-identification, and QA tied to risk.
Read
Image & Multimodal AnnotationModel-Assisted Labeling for Images: Label Less, Cover More
Model-assisted labeling lets a model draft image labels so annotators correct, not create. How pre-labeling and active learning cut work honestly.
Read
Image & Multimodal AnnotationMultimodal Annotation for Foundation Models: One Schema
Annotate aligned text, image, and audio in one schema for foundation models: cross-modal alignment, unified taxonomies, and QA that keeps labels in sync.
Read
Image & Multimodal AnnotationMultimodal Annotation: Linking Images to Their Reports
How to build multimodal image-text annotation datasets: pair reports to images, ground report phrases to regions, and link findings across modalities.
Read
Image & Multimodal AnnotationWhole-Slide Image Annotation: A Guide for Pathology AI
Whole-slide image annotation for computational pathology: the gigapixel challenge, the slide-to-cell label hierarchy, and pathologist-in-the-loop QA.
Read
Annotation Operations & Ethics
Running annotation projects end to end—guidelines, training, pay, quality control, compliance, and the people doing the work.
Annotation Operations & EthicsAnnotation Fatigue: Why Label Quality Drops Over Time
Annotation fatigue erodes label quality within a session. See the vigilance-decrement evidence and the breaks, caps, and catch-trials that hold it steady.
Read
Annotation Operations & EthicsAnnotator Burnout: Causes, Warning Signs, and Prevention
Annotator burnout has clear causes and warning signs. Learn the drivers, symptoms to watch for, and the workload and rotation practices that prevent it.
Read
Annotation Operations & EthicsEvidence Mode vs Instance Mode: Evidence-Based Annotation
Evidence mode rates the whole transcript once and cites its supporting spans; instance mode tags each occurrence. How the choice reshapes reliability math.
Read
Annotation Operations & EthicsEthical Data Annotation: A Practical Fair-Work Standard
Ethical data annotation means fair pay, conditions, contracts, management, and representation. Audit any vendor against the Fairwork standard.
Read
Annotation Operations & EthicsSensitive Content Annotation: A Humane, Practical Protocol
Concrete protections for sensitive content annotation: informed consent, exposure limits, grayscale and blur defaults, and debriefing. Get the protocol.
Read
Annotation Operations & EthicsMulti-Layer Annotation: Stacking Scales on One Transcript
Multi-layer annotation stacks several coding schemes and rating scales on one transcript. Keep layers independent, handle overlaps, score each separately.
Read
Annotation Operations & EthicsRating Scale vs Coding Scheme: Which Does Your Study Need?
Rating scale vs coding scheme: how dimensional severity and categorical coding change your unit of analysis, reliability statistic, and annotation UI.
Read
Annotation Operations & EthicsAnnotation Consensus Methods: Majority Vote to Dawid-Skene
How to turn many annotator labels into one gold label: majority vote, Dawid-Skene, MACE, STAPLE, and expert adjudication, with a table for when each fits.
Read
Annotation Operations & EthicsData Annotation Consent & Licensing: HIPAA, GDPR & IP
Data annotation consent explained: HIPAA de-identification vs a BAA, GDPR special-category data, who owns the labels, and how to license a dataset.
Read
Annotation Operations & EthicsCBCA vs Reality Monitoring: Two Credibility Methods
CBCA vs reality monitoring: how two statement-credibility methods compare on criteria, accuracy, and court limits—and why neither one detects lies.
Read
Annotation Operations & EthicsCriteria-Based Content Analysis: The 19 CBCA Criteria
Criteria-based content analysis (CBCA) rates 19 content criteria to judge whether a statement reads as experience-based—an aid, not a lie test. See all 19.
Read
Annotation Operations & EthicsData Cascades: Why Fixing Labels Beats Tuning Models
Data cascades are compounding downstream failures from upstream data problems. 92% of AI teams hit them—here's why fixing labels beats tuning the model.
Read
Annotation Operations & EthicsAnnotator Disagreement Is Signal, Not Noise to Discard
Forcing one gold label can throw away real information. See why annotator disagreement is signal, not noise, and when to reconcile versus preserve it.
Read
Annotation Operations & EthicsDoes Pay Improve Annotation Quality? What Research Shows
Does pay improve annotation quality? Higher pay lifts speed, participation, and fairness—but rarely accuracy. See what really drives label quality.
Read
Annotation Operations & EthicsExpert vs Crowd Annotation: When Expertise Really Matters
Expert vs crowd annotation: aggregated non-experts match experts on many tasks, but clinical judgment breaks the ratio. Get the decision guide.
Read
Annotation Operations & EthicsHow to Find Label Errors in Dataset Quality Audits
Label errors hide in almost every dataset. Learn how to find label errors in dataset audits with disagreement, model confidence, and confident learning.
Read
Annotation Operations & EthicsGold Questions & Honeypots: Annotation QA That Works
Gold questions, honeypots, and attention checks catch bad annotations—if thresholds spare legitimate disagreement. See how each QA method works.
Read
Annotation Operations & EthicsHow Many Annotators Per Item? Redundancy and the Cost Curve
How many annotators per item do you need? One label is risky. Compare majority vote, Dawid-Skene, and MACE, and see where extra labels stop paying off.
Read
Annotation Operations & EthicsThe Human Cost of Data Labeling: Ghost Work Behind AI
The human cost of data labeling: the underpaid, invisible ghost workers behind AI, the toll of the work, and what responsible sourcing looks like.
Read
Annotation Operations & EthicsWho Should Annotate Your Data? A Selection Framework
Who should annotate your data? Match annotator background to the task—expert, trained non-expert, or crowd—using a clear selection framework and matrix.
Read
Annotation Operations & EthicsIRF Classroom Discourse: Coding Teacher-Student Talk
IRF classroom discourse is the three-move Initiation-Response-Feedback exchange behind most teacher talk. Learn to code it and read the F move.
Read
Annotation Operations & EthicsImprove Annotator Quality: Screen and Train, Don't Filter
To improve annotator quality, screen candidates for aptitude and train them before coding—filtering bad labels later loses. See the evidence.
Read
Annotation Operations & EthicsSynthetic Data vs Human Annotation: When to Use Which
Synthetic data vs human annotation: when LLM-generated labels are cheap and good, where they miss the tail, and how to build a hybrid you can trust.
Read
Nothing matches that search
Try a different term—a scale, a method, or a tool name.
Tag anything. Guided by AI.
Turn these field notes into practice. Annotate transcripts against validated scales, with an agent that learns your codebook and keeps raters in agreement.
Try Tagaroo free