Template

Judge the expensive error separately from the common one

A thousand correct commas do not cancel one wrong recipient.

About 1 min read

Before you use it

Use when
  • An AI tool has an attractive average score but some mistakes would be much more costly than others.

Keep this limit in mind

  • Rare failures can be absent from a small test by chance. Consequence weights are judgments and should not be disguised as measured facts.

An idea to explore.

Try this idea.

Inspect the example, try the idea in your situation, then check your next move.

Keep reading

A template to use

Failure type: [error]. Consequence: [harm]. Observed cases and denominator: [evidence]. Required handling: [gate]. Owner of acceptance: [owner].

Why it matters

Separate error types before deciding whether the system is acceptable. For a classifier, begin with false positives and false negatives; for an assistant, define similarly concrete failure categories. Assign review or blocking rules to consequential failures rather than letting them disappear inside an average.

An example

A routing assistant's harmless category mistakes and messages sent to an unauthorized team are reported separately.

Check your result

The acceptance decision names the critical error counts and rules, not just the overall pass rate.

Save and practise this idea

My learning

Read. Recall. Repeat.

Save this idea and return to it on any device.

Use email
Keep exploring

Connected ideas

You are readingJudge the expensive error separately from the common one

Evidence and sources
Supports

A confusion matrix separates actual classes from predicted classes and exposes different error directions hidden by an aggregate accuracy figure.

The matrix does not determine the harm or acceptability of each error; those require task-specific judgment.

Metrics and scoring: quantifying the quality of predictions · Confusion matrix

All sources (1)
Edition and review

Version 0.1.0

No review details are supplied for this edition.

Read this idea as JSON · Release details