AI classification audit: review ten prewritten label pairs

Which label counts as positive in this review?

In this AI classification audit, positive means spam and negative means not spam. Positive does not mean desirable, correct or safe. Fix that label convention before inspecting any result. Otherwise a table can have plausible arithmetic while reversing the meaning of a missed message and a wrongly flagged message.

The reference column is the supplied answer key for a fictional exercise. The proposed column contains prewritten labels to review. No model was run to generate them, and no real email body, sender or address is involved. You are auditing agreement between two given labels, not deciding whether an actual customer message is spam.

Copy a boundary card: “Practice V1; B1–B10; positive 1 is spam; negative 0 is not spam; reference is the supplied key; proposed is prewritten; every ID counts once; neither label column changes during review.” A prediction is not its own reference answer. Google’s classification lesson makes that distinction explicit.

The practical artificial intelligence guide covers other starting tasks. This one produces an evidence ledger containing four outcome counts, three different denominators and decisions on three claims. It neither trains a classifier nor changes a live inbox, and it does not establish how a commercial product performs.

Preserve all ten fictional label pairs

Imagine a small business checking a report format before evaluating any real system. The following labels are intentionally constructed teaching data, not a representative sample. Do not describe the percentages as production estimates or compare vendors using them. All ten rows belong to the exercise, including the mistakes.

Count B1 through B10 once. Keep the reference and proposed column headings attached to their values. B3 and B4 have spam in the key but not-spam in the proposal. B5 has the reverse mismatch. Swapping the column names would change the error interpretation, even if a total agreement count remained unchanged.

Retain this source card and create a separate decision ledger. Do not improve the result by relabelling an inconvenient reference. If a real reference label were disputed, that would be an unresolved evidence question, not permission for the proposed output to mark itself correct. In this complete exercise, the key is stipulated so the counting task is unambiguous.

The zeros and ones are category codes, not quantities to add into a single business total. Their meaning determines the outcome box for each ID. You can write spam and not spam in full if that makes the review easier to inspect.

How do the four outcome boxes differ?

A true positive, TP, has spam in both columns. A true negative, TN, has not spam in both. A false positive, FP, is a not-spam reference incorrectly flagged as spam. A false negative, FN, is a spam reference incorrectly labelled not spam. Keep the direction of the disagreement visible rather than calling both types simply wrong.

The independent counts are TP 2, FN 2, FP 1 and TN 5. Their sum must equal ten. Each source ID occupies exactly one box. If the sum differs, repair the ID ledger before calculating percentages. A second person should be able to recover every count from the original pair of labels, without trusting your summary.

This four-outcome summary is called a confusion matrix. You need no programming to create it. The practical result is a distinction between two missed spam-labelled references and one wrongly flagged not-spam reference. Their real-world costs are not supplied here and must not be invented.

For broader record preparation, use the data analysis workflow. In this exercise, the important source unit is one labelled ID, not a message count, customer count or inferred event.

Why do the three percentages have different denominators?

Accuracy asks how many of all ten proposed labels match the reference. The correct labels are the two TP records and five TN records. Enter 7 divided by 10 multiplied by 100 in a calculator: 70%. The denominator includes both reference classes. It is not the number of actual spam references.

Precision asks what fraction of the proposed spam labels really match spam in the key. The proposal marks B1, B2 and B5 as spam. Two of those three match. Calculate 2 / 3 × 100 = 66.666…%, displayed here as 66.7% to one decimal place. Preserve the fraction next to the rounded figure.

Recall asks what fraction of reference spam records were found. The key marks B1 through B4 as spam, and the proposal finds B1 and B2. Calculate 2 / 4 × 100 = 50%. The denominator is four actual positive references, not the three proposed positives and not all ten records. Google’s metric definitions distinguish these questions.

Another card may have a zero denominator. With no proposed positives, precision cannot be calculated by this fraction. With no reference positives, recall cannot be calculated. With no records, accuracy is also undefined. For this exercise, write not calculated: denominator zero, rather than inventing 0% or 100%. The supplied ten-row card does not have that problem.

The resolution cohort audit examines different outcome evidence. It does not turn a resolved conversation into a correct classification. Likewise, the mean and median review separates other summary questions. Names and denominators must stay attached to the question actually answered.

Which three reporting claims should you reject or revise?

Use four headings for the claim ledger: proposed statement, decision, source IDs and corrected statement. These are deliberately prepared teaching claims, not observed responses from a tested assistant. Check them against the label pairs and fractions, rather than judging whether the wording sounds confident.

An accepted summary is: “For these ten fictional label pairs, accuracy is 70%, precision is 66.7% and recall is 50%; one false positive and two false negatives remain explicitly identified.” That sentence describes the exercise only. It does not say a real filter is safe, unsafe, accurate in production or suitable for automatic deletion.

A high agreement count can coexist with mistakes in the less frequent class. The useful response is to inspect the error types, not appoint one percentage the universal winner. This walkthrough neither changes a decision threshold nor proposes a product configuration. A live evaluation would require a different source and decision process.

If you need to prepare a separate learning test, the source-based quiz review addresses that task. A generated quiz answer is not automatically a trustworthy classification reference. Here the exercise key is already supplied, and the reader’s job is to preserve it.

What can you save and reopen in fifteen minutes?

The core task needs only the supplied text, a calculator and a note editor. On Windows, open Start, type Notepad and create a new document. Head it “Classification practice V1: not a model run.” Add the positive-label rule, reference/proposed definitions, all ten source rows and the separate per-ID outcome ledger.

Below the ledger, write TP 2, FN 2, FP 1, TN 5 and the three complete fractions. Add the claim decisions and source IDs. Press Ctrl+S, choose Documents and save as classification-review-V1.txt. Close the editor and open that file from the same folder. Check that B3/B4 remain FN, B5 remains FP, and denominators ten, three and four have not been merged.

For an optional AI second opinion, provide only the fictional card and this request: “Positive means spam. Reference is the supplied exercise key; proposed labels are prewritten, not a model observation. Put each ID in TP, FP, TN or FN using both labels as evidence. Reconcile the four counts to the ID total. Show three distinct metric denominators. If one is zero, mark not calculated. Change no source label and claim no live filtering action.” No product account or assistant output was tested for this article.

Compare any proposal with the human key. If access fails, record optional AI comparison not performed; the manual ledger can still be complete. A suggested allocation is three minutes for the source, four for boxes, four for rates and claims, and four for saving and reopening. This is a planning allowance, not measured speed. My earlier writing about perfection and effectiveness described work delayed by pursuing every detail. Applied here, that principle means finishing one traceable review, not claiming personal results from a classifier trial.