AI segment rate audit: when the total reverses the groups

What must stay comparable before you pool the counts?

This practice card has two mutually exclusive request groups: short and long. Every eligible fictional request belongs to exactly one, and together the groups cover all eligible requests in the supplied card. The outcome is one completed practice task per request at most. The outcome and eligibility definitions are identical across both groups and both tables.

A and B are names for two invented count tables, not randomised experimental assignments. No real organisation, customer record, model response or measured service result is involved. The question is descriptive: what do subgroup rates and the combined rate actually say when their group proportions differ? It is not a decision about which method caused better results.

Copy this boundary first: “Practice V1; two non-overlapping, exhaustive groups; same outcome and eligibility; each request counted once; A/B are table labels; source counts remain unchanged.” Without that card, an apparent reversal could simply be a change in the meaning of the numerator or denominator, which would be a different problem.

The customer segmentation guide covers creating and checking groups. Here those groups are already supplied. Your deliverable is a saved evidence note with four subgroup rates, two pooled rates, a group-mix explanation and three claim decisions.

Copy the four fictional count pairs

Imagine a small exhibition team practising how to review an internal report. Short and long are stipulated labels for the exercise, not categories inferred from real work. The following counts are deliberately constructed and not a representative sample. Do not treat them as productivity estimates, customer satisfaction evidence or a vendor comparison.

There are four aggregate rows, not four individual requests. Table A contains 20 short and 50 long requests; B contains 50 short and 20 long requests. Check those denominators before looking at the percentages. Equal group names do not imply equal group sizes.

Confirm that each completed-outcome count is no greater than its eligible-request count. Preserve every row under the heading fictional source V1. If an assistant rewrites 20 as 50 to make the groups comparable, reject that change. Equalising a hypothetical population would be another calculation with another question, not a correction to this supplied source.

The card contains neither missing outcomes nor overlapping group membership. A real file with those problems would need a source check before applying this exercise. Keep the current task bounded instead of inventing answers for an absent data collection process.

What do the four subgroup rates show?

Use a calculator to divide completed outcomes by eligible requests in each row and multiply by 100. For short requests, A has 18 / 20 = 90%, while B has 40 / 50 = 80%. For long requests, A has 10 / 50 = 20%, while B has 2 / 20 = 10%. Retain each fraction alongside the percentage.

Both subgroup comparisons favour A numerically. That sentence describes the fictional counts only. It does not establish that using A would improve anyone’s outcome. The card gives no random assignment, causal structure, collection method or external validation. Do not turn two correct comparisons into a claim about a real intervention.

Write the limited finding in your ledger before calculating the total: A is higher within each supplied request group. Leave the pooled conclusion blank for now. Moving from a subgroup to a combined population requires a new numerator and denominator; it is not a wording change.

For a broader source workflow, the AI data analysis guide separates calculations from interpretations. This exercise extends that discipline to one specific reporting trap: assuming a subgroup direction must survive aggregation unchanged.

Why does the combined comparison reverse?

For A, sum the outcomes: 18 + 10 = 28. Sum the eligible requests: 20 + 50 = 70. The pooled rate is 28 / 70 × 100 = 40%. For B, the outcomes sum to 40 + 2 = 42 and the denominator to 50 + 20 = 70, giving 42 / 70 × 100 = 60%. Both calculations preserve the supplied membership.

A is higher in each subgroup, while B is higher in the pooled card. Stanford’s explanation of Simpson’s paradox describes how associations can change when populations are combined or separated. There is no arithmetic contradiction here. The group sizes used in the combination differ.

A’s short group has weight 20/70 and its long group 50/70. B’s weights are 50/70 and 20/70. Short requests have higher completion proportions in both tables, and B contains a larger share of that group. This explains the numerical composition of the supplied totals. It does not establish why a real population acquired that composition.

The simple average (90% + 20%) / 2 = 55% is not A’s pooled rate for this card. It gives the two groups equal weight, unlike the source. The House of Commons Library’s explanation of weighted averages describes why differing group sizes matter. The most inspectable method here is to sum outcomes and denominators separately, then divide.

The mean and median audit concerns other summaries. Do not reuse its language as permission to choose whichever rate looks most favourable. Name the actual question and preserve the count-based method.

Which three conclusions need correction?

Create a claim ledger with four columns: proposed sentence, decision, source evidence and corrected wording. These claims are prepared teaching examples, not errors observed in a tested assistant. A fluent explanation does not override the fractions you have independently checked.

An accepted statement is: “For this fictional count card, A has the higher proportion within each request group; with the supplied group mix, B has the higher pooled proportion. These descriptions do not identify a causal winner.” Neither the pooled rate nor the subgroup rate automatically answers every decision question.

The reversal alone also does not prove selection bias, establish a reason for the different mix or justify a spending decision. Which comparison matters depends on the question and the data-generating context. This exercise deliberately stops at the source-backed description and identifies what the counts cannot establish.

If a figure is presented visually, the chart audit offers a separate check on whether labels and percentages preserve the underlying values. A clear chart cannot make an unsupported causal claim true. Keep the evidence note readable without needing a generated visual.

What can you save and reopen in fifteen minutes?

The manual exercise needs only the source card, a calculator and a note editor. On Windows, open Start, type Notepad and create a document headed “Segment and pooled rate practice V1: fictional data.” Paste the boundary, all four source rows, the four subgroup fractions and both pooled fractions. Add the two group-weight pairs without changing their denominators.

Append the three claim decisions and the accepted limited statement. Press Ctrl+S, choose Documents and save as segment-total-review-V1.txt. Close the editor and reopen the file from that folder. Check that A’s short denominator is 20, B’s short denominator is 50, and 28/70 and 42/70 remain attached to the correct tables. A note with percentages but missing counts is an incomplete handoff.

For an optional AI second opinion, supply only this fictional card and the request: “Groups are non-overlapping and exhaustive, with identical outcome and eligibility definitions. Show four subgroup rates, two sum-count pooled rates and the group weights. Do not average percentages equally as a substitute for the supplied pooled rate. Review the three claims using the counts. Invent no randomised experiment, causal winner, proven selection bias or real company result.” No product account or model response was tested for this article.

Compare any response with the human answer key. If the assistant is unavailable, record optional AI comparison not performed. The manual ledger can still be complete. A suggested allocation is three minutes for the source, four for calculations, four for claims and four for saving and reopening; this is a planning allowance, not measured speed.

In my earlier writing about perfection and effectiveness, I described work delayed by pursuing every detail. Applied here, that principle means finishing one reproducible comparison before polishing a dashboard. It is not evidence of a personal marketing or model-evaluation trial. For other starting tasks, see the artificial intelligence guide.