Measure Before You Automate: Setting a Real Baseline

You cannot improve what you haven't measured, and you cannot prove an automation worked if you never recorded what "before" looked like. This episode covers the baseline discipline, the single most skipped step in SMB AI adoption.

Why teams skip this step

Baselining feels like overhead when you're eager to ship the fix. It isn't overhead. It's the only thing standing between "we think this is faster" and "this is 34% faster and here's the log." Skip it and you'll be defending your automation spend with vibes instead of numbers in six months.

The two numbers that matter

For the task you selected in Episode 1, track exactly two things for at least two weeks (or 30 occurrences, whichever comes first):

1. Time-on-task. Wall-clock minutes from start to finish, measured by the person actually doing the work, not an estimate given from memory afterward. Use a simple timer or timestamp log. Estimates run 30-50% optimistic in practice; real numbers surprise people almost every time.
2. Error rate. Define what "error" means for this specific task before you start counting: a wrong field, a missed step, a customer callback, a redo. Write the definition down so it doesn't drift mid-measurement. Count every instance, including the ones caught and fixed internally (a caught error still cost time).

Recording the baseline

Keep it in a plain spreadsheet: date, operator, minutes, error count, and a one-line note on anything unusual (system outage, new hire doing the task, holiday rush). The note matters. Baselines taken during atypical weeks will mislead you later.

Compute and record:

• Median minutes per occurrence (median, not average; outliers distort averages badly on small samples)
• Error rate as a percentage of occurrences
• Volume per week (how many times this task actually happens; you'll need this to project savings)

Setting a success threshold before you build

Before touching any automation or AI tool, write down what "success" means in the same units as your baseline. Example: "Median time drops from 14 minutes to under 6 minutes, with error rate at or below the current 4%." Commit to this threshold in writing, before the tool is in the room. It is far too easy to move the goalposts after the fact once you're emotionally invested in a rollout.

Re-measuring after deployment

Run the identical measurement protocol for two weeks after the automation goes live: same definitions, same log format. Compare medians directly. If error rate went up while time went down, you haven't won yet; you've just moved the cost downstream to whoever cleans up the errors. Both numbers have to hold or improve.

A note on soft costs

Some costs won't show up in minutes or errors: operator frustration, context-switching cost, onboarding time for new staff. These are real, but they're secondary. Don't let a "this feels better" narrative substitute for the two hard numbers above. Feelings are useful signal for where to look next, not proof of anything on their own.

Field exercise

For the task carried over from Episode 1, build a baseline log template with columns: date, operator, minutes, error count, notes. Run it for five real occurrences this week (even if that means partial data), and write down your target success threshold in one sentence before you read Episode 3. Bring both to the next session.