How to Monitor What AI Says About Your Business

To monitor what AI says about your business, run the same fixed set of customer questions against the same AI systems every month, and log each answer: mentioned or not, position, what was said, and which sources were cited. Consistency is the whole method — a repeated measurement shows trends; scattered one-off checks show noise.

This page gives you the full DIY protocol, the exact things worth tracking, and an honest account of where the manual version stops scaling.

Why a one-off check misleads you

Ask ChatGPT for “the best plumber in Mesa” once and you’ve collected an anecdote, not a measurement. Three kinds of variance stack up against you:

• Sampling variance. Models compose each answer fresh. The same prompt, minutes apart, can name different businesses or the same businesses in a different order.
• Phrasing variance. “Best plumber in Mesa,” “who do I call for a burst pipe in Mesa,” and “top-rated Mesa plumbing companies” are one customer need and three retrieval paths, which can surface three different shortlists.
• Time variance. The pages behind the answers change as reviews land, listings update, and competitors publish. This month’s answer is not last month’s.

A single check can’t tell you which variance you just observed. If you asked once in January and were named, and once in June and weren’t, you know nothing — maybe you declined, maybe you hit an unlucky sample. Monitoring exists to separate signal from noise, and it does that in only one way: same questions, same systems, repeated on a schedule. The moment you improvise new prompts each month, you’re comparing different measurements and the trend line means nothing.

The DIY monthly protocol

Here’s the version you can run yourself for free, in roughly an hour a month.

Fix your prompt set. Write 8–12 questions a real customer in your market would ask, covering four intents: discovery (“HVAC companies in Tucson”), recommendation (“who’s the best mortgage broker in Boise?”), comparison (“compare the top realtors in Bend for first-time buyers”), and task-specific (“my AC died and I need someone in Plano today — who do I call?”). Add one name check: “Tell me about [Business Name] in [City].” Write them down verbatim. These are now your battery — don’t edit them month to month, or you break comparability.

Fix your systems. Pick two or three and stick with them: ChatGPT with search enabled, Perplexity, and Google’s AI answers is a sensible trio. They retrieve and weight sources differently, so a gap in one and not another is diagnostic, not random. (If you want the mechanics, see what sources ChatGPT uses for local recommendations.)

Log every answer. A spreadsheet, one row per question per system per month:

Column	What goes in it
Date (with time)	Answers drift; timestamps make rows comparable
System	ChatGPT, Perplexity, Google AI
Prompt	Verbatim, copied from your fixed list
Mentioned?	Yes / no
Position	First named, mid-list, afterthought
Competitors named	Who took the spots you didn’t
What it said	Verbatim quote — especially anything wrong
Sources cited	The domains, one by one

Copy answers verbatim. Paraphrase is where wrong facts hide — “it said something about our old location” is not evidence; the exact sentence is. This is the same rule our own audits run on: every response stored raw, timestamped, quoted verbatim, never edited.

Same day each month. Put it on the calendar. The protocol’s value compounds only through repetition.

What to track (and what each number tells you)

Four things are worth computing from your log:

• Mention rate. The share of answers that name you at all. This is the headline number — it’s why mention rate carries 50% of the weight in our Visibility Score. If you’re named in 4 of 24 answers this month and 9 of 24 in three months, something you fixed is working.
• First-mention rate. How often you’re named first. AI answers, like customers, have a shortlist and a favorite. Moving from “mentioned fifth” to “mentioned first” is real progress the raw mention rate won’t show.
• Wrong facts. Every stale or incorrect claim, with the citation that carried it. A rising mention rate with a wrong phone number attached is not a win. When you find one, the correction workflow is in AI says wrong things about my business.
• Cited sources. Tally which domains keep appearing in citations across all answers in your market — not just answers about you. That tally is your to-do list: the pattern that predicts visibility best in the local-business audits we run is presence on the sources AI answers already cite, plus consistent name/address/phone data and a complete Google Business Profile.

Resist tracking more than this. A dozen metrics you abandon in month two are worth less than four you keep for a year.

Reading the trend honestly

Judge in quarters, not weeks. Source fixes take days to weeks to propagate into answers, so the earliest a change can plausibly show up is your next monthly run, and a real trend takes two or three.

Expect noise even with a fixed protocol. Mention rate moving from 40% to 45% in one month is probably sampling; moving from 20% to 50% over three months is probably real. And watch competitors in your log as closely as yourself — a competitor who suddenly appears in every comparison answer has usually fixed something you can identify by reading the new citations.

One more honesty note: monitoring measures, it doesn’t move. If the numbers are flat, the work is upstream — profiles, listings, reviews, your site — and no amount of extra checking changes them.

When DIY stops scaling

The manual protocol genuinely works, and for a single-location business with time, it may be all you need. It strains in predictable places:

• Sample size. An hour a month buys you roughly 24–36 answers across two or three systems. Detecting a change from a 30% mention rate needs more samples than patience usually allows.
• Coverage. Customers also ask Gemini, Claude, and DeepSeek-class systems. Each additional system multiplies your hour.
• Discipline. The protocol only works if it runs the same way every month — including months you’re busy. Skipped months put holes in the trend line exactly when something changed.
• Verbatim storage. Copy-pasting 30 answers with citations into a spreadsheet is the step humans quietly stop doing around month three.

This is the part we built the paid version for, so weigh the pitch accordingly: our audits run a 20-prompt battery — four intents, from discovery to ready-to-buy — against 6 AI systems, producing 100+ timestamped answers per run, scored 0–100 with documented weights and a citation-source inventory for your market. Monthly tracking re-runs the identical battery for $99/mo, so the trend line is measured, not remembered. The methodology is public on how it works — take the protocol on this page and it’s the same thing, scaled up and automated.

What to do this week

1. Write your fixed prompt set. 8–12 customer questions across the four intents, plus the name check. Save them verbatim in your spreadsheet.
2. Pick your systems. ChatGPT with search enabled, Perplexity, and Google’s AI answers. Commit to those three.
3. Run month zero. Ask everything, log every row, copy answers and citations verbatim. Budget an hour.
4. Compute your baseline. Mention rate, first-mention rate, wrong facts found, and the top five cited domains in your market.
5. Act on the two obvious outputs. Fix any wrong fact at its source; claim your listing on the most-cited domain you’re missing from.
6. Schedule month one. Same day next month, on the calendar, before you close the spreadsheet.

If you want a measured baseline before your first manual run — mention count, competitors, and a shareable results page — start with the free AI visibility scan.