An agency buys a tool, someone types a question into it, and out comes an answer that sounds like it was written for a bank. The tool gets a reputation inside the office within a week. AI doesn't really work for insurance, not for our kind of accounts anyway, and everybody moves on.
Nothing in that agency ever told the tool what it does. No book of business, no sample emails, no standards, nobody checking the output and correcting it. It got a question and was expected to know the rest.
Then look at how the producer hired in March got started. A login, a desk, and 2 days of shadowing whoever had the lightest calendar that week. Same non-training, and it costs a lot more.
Bad input, confident output
I've been inside enough independent agencies to know that the AI disappointment is almost always an input problem. These tools are exactly as confident when they're wrong as when they're right, so a thin input doesn't come back as an error message. It comes back as a clean paragraph that's subtly not your agency. The tone is off, the coverage language is generic, and it recommends something you'd never recommend. People read that, decide the technology is hype, and go back to typing the renewal email by hand for the ninth time this week.
The tools that are working inside agencies right now got fed. Somebody gave them the agency's real emails, the real workflow, the real vocabulary, and somebody read the output before a client saw it.
People don't learn by osmosis either
Across 49 agency tech stack audits, half of them had no formal training tool of any kind. Whatever training happened lived in somebody's head, and it walked out the door when they did. Average AMS utilization across those same agencies came in at 65%, and I don't read that as a software problem. People use the parts of a system somebody showed them. The other 35% is the part nobody had time to explain, so it may as well not be paid for.
A new producer with no context, no examples and no feedback loop produces the same thing a starved AI tool produces. Work that's technically fine and doesn't sound like your agency, until somebody catches it in front of a client.
Where I got this wrong
Early on in my career I ran an onboarding that was basically a tour. Here's the AMS, here's the rater, here's where the carrier logins live, call me if you're stuck. I thought staying available was the generous thing to do. What I'd actually built was a system where every question cost that person something, because asking meant interrupting me. The quiet ones stopped asking. I lost 2 people who could have been good, and it took me too long to connect those 2 things.
Don't be that agency
The same five things fix both. Context, examples, standards, correction, and a review step.
Write down how your agency does the 5 jobs that come up every day: the quote, the renewal call, the certificate request, the claim intake, the handoff to service. The way your best person actually does it, in their words, one page each. Record them talking through it if writing is the thing that never happens in your office. Ten recordings and a transcription tool gets you further in a week than a training platform gets you in a year. That same page is what you paste into the AI tool and what you hand the new hire on day one.
Give both of them 10 examples of the work done right. Your best renewal emails. A quote presentation that won. A tough claim conversation that went well. Nobody hits a standard they've never seen, and that goes for the software and the person.
Then read the output for the first 90 days, from either one, before it reaches a client. A correction in week 2 is cheap and a correction in month 8 is a phone call with an angry account. You already proof a cert before it leaves the office. Same instinct, applied to the 2 newest things in your agency.
This is what I mean when I say less tech that does more. Nothing in that list requires a purchase. The agencies pulling ahead right now are the ones where a new person is producing in 90 days instead of 9 months, and where the tools they already pay for got 20 minutes of setup instead of a demo and a shrug.
If you're the producer reading this
You may not get any of the above. Build it for yourself and be obvious about it.
Ask for the 10 examples. Ask the top producer if you can record 3 of their calls and keep the transcripts. Write your own one-pager for every process you figure out the hard way, because in 2 years you'll be the one training somebody and that file is what you'll hand them. Ask for a 15-minute review of your first renewals instead of waiting for an annual review to find out how you're doing.
The producers I watched pull ahead did this without anybody telling them to. That's the whole trick, and it's available to you in your second week.
What it comes down to
Nothing learns your agency by watching from across the room. Not the software you bought last quarter, and not the person you hired in March. Both of them need the same five things from you, and only one of them will still be here in 5 years.
If you're sorting out which AI to actually run in your agency this quarter, I went through it tool by tool here.