AI native or AI bolted on: the ten-minute test

We build AI into other people's software for a living, and we get called into the aftermath often enough to see the pattern. Two tools with identical marketing pages. One is worth the premium. The other charges you for a public chat bot with a logo on it.

The difference is rarely the model underneath. It's whether the vendor did the unglamorous work of connecting that model to your actual business. You can tell which one you're holding in about ten minutes, with three questions and no technical help. Run them before the next renewal, not after.

Why the premium deserves a hard look

AI features arrive on pricing pages as recurring add-ons, per seat, month after month. Across a three-year contract that line item quietly becomes real money, and the only question that matters is whether it does work a free chat bot couldn't.

The premium is worth paying in exactly one case: the feature does work that would otherwise take a person time. A good write-back routine running daily across a sales or service team buys back hours every week, and the add-on earns its line item. A bolted-on chat box bills you for the label while you supply the labor. Same invoice, opposite outcomes.

Here's the tell we run into constantly in integration work. A tool already holds your data in the database it owns. Your clients, your order history, and the notes nobody cleaned up from last March are sitting right there. The AI feature can't see any of it. You ask it a question and it answers like a well-read stranger, because the vendor connected a model to the open internet and then parked a chat window beside your records. Two fields away, and the feature is blind.

The marketing never says this. The demo won't show it either, because demos run on scripted sample data. Only your account, with its messy history, exposes the gap.

There's a shortcut for demo day. Ask the sales engineer to point the trial at a sandbox copy of your own data instead of the sample company, then run the first question live. Good vendors agree easily. The ones that can't will explain why that's hard, and the explanation is usually the answer.

Question one: does it know your business?

Open the tool and ask it something only your company could answer. What did that difficult client order last quarter? Which service line carries the best margin? What usually goes wrong during our onboarding? No pasting, no context. Just the question, the way it occurred to you.

A native feature pulls from your records and shows where the answer came from. It names the client, reaches into the history, and either gets the specifics right or fails honestly. A bolted-on one produces a fluent paragraph that could describe any company on earth, or asks you to paste in the details it should already have.

That single exchange tells you whether the vendor connected the model to your system of record or just to the internet. Everything else follows from it, because an AI that can't see your data can only ever produce generic content in your branding.

There's a middle result worth knowing about. Some tools retrieve from your records but only from a narrow slice, the ticket bodies and not the notes, the invoices and not the emails. That's a scoping decision the vendor made, and it's fixable on their side, so report precisely what the feature could and couldn't see. You're doing their roadmap a favor, and you're building a paper trail for the renewal conversation.

Question two: does the answer go anywhere?

Make the tool produce something genuinely useful. A draft reply to a real client email. A summary of a long project thread. A triage of twenty open support tickets.

Then watch what happens to the output.

If the feature is native, the result lands back where the work lives. The draft saves to the record, the summary attaches to the thread, the tickets get sorted, and the next person to open the account finds the work already done. If it's bolted on, the answer dies in the chat window and a human ferries it to the right place, every single time. That ferry ride is a labor cost the pricing page doesn't list. Multiply it across the team, ten people moving an answer twice a day, and you've bought forty hand-offs a week nobody budgeted for.

When clients call us to "add AI," the model is almost never the hard part. Wiring the output back into the record, with the right permissions and a trail of what the system did, is the actual project. A vendor who skipped that step sold you the easy fifth of the feature and called it done.

The write-back also buys you something quieter: accountability. When the AI files the summary or drafts the reply inside the system, there's a record of what changed and who approved it. When the same work happens through copy-paste, the trail lives in nobody's memory and resolves in an argument. Teams that get audited, or that bill by the hour, feel this difference first.

Question three: where does your data end up?

This one matters more as the other two pass, because a genuinely native AI has deep access to your business. Deep access cuts both ways.

Ask the vendor, in writing, what the AI can read in your account, whether your inputs train their models, how long prompts and outputs are kept, and whether any of it can be switched off. A vendor who did the integration work properly gives boring, specific answers with real limits in them. A vendor who bolted on a chat box answers in adjectives. Enterprise-grade. Secure. Privacy-first.

Push past the adjectives with paperwork. Ask for the data processing addendum and the subprocessor list, then look for the line where the AI provider appears. Very often the model vendor is a subprocessor you never knew you had, receiving whatever your team typed into that helpful little box. A seller who can't name that line hasn't done the integration work either.

Vague answers here are not a paperwork nuisance. They mean a second copy of your client records is flowing somewhere you can't audit, and you signed for it. American securities regulators have already charged firms over inflated AI claims and coined a phrase for the practice, AI washing, which should tell you how common the inflation has gotten.

What good looks like when it's real

The tools that pass all three questions share a shape. The AI reads the system of record, not a generic web page. Its output writes back where the workflow lives, so the value compounds instead of evaporating in a chat window. Access is scoped so a summarization feature can't read payroll, every automated change leaves a trail, and a human approves anything that leaves the building. None of that photographs well in a demo. All of it is the difference between an assistant and a sticker.

In defense of the honest wrapper

A thin layer over a good public model is not automatically a rip-off. We have built exactly that for a client who needed three people drafting quotes faster and nothing more, and it paid for itself in weeks. When the underlying model improves, a thin wrapper inherits the improvement for free, and sometimes that's the right engineering call.

The sin isn't the wrapper. The sin is wrapper value at native prices, or a wrapper marketed as a system. Pay for what the feature does this quarter, not for the two letters on the pricing page.

Four questions to put in writing

Before the renewal, send these to the vendor and keep the replies with the contract.

1. Exactly which data can the AI read in our account, and at what scope?
2. Where does the output go? Does anything write back to our records, or does everything stay in the chat?
3. Are our inputs used for training, how long are they retained, and can we opt out?
4. If you switch model providers or a model changes underneath us, what happens to the features we're paying for?

Slow, fuzzy answers teach you the same lesson as the ten-minute test, just from the other direction.

The decision rule

Score the tool honestly. If it passes all three, keep paying the premium and lean on the feature harder than you have been, because compounding value like that is rare. If it fails two of three, you're renting a public model at a markup, and your options are to renegotiate the add-on away, switch tools, or have someone connect the model to your records properly.

That connection work is unglamorous, and it is most of what we do. If a tool you otherwise like fails the test, closing the gap between the chat window and your actual data is exactly the kind of AI integration project we take on. Your software should read your business, write its answers back where the work happens, and tell you plainly where your information goes. Anything less is a sticker.