Be clear about what kind of AI work you are buying
AI development covers several different kinds of work that need different skills. Business projects today usually build on existing language models, and a model trained from nothing is seldom needed. Know which kind you need, because a company strong in one may be weak in another.
| Kind of work | What it is | The skill that matters most |
|---|---|---|
| An assistant over your own documents | Answers questions from your policies, manuals or records, with sources | Preparing and retrieving your content well, and measuring answer quality |
| Workflow automation with agents | Software that reads, decides and acts inside your systems, with approval where needed | Integration, permissions and safe handling of failure |
| AI features inside a product | Summaries, search, drafting or classification within an application | Product design, cost control and response time |
| A custom or fine-tuned model | A model adapted to your data for a narrow task | Machine learning engineering and a large, clean dataset |
| Strategy and assessment | Deciding where AI is worth using at all | Knowledge of your operations, and the honesty to say no |
If you have not yet decided where AI belongs in your business, start with how to identify business workflows suitable for AI.
What separates real capability from a good demo
A convincing demonstration can be assembled quickly with current tools. Making the same system dependable on your real cases, at a cost you accept, is most of the work. These eight criteria show which kind of company you are talking to.
| Criterion | What good looks like | Warning sign |
|---|---|---|
| Problem framing | They ask which decision or task the system supports and what a mistake costs | They start from the technology and look for somewhere to use it |
| Data handling | They ask where your data lives, how clean it is and who may see it | They assume the data is ready |
| Evaluation | A test set built from your real cases, with scores tracked for every change | Quality judged by trying a few examples in a meeting |
| Failure handling | Defined behavior when the model is unsure or wrong, with human review where the stakes are high | The claim that the model rarely makes mistakes |
| Cost and speed | An estimate of running cost per task, and a design that controls it | No mention of usage costs until the invoice arrives |
| Security and privacy | Clear answers on what is sent to which model provider and what is stored | Vague comfort about data being safe |
| Model independence | A design that can switch models as prices and quality change | Everything tied to one provider with no reason given |
| Ownership | The agreement says who owns the prompts, test sets, code and logs | The logic sits inside their platform, on subscription |
Questions that expose the difference
- 01
How will we know the system is good enough to launch?
A strong answer coversMeasures agreed in advance, a test set from your own cases, and a pass mark you set together
- 02
What happens when it is wrong?
A strong answer coversHow uncertainty is detected, what the user sees, when a person reviews, and how errors are fed back into testing
- 03
What will it cost to run each month at our volume?
A strong answer coversA calculation from your expected usage, with the main factors and how they can be reduced
- 04
Which of our data leaves our systems, and where does it go?
A strong answer coversA plain list of providers, what each receives, whether it is retained and under which terms
- 05
Can you show us an evaluation report from another project?
A strong answer coversA real one with private details removed, showing scores over time and examples of failures
- 06
What would make you advise us not to do this?
A strong answer coversSpecific conditions, such as too little data, too high a cost of error or a simpler fix
- 07
How does the system change when a better model appears?
A strong answer coversThe model is a replaceable part, and the test set shows at once whether a change is an improvement
What weak and strong answers sound like
One question, two answers
A logistics company asks two vendors how they would know that an assistant answering customer delivery questions is working. Both answers are invented.
- Weak answerWe use the most advanced model available, so accuracy is very high. We will test it thoroughly before launch and you can try it yourself.
- Strong answerWe would take a few hundred real questions from your support inbox, with the answers your team gave. That becomes the test set. We measure how often the assistant matches the correct answer, how often it declines when it should, and how often it invents something. You set the pass mark before we start. After launch, a sample of live conversations is reviewed each week and added to the test set.
- What the difference tells youThe first vendor is relying on the model. The second has a method, and the method is what you are paying for.
Notice that the strong answer never mentions which model it would use. The measurement comes first, and the choice of model follows from it.
Ask for a small, measured pilot
The best protection against a poor choice is a pilot with success criteria written before it begins. One workflow, real data, a fixed period and a clear decision at the end. A company that is confident in its method will welcome this. One that is selling a demo will push for a larger commitment.
Our guide to planning an AI pilot with measurable success criteria explains how to set one up.
Good signs and warning signs
Good signs
- They ask to see real examples of the work before proposing anything
- They talk about evaluation more than about models
- They suggest keeping a person in the loop where errors are costly
- They tell you when a rule or a simple automation would do the job without AI
- They can explain their system in plain language
Warning signs
- Promised accuracy figures before seeing your data
- No plan for wrong or uncertain answers
- A proprietary platform you cannot leave
- No mention of running costs
- Every problem answered with the same product
A checklist to run before you sign
AI project contract checklist
0 of 9 checkedWhat to do next
Choose one workflow, collect twenty real examples of it and take them to the companies you are considering. How each one reacts to real cases will tell you more than any presentation.
You can see how we describe our own work, including which solutions are demonstrated and which are proposals, in our AI agent solutions catalogue, and how we build in AI product development and AI integration. Or send us your examples and we will tell you plainly whether we think AI is the right tool.
Common questions
Do we need a company that trains its own AI models?
Rarely. Business applications today are commonly built on existing models from major providers, combined with your data and your systems. Training or fine-tuning a model is worth it for narrow tasks with large amounts of clean data. A company that pushes custom training for an ordinary use case is selling the wrong thing.
Is our data safe when it is sent to a model provider?
It depends on the provider, the plan and the settings, so ask for specifics. You want to know what is sent, whether it is stored, for how long, and whether it may be used to train models. The business terms of the major providers set out their position on each point, so read them for the plan you would use. Sensitive data can also be masked, or processed by models that run in your own environment.
How much data do we need to start?
Less than most people expect for work built on existing models. A few hundred real examples of the task, with correct outcomes, are often enough to build a test set and begin. What matters more than volume is that the examples are real, varied and include the difficult cases.
Should we build an AI team in house instead?
If AI will be central to your product for years, you will want the capability inside. An outside company is useful for getting a first system live, proving its value and setting up the method. Agree at the start that knowledge and ownership transfer to you, so that both paths stay open.
What does an AI system cost to run?
Running cost depends on how many tasks it handles, how much text each involves and which models are used. It can be estimated before building from your volumes. Ask for that estimate, and for the design choices that keep it down, such as using smaller models for simple steps.
