Generative AI consulting: what to look for in a partner

Most “AI Consulting” is just a series of expensive workshops and a 60-slide deck that tells you your business needs to be “AI-enabled.” It’s high-level, vague, and fundamentally useless because it focuses on the what instead of the how.

If you’re looking for generative AI consulting, you aren’t looking for a strategy. You’re looking for a production-grade implementation. There is a massive gap between a chatbot that “feels” right in a demo and a system that reliably handles customer data without hallucinating in front of a client. The difference is in the engineering.

Why generative AI consulting is different from classic AI consulting

Classic AI consulting was about prediction. You gave a firm a massive dataset of historical churn, and they built a regression model to tell you who was likely to leave. It was a linear process: Data → Model → Prediction.

Generative AI is about reasoning and orchestration. It’s not just about a model; it’s about the entire system surrounding that model. You aren’t just predicting a value; you’re building a system that can read your internal documentation, reason through a complex request, and execute a task across three different APIs.

Because of this, the skill set has shifted. You no longer need a team of PhDs to tune a hyperparameters for six months. You need engineers who understand context windows, retrieval strategies, and the volatility of non-deterministic outputs. If your partner is still talking about “training models from scratch” for a business process, they’re playing the old game.

The five capabilities to look for

When vetting a partner, move past the portfolio. Ask them specifically how they handle these five technical pillars. If they give you a vague answer about “best practices,” they aren’t building in production.

• LLM Evaluation (The Anti-Vibe Check): Most people test AI by typing three prompts and saying, “Looks good.” That’s a vibe check, and it’s how systems fail. A real partner uses evaluation frameworks (like RAGAS or DeepEval) to create a “golden dataset” of queries and expected answers. They should be able to show you a precision and recall score, not a “thumbs up” from a stakeholder.
• RAG Architecture: Retrieval-Augmented Generation (RAG) is how you stop an AI from lying. It forces the model to look at your specific data before answering. Look for a partner who talks about chunking strategies, vector database selection, and re-ranking. If they just say “we connect it to your docs,” they’re building a toy.
• Agentic Orchestration: A single prompt is a tool; a chain of agents is a workflow. Agentic AI allows the system to plan, execute a tool, observe the result, and correct itself. Your partner should know when to use a simple chain and when to deploy a multi-agent system to handle complex, multi-step business processes.
• Prompt Engineering at Scale: Writing a great prompt in a chat window is easy. Managing 500 interconnected prompts across a production system is a nightmare. Look for a partner who treats prompts as code: version-controlled, tested, and decoupled from the core application logic.
• Governance and Guardrails: AI is non-deterministic, which is a polite way of saying it’s unpredictable. You need a partner who implements “guardrails” (like NeMo Guardrails or custom validation layers) to ensure the AI doesn’t go off-script, leak sensitive data, or promise a customer a free car.

Common pitfalls of generic AI consultancies trying to retrofit GenAI

There is a growing trend of “Classic” consultancies adding “Generative AI” to their website overnight. They are trying to retrofit a high-margin strategy model onto a fast-moving engineering problem. Here is how to spot them:

The “Discovery Phase” Loop: They want to spend three months “mapping the opportunity” and “aligning stakeholders.” In GenAI, the tech moves faster than the slides. A native GenAI partner will prototype a “lean” version of the solution in week two to prove it’s even possible before spending a penny on a roadmap.

The Wrapper Trap: They build you a “custom solution” that is actually just a thin wrapper around a ChatGPT prompt. You aren’t buying an asset; you’re renting a prompt. If they can’t explain the data architecture or the evaluation loop behind the interface, you’re paying a premium for a UI that you could have built in an afternoon.

The “Black Box” Delivery: They deliver a system that works, but they keep the “secret sauce” (the prompts, the retrieval logic, the eval sets) in their own environment. You become dependent on them for every minor tweak. A real partner builds the system in your environment, with your ownership of the intellectual property.

Engagement formats that work

Avoid the “Fixed-Price Project” trap. In traditional software, you can define a spec, build it, and hand it over. In Generative AI, that’s a mistake. A system that works today might drift tomorrow because of a model update or a change in your data structure. A “finished” AI project is actually just the start of the decay curve.

The most successful implementations follow a Partner-Extension model. Instead of a one-off delivery, you bring on a team as an extension of your own, working on a retainer basis to build, optimize, and evolve the system in real-time.

This works for three reasons:

• Continuous Optimization: The first version of a prompt is never the final version. A retainer allows for a constant loop of: Deploy → Monitor → Evaluate → Refine.
• Agility at Pace: When a new model drops or a new technique (like Agentic RAG) becomes viable, you don’t need to negotiate a new contract. Your team just pivots the roadmap and implements it.
• Knowledge Transfer: Because they are embedded in your team, the expertise doesn’t leave when the project “ends.” Your internal processes evolve alongside the technology.

What a good first project looks like

Don’t start by trying to “Transform the Enterprise.” That’s a recipe for a six-month failure. A good first project is the “wedge” that proves the value and justifies the ongoing partnership. It should be high frequency, low risk, and high manual effort.

The Ideal Candidate: A task that a smart human does 50 times a day, which takes 20 minutes of “searching and summarizing” per time, and where a mistake is easily caught by a human reviewer. Examples include internal knowledge base queries, initial lead qualification, or draft generation for standard reports.

The goal of the first project is to build the “AI muscle” within your organisation. Once the first win is live, the retainer shifts from “proving the concept” to “scaling the impact” across the rest of the business.