AI Agent Development Company: Questions to Ask Before You Hire

The model isn't the risk anymore. The vendor is.

Every week, someone inherits an abandoned AI project. No documentation. No handover. Just a codebase that the previous vendor won't return calls about.

The agent half-works. Nobody knows why. Nobody knows how to fix it without starting over.

That's not a technology failure. That's a vendor selection failure.

Greensighter exists partly because of how often we get that call. We've rebuilt more abandoned agents than we'd like to count. The questions below are the ones that would have prevented most of them.

This guide gives you the actual questions to ask before you sign anything. Not the polite ones. The ones that get you a real answer.

‍

Why the Vendor Matters More Than the Model

Every serious AI agent development company today has access to roughly the same models.

Research from McKinsey and the University of Oxford on large IT projects found that half of them blow their budgets, running 45% over cost on average.

That study predates AI agents entirely. It's also still true. The technology changed. The way projects fail didn't.

GPT, Claude, Gemini, Llama, take your pick. The frontier labs aren't the differentiator anymore.

What actually separates a good build from a bad one is judgment. Knowing when to give an agent more autonomy, and when to pull it back.

Knowing which integration will quietly break in production, before it breaks. Knowing how to test something that doesn't behave the same way twice.

That judgment lives in the team, not the model. 

Which is exactly why vendor selection deserves more scrutiny than most companies give it.

‍

Why Hiring the Wrong AI Agent Development Company Gets Expensive

A bad hire in a regular software project is recoverable. A bad hire in an AI agent project compounds.

Here's why.

An agent that's poorly scoped doesn't just fail to deliver value. It actively creates problems: actions taken on bad data, integrations that break silently, audit logs that don't exist when a regulator asks for them.

Rebuilding it means paying twice. Once to the original vendor. Once to fix what they built.

Then there's the data problem. If your vendor co-mingles your data with other clients, or trains a shared model on your workflows, you've lost something you can't get back. Contracts that don't address this upfront almost never address it after.

And the longer a poorly-built agent runs in production, the more downstream systems depend on its broken behaviour. By the time you realise it's wrong, untangling it is a project of its own.

The questions below are designed to surface these risks before you sign, not after.

‍

Questions About Their Development Process

Here’s a list for you:

• "Walk me through your discovery phase."

Good answer looks like: "We spend the first one to two weeks mapping your actual workflow, the systems involved, the edge cases, and the humans who still need to be in the loop. No code gets written before that's done."

Example: A vendor who's built in healthcare will immediately ask about your EHR, your approval workflows, and your compliance requirements before touching a timeline.

Red flag: "We start coding in week one." Discovery skipped means assumptions baked in, and you'll pay to undo them later.

• "Who actually builds this, and where are they?"

Good answer looks like: Named engineers, their roles, their time zones, and a clear explanation of who owns what.

Example: "Our lead engineer is based in [location], she'll own the integration layer. Our prompt engineer works closely with her and is available during your core hours."

Red flag: Vague answers like "our team" or "our partners." If they can't name who's touching your codebase before the contract is signed, they're hiding a subcontracting arrangement.

• "How do you test something that gives a different answer each time?"

Good answer looks like: A described evaluation process, scored test runs against defined expected outputs, an acceptable failure rate, and a monitoring plan for production drift.

Example: "We build an eval set of 200 to 300 representative inputs before launch and run the agent against them on every major prompt change. We define what 'good enough' looks like before we start, not after."

Red flag: "We test it manually before delivery." Manual spot-checking isn't an evaluation framework. It's vibes.

• "Can you show me the agent's reasoning, not just its output?"

Good answer looks like: A demonstration of logging or tracing that shows which tools were called, in what order, and why, not just the final answer.

Example: A vendor who's thought about this will show you a trace from a previous project, redacted if needed, that makes the agent's decision path legible.

Red flag: "The model decides internally." That's a black box. In any regulated industry, that answer alone should end the conversation.

• "What's the realistic timeline, and what slows it down?"

Good answer looks like: A breakdown by phase with named variables, integration complexity, data quality, compliance requirements, and approval workflows.

Example: "Eight weeks for a single-task agent with clean data. Twelve to sixteen if you're connecting to an EHR system that needs custom auth, which is common in healthcare."

Red flag: A firm number with no explanation of what's inside it. A vendor who can't tell you what slows a project down hasn't scoped enough projects to know.

That's the one worth a closer look. We'll sit down and map your actual workflow with you first, free, before you sign with anyone.

Get a Free Workflow Map with us today!

‍

Questions About Cost and Contracts

• "What exactly is included in this quote?"

Good answer looks like: A line-by-line breakdown covering the build, token, and API costs, hosting, monitoring, post-launch tuning, and integration maintenance.

Example: "Our quote covers build and deployment. Token costs are separate and depend on usage; here's a range based on your expected volume. Hosting is X per month. Post-launch support is covered for 90 days, then moves to a retainer."

Red flag: A single number with no breakdown. Everything that isn't named in the quote will show up as a surprise invoice.

Our guide to AI agent development cost breaks down what a complete quote should actually cover, line by line.

• "Who owns the code when this is done?"

Good answer looks like: "You own it entirely. Full source code, all documentation, transferred to your repository on final payment. We retain no rights."

Example: A clean contract has a one-paragraph IP assignment clause. It's unambiguous. Any vendor with experience will have one ready.

Red flag: Phrases like "proprietary framework," "we license the core," or IP language that requires a lawyer to parse. If you have to ask twice, the answer is no.

"Code" is the easy part. Make sure the contract explicitly covers:

Prompts. The system prompts and instructions that make your agent behave the way it does are intellectual property. Some vendors consider these proprietary. They shouldn't be — you paid for them.

Workflows. The logic that defines how your agent makes decisions, routes tasks, and escalates to humans. This is business logic. It belongs to you.

Integrations. Custom connectors built to link your agent to your specific systems. If a vendor built a bespoke Salesforce integration for your workflow, you own that too.

Infrastructure configuration. Hosting setup, environment variables, deployment scripts. If these live only in the vendor's accounts, you're dependent on them even after the project ends.

Ask for a complete asset list before signing. If it isn't in the contract, assume it stays with them.

That includes any clause letting the vendor reuse your custom logic for other clients.

Greensighter's own breakdown of what makes a vibe-coded MVP a liability instead of an asset covers this exact risk.

It applies just as much to a custom agent as it does to an app.

• "What happens to our access if you go out of business?"

Good answer looks like: An explanation of source code escrow, or a clear process for transferring all assets on dissolution.

Example: "We use a third-party escrow service. If we cease operations, the code is automatically released to you within 30 days."

Red flag: "That won't happen to us." It's not about likelihood. It's about whether they've thought through the contingency at all.

• What recurring costs should we expect after launch?"

Good answer looks like: A breakdown of ongoing costs across four categories: infrastructure (hosting, storage, compute), model usage (tokens per thousand requests, at your expected volume), maintenance (monitoring, prompt tuning, dependency updates), and integrations (API fees from connected systems, plus the cost of fixing them when those systems change).

Example: "At your expected usage, you're looking at roughly $X in token costs per month, $Y in hosting, and we recommend budgeting $Z monthly for maintenance. Integration fees depend on your Salesforce and HubSpot tiers. Here's what to check."

Red flag: "Ongoing costs are minimal." They're not. Token costs scale with usage. Models need retuning. Integrations break when connected systems update. A vendor who waves this off hasn't run a production agent long enough to know what it costs to keep one running.

• "What are the payment milestones?"

Good answer looks like: Four to six milestone payments tied to working, demonstrable deliverables, not calendar dates.

Example: "Twenty percent on signed contract, twenty percent on architecture sign-off, thirty percent on working integration demo, thirty percent on final delivery and handover."

Red flag: Fifty percent or more upfront, or milestones tied to dates rather than deliverables. Incentives that don't require the work to be done don't require the work to get done.

‍

Questions About Security, Data, and Compliance

• "Where does our data actually go?"

Good answer looks like: A named list of subprocessors, model providers used, data retention policies, and an explicit statement that your data is not used for model training.

Example: "We use Claude via Anthropic's API. Your data is processed but not retained or used for training under their standard API terms. We don't use any third-party fine-tuning service on your data."

Red flag: "We use AI responsibly." That sentence means nothing. Ask for the subprocessor list in writing.

• "What compliance frameworks do you actually work within?"

Good answer looks like: Named frameworks with evidence, a SOC 2 report, a BAA for HIPAA, documented GDPR processes, not just a checkbox.

Example: "We're SOC 2 Type II certified. For healthcare clients, we sign a BAA and scope the build to HIPAA's technical safeguard requirements from day one."

Red flag: "We follow all relevant regulations." Ask which ones. If they hesitate, they don't have documentation to back it up.

ISO/IEC 42001, the first international AI management system standard, is becoming the reference point serious AI vendors are starting to align with, even without full certification.

• "What's your incident response plan if something goes wrong?"

Good answer looks like: A documented process: who gets notified, within what timeframe, how the agent gets isolated, and how you're kept informed.

Example: "We have a defined SLA for critical incidents, four-hour response, twenty-four-hour resolution target. The agent's write access can be killed via a single config flag. You're notified within the hour."

Red flag: "That hasn't happened to us." Past luck isn't a plan.

• "How long is our data stored, and where?"

Good answer looks like: A specific retention period, named storage locations, and a clear process for deletion, on request and at contract end.

Example: "Prompt inputs and outputs are retained for 30 days for debugging purposes, stored in an encrypted AWS bucket in [region], and deleted automatically after that window. On contract termination, we run a full deletion and send you a confirmation."

Red flag: "We don't store your data." Almost every system does, even temporarily. Logs, caches, and debugging output data touch storage at multiple points in a production agent. A vendor who claims otherwise either doesn't know their own architecture or isn't being straight with you

‍

Questions About Long-Term Support

• "What does support actually look like after launch?"

Good answer looks like: Named response times, a defined scope, and a cost structure that's agreed before delivery.

Example: "We offer a 90-day included support window covering bugs and integration issues. After that, we move to a monthly retainer covering monitoring, prompt tuning, and up to X hours of changes."

Red flag: "We'll be available if you need us." Availability without a defined scope means you have no recourse when you do need them.

• "What happens when the underlying model gets deprecated?"

Good answer looks like: A documented migration plan, how the vendor handles model version upgrades, and who pays for the retesting required.

Example: "We've migrated three clients from GPT-3.5 to GPT-4 and one from an early Claude version. The process involves re-running our eval set against the new model and updating prompts where behaviour changed. We scope it as a fixed-fee engagement."

Red flag: A blank look, or "models don't change that often." They do. Any vendor who hasn't managed a model migration hasn't been in this long enough.

• "If we want to leave, what does that actually involve?"

Good answer looks like: Full documentation, all credentials transferred, code in your repository, a structured handover process.

Example: "Our handover package includes architecture documentation, prompt versioning history, integration credentials, and a two-week overlap period where we answer questions from whoever takes over."

Red flag: Defensiveness, vague answers, or contract clauses that make departure expensive. A vendor who makes leaving hard is counting on you staying reluctantly.

Greensighter's own guide on how to fire your development team without losing your code walks through exactly what that process should look like, before you ever need it.

‍

Common Vendor Red Flags

A few patterns show up again and again with vendors you should avoid:

1. No named case studies. "We can't share client names" is sometimes legitimate. It's also the easiest cover for a thin portfolio.
2. Promises of full autonomy on day one. Anyone skipping read-only and shadow-mode rollout phases is optimizing for your wow moment, not your stability.
3. Vague answers on IP ownership. If you have to ask twice, that's the answer.
4. No questions back at you. A vendor who doesn't ask about your compliance needs, your existing systems, or your risk tolerance isn't scoping your project. They're scoping their own template.
5. Resistance to a paid pilot. Confidence in the work means confidence in starting small.
6. Heavy team turnover mid-project. If your point of contact changes three times before launch, expect the same after.

Picture Dana, the account exec who pitched you a senior team on the sales call.

By kickoff, you're on Slack with two contractors whose names you've never seen before. That's not a hypothetical; it's the most common bait-and-switch in this industry.

‍

How to Actually Evaluate the Answers

Asking the right questions only gets you halfway. You also need a way to check the answers.

Call two references, not one. Ask each one specifically what went wrong, not just what went right.

Request a small paid pilot before the full engagement. A vendor confident in their AI agent development services will agree without much friction.

Ask about a past project that failed or fell short. How they talk about it tells you more than any portfolio piece.

‍

What Actually Predicts Long-Term Success

The partnerships that hold up past launch share a pattern.

They started with a narrow, well-defined first agent, not an ambitious multi-agent system on day one. They had a named team, not a rotating cast.

They built in a maintenance plan before launch, not after something broke.

And they were honest, upfront, about what the project would cost to run, not just to build.

The technology will keep changing. The questions that protect you from a bad partnership mostly won't.

Ready to vet your shortlist the right way?

We'll review the proposals you already have, with no obligation to pick us, and flag anything that should give you pause.

You'll walk into your next vendor call knowing exactly what to ask.

Get a Second Opinion on Your Shortlist.