AI Vendor Risk Assessment: The Questions to Ask Before You Buy an AI Tool
A practical AI vendor risk assessment for small businesses: what it is, the twelve questions that cover data, security, access, and exit, how to read the answers, the red flags, and a one-page record you can keep for every tool you approve.
An AI vendor risk assessment is a short, structured check you run on an AI tool before your business starts using it. It answers four questions: what happens to the data you put in, how well the vendor protects it, what the tool will be able to reach inside your business, and how hard it will be to leave. For a small business it does not need a consultant or a scoring spreadsheet. It needs twelve questions, the vendor’s own documents, and about an hour per tool.
The timing is the point. Once a tool is connected to your email, your drive, or your customer records, changing course is expensive. Before you sign, you still have all the leverage.
How is this different from an AI security audit?
An AI security audit reviews AI systems you have already deployed: what they can read, what they can do, how they handle hostile input. A vendor risk assessment comes earlier. It decides whether a tool should be deployed at all, and on what terms.
The two fit together. The assessment catches problems you can still avoid by choosing a different vendor or a different plan. The audit catches problems that appeared after setup, usually through permissions that grew over time. If you have a governance framework that says who approves new tools, this assessment is what the approver should look at before saying yes.
Why AI tools need their own vendor check
You probably already accept software vendors without much review, and for most of them that is fine. AI tools differ in three ways that justify an extra hour.
They see more. An AI tool is useful in proportion to how much context you give it, so people paste in contracts, client emails, and financial details that would never go into a project management app.
Training and retention terms vary. Some vendors use customer inputs to improve their models by default, some only on consumer plans, some never. Retention periods for prompts and outputs range from zero to indefinite. The same vendor often has different terms on different plans.
They increasingly ask for connections. Many AI tools now want access to your mailbox, calendar, file storage, or CRM. A connected tool inherits the reach of whatever account authorized it.
The twelve questions
Ask these in writing, or find the answers in the vendor’s terms, privacy policy, data processing agreement, and security or trust page. Most reputable vendors publish nearly all of it. Write the answer and its source next to each question.
Data use
1. Is our data used to train or improve your models? You want a clear “no” for your plan, stated in the contract or data processing terms, not only in a blog post. If the answer is “not by default,” find out who can turn it on and whether an individual user can opt your company in without you knowing.
2. How long are prompts, files, and outputs retained, and can we delete them? Look for a specific number of days and a deletion mechanism. Note whether retention differs for abuse monitoring, which some providers keep separately for a limited period.
3. Where is the data processed and stored? This matters if you have clients with contractual location requirements, or regulated data. I covered the details in AI data residency.
4. Which sub-processors handle our data? Most AI tools are built on top of a model provider, so your data passes through at least two companies. Ask for the sub-processor list. A vendor that cannot tell you which model provider it uses cannot tell you where your data goes.
Security
5. What independent security attestation do you have? For US vendors, the common one is a SOC 2 report. A Type II report covers whether controls actually operated over a period of months, which is stronger evidence than a Type I, which only describes the design at a point in time. ISO/IEC 27001 certification is the common international equivalent. Ask for the report or the certificate, and check the date. Small vendors may have neither, which is not automatically disqualifying, but then the remaining answers need to be stronger.
6. Do you support single sign-on or enforced multi-factor authentication? Logins are the most common way business accounts get compromised. If you cannot require MFA for everyone on your team, every user’s password becomes your weakest link.
7. How do you notify customers of a security incident, and how fast? Look for a stated commitment in the contract. “We take security seriously” is not an answer.
Access and control
8. What does the tool need access to, and can it be narrowed? If the tool connects to email or file storage, check the permission screen before approving it. Can you limit it to one folder, one mailbox, or read-only access? A tool that requires full access to your whole drive for a feature that only needs one folder is a design decision you should weigh.
9. Can it take actions, or only read and suggest? Drafting an email is different from sending one. For any tool that can send, change, delete, or pay, ask whether actions can require human approval. This is the same excessive-agency risk the AI security audit checks for after deployment. It is far easier to choose a tool that supports approvals than to add them later.
10. Can we see what users asked and what the tool did? Admin logs let you answer a client’s question, investigate a mistake, and remove a departed employee’s access with confidence. Consumer plans usually have none.
Exit and fit
11. How do we get our data out, and what happens to it when we leave? Ask for the export format and the deletion timeline after cancellation. If you are building workflows, prompts, or knowledge bases inside the tool, you want to be able to take them with you.
12. Will you sign the agreement our data requires? If you handle health information, you need a Business Associate Agreement under HIPAA before patient data goes in. If you work with European clients, you may need a data processing agreement. If a client contract restricts subcontractors or data locations, check the AI vendor against it. A vendor that will not sign the required agreement is ruled out for that data, however good the tool is.
How to read the answers
You are not looking for perfect scores. You are looking for a match between the tool’s risk and the data you plan to give it.
A writing assistant that will only ever see marketing copy can pass with modest answers. A tool that reads client files, connects to your mailbox, and can send on your behalf needs strong answers on every question, especially 1, 5, 8, and 9.
A useful habit is to decide the data class first. Write down “this tool will see public content,” or “internal business data,” or “client confidential or regulated data,” then judge the answers against that. It turns a vague feeling into a decision you can explain later.
Red flags
- Training terms that differ between the marketing page and the contract. The contract wins, so read it.
- No sub-processor list, or an answer that the underlying model provider is confidential.
- Business features only on a plan you are not buying. Many vendors offer no-training, admin controls, and SSO only on business or enterprise plans. Confirm the plan you are pricing actually includes them.
- All-or-nothing permissions for connected tools, with no way to limit scope.
- No way to export your data, or deletion “upon request” with no timeline.
- Pressure to connect everything during the free trial. Trials are for testing on non-sensitive data.
One red flag is a question for the vendor. Several is an answer.
The one-page record
For every AI tool you approve, keep a short record: the tool and plan, the data class it is approved for, the twelve answers with their sources, who approved it and when, and a review date a year out. A shared document or a tab in a spreadsheet is enough.
This record pays for itself the first time a client sends a security questionnaire asking which AI tools touch their data. It also gives the person running your next AI security audit a starting inventory, and it makes it obvious when a vendor changes its terms and the approval needs another look.
When to bring in outside help
You can run this assessment yourself for most tools. Outside review earns its cost when the tool will handle regulated data, when it will have write access to systems that touch customers or money, when a client contract imposes specific security terms, or when you are choosing a platform that a lot of future work will be built on. In those cases a second opinion on the architecture can matter more than the vendor’s answers. If you are weighing whether to run AI in your own cloud instead, Azure OpenAI consulting covers that decision.
Frequently asked questions
What is an AI vendor risk assessment?
An AI vendor risk assessment is a review of an AI tool before a business adopts it, covering how the vendor uses and retains customer data, where data is processed, which sub-processors are involved, what security attestations the vendor holds, what access the tool requires, whether it can take actions, what logging it provides, and how data can be exported and deleted on exit.
What questions should I ask an AI vendor?
Ask whether your data is used for training, how long it is retained, where it is processed, which sub-processors handle it, what independent security reports the vendor has, whether MFA or SSO can be enforced, how incidents are reported, what access the tool needs and whether it can be narrowed, whether it can take actions without approval, what admin logs exist, how to export and delete data, and whether the vendor will sign the agreements your data requires.
Does a small business need an AI risk assessment?
For any AI tool that will see client information, internal financials, or regulated data, yes. The assessment takes about an hour per tool using the vendor’s published documents. For tools that only handle public content, a quick check of training and retention terms is usually enough.
What is the difference between SOC 2 Type I and Type II?
A SOC 2 Type I report describes a company’s security controls and whether they are suitably designed at a single point in time. A Type II report also tests whether those controls operated effectively over a period, typically several months to a year. Type II is the stronger evidence.
Is a free AI tool riskier than a paid one?
Often, because free and consumer plans are more likely to use inputs for training, to lack admin controls and logs, and to sit outside any business agreement. Price alone does not settle it, though. Check the terms of the specific plan you would use.
The one-sentence version
Before any AI tool touches business data, find out in writing what the vendor does with that data, how they protect it, what the tool can reach and do inside your business, and how you get out, then match those answers to the most sensitive data the tool will see.
If you want help evaluating a tool before you commit, or you would rather have an AI work environment where these questions are settled by design, that is what my secure AI work covers.
Jose Lugo is a CISSP-certified security engineer with 12 years of U.S. Army intelligence experience. He builds secure AI work environments and fast, maintainable websites for businesses at josecustom.ai. See his portfolio of 13 live client systems at portfolio.josecustom.ai.