Menu
Close menu

Data for AI teams

Illustration

AI red teaming: test your AI before customers do

AI red teaming and evaluation for chatbots, agents and LLM apps: prompt injection, data leaks, harmful or wrong answers, a findings report and a retest.

A free call, then a fixed quote in writing. You don't have to go ahead.

Unsafe answer caught before launch

The short answer

AI red teaming is planned attack testing of a chatbot, agent or language model before customers find its weak spots. Web Vijayi's trained reviewers try to make it leak data, break its rules or give harmful or wrong answers, then score it against pass marks you set. It suits companies whose AI talks to many customers or staff.

Red teaming is one of three ways we check an AI system with people, before launch and after every change. In red teaming, reviewers try to make it fail on purpose. In RLHF they rate and rank its answers so it learns what a good answer looks like, and in evaluation they score the results against your pass marks. You get the findings, the rated data and a list of what to fix.

Key facts

  • Pass marks agreed with your teams
  • Each finding comes with the conversation that caused it
  • People check a sample of every automated score
  • Agreed tests rerun after changes

Your data

  • Data rules agreed in writing before anyone gets access
  • A test copy with made-up customer records where we can

Who is it for?

It's for companies whose AI talks to a lot of people or helps make decisions that matter: a bank's customer assistant, an insurer's claims helper, a retailer's shopping assistant, or an assistant that staff across a large company rely on for policy answers. It's also for teams that build or fine-tune their own models and need preference data or safety testing in volume.

The usual moment is just before launch. The product team has a demo that works, and the risk, legal or security team wants evidence before it reaches customers. Or the AI is already live, complaints are coming in, and nobody can say how often it gets things wrong.

What does AI alignment cover?

AI alignment is the work of making an AI system do what the business intends and avoid what it must not do. Most of it is people checking the system's answers at scale and changing the model or its rules when they find a problem.

RLHF: human feedback that teaches the model

RLHF stands for reinforcement learning from human feedback. Trained reviewers write good example answers, then compare pairs of the model's own answers and pick the better one against your written guidelines. Those choices are then used to train the model.

You get the preference data in the format your team trains on, the guidelines the reviewers followed, and a note on every question where reviewers disagreed. Research on preference data finds most disagreement comes from unclear instructions or differences in style, so we read each one and tighten the guidelines where they caused it.

RLHF is mostly for companies that train or fine-tune their own models, because preference data is what that training runs on. If you use a provider's model and cannot retrain it, the same human ratings still show where your instructions, rules or source documents need changing, and they make a good evaluation set.

Red teaming for AI: trying to break it on purpose

Red teaming for AI is a planned attempt to make your system give harmful, wrong, biased or confidential answers before real users find a way. Reviewers use attacks from the OWASP list of top risks for AI applications, such as prompt injection (hiding orders in text the AI reads), coaxing out its hidden instructions, and tricking an assistant with tools into acting beyond its permissions. They also hold long conversations, because an assistant that refuses a bad request once can give way after several turns of persuasion.

Each finding comes with the exact conversation that caused it, a severity rating and a suggested fix. Some testing is targeted, working through an agreed list of harms. Some is open-ended, looking for problems nobody has listed yet. Most projects need both.

Evaluation: measuring whether it is good enough

Evaluation turns "it seems fine" into scores you can track from one release to the next. We build a test set from your real use cases, agree a pass mark for each risk with you, and score every version of the system against it. Where an AI judge does the bulk of the scoring, people score a sample as well, so you know how far to trust the automated scores.

How does it work?

  1. We agree what could go wrong. With your product, risk and legal teams we list the harms that matter for your use, who the users are, and the pass mark for each risk. There's no universal standard: a medical assistant needs a stricter bar than an order-tracking chat.
  2. We write the guidelines and run a pilot. Reviewers get written instructions and worked examples. A small pilot shows where the instructions are unclear before the full volume starts.
  3. We choose the reviewers for the task. A strong red team mixes people who know your field, people who know attack techniques and people like your real users. We agree the mix, the languages and the markets with you.
  4. We do the work and check it. Each reviewer's ratings are compared with answers your experts have agreed and with other reviewers' ratings. Automated attack tools and AI judges handle volume, and people check a sample of everything they score.
  5. We report, and you decide. You get the findings, the scores and the data, with a plain summary for leaders. Your team decides what to fix and when.
  6. We test again after changes. A new model, prompt or tool connection can open a new weakness, so we rerun the agreed tests and compare the results with the last round.

What you get

Screens are mock-ups

You stay in control

Everything we set up works with a person in charge. Our reviewers work only to the guidelines and limits you approve, anything unusual comes to you or your team, and we keep a record of what was done. You can check the work, change it, pause it or stop it whenever you like.

What we need from you

Where it fits

If you need data to fine-tune a model, see LLM fine-tuning and AI training data. For labelling images, audio or text, see data annotation. If the worry is your wider systems and networks, see cybersecurity.

Red teaming suits any AI that talks to people or acts for them, such as a chatbot on your website, voice agents that answer calls or AI agents that take actions in your other systems.

Price

We quote every project in writing after a free consultation, and we agree the timeline in the same quote. The price depends on how many risks we test, how many answers need rating, the languages, and how much specialist knowledge the reviewers need. You'll find full details on the pricing page.

Download the brochure

AI red teaming and evaluation

Four A4 pages on how it works, what you get, what we'll need from you and the questions people ask, ready to pass round your team.

Sources

FAQ

How is AI red teaming different from a penetration test?

A penetration test probes your networks, servers and code for a way in. AI red teaming goes after the model and whatever it is connected to, using crafted prompts, long conversations and planted documents to make it misbehave. A pen test covers the systems around your AI, and red teaming covers how the AI itself behaves.

What is prompt injection?

Prompt injection is text that makes an AI follow an attacker's instructions instead of yours. Someone can type it straight into the chat, or hide it in an email, web page or file the AI reads later. It is the first risk on the OWASP Top 10 for LLM Applications (2025).

What is the difference between red teaming and evaluation?

Red teaming looks for ways to make the AI fail, and the attacks change as reviewers learn. Evaluation measures how often it gets things right on a fixed set of test questions, so you can compare one version with the next. After a round of red-team fixes, we rerun the evaluation to check the fixes worked.

Can automated tools do this instead of people?

They do part of it. Automated tools can write many attacks and score answers quickly, so we use them for volume. They miss subtle harms and mistakes in specialist fields, and AI judges score unevenly, so people check a sample of every automated score.

How often should we retest our AI?

Retest after any change to the model, its instructions, the tools it can use or the data it can reach, because each of those can open a new weakness. We rerun the agreed tests and compare the results with the last round. Many teams also run light automated checks on a schedule between full rounds.

Which frameworks and laws apply, including the EU AI Act?

Our reviewers use attacks from the OWASP Top 10 for LLM Applications (2025). NIST AI 600-1, the Generative AI Profile from the US standards body NIST, defines AI red teaming and asks for written instructions to testers. Article 55 of the EU AI Act says providers of general-purpose AI models with systemic risk must carry out and document adversarial testing. Those duties have applied since 2 August 2025, and models already on the market before that date have until 2 August 2027. Our reports record what was tested, how, and what was found, so you can keep them on file as evidence. Your legal team decides what the law asks of you.

Next step

Tell us what your AI does and who uses it. We'll suggest where testing should start and what a pilot would cover.

Book a free consultation

Last updated

Web Vijayi

Your free AI plan in two minutes

You get three things worth automating for your business, each with a reason, on screen before we ask for anything. The written quote comes later, after a free consultation.