AI evaluation & quality intelligence

Know where your AI fails
before your users do.

Latent Lens tests LLM apps, RAG pipelines and AI agents for factual accuracy, hallucinations and instruction adherence, then tells your team exactly how to fix them.

13FAILURE TYPES CLASSIFIED
100%SCORES COMPUTED IN CODE
3FREE CHECKS A DAY
sample verdictrefund bot
Faithfulness40
Instruction adherence50
Overall51
Numeric drift · "refund within 60 days" vs source: 30 days
Model-agnostic · checks answers from
  • OpenAI
  • Anthropic
  • Google Gemini
  • Meta Llama
  • Mistral
  • Cohere
  • Amazon Bedrock
  • Azure OpenAI
Names are trademarks of their owners. No affiliation or endorsement implied.
The problem

Your model sounds confident. That is the problem. A chatbot is built to produce answers. An evaluation platform is built to judge them, with fixed rubrics, separated judges and scores you can reproduce next week.

Not a chatbot wrapper.An instrument.

The core architectural difference: a chatbot optimises the next reply. Latent Lens treats every response as evidence and runs it through a deterministic, versioned evaluation pipeline.

Chatbot

Generates

  • One model writes and grades its own output
  • Stateful conversation, temperature > 0
  • Free-text answers, nothing to compare
  • Success = the user stopped asking
Latent Lens

Measures

  • Judge is isolated from the generator
  • Pinned rubric and model versions
  • Structured labels per claim and per instruction
  • Every run stored, comparable and exportable
Platform

Six lenses onevery response.

01 / FAITHFULNESS

Claim-level grounding

Responses are split into atomic claims. Each one is checked against your ground truth and labeled supported, partial, contradicted or unsupported, with the exact evidence quoted.

02 / HALLUCINATIONS

Drift & fabrication

Numeric drift, invented entities and flipped polarity are caught and classified, not just flagged. The HHEM model checks meaning on top.

03 / INSTRUCTIONS

Instruction adherence

Length, format, language, exclusions and required items are extracted from the prompt and verified one by one.

04 / CALIBRATED SCORES

Labels in, scores in code

Models only assign labels. Scores are computed in code from those labels with a versioned rubric, so a run is reproducible and the rubric can be audited against human labels.

05 / TAXONOMY

Failure taxonomy

Every defect is mapped to a type and a severity, so you can see which kinds of failure are growing across releases.

06 / REMEDIATION

Actionable fixes

Each failure type comes with a concrete remediation step your engineers can ship: a prompt change, a retrieval fix or a post-generation guard.

How it works

Five frames.One verdict.

Each evaluation is an async job: queued, processed by workers, stored in Postgres and sent back to your screen and your dashboard.

01INGEST

Prompt, response, ground truth.

Submitted from the checker. Validated, hashed and queued as an evaluation job.

POST /v1/evaluations { prompt, response, context? } -> 202 { id, status: "queued" }
02DECOMPOSE

Claims and constraints.

The response is split into atomic claims. Explicit instructions are extracted from the prompt.

claims: 5 constraints: 2 "refund within 60 days" "avoid: store credit"
03JUDGE

Deterministic checks + HHEM.

Numbers, entities and contradictions are caught in code first. An open hallucination model checks meaning; an LLM judge can be added.

label: "contradicted" evidence: "...within 30 days..." failure: "Numeric drift"
04CALIBRATE

Scores computed in code.

Labels become scores through a fixed, versioned rubric. A hand-labeled test set runs on every engine change.

faithfulness = (sup + .5*partial) / n rubric: ll-rubric-0.4
05REPORT

Verdict, taxonomy, fixes.

Stored per user, comparable across runs and exportable as CSV, JSON or a branded PDF.

overall: 51 verdict: "High risk" failures: 4 remediation: 4
Illustrative sample report

A refund bot,under the lens.

Sample dataA fictional refund bot, scored by the real engine. Hover the highlighted claims, or switch tabs to see the failures, the fixes and the structured output your CI pipeline would read.

supportedpartialcontradicted / unsupported
You can request a full refund within 60 days of purchase. Refunds go back to your original payment method within 5 business days. Digital products are refundable only if not downloaded. Shipping fees are also refunded in full. If you prefer, we can issue store credit instead, approved by our Lisbon support team.
✕
You can request a full refund within 60 days of purchase.Figure "60" does not match the source. Evidence: "Customers can request a full refund within 30 days of purchase."
contradicted
✓
Refunds go back to your original payment method within 5 business days.Every key term is present in the source. Evidence: "Refunds are issued to the original payment method within 5 business days."
supported
✓
Digital products are refundable only if not downloaded.Every key term is present in the source. Evidence: "Digital products are refundable only if they have not been downloaded."
supported
✕
Shipping fees are also refunded in full.Polarity is flipped against the closest source sentence. Evidence: "Shipping fees are non-refundable."
contradicted
✕
If you prefer, we can issue store credit instead, approved by our Lisbon support team.Entity "lisbon" is not in the source.
unsupported
5 claims2 instructionsrubric ll-rubric-0.4engine heuristic-1.3

Want a human expert to check your AI properly?

One report shows one answer. An audit runs a sample of your real conversations and tells you which fixes matter first.

Request an expert review

Trust is measured,
not assumed.

Architecture

Built likeinfrastructure.

Next.js front end, FastAPI evaluation service with a Redis job queue, Supabase for accounts and Postgres. Judges are swappable behind one interface. Hosted in the EU.

Next.js appFastAPISupabase AuthJob queueEval workersPostgresTS + Tailwind/v1/evaluationsJWT / RLSRedisrules + HHEM + judgeruns, reports, leads
FrontendNext.js 16, TypeScript, Tailwind, server actions
Evaluation APIFastAPI, Pydantic schemas, Redis queue and workers
Data & authSupabase Postgres and Auth, row-level security
JudgesHHEM-2.1-Open, optional OpenAI, Claude or Gemini
Honest status

What's live,what's next.

No vapourware. Everything is marked by where it stands today, so you know exactly what you can use now.

Live now
  • Free Response Checker with the evaluation engine
  • Claim grounding, instruction checks, 13 failure types
  • HHEM meaning check on grounded claims
  • Accounts, history, run comparison and exports
  • Branded PDF reports
  • Expert review on every result
  • Lens Assistant with hand-off to a person
Available in a pilot
  • Evaluation on your own datasets and scenarios
  • RAG evaluation against your documents
  • LLM judge (OpenAI, Claude or Gemini) for meaning-level checks
  • Human expert review of every finding
Roadmap
  • Batch evaluations from CSV and API
  • Agent trace evaluation
  • CI regression gates per release
  • Shareable report links
  • Team workspaces and billing

Ask about any of these as part of a Custom Pilot.

Three ways to work with us

Start free.Scale with proof.

Automated evaluation

Free Checker

Free

Check individual AI responses and get scores, identified failures, explanations and recommendations in seconds.

  • Scores for faithfulness, adherence and relevance
  • Failures classified, with explanations
  • A recommended fix for every issue
  • History, exports and PDF in your dashboard
Open the checker
Human-led AI evaluation

Quality Audits

Fixed fee

Our evaluation experts assess your AI outputs against defined quality criteria. The core of the audit is expert human judgement.

  • Findings validated by evaluation experts
  • Errors classified, failure patterns analysed
  • Detailed evaluation report with priorities
  • NDA available on request
Request an audit
Evaluation system for your business

Custom Pilots

Scoped per project

We test your specific AI application or workflow against requirements tailored to your business.

  • Your datasets and multiple test scenarios
  • Automated evaluation pipelines and integrations
  • Ongoing testing as your AI changes
  • Human evaluation where it is needed, NDA on request
Discuss a pilot
Audits & pilots

Tell us what you ship. We will show you where it breaks.

Every request goes straight to our team. Tell us about your AI product and we come back with a clear next step.

What do you need?
We reply within one business day. Your details are only used to answer you (privacy).
FAQ

Straightanswers.

Written to be quoted. Every answer here is also published as structured data for search engines and AI assistants.

What is the difference between an AI chatbot and an AI evaluation platform?

A chatbot generates answers. An evaluation platform judges answers produced by another system, using an isolated judge, pinned rubric and model versions, schema-validated outputs and stored runs that can be reproduced and compared.

How does Latent Lens detect hallucinations?

The response is split into atomic claims and each claim is checked against the supplied ground truth. Claims are labeled supported, partial, contradicted or unsupported, and failures such as numeric drift, fabricated entities and contradictions are classified with a remediation step. An open hallucination model (HHEM) also checks meaning.

Do I need an OpenAI API key?

No. The evaluation engine runs without any key. An LLM judge from OpenAI, Anthropic or Google can be switched on later for deeper meaning-level checks, and the scoring stays the same.

How are evaluation scores calibrated?

Models only assign labels. Scores are computed in code from those labels, the rubric version is stamped on every run, and the engine is tested against a hand-labeled set on every change, so the same input always gives the same score.

Is the Response Checker really free?

Yes. You get 3 free checks a day without an account. Create a free account to save your checks, compare runs and export them as CSV, JSON or PDF.

Do you store what I paste into the checker?

Not without an account: anonymous checks never store your prompt, response or context. Signed-in users can turn off input storage in their settings or for a single check, and can delete their account and all data at any time.

Ready when you are

Put your AI
under the lens.