Applied AI Studio · Nice, France
Raphaël Attal
builds on Claude.
An independent studio that ships LLM products for small businesses, and builds the tooling to prove those products actually work.
Flagship project: Contre-expertise
Contre-expertise audits LLM evaluations themselves. Teams trust their eval suites to catch regressions, yet those suites often miss the errors that matter. The tool measures how good an eval really is:
- mutate: injects realistic errors (numbers, dates, negations, inventions, omissions, swaps) into answers and checks whether the eval catches them;
- gap: replays the eval on realistically degraded documents to expose the distance between benchmark and production;
- labels: verifies that every ground-truth label is actually written in its source document.
First measured result (French contracts)
18%: share of injected errors detected by a widely used off-the-shelf F1 evaluator (8 out of 45). An extraction model that scored 7/8 in testing fell to 3/8 on realistic documents.
Status: private repository, version 10. A public study across about 30 open evaluation suites is planned.
Claude in production
- Customer-facing Claude chatbots deployed on three websites of an industrial inspection group.
- Internal agents for monitoring and daily digests built on Claude Haiku (scientific literature, cybersecurity news).
- Day-to-day engineering with Claude Code: every project in this studio is built with it.