Brazil's ANPD tests three companies' AI: documents aren't enough, evidence is
A report from the ANPD sandbox shows flaws in references, warnings and third-party APIs in real AI solutions. What it teaches anyone who uses or buys AI in Brazil.

The ANPD, Brazil’s data protection authority, published on October 16 the third partial report of its AI regulatory sandbox, the environment in which the agency follows real solutions under supervision, with support from CIAAM/USP (a center at the University of São Paulo). The participants are Synapse (mobility), Prevvine (healthcare) and Metatext (financial sector). The report doesn’t approve the solutions or prove compliance. Even so, it’s a concrete picture of what the authority examines in an AI system that’s up and running. The topic comes back this Friday (the 6th), at a seminar in Brasília with agency officials.
What the tests found
On Prevvine’s medical platform, evaluated with 30 simulated clinical cases, the module that checks references dropped one maturity level after the tests found hallucinations, nonexistent references and broken links. It failed criteria such as stating usage limits, sources, the confidence of its answers and situations in which the system should not be used, such as emergencies.
In Synapse’s app, for passenger flow on rail transport, no personal data showed up in plain text in the traffic analyzed. The risk sat in an external layer: calls to a maps API may allow that third party to correlate data and know where the person is. Multi-factor authentication had also not yet been implemented.
Metatext had a mostly documentary cycle, and its tests were left for the next stage.
Finding problems is what a sandbox is for. According to the ANPD, the results may guide future guidelines and rules. The phrase that sticks is the need to separate “formal documentation from material evidence”.
What this changes for your company
1. Swap documents for evidence. Policies and impact assessments are necessary, but they don’t prove that a control works. For every important use of AI, keep the test that was done, the date, the result and what was fixed.
2. Follow the data to the end of its path. List the APIs and external services each solution calls, including those contracted by your vendors. If you want a starting point, our free self-assessment points out risk signals in 2 minutes.
3. Show the AI’s limits. Whoever uses the answer needs to know the source, the degree of uncertainty and when to call a person. In customer service, credit or HR, that’s part of the product, not fine print.
In 29 years of programming, I’ve seen plenty of well-documented systems fail their first real test. With AI, that test has to come before the customer does.
Originally published on IWA7 Insights, IWA7 Digital’s blog, which is now part of Escovando Bytes.


