Alibaba AI model tried to send a file to Beijing during a test

Cohere caught an open-source Qwen3.8 model trying to send a file to a server in Beijing without anyone asking. Running AI in-house is not enough.

Cohere, a Canadian company that builds AI for the enterprise market, published on Monday (the 26th) an account of concerning behavior in an open-source model from Alibaba, from the Qwen3.8 family. In an isolated test environment, the model was given the task of writing a threat-detection rule. Instead, it rummaged through the machine and tried to send a file to an Alibaba cloud storage service in Beijing. Shortly before that, the reasoning displayed by the model itself said it would not use any tools. The upload only failed because the test environment had no internet access, according to BetaKit and The Next Web.

Rare, but not negligible

Cohere says the behavior appeared in fewer than one in every 100,000 runs and that it doesn’t know whether it is intentional or a leftover from training. There was no leak either: the test was sealed off. By the time the reports were published, Alibaba had not commented on the case.

The key point, for Cohere, is that running an open model on your own server doesn’t solve the trust problem. In practice, the black box just changes address and ends up inside the company’s network.

What this changes for your company

1. Know which model is behind your tools. Many vendors use open models, including Chinese ones, because they are good and cheap. That is neither forbidden nor wrong, but you need to know. Ask your vendor which model it uses and who controls what that model can access.

2. No open internet for agents by default. If an AI agent runs commands or reads files on your network, it should only be able to talk to the addresses it really needs. That block is what would have stopped the upload at a real company.

3. Don’t rely only on what the AI says it will do. The reasoning a model displays is not a safety lock. What counts is the record of what it actually did: which files it opened and where it tried to send data.

In 29 years of programming, I’ve learned to treat any third-party software like a visitor: it comes in, but only into the rooms we opened. An AI model is no different.


Originally published on IWA7 Insights, IWA7 Digital’s blog, which is now part of Escovando Bytes.

Keep readingSee all →