Does Together AI Work in China? Prompts, Fine-Tuning Data, the Generative-AI Filing & Cross-Border Transfer
Together AI serves 200+ open-weight models and fine-tuning from infrastructure outside the mainland — its own docs offer data-residency deployments only in EU regions, with no mainland-China region — so every prompt, RAG document, and fine-tuning corpus you send it leaves China. For a China-facing product the decision isn't whether the API responds: it's whether your data may lawfully cross the border, and whether a public generative-AI feature needs a CAC filing. A compliance-first look at residency, cross-border data, the filing gate, and the lawful China AI path.
Does Together AI work in China?
Together AI has no mainland-China region — its own documentation offers data-residency deployments only in EU regions — so reaching it from China sends your prompts, fine-tuning data, and outputs to offshore infrastructure, and that cross-border transfer, not latency, is the compliance problem.
Together AI is an inference and fine-tuning cloud for 200-plus open-weight models (Llama, DeepSeek, Qwen, FLUX and more), run from infrastructure outside the mainland; by default it stores the prompts you send and the responses models return. A prompt, a retrieved RAG document, or a whole fine-tuning corpus routinely carries personal information and trade secrets, so sending it to Together's offshore inference is a PIPL cross-border transfer (Articles 38–40: notice, a separate consent, and a transfer mechanism). Offering a public generative-AI feature to mainland users also turns on a CAC filing under the Interim Measures for the Management of Generative AI Services, which an upstream API vendor does not hold for you — and serving a Chinese model such as DeepSeek or Qwen from a US data center changes neither gate.
This is a risk map, not a ruling — settle the specifics with counsel. 21YunBox maps your exposure, localizes onto a CAC-filed domestic model or an in-China sovereign cloud (never a route to the offshore one), and delivers the China-facing surface in-country on ICP-filed infrastructure. Our China team can map your exposure →
What Together AI's own documentation says about China
| Fact | Primary source |
|---|---|
| By default, Together AI stores your prompts and the model's responses. Together's documentation states that “ZDR is not enabled by default” and that “Unless your organization enables ZDR, Together stores the prompts you send and the responses models return.” Zero data retention must be switched on per organization; using your data to train models is “a separate opt-in and is off by default.” | Together AI Docs — Zero data retention (docs.together.ai), retrieved 2026-10-10 |
| Together offers data-residency deployments only in EU regions — there is no mainland-China region. Its documentation says Together “supports private networking and VPC-based deployments, including in EU regions, on dedicated endpoints,” the only residency regions it names; its Privacy Policy adds that your information “may be transferred to, and maintained on, computers located outside of your … country … where the data protection laws may differ.” | Together AI Docs — Zero data retention, and Privacy Policy §2.5 (together.ai), retrieved 2026-10-10 |
| Sending a mainland user's prompts or data offshore is a PIPL cross-border transfer. China's Personal Information Protection Law (Articles 38–40) requires a handler moving personal information out of the mainland to give notice, obtain a separate consent, and use one transfer mechanism — a CAC security assessment, the CAC standard contract, or certification. The duty sits on you, not the model vendor. | Personal Information Protection Law of the PRC, Articles 38–40 (cac.gov.cn), retrieved 2026-10-10 |
| A public generative-AI feature for the mainland needs a CAC filing. China's Interim Measures for the Management of Generative AI Services (生成式人工智能服务管理暂行办法, CAC, in force since Aug 15, 2023) govern generative content services offered to the public within China's territory, with training-data, content-labeling, and personal-information duties and a security assessment and algorithm filing for services able to shape public opinion. An upstream API vendor holds none of this for the feature you build. | Interim Measures for the Management of Generative AI Services, CAC (cac.gov.cn), retrieved 2026-10-10 |
Sources verified by the 21YunBox compliance team on 2026-10-10.
For a mainland-China audience, the first question about Together AI is not whether the API answers or how fast a token streams back — it is whether what you hand the model may lawfully leave the country, and whether the feature you build on top of it needs a Chinese filing. Together AI is an inference and fine-tuning cloud that serves 200-plus open-weight models — Llama, DeepSeek, Qwen, FLUX and more — from infrastructure outside the mainland; its own documentation offers data-residency deployments only in EU regions, and there is no Together region or entity inside China. What you send it is never just a question: prompts, retrieved documents, and whole fine-tuning corpora carry personal information and, often, trade secrets. By default, Together even stores the prompts you send and the responses the model returns. So the decision here is a compliance one, not a speed one.
Together AI in China at a glance
| What decides it | In Together AI's own terms — and China's law |
|---|---|
| Where the inference actually runs | Together serves open-model inference, fine-tuning, and GPU clusters from infrastructure outside the mainland. Its own docs offer data-residency deployments only "in EU regions, on dedicated endpoints" — there is no mainland-China region. Even the Chinese open-weight models it hosts (DeepSeek, Qwen) run from that offshore infrastructure, which does not make a call to them China-local. |
| What you send it, and why it is personal information | Prompts, system context, retrieved RAG documents, fine-tuning files (often an entire proprietary corpus), and the model's outputs. These routinely carry names, contact details, account and order records, and the secrets embedded in source — personal information under PIPL, and frequently trade secrets. Where IDs, financial, health, biometric, or minors' data appear, that is sensitive personal information under PIPL Article 28. |
| Your China users' prompts and your corpus = a cross-border transfer | Sent from a user in the mainland to Together's offshore inference, each of these is a cross-border transfer of personal information under PIPL (Articles 38–40): notice, a separate consent, and one transfer mechanism — a CAC security assessment, the CAC standard contract, or certification. By default Together stores the prompts you send and the responses models return; zero data retention is opt-in and off by default. |
| The generative-AI filing gate for a public mainland feature | Offering a public-facing generative-AI service inside the mainland turns on China's Interim Measures for the Management of Generative AI Services (生成式人工智能服务管理暂行办法, CAC, in force since Aug 15, 2023) — training-data and content-labeling duties, personal-information handling, and a security assessment and algorithm filing for services able to shape public opinion. An upstream API vendor holds none of this for you. |
| Reachability is not the axis | That the endpoint responds settles nothing. The decision turns on the cross-border transfer and the filing gate above — and the China-facing surface that carries the feature (the app, the chat UI, the API edge your mainland users hit) is itself a public service in China, so it needs an ICP filing (备案) and compliant in-country delivery. |
No mainland region, so your prompts, fine-tuning data, and outputs leave the country
Together’s position is set by where it runs, not by a load-time test. Together AI is a US company; its serverless inference runs offshore, and its documentation offers data-residency deployments only “in EU regions, on dedicated endpoints” — it publishes no mainland-China region and no in-country service there. A revealing detail follows from the model catalog: Together serves popular Chinese open-weight models — DeepSeek and Qwen among them — but from that same offshore infrastructure. Running a Chinese-origin model from a US or EU data center does not make the call China-local or China-compliant, because what governs the decision is where the data is processed and under whose filing the service is offered, not where the model was first trained.
For that reason this page publishes no first-party China latency figure for Together: a number captured on one day from one vantage point would say nothing durable, and speed is the wrong axis for a service processed in another jurisdiction. There is no lawful way to turn an offshore endpoint into an in-country one; the productive question is a different one — how to put generative AI in front of Chinese users lawfully, on infrastructure that sits inside the country.
What you send it is personal information — and often a whole proprietary corpus
A prompt is rarely just a question. On Together you send prompts, system context, retrieved RAG documents, and — distinctively — fine-tuning files that can amount to an entire proprietary corpus, plus the outputs the model returns. These routinely carry names, contact details, order and account records, internal documents, and, for AI coding and data work, source and the credentials and PII embedded in it. Under China’s Personal Information Protection Law that is personal information, and much of it is also trade-secret material. Where a prompt or a dataset includes government IDs, financial, health, or biometric data, or anything about minors, it is sensitive personal information under PIPL Article 28, which raises the handling bar further.
Two honest details about Together’s own handling. By default it stores the prompts you send and the responses models return — zero data retention is a per-organization setting that is off by default — and using your data to train models is a separate opt-in that is also off by default. Those controls matter, but they govern what Together keeps, not where your data goes: sent from a user in China to Together’s offshore inference, every prompt, document, and fine-tuning file is a cross-border transfer of personal information under PIPL, and the duty sits on you, the handler, not the model vendor. PIPL Articles 38–40 require notice, a separate consent distinct from the user’s agreement to use the feature, and one transfer mechanism — a CAC security assessment, the CAC standard contract, or certification. For a critical information infrastructure operator, the data-localization duty in Cybersecurity Law Article 39 (formerly Article 37) — the 2025 Cybersecurity Law amendment, in force January 1, 2026, renumbered the data-localization article from 37 to 39, substance unchanged — requires personal information and important data to be stored inside China, which an offshore endpoint cannot satisfy.
The filing gate — and why trimming retention doesn’t close the door
Suppose you settle the data question and still want a “generate” button, an assistant, or a chatbot in front of mainland users. The first gate is not which model you picked on Together — it is whether you may offer a public-facing generative-AI service in China at all. China’s Interim Measures for the Management of Generative AI Services (生成式人工智能服务管理暂行办法, Cyberspace Administration of China, in force since August 15, 2023) are the national rules written for exactly this. They govern providing services that generate text, images, audio, and video “to the public within the territory of the People’s Republic of China,” and they set obligations on training data, on labeling AI-generated content (reinforced by the 2025 AI-content labeling rules), on personal-information handling, and — for services able to shape public opinion — a prior security assessment and an algorithm filing. Where your feature also ranks or recommends content, the Algorithm Recommendation Provisions filing may apply too. Together, an upstream API vendor, holds none of these filings for you; the duty attaches to the operator who actually serves the feature to the public in the mainland.
This is why trimming retention does not close the door. Enabling zero data retention, or choosing an EU dedicated endpoint, changes what crosses the border and how long it lives — not that it crosses. The prompt still travels from the mainland to offshore infrastructure to be served, the residency position is unchanged, and the filing duty on a public mainland feature is untouched. Underneath both, the China-facing surface that carries the feature is itself a public service in China, so it carries an ICP filing (备案) duty and needs compliant, in-country delivery like any other China-facing property. None of this turns on how fast a completion streams back; it turns on whether the data had a lawful basis to leave the country and whether the public feature is filed. This page maps exposure; it is not a ruling — whether and how the Measures, PIPL, and the ICP duty apply to your specific feature is something to settle with qualified counsel against what you actually ship.
The lawful path — map, localize, deliver
There is a lawful way to put a Together-style AI feature in front of Chinese users, and it has a shape: the model is served from inside China by an operator that holds the filings, and the data stays on a compliant footing. Because Together already serves Chinese open models from offshore, the localize move is often a clean one — not a route back to Together’s endpoint, but the same class of model stood up in-country through its domestic operator: a CAC-filed domestic model such as Alibaba’s Qwen (通义千问) or Baidu’s Ernie (文心一言), or an in-China sovereign-cloud offering operated by a licensed local operator — each subject to its own availability and terms, which you confirm with the operator and with counsel before you build.
Underneath that choice sits the part 21YunBox owns, and it is more than advice. Our China team does three things. We map your PIPL cross-border, generative-AI filing, sensitive-personal-information, and ICP obligations against your entity, your data volumes, and who your users are. We localize — standing up and integrating a CAC-filed domestic model or an in-China sovereign-cloud model in place of the offshore Together call that cannot lawfully serve mainland users, keeping consented, in-country processing and storage for what must stay on Chinese soil (and self-hosted where a model’s license allows). And we deliver any China-facing surface — the app, the chat UI, the API edge your mainland users hit — over ICP-filed, in-country infrastructure, the 21YunBox Optimizer, in front of the stack you already run, with no rebuild and no second codebase. 21YunBox is a compliant overlay and a partner to the vendors you use, not a migration and not a competitor. For a service with no mainland region, localize means standing up a lawful China-legal alternative, never a route to the offshore model: 21YunBox never uses or suggests circumvention of any kind. The result is a generative-AI feature that runs legally and compliantly for your users in China.
Related reading:
- China’s Interim Measures for the Management of Generative AI Services
- Cross-border data transfers under PIPL
- How to get an ICP filing for China
- China’s Cybersecurity Law — data localization (Article 39)
