Sovereign AI: GDPR-compliant AI in Europe

The European AI regime has entered its concrete application phase. The AI Act, adopted in 2024, deploys its obligations on a staggered calendar. Prohibited practices (social scoring, manipulation, real-time biometrics in public spaces) have applied since February 2025. Obligations on general-purpose models (GPAI) — training-corpus transparency, technical documentation, copyright compliance — have applied since August 2025. The full obligations on high-risk systems (HR management, justice, critical infrastructure) enter into force in August 2026. On top of that come data governance, audit, and traceability requirements.

The GDPR remains the underlying matrix. An AI inference pipeline that processes personal data triggers the same obligations as a classic processing operation: lawful basis (art. 6), minimisation (art. 5), data-subject rights (art. 15 to 22), processor delegation (art. 28). A US API receiving French candidates' CVs creates an international transfer subject to appropriate safeguards (art. 44 to 49), and the Schrems II case law severely limits the viable options.

The CNIL, in its AI recommendations published between 2024 and 2026, has laid down a clear doctrine: reinforced anonymisation of training corpora, a detailed register of AI processing operations (per art. 30), a mandatory impact assessment (DPIA) for any system that profiles, scores, or processes sensitive data. The CNIL has also published an experimentation framework (sandbox) that allows testing an AI system under supervision — a non-negligible benefit for an SMB that wants to get started.

For a French CIO or CTO in 2026, the reality is simple: as soon as personal or sensitive data enters an AI chain, the architecture must be designed to answer these three frameworks simultaneously. Sovereign AI is not a slogan: it is the set of technical decisions that make this compliance possible without penalising production.

Sovereignty does not mean technological nationalism. It is a set of four verifiable technical properties. First, data residency: no personal data transits or is stored outside the European Union during inference, fine-tuning, or evaluation. The payload containing a CV, a support conversation, a medical file stays within an EU perimeter at every step.

Second, model governance. A sovereign model is a model whose training corpus you know (at minimum its categorical composition), usage licence, evaluation process, and update scheme. Mistral Large, Llama 3.3, and Gemma 2 publish their model cards and usage policies. A closed proprietary model behind an API operated outside the EU gives none of these guarantees, and the CNIL considers this a compliance risk.

Third, the audit trail. Every inference is logged: timestamp, user identifier, model version, input prompt (or its hash if too sensitive), produced answer. Without this trace, a GDPR access request on an AI processing operation is impossible to honour, and a CNIL inspection becomes a direct risk. On our projects, these logs go into a dedicated encrypted bucket, 13-month retention, access restricted to the DPO and the security team.

Fourth, absence of structural dependency. If your AI pipeline relies entirely on a proprietary non-EU API, you are at the mercy of a terms-of-service change, a price increase, a service interruption, or a Cloud Act order granting access to the data. Diversification (multiple EU suppliers, failover capability) is operational insurance, not a luxury.

The model landscape has thickened. Mistral AI, with Mistral Large 2 and its successor Mistral Large 3, Codestral for code generation, and Pixtral for vision-text multimodality, has become the reference European player. The company offers both proprietary models via its API (hosted in France) and open-weight models (Mistral Small, Mixtral) under Apache 2.0 or Mistral Research licences, deployable on the infrastructure of your choice.

Aleph Alpha, in Germany, positions its Luminance models on sovereign segments (public sector, defence, finance) with an offering that includes audit and certification. Silo AI, acquired by AMD, offers Scandinavian open-source models like Poro and Viking, optimised for non-English European languages. The Hugging Face catalogue, though American, now hosts the majority of European open-weight models and has provided EU inference infrastructure since 2025.

On the open-weight side, the trio Mistral, Llama 3.3 (Meta, but deployable on an EU cloud), and Gemma 2 (Google, same logic) covers 90% of SMB use cases. Mistral Small 3 rivals GPT-4 mini on French-language tasks, Llama 3.3 70B remains the most versatile option for controlled self-hosting, and Gemma 2 9B covers light workloads with modest infrastructure cost.

The winning strategy in 2026 is no longer picking a single model. It is building an abstraction layer (an LLM router, like LiteLLM or an in-house router) that can switch between several models based on cost, latency, and the sensitivity of the request. Mistral Large for complex tasks, Mistral Small for routing, a self-hosted Llama 3.3 70B for highly sensitive data. This agility is the key to resilience.

The European cloud now offers credible AI coverage. OVHcloud, the historic French player, exposes its AI Deploy and AI Training offerings on H100 and H200 GPU instances, with a choice of regions (Gravelines, Roubaix, Limburg, Warsaw) all inside the European Union. Scaleway, an Iliad subsidiary, offers hourly H100 GPU instances and a serverless Managed Inference that simplifies deploying Llama or Mistral in minutes. Hetzner and Outscale round out the offering — the former more cost-oriented, the latter sovereignty-oriented.

For sensitive workloads, the SecNumCloud certification (level 2, issued by ANSSI) is the reference framework. It guarantees the absence of non-European access to data, transparency on the applicable legal terms, and a strict separation between clients. OVHcloud (Sovereign Cloud offering) and Outscale hold this certification. Bleu, the Orange-Capgemini-Thales consortium formed in 2024, is ramping up on this segment with a secured sovereign AI offering.

The GPU question remains the friction point. Europe does not have Nvidia's production capacity, and the paradox is real: even sovereign clouds buy American H100s. Initiatives are multiplying. LUMI, the European supercomputer installed in Finland, provides GPU capacity for research and industry. EuroAI, the AI consortium announced in 2025, aims to federate a full supply chain from chip to application. Availability remains lower than in the United States but improves continuously, and the price/performance ratio of EU instances has caught up since 2024.

For an SMB, the decision criterion is clear: if your data is sensitive (health, defence, HR, public sector), require SecNumCloud or at minimum EU hosting with a signed DPA and contractual guarantees. If your data is non-sensitive (marketing, public code), an EU region of a generalist cloud suffices. The premium for a certified sovereign cloud is 20 to 35% on average — to be weighed against the legal and reputational risk.

Two economic models compete. The first, the commercial API (Mistral Large, Aleph Alpha Luminance), bills per usage. An input token costs between 0.2 and 2 euros per million depending on the model, an output token between 0.6 and 6 euros. For an SMB processing 10,000 requests per day averaging 1,500 tokens, the monthly Mistral Large bill runs around 1,500 to 3,500 euros. Advantages: zero capex, production in days, no infrastructure management.

The second, self-hosting an open-weight model (Llama 3.3 70B, Mistral Small 3), radically changes the cost structure. The model runs on 2 to 4 H100 GPUs (the 70B needs about 140 GB of VRAM in FP16, i.e. 2 H100s). On OVHcloud or Scaleway, an H100 instance costs 1.8 to 2.5 euros per hour, roughly 1,500 to 2,200 euros per GPU per month. A highly-available 4-H100 setup therefore represents 10,000 to 18,000 euros per month, plus 1 FTE for operations (MLOps, supervision, updates, vLLM/SGLang tuning).

The switch-over threshold sits around 100,000 to 150,000 tokens per minute of sustained load. Below that, the Mistral API wins hands down. Above, self-hosting becomes economically competitive and offers three structural advantages: total data control (no potential leakage to a vendor), latency control (typically 200 to 400 ms locally vs 600 to 1,200 ms via API depending on the zone), and the ability to fine-tune the model on your business corpora.

Our recommendation for an SMB in 2026: start with the Mistral or Aleph Alpha API on non-critical cases (3 months to validate usage), then migrate to self-hosted Llama 3.3 70B only for critical sovereign cases where the data must never leave your perimeter. The hybrid model is king: 80% of usage on API, 20% self-hosted, managed by an LLM router that routes based on the request's sensitivity classification.

The whole difficulty of a sovereign AI project is avoiding two symmetrical pitfalls. The first: going fully sovereign on principle, accepting a 30% premium and operational debt the team cannot carry. The second: sending everything to American APIs and suffering a GDPR incident within 18 months. The 80/20 method avoids both, by segmenting use cases by level of requirement.

Three categories emerge. Category S1 (sovereignty required) covers: HR data (CVs, evaluations, interviews), health (patient records, telemedicine), defence and national sovereignty, public sector and public procurement, regulated finance, sensitive client data (banking, contracts). For these uses: Mistral or Aleph Alpha API, or failing that self-hosted Llama 3.3 70B on SecNumCloud infrastructure, signed DPA, documented DPIA, timestamped audit logs.

Category S2 (sovereignty desirable) concerns: customer support with moderate personal data, AI-enriched CRM, personalised marketing content generation, customer-feedback analysis. For these uses: a European API (Mistral, Hugging Face EU) or an American API with appropriate safeguards (SCCs, in-transit encryption, prompt anonymisation where possible). This is the grey zone where the cost/benefit ratio must be computed case by case.

Category S3 (sovereignty not required) covers: public marketing content generation, code assistance on public codebases, translation of non-sensitive content, technical documentation writing from public sources, creative ideation. For these uses, a generalist API (Claude, GPT, Gemini) is acceptable and economically optimal. The empirical rule on our 2026 SMB projects: about 20% of uses in S1, 30% in S2, 50% in S3. The 80/20 ratio (80% non-sovereign usage, 20% sovereign) reflects reality and lets you focus investment where it counts.

The practical roadmap fits in five steps over one quarter. First, map existing and planned AI use cases, classifying them S1/S2/S3. Second, sign the DPAs with EU suppliers (Mistral, OVHcloud, Scaleway). Third, set up an LLM router with automatic sensitivity classification. Fourth, deploy the audit logs and the AI processing register. Fifth, train the product and technical teams on the S1/S2/S3 grid, so they apply it upstream of any new project. At the end, you have a solid, auditable base — ready to answer a CNIL inspection without friction.