Why the question arises now
Hosted AI services run on formidable infrastructure, and that is exactly what makes them good. Nothing about them is simple except the path your data takes: it leaves your network, is processed on servers that are not yours, and comes back as an answer. For marketing copy, that has no consequence. For a medical report, it means health data, a special category under Article 9 of the GDPR, has been disclosed to a processor, sometimes established outside the Union.
Providers offer contractual guarantees: no training on your data, European hosting, deletion commitments. Those guarantees are real and often sufficient. But they remain contractual undertakings, not technical impossibilities. A data protection officer, an auditor or a hospital client may legitimately ask for more: a demonstration that the data cannot leave, rather than a promise that it will not.
That is what sovereign AI shifts. The question is no longer whether the third party processing your data can be trusted, but how few third parties can reach it at all, down to none in an on-premise installation.
What “sovereign” actually means
The word is used loosely. In our projects it covers four verifiable requirements:
- Local execution. The model runs on a machine you own or rent outright. No request is sent to a third-party AI service.
- Open-weight models. You hold the model itself, not access to a service. It keeps working if the provider changes its pricing, its terms, or disappears. Licences vary: at framing stage we check that the one attached to the chosen model covers your use.
- Traceability. Every operation is logged: which document reference, which model, when, for what purpose. That is what makes an audit answerable. The log itself is minimised and given a retention period.
- Control over retention. You decide what is kept and for how long, including the intermediate data that hosted services often retain by default.
The possible architectures
Fully on-premise
The model runs on your servers, behind your firewall. This is the strictest configuration, and the only acceptable one in some hospital or legal settings. It requires hardware: a GPU server, sized according to the model and the number of concurrent users. For many internal uses, a single professional graphics card is enough.
Dedicated European hosting
The model runs on a machine rented from a European host, dedicated to your organisation. You buy no hardware, and the data stays within a European legal framework, with no transfer outside the EU. The host remains a processor under the GDPR: a data processing agreement is required, and so is checking that no parent company subject to non-European law can reach the infrastructure. This is the most common compromise for organisations without a server room.
Hybrid architecture
Sensitive processing stays in-house; processing that is not sensitive can use hosted services, which are cheaper and more powerful. The difficulty is drawing a clear, documented boundary between the two, and making sure it cannot be crossed by accident.
What can genuinely be done locally
The usual objection is performance: open models would be too weak for serious use. That was true. It no longer is for most business uses.
- Retrieval-augmented search (RAG). Query thousands of internal documents (procedures, case files, contracts) in natural language and get a sourced answer. This is where the gap with hosted models is narrowest, because quality depends mostly on how the documents are indexed.
- Extraction and structuring. Turn reports, letters or forms into usable data. A repetitive task, expensive in human time, and perfectly workable locally.
- Classification and detection. Sort, route and flag items in a stream of documents or messages.
- Assisted drafting from templates. Produce a first draft from structured data, which a human reviews and approves.
For complex reasoning on open-ended subjects or high-end creative generation, hosted models still lead. A well-designed project starts by establishing which category your need falls into, before choosing an architecture.
How a project runs
- Framing. Which data, which regulatory constraints, which use, what volume. This is where we establish whether local is necessary, sufficient, or oversized.
- Model evaluation. We test several open models on your real data, against your criteria. Public leaderboards say nothing about your particular case.
- Sizing. Which hardware, at what cost, with what response time. Quantified before anything is committed.
- Deployment. Roll-out, role-based access control, logging, monitoring, backup.
- Handover. Documentation and training, so your team can run the system without us.
Who it is for
The sovereign AI projects we run mostly concern healthcare (health data, medical confidentiality, supervisory authority requirements), legal professions, manufacturers whose processes are their core assets, and public bodies subject to data localisation requirements.
What they share is not size, but the nature of the constraint: data that, if it leaked, would create a real legal or competitive problem, not merely discomfort.
Frequently asked
Is it much more expensive?
The cost shifts rather than rises. A hosted service is paid per use, indefinitely; a local installation requires an upfront hardware investment, then costs essentially electricity and maintenance. Above a certain usage volume, local becomes cheaper. Below it, the justification is regulatory, not economic.
Do we need a technical team in-house?
Not for everyday use: a properly delivered system works like any web application. Yes for updates and monitoring: that is why we offer either knowledge transfer or a maintenance contract.
What if open models improve?
That is precisely the point of the architecture: the model is a replaceable component. A well-designed installation lets you swap it without rebuilding the rest of the system.
Can we start small?
We recommend it. A proof of concept on one specific use, with a limited dataset, answers the question that matters within a few weeks: does the quality obtained justify the investment. See also our full range of services.