Private Inference for Open-Weight Models

It means the application your scientists log into runs on infrastructure we control on your behalf, rather than on a consumer AI service, and that the model provider behind it is configured under a contract with no training rights and no prompt retention. Your conversations, uploaded documents, search index, and embeddings live in a dedicated database on a dedicated machine, not in a multi-tenant SaaS account. The prompt and the model response still travel to a hosted inference provider — that is what makes running a 500-billion-parameter model economically possible — but they travel under commercial terms you can read, to an organisation account we configure with prompt storage and training explicitly switched off. If your requirement is that no text ever leaves your own network at all, that is a different and much more expensive architecture; see the on-premises question below.

Three reasons that matter in a regulated R&D setting. First, cost: an open-weight model of comparable capability typically runs a fraction of the per-token price of a frontier API, which changes what you can afford to do at volume — screening literature, drafting summaries across a whole portfolio, processing thousands of documents. Second, portability: an open-weight model can be moved between hosting providers, or brought fully in-house later, because the weights are published under licences such as MIT and Apache 2.0. You are not architecting around one vendor's roadmap or pricing. Third, choice: different models are genuinely better at different jobs, and this setup lets a scientist switch between them mid-project rather than accepting one vendor's single answer.

The full open-weight catalogue in the table above is available, and we curate a short list per deployment rather than exposing hundreds of options. In practice most teams settle on two to four: a strong general reasoning model as the default, a long-context multimodal model for figures and scanned pages, a cheap fast model for bulk work, and sometimes a code-capable model for bioinformatics scripting. Curation is deliberate — an unfiltered picker of 200-plus models produces support tickets, inconsistent output quality, and unpredictable spend. The list is a configuration change, so it moves as the field moves; when a materially better open-weight model ships, we test it and swap it in.

Through your existing identity provider using single sign-on, so there is no separate password and no parallel user list to maintain. We integrate with Okta, Microsoft Entra ID, Google Workspace, and other standard SAML or OIDC providers, and the workspace inherits whatever multi-factor and conditional-access policy you already enforce. Self-service registration is switched off entirely: nobody can create an account by finding the URL. Access is decided by policy on email domain and group membership, which means your joiner-mover-leaver process governs the workspace automatically — when your directory disables an account, access ends with it rather than waiting on a ticket.

Yes, and this is one of the more useful things the identity layer does. Login method can be enforced per email domain, so your own staff can be required to authenticate through your corporate identity provider specifically — a user with a company address who tries to authenticate by any other route is denied rather than let in through a weaker path — while a named group of external collaborators or consultants uses a different, tightly scoped method. That distinction matters in life sciences, where a CRO analyst, a university collaborator, and an employee often all need access to the same project but should not be governed by the same rule. Policies are enforced at the edge before a request reaches the application.

The workspace has no public IP address, no inbound firewall rule, and no port exposed to the internet. Instead, a lightweight connector inside the environment establishes an outbound-only tunnel to an edge network, and the edge publishes your private hostname. Every request is authenticated at the edge before it is allowed through that tunnel, so an unauthenticated visitor never reaches the application at all — they are stopped at the identity gate. This removes an entire class of exposure: there is no login page on the open internet to brute-force, no service banner to fingerprint, and no listening port for a scanner to find. It also means remote and travelling staff need no VPN client; the browser is the access method.

There is a second, independent layer, because a single control that can be misconfigured in one dashboard is not a security architecture. Inside the environment a small fail-closed proxy validates the signed identity token on every request and rejects anything unsigned, expired, or issued for a different application — so even if the edge policy were deleted or misconfigured, the application still refuses the request rather than serving it. The application itself binds only to the local loopback interface and is not reachable on the network even from inside our own infrastructure. Defence in depth here is cheap, and the failure mode it prevents is total.

Conversations, user records, uploaded files, the search index, and the vector embeddings all live in a dedicated database on a dedicated virtual machine, in a location we agree with you up front. There is no shared multi-tenant store and no other client's data on the same instance. We can deploy into infrastructure we operate, into your own cloud tenancy, or into your datacentre, depending on your data residency obligations — a requirement to keep data in the EU under the GDPR, or in a named region under a contractual commitment, is a normal constraint we design around rather than an exception.

No, and this is a configured control we verify rather than a claim we repeat from a marketing page. The inference organisation account is set so that prompt and response storage is off and training on submitted data is off. We also verify that the specific models you use are served natively by that provider rather than being transparently relayed to a third-party upstream, because a relayed model would place your text under a different organisation's data policy and quietly break the guarantee. That verification is repeated when a model is added or changed, since the routing behind a model name can change without the name changing.

Yes. Documents can be uploaded and searched, with answers carrying clickable citations back to the source page so a reader can check the claim rather than trusting it. Retrieval runs against a vector database inside the same private environment, and embeddings are generated under the same no-retention terms as the chat itself. This is the feature most teams actually adopt first, because the highest-value everyday task is rarely open-ended conversation — it is "what does this 180-page protocol, dossier, or guidance document actually say, and where." Bear in mind that retrieval quality depends heavily on document quality; scanned pages without a text layer need OCR first.

A private research workspace should be narrower than a consumer product, and we disable capabilities that add risk without adding value for a regulated team. Long-term memory across conversations is off, so the system does not silently accumulate a profile of what a user has asked. Arbitrary code execution is off. Public link sharing is off, so a conversation cannot be exposed to the open internet by a single click — sharing works only between authenticated users of your instance. Image generation is off. Each of these is a configuration we can revisit with you; the default is closed, and opening something is a decision with a reason attached.

Yes, through a centrally credentialled search provider, so your scientists are never asked to supply their own API key and no personal keys end up scattered across the organisation. Web search is genuinely useful for literature triage and competitive monitoring, but it is worth being clear about the trade-off: a query sent to a search provider leaves your environment by definition. For teams whose material is sensitive enough that even a search string is a disclosure risk, we can leave web search off and rely on retrieval over your own documents instead.

Not out of the box, and any vendor claiming a "GxP validated" AI product is describing something that does not exist. Validation is an activity performed against your intended use, your procedures, and your risk assessment. What we provide is a qualified infrastructure layer and the evidence that makes your own validation tractable: a documented and version-controlled build, change control over the environment, access logs, and a defined configuration baseline. Your quality organisation defines intended use and runs acceptance testing, guided by GAMP 5 and, where electronic records are in scope, 21 CFR Part 11. Our computer system validation practice runs that as a separate engagement.

Technically yes, and for some organisations it is the right answer, but the economics deserve a clear-eyed look before anyone commits. Serving a large open-weight model at production quality and reasonable latency needs multiple high-memory accelerators per replica, plus redundancy, plus somebody who can operate an inference stack and keep up with a fast-moving field. That is a capital purchase and a permanent staffing commitment, and it usually only makes sense above a sustained token volume that most R&D teams do not reach. The pragmatic middle ground — a private workspace and private data stores, with inference bought under no-retention terms — delivers most of the control at a small fraction of the cost, and keeps the option of moving in-house later precisely because the weights are open.

Usage is metered per token, so cost tracks real activity rather than seat count, and cheaper models can be made the default for routine work while expensive reasoning models are reserved for tasks that justify them. Prepaid credit is the normal arrangement, which caps exposure by construction. Be aware of the honest limitation: a shared organisation balance with automatic user provisioning means many users can draw on one pool, so agreeing a monitoring and top-up process at the outset is part of the deployment rather than an afterthought. We will raise this during scoping rather than let it surprise you later.

You keep your data and we hand over cleanly. The identity gate, the hostname, and the tunnel can be operated in your own accounts from the start if you prefer, in which case you can revoke our access unilaterally at any time by changing one policy. Conversation data can be exported from the database, and because the underlying components are open-source and the models are open-weight, there is no proprietary format holding your content hostage. We would rather set this up so that leaving is straightforward, because a client who stays only because leaving is painful is not a reference.

A working pilot for a defined group is typically live in one to two weeks, and the pacing item is almost never the technology. It is your security review, the data classification decision about what may be pasted into the workspace, and the identity provider integration — which usually needs a named person in IT for an hour or two rather than a project. We run those three tracks in parallel from day one because sequencing them is the main reason this kind of rollout stretches into months. A pilot should include real work from real projects; a demo with sample data tells you the software runs, not whether your scientists will adopt it.