Good AI starts with an open connection.
Why we’re thinking about a shared AI gateway, and what it should simplify for organizations.
Models evolve faster than processes
New models create new possibilities. Organizations also need reliable workflows and clear data boundaries. A shared connection layer can bring these different speeds together.
A shared gateway could be especially useful where an organization considers different model paths for different tasks. An internal summary may have different requirements from a draft for external communication or processing on the organization’s own infrastructure. Before turning that into routing, the team needs to define the differences: Which data is needed, which results are expected, and which execution locations are acceptable?
An idea built on an existing foundation
AIOS already connects different model providers and self-hosted runtimes. AI Gateway is our concept for bringing model access, routing and usage into a shared control layer.
A useful provider comparison starts with representative tasks and agreed review criteria. Where possible, use the same permitted documents and brief, then assess results for evidence, completeness and usefulness. Also record unanswered questions about runtime and usage. Such a comparison shows which decisions a shared control layer would need to make understandable, without assuming automatic selection already exists.
AIOSThe right choice remains yours
Quality, cost, runtime and data location matter differently for each task. A gateway should make those choices visible and configurable. Different tasks can use different models.
The product direction also needs to address cases where the preferred path is unavailable. Should a task wait, return an error or be allowed to use another provider? An alternative path may involve different data boundaries and therefore needs its own decision. These questions help make access, observability and provider-switching requirements concrete; they describe desired behavior, not a gateway capability already promised.
Developing it together
The gateway is a product direction. We’re speaking with teams about their needs around provider changes, access and observability before defining its scope.
Try it yourself
Adapt the placeholders to your task. This example is a template for your work with AIOS.
Task: [Representative use case]
Inputs: [Which information is needed?]
Permitted execution: [Local / named external providers / unresolved]
Expected result: [Format and practical use]
Quality review: [Evidence, completeness and other task-specific criteria]
Runtime and usage requirements to clarify: [Requirements or open questions]
If unavailable: [Wait / report an error / explicitly agreed alternative path]
Traceability needed: [Which details must the team be able to review later?]
Decision owner: [Person/team]
Discussion template for the proposed gateway; not a configuration file or API.Decisions a gateway should make visible
| Dimension | Question for the team |
|---|---|
| Data and execution location | Which information may use which model path? |
| Result quality | Which concrete tasks and evidence will be used to review a result? |
| Runtime and usage | Which requirements need to be established for this use case? |
| Unavailability | Is an alternative path permitted, and under which conditions? |
| Traceability | Which decision and result need to remain understandable later? |