AWS Publishes Blueprint for Sharing a Single HyperPod EKS Cluster Across Teams

AWS has published a reference architecture for safely giving several teams access to a single Amazon SageMaker HyperPod cluster orchestrated through EKS. The design has five layers: single sign-on through AWS IAM Identity Center, separate SageMaker Domains and Kubernetes namespaces for isolation, HyperPod Task Governance for fair allocation of accelerators, and namespace-level cost tracking for internal billing. It answers a common problem: GPUs are expensive and scarce, and giving each team its own cluster is wasteful.

Why share one cluster

When every ML team has its own cluster, utilization becomes lopsided. Some teams' accelerators sit idle at night and between experiments, while other teams wait days in the training queue. Platform engineers, meanwhile, administer several environments instead of one. A shared cluster solves both problems but raises three new questions. How do you keep one team from accessing another team's data and jobs? How do you keep one team from taking everything? And who pays for what?

The AWS architecture answers each question with a separate component.

How isolation works

The entry point is AWS IAM Identity Center. Users and groups come from the corporate directory, and each group maps to a team. There is no need to create separate AWS accounts, and permissions can be revoked in one place.

From there, a dual boundary applies. Each team gets its own SageMaker Domain, meaning its own working environment with user profiles and execution roles. In parallel, the team gets a Kubernetes namespace in the shared EKS cluster, where its jobs run. The Domain controls what a person can see and launch. The namespace controls where those jobs run on the shared hardware and under what constraints.

Fairness and money

Isolation is not enough: without rules for allocating compute, a shared cluster turns into a fight over resources. This is where HyperPod Task Governance comes in. It lets you set a compute quota and job priorities for each team. Teams can temporarily borrow their neighbors' idle capacity, and when the owner returns, lower-priority jobs give way. This raises cluster utilization while preserving each team's guaranteed share.

The last layer is financial. Costs are tied to a namespace, and therefore to a specific team. This is the basis of chargeback: finance and department heads can see who consumed how much and can issue internal invoices without manual spreadsheets.

Key elements of the design:

• Cluster: one Amazon SageMaker HyperPod cluster orchestrated through EKS.
• Authentication: AWS IAM Identity Center, with groups mapped to teams.
• Isolation: a separate SageMaker Domain and Kubernetes namespace for each team.
• Fairness: HyperPod Task Governance with quotas and priorities.
• Finance: cost allocation at the namespace level.

Where to start with adoption

The source describes only the architecture, so what follows are my own practical conclusions, not AWS data.

Start with a map of your teams and their actual consumption. Derive quotas from utilization history rather than from managers' wishes, or you will have to redistribute them right after launch. Then agree on priority classes: what counts as production training, what counts as experiments, and what counts as interactive work. Preempting jobs is painless only when checkpoints are regular, so it is worth requiring teams to save state in advance.

Assess the level of isolation separately. Namespaces and Domains separate teams logically, but physically they remain on a shared cluster. Data subject to strict regulatory requirements may need additional controls or a separate cluster. Finally, agree on the chargeback model with the finance department before launch: what exactly counts as a team's cost, and how shared costs for idle capacity and administration are split. Technically, setting up per-namespace accounting is easier than reaching that agreement inside a company.