Services
GPU clusters and HPC
From hardware selection to scheduling and monitoring, the deliverable is a cluster that can be operated; delivery ends with documentation and a handover.
Engineering field guide
Design the cluster around its jobs.
GPU count is only part of the configuration. Data delivery, node communication and job recovery determine whether the system is useful.
Start with the workload
Describe hardware needs through real jobs, rather than individual GPU specifications.
- Distributed training & tuning
- Provide the model, precision, sequence length, parallelism plan and checkpoint cadence. Evaluate local interconnect, cross-node communication and checkpoint writes together.
- Serving & batch compute
- Separate latency-sensitive serving from queueable batch work. Define capacity, isolation, scheduling priority and recovery behavior.
- Shared research & HPC
- Specify users, job types, software stacks and storage access. Define queueing, quotas, reproducible environments and operational permissions.
Compute, data and control together
Connect the physical topology to operations so every component has a defined responsibility.
Compute plane
Size GPU, CPU, host memory and local interconnect for the job. Record driver and runtime versions and retain a single-node baseline.
Data plane
Plan dataset reads, caches, checkpoints and archives separately. Test concurrent access and recovery; network bandwidth alone does not validate data delivery.
Control plane
Configure queues, quotas, monitoring and alerts. Define node draining, failed-job retries, maintenance windows and administrator access.
Accept the workload, not just the hardware
Agree test cases and thresholds before implementation; record hardware, software and test conditions.
- Nodes & interconnect
- Check device discovery, topology and link state. Record local and cross-node communication tests and investigate abnormal links.
- Representative jobs
- Run agreed training, serving or compute workloads; inspect throughput, waiting, memory use, data reads and failures.
- Recovery & handover
- Exercise interrupted jobs, checkpoint recovery, node maintenance and alerts. Deliver topology, configuration, runbooks and ownership boundaries.
Before planning the build
The cluster must fit both the site and the team that will operate it.
Can existing hardware be reused?
Inventory models, memory, interconnect, storage, drivers and support status, then test the target job. Mixed hardware depends on scheduler and framework constraints.
Who operates the cluster after handover?
The customer team takes over by default. Ongoing operations, on-call coverage, hardware maintenance and spares need a separate agreement.
What does planning need?
Example jobs and growth expectations, existing assets, power and cooling, network and storage constraints, procurement scope and the receiving team.
Continue reading
Explore compute and engineering scope4ROUTER / SYSTEMS RESEARCH
Technical foundations of the service
Understand how operators, KV state and scheduling shape an inference path.
Read the systems notes in Resources ↗KV cache: from tensor shapes to capacity budgets
Context length is a state budget that grows with concurrency.
↗TIERS / TRANSFERLMCache: placing KV state in a storage hierarchy
Avoiding prefill requires paying for state access.
↗SCHEDULING / SLOFrom prefill to decode: designing the scheduling experiment
When throughput rises, know which requests are waiting.
↗Let’s talk infrastructure.
Connect to the platform, or discuss models, performance and deployment.