Home / Solutions / Industrial Networking & Datacom / AI Computing Datacenter Network

AI Computing Datacenter Network: When the GPUs Are Idle, It Is Usually the Network

You bought the accelerators and the utilisation graph still sits at half. That is rarely the cards — it is a fabric that drops packets under collective communication, and a single drop makes the whole job wait for a retransmit. We design AI cluster networks the way the job actually behaves: a spine-leaf fabric with no oversubscription, lossless Ethernet with RoCEv2, PFC and ECN tuned for your cluster rather than copied from a guide, and an out-of-band management plane that still answers when the fabric does not. Sized honestly from a single 8-card rack to a multi-room build — and budgeted for the optics and NICs, not just the switches.

Why AI Fabrics Fail Differently

Four problems we find in almost every cluster that calls us:

The GPUs are idle and nobody knows whyUtilisation sits at half and the vendor says the cards are fine. They usually are. The cards are waiting on each other, and what makes them wait is the fabric between them — which nobody instrumented, because it was assumed to be plumbing.

One drop and the run starts overA training job is not a web server. It does not degrade gracefully — a loss during a collective operation stalls every rank until the retransmit lands, and in the worst case a long run has to be restarted from the last checkpoint. Hours of accelerator time for one dropped frame.

Ordinary Ethernet was never asked to do thisCollective communication sends everything at once, to everyone, in a burst. A fabric designed for average office traffic answers that with a full buffer and a discard — it is behaving exactly as designed, which is the problem.

When the fabric breaks, you cannot get inManagement rides the same fabric it is supposed to fix. The night it congests, the console is unreachable too, and someone drives to the site to plug into a console port — which is a lesson everybody learns exactly once.

Architecture: Spine-Leaf Lossless Fabric + Out-of-Band Management

A flat, non-blocking fabric built around one requirement — do not drop the packet:

Architecture drawn by AtlasCommTech following carrier-grade design practice. Diagram labels are kept in English for engineering clarity.

Why us: our founder spent 13 years inside the Huawei partner ecosystem delivering carrier networks. Carrier work is where you learn that a fabric is judged by its worst moment, not its average one — which is exactly the discipline an AI cluster needs, because the training job is only as fast as its slowest collective operation.

Equipment Options

The solution is sized to your requirements and budget first — the same architecture can be delivered on several vendors' product lines. We help you choose by supply availability in your destination country, budget and your team's operating habits.

Huawei — enterprise campus, WAN and security linesMature ecosystem with a global service network.

ZTE Wantong — comparable datacom linesPrice-performance direction; supply runs smoother in some markets.

H3C — campus and data-center linesWidely deployed campus and data-center portfolio.

Zhongxin Shutong industrial switches — industrial-scenario access layerOur own industrial line — compatible with any brand's core layer.

What the Design Delivers

Six things an AI fabric does that an ordinary data-center network is never asked to:

No oversubscription in the fabricEvery leaf uplinks to every spine, and the uplink capacity matches what the leaf can receive. Statistical multiplexing is an assumption about average traffic — a collective operation is the one workload that refuses to be average.

Lossless transport, tuned not copiedRoCEv2 with PFC per priority class and ECN marking, with the thresholds set for your cluster's size and traffic. A PFC configuration copied from a guide is not lossless — it is a deadlock waiting for the right week.

Flat topology, one hop between leavesAny GPU can reach any other GPU through exactly one spine. Latency is uniform, so the job scheduler does not have to know the cabling — and adding a rack does not change anyone's distance to anyone else.

High-density 100G / 400G portsEvery accelerator gets its own high-speed port into the leaf, because a shared port makes the cards queue behind each other before the fabric even sees the traffic. Port speed is chosen from your node's NIC layout, not from a catalogue.

An out-of-band plane that still answersSeparate switches, separate cabling, separate addressing — reachable precisely when the fabric is not. You do not debug a congested fabric through the congested fabric, and the night you find that out is not the night to learn it.

Instrumented before it is blamedPer-queue counters, PFC and ECN statistics and drop counters exported from day one. When the utilisation graph dips, the question "was it the network?" has an answer instead of an argument.

Three Sizes, One Design Logic

Tell us the card count, the node layout and what your racks can actually power and cool — the tier tells you the shape of the fabric:

Numbers we design around:

Every accelerator gets its own high-speed port — the collective runs at the speed of the slowest rank

Zero oversubscription between leaf and spine — statistical multiplexing is an assumption this workload breaks

Optics and NICs usually cost more than the switches — a budget counting only switches is short before the first job runs

Scale tier	Typical site	What the design includes
Single rack, from 8 cards	One node or a few · a pilot or a small research cluster · one room	A pair of leaf switches, no spine layer at all — at this size the leaves talk to each other directly and a spine is a layer bought for a diagram. Lossless configuration and out-of-band management are still in, because they are what makes it a cluster rather than eight expensive computers in a rack.
Mid-size GPU cluster	Several racks in one room · production training jobs · a scheduler in front	A full spine-leaf fabric with no oversubscription, one leaf pair per rack, RoCEv2 with PFC and ECN tuned against your actual traffic rather than a template, a separate storage path so checkpoint writes do not collide with the training traffic, a full out-of-band plane, and per-queue telemetry exported from the start.
Multi-room build	More racks than one room can power · several halls or buildings · long spans between them	Fabric per room with an inter-room layer above it, span distances driving the optics choice and the optics choice driving a large part of the budget, a job-placement conversation with whoever runs the scheduler so a single job does not straddle rooms unnecessarily, and a power and cooling reality check before the topology is drawn — because at this size the building decides the design, not the other way round.

Equipment Roles (Categories, Not Models)

The solution is built from these equipment categories — the brand is chosen with you at design stage. Exact models depend on your card count, node NIC layout, port speeds and country — so we spec models after your requirements list, not before.

Role	What it does
Leaf switches (rack access)	One high-speed port per accelerator, deep enough buffers to absorb a collective burst, and the lossless configuration that actually holds it. Sized by the NIC layout of your node, not by what fits a rack diagram.
Spine switches (fabric core)	Give every leaf a one-hop path to every other leaf with no oversubscription. Port density here decides how many racks the cluster can ever grow to — so it is sized for the cluster you are heading towards, not the one you are starting with.
Optics and cabling	Every high-speed link needs two transceivers and a cable, and the count runs to hundreds fast. Reach, connector type and rack distances decide the choice — and this line item is usually larger than the switch line item, which is why we insist on sizing it early.
Out-of-band management switches	A small, cheap, separate network reaching every management port in the room. It is the least interesting equipment on this page and the first thing you will be grateful for — because it is the only path in on the night the fabric is the problem.
Storage path switching	Checkpoint writes and dataset reads are bursty in their own way, and putting them on the same queues as the training traffic is how a well-tuned fabric starts dropping again. Separated by path or by priority class, decided from your checkpoint cadence.
Telemetry / management platform	Exports per-queue depth, PFC pause counts, ECN marks and drops, so a dip in utilisation can be attributed instead of debated. Without this the network is guilty by default in every meeting, and usually not guilty in fact.

Send us your card count, your node NIC layout, your rack power and cooling limits and your checkpoint cadence — and the model list follows. That order keeps the design honest.

Design Notes & Honest Limits

Read this before you commit:

On this kind of network the optics and the NICs usually cost more than the switches — do not budget switches only. Every high-speed link needs two transceivers and a cable, and the count runs into the hundreds quickly. A budget built from a switch quote will be short before the first job runs, and it will be short by an amount nobody wants to explain. We size the whole thing at design stage so the number arrives in the plan.
Power and cooling decide the topology, not the other way round. At this density a rack's electrical and thermal limit sets how many nodes fit in it, which sets how many racks you need, which sets the fabric. If your facility cannot power the layout, no network design rescues it — so the first question we ask is about your rack, not your switch.
A lossless fabric is tuned, not switched on. PFC and ECN thresholds depend on your cluster's size, your traffic pattern and your NICs, and a configuration copied from someone else's guide can be worse than no configuration at all — it turns a drop into a stall that spreads. Budget engineering time for tuning and validation, not just installation days.
Licensing policy and product availability differ by brand and destination country, and high-speed optics in particular have their own supply reality — lead times on some port speeds are measured in months, not weeks. We check and confirm both for your country at the design stage, before you commit to anything, because an optic that arrives after the accelerators is a rack of idle hardware.
If your utilisation problem is not the network, we will tell you. Data loading, a scheduler that packs jobs badly, an undersized storage tier or a model that simply does not parallelise all look exactly like a network problem from the utilisation graph. We instrument first and conclude second — and if the fabric turns out to be innocent, that finding is the deliverable, not a switch quote.

FAQ

How is this different from your Resilient Data Center Network solution?

Different failure you are buying protection against. The resilient data center page is about business systems staying available: if a link, a switch or a room dies, the application keeps serving users, and a brief re-convergence is an acceptable outcome. This page is about a training job not losing packets: nothing has failed, everything is up, and a single drop under collective communication still stalls every rank and can cost you hours. One design optimises for surviving a failure; this one optimises for a fabric that never drops while everything is working perfectly. If you are hosting business applications, that page is your page.

Can Ethernet really carry AI training traffic?

Yes, when it is configured to be lossless. Plain Ethernet handles congestion by discarding, which is fine for a web server and fatal for a collective operation. RoCEv2 with PFC and ECN changes that behaviour: the fabric applies back-pressure and marks congestion instead of dropping. The important word is configured — the protocol does not make the fabric lossless, the tuning does, and the tuning has to match your cluster. This is why we treat validation as part of the build rather than something the customer does afterwards.

We only have one rack. Do we need a spine layer?

No, and we will not sell you one. With one rack the leaves talk to each other directly and a spine adds cost, cabling, optics and latency in exchange for a diagram that looks like a real cluster. What you should still buy at that size is the lossless configuration, one high-speed port per accelerator and the out-of-band plane — those are what turn eight expensive computers into a cluster. The spine becomes worth its money when you have enough racks that direct leaf-to-leaf cabling stops scaling, and we will tell you when you are at that point.

Can we prove it was the network before we buy anything?

That is the right question to ask first, and yes. Per-queue depth, PFC pause counts, ECN marks and drop counters on the existing fabric will tell you whether the accelerators are waiting on the network or on something else — often it is data loading or a scheduler that packs jobs badly, and both look identical on a utilisation graph. We would rather run that measurement with you and find the fabric innocent than sell you a rebuild that fixes nothing. If it turns out the network is fine, that is a result, and it is a cheaper result than the alternative.

Can we start small and grow the cluster later?

Yes, and most people should — but the growth has to be designed in now, in exactly two places. The spine port density decides how many racks the fabric will ever reach, and the room's power and cooling decide how many racks the building will ever allow. Both are hard to change later and cheap to plan for today. Everything else — more leaves, more nodes, more optics — is addition rather than redesign. Tell us the cluster you are heading towards, not just the one you are starting with, and we will size those two things for the destination and the rest for today.

Send us your card count and your rack power limits

An engineer replies with a fabric design, an optics and NIC count and the equipment-category list. Send us your requirements list — the model list follows.

WhatsApp an engineer →