Skip to main content

Domestic GPU Bypasses NVIDIA NIC: IBGDA Direct-RDMA Doubles Throughput, Halves Latency

At WAIC 2026 (July 17–20, Shanghai), a seemingly modest technology demonstration drew outsized attention from China's domestic AI infrastructure community: Singularity Micro, in partnership with Biren Technology, unveiled an IBGDA solution that, for the first time, enables a domestic GPU to communicate directly with a domestic RDMA NIC — bypassing the CPU entirely. Benchmarks showed throughput nearly doubling and All-to-All communication latency dropping by half.

This is more than a handshake between two pieces of domestic hardware. It is a surgical fix for what has long been the Achilles' heel of domestic AI compute clusters: system-level communication bottlenecks.

The Real Bottleneck: Strong Chips, Weak Systems​

The core challenge facing China's domestic GPU ecosystem is not raw chip performance. In recent years, vendors such as Biren, Moore Threads, and Iluvatar Corex have delivered steady improvements in single-chip compute capability, narrowing the gap with international counterparts. But when these chips are assembled into large-scale training and inference clusters, a structural imbalance emerges — "strong at the node, weak at the system."

The culprit is interconnect. In large-model training and especially in MoE (Mixture of Experts) inference, inter-node communication becomes the critical performance bottleneck. MoE expert-parallel communication involves two core stages — Dispatch and Combine — each requiring massive volumes of fine-grained data exchange. In traditional cross-node communication, data follows the path GPU → CPU → NIC. The CPU acts as a middleman, forwarding every instruction, which adds latency and burns CPU cycles.

NVIDIA recognized this bottleneck early and addressed it through tight integration between its GPUs and ConnectX-series NICs, pioneering the IBGDA (InfiniBand Gather Direct Accelerate) mechanism — allowing the GPU to initiate communication directly with the NIC, bypassing the CPU entirely. This has become standard equipment in NVIDIA-powered AI clusters.

On the domestic side, however, IBGDA support has largely been limited to pairing domestic GPUs with NVIDIA NICs. The combination of a domestic GPU directly communicating with a domestic RDMA NIC remained a near-blank spot in production deployments. This gap has been the weakest link in the chain of full-stack domestic autonomy.

IBGDA: Letting the GPU Talk Directly to the NIC​

Singularity Micro's IBGDA solution is built on its Kiwi SNIC 800G platform architecture, featuring a single-channel 400G RDMA ASIC engine. The engine has completed software-hardware co-development, is compatible with collective communication libraries optimized for MoE models (counterpart to DeepEP), and natively supports the IBGDA mechanism — meaning the GPU can directly initiate communication requests without traversing the CPU.

At the WAIC live demonstration, Singularity Micro and Biren Technology jointly built a test platform benchmarked against NVIDIA's IBGDA test standards. The measured results told a compelling story:

  • Throughput leap: Under the small-packet, high-frequency, tightly synchronized conditions typical of MoE inference, single-message bandwidth surged dramatically. Both raw small-packet throughput and combined transmission throughput nearly doubled.
  • Latency halved: All-to-All communication latency was reduced by approximately 50%.
  • Small-packet penalty eliminated: The notorious "small-packet penalty" — where small messages incur disproportionately high latency — was completely eradicated.

In other words, on a fully domestic GPU + domestic NIC stack, communication is no longer the bottleneck holding back compute utilization. Crucially, the solution is built entirely on an open Ethernet ecosystem, inheriting Ethernet's deployment flexibility and cost advantages while providing domestic AI clusters with a high-performance, fully autonomous Scale-Out networking path.

From Technical Breakthrough to Ecosystem Building​

The IBGDA demo was just one piece of a broader picture. At the "AI Network Frontier Ecosystem Forum" hosted by Singularity Micro, two additional initiatives of far-reaching significance were launched in parallel.

First, the "Co-Packaged Optics (CPO) Technology White Paper" was officially released. Initiated by the Hong Kong University of Science and Technology (Guangzhou) and co-authored by Singularity Micro, Biren Technology, Turing Quantum, Singularity Photonics, and other institutions, the white paper systematically charts the technology evolution path from pluggable optical modules to NPO, CPO, and ultimately OIO (Optical I/O) — where optical interconnect functionality is integrated directly into the compute die. It provides an authoritative reference for the standardization and industrialization of domestic optical interconnect technologies.

Second, the "Domestic SuperNode Ecosystem Co-Building Initiative" was formally launched. Under the witness of the Shanghai Municipal Commission of Economy and Informatization, the Shanghai Computing Network Association, Shanghai AI Laboratory, and the East China Branch of CAICT jointly initiated the effort, with participation from ZTE, Biren Technology, MetaX, Iluvatar Corex, Enflame Technology, SenseTime, Sugon, Singularity Micro, and other industry heavyweights. This marks a pivotal shift for China's domestic AI compute industry — from isolated efforts toward standardized, collaborative ecosystem building.

Interconnect Is No Longer a Supporting Actor​

With three exponential demand curves — training Scaling Laws, test-time scaling for inference, and the explosion of Agent-driven API calls — converging simultaneously, single-chip performance improvements alone can no longer carry the industry's weight. The variables that increasingly decide the competitive landscape are interconnect, system integration, software, and cross-layer coordination.

Singularity Micro's IBGDA solution does more than prove with hard benchmarks that China's domestic AI compute ecosystem now possesses the capability for low-level interoperability and system-level co-optimization. It illuminates a viable path forward for domestic AI infrastructure: break down technology silos with open interconnect standards, unlock single-node compute power through full-stack coordination, and replace go-it-alone approaches with ecosystem-wide collaboration.

When interconnect graduates from a supporting role to the backbone of system performance, the next chapter of domestic AI compute is only just beginning.