Skip to main content

The Unified Measurement of Computing Power: TOPS, TFLOPS, and the Battle for a Standard in AI Chips

Introduction​

In 2026, the global AI computing power arms race is escalating. NVIDIA's Blackwell Ultra GPU claims XX PFLOPS at FP8, Huawei's Ascend 910C boasts XX TFLOPS peak compute, Apple's M4 Ultra NPU delivers XX TOPS, and Qualcomm's Snapdragon Gen 5 sets new records for mobile AI performance.

Yet there is an awkward reality: none of these numbers are directly comparable. One vendor quotes FP16 peak compute, another uses INT8 inference TOPS. Some report sparse compute, others report dense. Some cite theoretical peaks, others cite measured performance. "30 TFLOPS" and "30 TOPS" differ by nearly an order of magnitude, yet they are often placed side by side on comparison charts.

This article aims to answer a core question: Is there a unified "measurement standard" for AI computing power? And if not, how should we properly understand these numbers?

I. Understanding the Fundamentals​

1.1 The Essence of Computing Power​

Computing power is simple at its core: how many computational operations a chip can complete per unit of time. Just as speed is "meters per second," compute is "operations per second." But the definition of "one operation" varies dramatically depending on data type and computation scenario.

1.2 The Two Dominant Units: TOPS vs TFLOPS​

UnitFull NameMeaningUse Case
TOPSTera Operations Per SecondTrillions of integer operations per secondAI inference (INT8 quantized)
TFLOPSTera Floating-point Operations Per SecondTrillions of floating-point operations per secondAI training, scientific computing

A critical distinction often overlooked: TOPS and TFLOPS measure different physical quantities. Integer operations (OPS) and floating-point operations (FLOPS) have completely different hardware costs and computational complexity. They cannot be directly converted or compared.

1.3 The Precision-Dependent Gap​

The theoretical compute of the same chip varies dramatically across data types:

  • INT8 compute ≈ 2 × FP16 compute ≈ 4 × FP32 compute
  • The reason is straightforward: fewer bits per data element means more parallel computations per cycle

Take the NVIDIA H100 GPU:

  • FP32 (32-bit): ~67 TFLOPS
  • FP16 (16-bit): ~134 TFLOPS (with Tensor Core)
  • INT8 (8-bit): ~268 TOPS

If you only look at the numerical values, 268 > 134 > 67—but this is entirely because the units are different. Some vendors further compound the confusion by quoting sparse compute, which skips zero-valued elements in matrices and effectively "doubles" the claimed throughput.

II. The Three Layers of Measurement Confusion​

2.1 Layer One: Mixed Precision​

A common trap: one vendor compares their INT8 TOPS against another's FP16 TFLOPS, or sparse against dense.

For example:

  • Vendor A claims "200 TOPS" (INT8 inference)
  • Vendor B claims "100 TFLOPS" (FP16 training)
  • Both are "level 100" numbers—but 200 TOPS ≠ 100 TFLOPS!

TOPS and TFLOPS measure fundamentally different dimensions. It's like comparing kilometers per hour to nautical miles per hour—both are speed units, but with different baselines.

2.2 Layer Two: Theoretical Peak vs. Real-World Performance​

Theoretical peak compute (Peak FLOPS/TOPS) is where vendors have the most room for inflation. These are maximums the chip could theoretically achieve under ideal conditions—rarely reached in practice.

  • Theoretical Peak: All compute units running at full throttle, ignoring data movement, memory bandwidth, and thermal constraints
  • Real-World Performance: Affected by memory bandwidth, tensor shapes, model characteristics, software stack optimization—typically 30%-70% of theoretical peak

NVIDIA's H100 achieves only about 40-50% of its theoretical FP8 TFLOPS on real Transformer training workloads. Huawei's Ascend 910B shows an even wider gap between theoretical specs and actual measured performance on certain models.

2.3 Layer Three: Cross-Architecture Comparison​

Different chip architectures make direct comparison unreasonable—even when precision and test conditions are unified:

  • NVIDIA GPU: Tensor Cores optimized for matrix multiplication, dominant advantage in LLM workloads
  • AMD MI300X: CDNA architecture with distinct characteristics for massively parallel computing
  • Huawei Ascend: Da Vinci architecture with specific operator-level optimizations
  • Apple M Series: Unified memory architecture offering unique advantages for on-device inference
  • Qualcomm/MediaTek: Focused on mobile power efficiency and lightweight inference

A typical case: Apple's M4 Ultra NPU claims 80 TOPS (INT8), while the RTX 4090 claims 1321 TOPS (INT4 sparse). The numbers differ by 16×—but they use completely different precision levels and sparsity strategies. Putting them in the same chart is meaningless.

III. Standardization Efforts and Their Challenges​

3.1 MLPerf: The Industry's Most Credible Benchmark​

MLCommons' MLPerf benchmark suite is currently the most authoritative standard for fair compute comparison. It covers both Training and Inference scenarios, using standardized models, datasets, and rules to enable apple-to-apple hardware comparison.

MLPerf's core philosophy: instead of theoretical peak FLOPs, it measures the time required to complete a specific task—how many minutes to train BERT-Large, or how many images per second for inference.

3.2 Latest Developments: MLPerf Client and AI PC Standards​

In 2025-2026, MLCommons launched MLPerf Client, a benchmark specifically designed to evaluate LLM inference on PCs and mobile devices. It covers, for the first time, real-world LLM performance on edge devices—including text generation speed, time-to-first-token (TTFT), and token generation rate.

3.3 The Standardization Dilemma​

Despite MLPerf's success, full unification of compute measurement faces major challenges:

  1. Diversity of Scenarios: Training, inference, edge deployment, and mobile have completely different requirements. One "compute score" cannot cover all.
  2. Vendor Conflicts: Each vendor prefers metrics that showcase their own strengths
  3. Rapid Technology Evolution: New architectures (AMD CDNA 4, Intel Gaudi 3, Google TPU v6) emerge constantly, requiring benchmark updates
  4. Fragmented Ecosystems: AI performance across Android, iOS, and HarmonyOS cannot be easily measured cross-platform

3.4 China's Compute Standard Initiatives​

Between 2024-2026, China has accelerated its compute standardization efforts. The China Academy of Information and Communications Technology (CAICT) published a series of "AI Chip Compute Evaluation Specifications," covering testing methodologies for general computing, AI training, and AI inference scenarios. The "East Data West Computing" project has also driven national standards for compute scheduling and measurement.

IV. How to Properly Understand and Compare Compute​

4.1 The Compute Comparison Checklist​

When faced with two chip spec sheets, ask at least these questions:

  1. What precision? FP32/FP16/BF16/INT8/INT4?
  2. TOPS or TFLOPS? Integer or floating point?
  3. Theoretical peak or measured performance? Check MLPerf or real benchmarks
  4. Dense or sparse? Sparse claims of 2× speedup are rarely achievable in practice
  5. What workload? Training LLMs or running vision inference? Different workloads have very different hardware demands

4.2 Three-Tier Evaluation Framework​

LevelMetricPurposeExample
TheoreticalPeak TFLOPS/TOPSQuick upper bound checkH100 FP8: ~1979 TFLOPS
BenchmarkMLPerf scoresFair cross-comparisonH100 BERT training: ~X minutes
ApplicationReal-world performanceProcurement decisionRunning LLaMA-70B in Token/s

Core Principle: Use theoretical for upper bounds, benchmarks for cross-comparison, and application-specific testing for final decisions.

V. Future Outlook: Toward True Standardization​

  • MLPerf dominance will continue: With MLPerf Client and Endpoints, coverage will become more comprehensive
  • Energy efficiency becomes a new dimension: Raw peak compute is increasingly meaningless. Performance per Watt is gaining attention.
  • Edge standardization accelerates: MLPerf Client and industry consortia (like Linaro ML Working Group) are pushing mobile and PC AI testing standards
  • Chinese and international standards converge: CAICT standards and MLPerf mutual recognition are progressing

5.2 Advice for Practitioners​

  1. Don't trust peak compute alone—chips don't run on roofline models; real workloads vary enormously
  2. Always require precision context—every compute number must carry its precision label (FP16/BF16/INT8)
  3. Prefer MLPerf data—it's much more trustworthy than vendor self-reported numbers
  4. Focus on TOPS/W, not just TOPS—for edge deployment, energy efficiency is the real constraint

Conclusion​

The computing power industry still lacks a universally recognized standard comparable to the "meter" or "kilogram." TOPS, TFLOPS, FP32, INT8, dense, sparse—these concepts form a bewildering "measurement jungle" for practitioners.

Just as the unification of physics required standardized measurement, the maturation of the AI industry demands compute standard unification. MLPerf is currently the closest thing to an "industry referee," but complete unification may never arrive—because technological diversity is itself a driver of progress.

For every AI practitioner, understanding the meaning behind these numbers is far more important than memorizing the numbers themselves.

References: CSDN "TOPS, TFLOPS, FP32/FP16/INT8 Complete Guide", MLCommons MLPerf Official Documentation, NVIDIA/AMD/Huawei Hardware Technical Whitepapers, CAICT AI Chip Evaluation Specifications, Zhihu "Reading FLOPs: Understanding the Measurement Units of Large Models"