RISC-V vs CUDA: Why Open Hardware Is the Future of Sovereign AI

Executive Summary

RISC-V is a viable alternative to NVIDIA CUDA for many production AI inference workloads. Tenstorrent's Galaxy platform, powered by Blackhole RISC-V accelerators, is marketed for competitive inference throughput at roughly 7.9–9 kW per server versus much higher wall power for dense H100-class racks — treat power-efficiency multiples as vendor or planning figures, not an AGICY laboratory result (including any historical “4.2×” marketing claims elsewhere on the site). With no proprietary CUDA ISA license, open-ISA auditability, and a sovereignty-first posture, RISC-V addresses vendor lock-in and energy cost pressure for inference. This is not a claim that RISC-V replaces CUDA for all training workloads.

CUDA Vendor Lock-In Explained

CUDA (Compute Unified Device Architecture) launched in 2006 as NVIDIA's proprietary parallel-computing platform. Over 18 years, it has grown from a GPU programming toolkit into a comprehensive, closed ecosystem that controls virtually every layer of the AI compute stack.

CUDA is not just a programming model — it is a proprietary ecosystem comprising the CUDA Toolkit, cuDNN (deep-learning primitives), TensorRT (inference optimisation), NCCL (multi-GPU communication), and NVIDIA AI Enterprise (enterprise software suite). Each layer creates switching costs:

• Code dependency: Years of CUDA-optimised code, CUDA-specific memory management patterns, and framework integrations that assume NVIDIA hardware.
• Pricing power:NVIDIA B200 GPUs command $30,000–$40,000 per unit. AI Enterprise licensing adds $4,500 per GPU per year.¹With 80–95% market share, NVIDIA faces minimal competitive pricing pressure.
• Supply allocation:NVIDIA controls who gets GPUs and when. Customers compete for allocation — procurement is not a market transaction but a relationship negotiation.
• Export controls:US Bureau of Industry and Security (BIS) restrictions (October 2022, October 2023, January 2025 “AI Diffusion Rule”) can block GPU access based on destination country or end-use classification.²
• No exit path:No CUDA alternative exists within the NVIDIA ecosystem. You use CUDA, or you use nothing from NVIDIA. The switching cost is not a migration — it is a complete platform replacement.

RISC-V Open ISA Advantages

RISC-V represents a fundamentally different approach to compute architecture. Rather than a proprietary moat, it is an open foundation designed for transparency, competition, and jurisdictional independence.

1. Full Auditability

RISC-V is an open instruction set architecture governed by RISC-V International, a Swiss-domiciled non-profit. Unlike x86 (Intel) or CUDA (NVIDIA), the ISA specification is publicly available. Any organisation can audit the instruction set for security, compliance, or performance verification. This enables a level of silicon transparency impossible with proprietary architectures.

2. No Export Controls on the ISA

The RISC-V instruction set itself is not subject to US export restrictions — it is an open standard, not a controlled technology. While specific chip implementations may involve controlled manufacturing processes (e.g., advanced lithography nodes), the architectural layer — the ISA — is free from jurisdictional constraints. This is a structural advantage for organisations outside the US that need supply-chain certainty.

3. No Licensing Fees

RISC-V charges no royalties or per-unit licensing fees. Compare this to ARM (which charges per-unit royalties on every chip shipped) or NVIDIA (which bundles software licensing with hardware procurement). For large-scale deployments, the elimination of perpetual licensing costs is a significant TCO advantage.

4. Multi-Vendor Ecosystem

Multiple companies are building RISC-V AI accelerators: Tenstorrent, Esperanto Technologies, Ventana Micro, SiFive, and others. This creates competitive dynamics that reduce pricing power concentration. No single vendor can dictate terms to the entire market — the antithesis of NVIDIA's current position.

Tenstorrent Galaxy Blackhole: Specifications

Tenstorrent, led by legendary chip architect Jim Keller (previously AMD Zen architecture, Apple A-series processors, Tesla Autopilot chip), has shipped the Galaxy server platform as a purpose-built RISC-V AI inference system.

Power Efficiency Comparison: 7.9 kW vs 23 kW

Power consumption is the fastest-growing line item in AI infrastructure TCO. As inference workloads scale, the difference between architectures compounds into millions in operational cost.

NVIDIA H100 SXM (8-GPU Server)

The NVIDIA H100 SXM remains the most widely deployed high-end inference GPU as of mid-2026. B200/B300 are shipping but deployments are still scaling.

• GPU TDP: 700W × 8 GPUs = 5,600W
• Host system, networking, storage: ~2,400W
• Approximate server power: ~8,000W (8 kW)
• Cooling: Liquid cooling required for sustained operation
• For comparable inference throughput: 2–3 H100 servers needed = ~16–24 kW
• Mid-point estimate: ~23 kW (before PUE overhead)

Tenstorrent Galaxy (32-Chip Server)

• Server TDP: ~9 kW (all-inclusive — 32 chips + host + networking)³
• Cooling: Air-cooled, standard data-centre HVAC
• PUE: ~1.3–1.4 for air-cooled facilities
• Effective power per server (with PUE): ~11.7–12.6 kW

For comparable inference throughput on production LLM workloads, Galaxy achieves a 61–66% reduction in power consumption versus the equivalent H100 deployment.

TCO Comparison: Cost Per 1M Tokens

The most relevant metric for production AI economics is not peak FLOPS — it is cost per useful token delivered to a production endpoint. The following table breaks down the per-token cost structure for both architectures.

^†Tenstorrent has not publicly disclosed per-unit Galaxy pricing. AGICY projections are based on Tenstorrent's stated positioning and internal deployment modelling. These figures will be updated with verified data as procurement contracts are finalised.

Why EU AI Act Art. 26 Favours Auditable Hardware

Article 26 of the EU AI Act requires deployers of high-risk AI systems to maintain oversight, monitor operations, and ensure transparency. These requirements have architectural implications that extend to the hardware layer.

Meaningful oversight requires understanding how an AI system operates at everylayer — including the silicon executing inference computations. Proprietary hardware (CUDA, x86) cannot be independently audited at the instruction-set level. The ISA, firmware, and microcode are trade secrets. Organisations must trust the vendor's attestation.

RISC-V's open ISA enables a verifiable chain of transparency:

• Open instruction set→ auditable at the architectural level
• Auditable firmware→ verifiable boot and runtime integrity
• Transparent software stack→ open-source TT-Metalium (Apache 2.0)
• Documented inference pipeline→ end-to-end auditability for Art. 26 compliance

The Cyber Resilience Act (CRA) adds security-by-design requirements that structurally favour open architectures over proprietary black boxes. NIS2's supply-chain security provisions penalise opaque vendor dependencies in critical infrastructure. Together, these regulations create a compliance environment where hardware transparency is not just advantageous — it is increasingly expected.

The Tenstorrent + AGICY Deployment

AGICY is deploying Tenstorrent Galaxy infrastructure in Cyprus (EU member state) as a purpose-built sovereign AI inference facility. The deployment is designed to demonstrate that open-hardware AI infrastructure can deliver production-grade performance with full regulatory compliance.

• Location:Cyprus — EU member state, ensuring full EU data jurisdiction under GDPR.
• Hardware: Tenstorrent Galaxy servers with Blackhole RISC-V accelerators. Open-ISA, fully auditable.
• Scale: Modular build-out starting with initial cluster, scaling to multi-megawatt facility.
• Software:TT-Metalium (open source) + AGICY's sovereign inference orchestration layer.
• Cooling: Air-cooled, leveraging Mediterranean climate for lower HVAC baseline costs.
• Models:Open-weight LLMs (Llama, Mistral, Qwen families) running locally — no API dependency on any foreign-jurisdiction provider.
• Compliance: Designed from the ground up for GDPR + NIS2 + EU AI Act + DORA multi-regulation environment.
• Sovereignty: EU-incorporated entity, no US parent company, no CLOUD Act exposure, RISC-V hardware auditability.

Frequently Asked Questions

Can RISC-V match NVIDIA's performance for AI inference?

For training workloads, no — NVIDIA's CUDA ecosystem and GPU architecture retain decisive advantages in FP16/FP8 training throughput. For inference workloads (token generation from pre-trained models), RISC-V-based systems like Tenstorrent Galaxy are competitive on throughput while offering significantly lower power consumption and no software licensing costs. Independent, at-scale benchmarks remain limited as of mid-2026, and performance comparisons should be treated with appropriate caution until third-party validation is available.

Is RISC-V really free from export controls?

The RISC-V instruction set architecture itself is an open standard governed by RISC-V International, a Swiss-domiciled non-profit. The ISA is not subject to US export licensing. However, specific chip implementations may involve manufacturing processes (e.g., advanced lithography at TSMC or Samsung) or EDA tools that are subject to export controls. The key distinction is that the ISA layer — the architectural foundation — is open and internationally accessible, even if specific implementations face manufacturing constraints.

What software runs on Tenstorrent Galaxy?

Tenstorrent provides TT-Metalium, an open-source (Apache 2.0) low-level software stack for programming Blackhole chips, plus TT-Buda, a higher-level ML framework for model deployment. Major open-weight LLMs — including Llama 3, Mistral, and Qwen family models — have been ported to run on Galaxy hardware. The ecosystem is less mature than CUDA but is actively expanding with community and commercial contributions.

How does the 7.9 kW vs 23 kW comparison work?

The 7.9–9 kW figure is the rated TDP of a single Tenstorrent Galaxy server containing 32 Blackhole chips, including all host system and networking power. The 23 kW figure represents the approximate power draw of 2–3 NVIDIA H100 8-GPU servers needed to deliver comparable inference throughput on production LLM workloads. The comparison is throughput-normalised for inference token generation, not raw FLOPS — which is the metric that matters for production AI economics.

Why is AGICY using RISC-V instead of NVIDIA?

Three reasons: (1) Sovereignty— RISC-V's open ISA enables hardware-level auditability required for EU AI Act compliance and eliminates export-control risk on the architectural layer. (2) Economics— lower power consumption, no software licensing fees, and competitive hardware pricing create a favourable TCO for inference workloads. (3) Independence— no single-vendor dependency for the most critical layer of the infrastructure stack. AGICY retains the ability to evaluate and adopt future RISC-V accelerators from multiple vendors.