TuringData launches ContextCube to accelerate GPU inference clusters
Announced at Tech Week Singapore 2026, the ContextCube appliance delivers a shared, persistent KV cache pool to optimize GPU cluster inference.
At Tech Week Singapore 2026 on Sept. 29, 2026, TuringData introduced ContextCube, a purpose-built key-value cache appliance engineered to equip AI inference clusters with a shared, persistent context pool. The system is designed to streamline context management across participating GPU nodes handling artificial intelligence workloads.[1][2]
As input prompts grow longer and incoming requests are distributed across multiple servers, critical KV cache is frequently evicted or left inaccessible because it remains stored locally. ContextCube directly addresses this limitation by allowing previously computed context to be shared and reused across the entire inference cluster.[1]
Through the use of RDMA, participating inference nodes can retrieve matching cached KV data instead of repeatedly calculating or storing duplicate context. This mechanism reduces time to first token, preserves vital GPU cycles for token generation, and enhances overall cluster concurrency and throughput. The appliance is built on the NVIDIA CMX inference-storage reference design.[1]
