Tensormesh Launches Tensormesh Platform to Cut LLM Inference Costs by Reusing KV Cache at Enterprise Scale
For best results when printing this announcement, please click on link below: Tensormesh Launches Tensormesh Platform to Cut LLM Inference Costs by Reusing KV Cache at Enterprise Scale Tensormesh ( ), the creators of LMCache, the world’s number one open-source KV caching technology, today announced Tensormesh Platform, the data management layer for AI inference. This multi-tier, scalable KV cache management software deploys across infrastructure with an enterprise Kubernetes deployment. With intelligent, AI-native data at its core, Tensormesh Platform enables enterprises to optimize the efficiency, cost, and quality of every inference they run. This marks the arrival of the first enterprise-ready KV cache management offering that works with any LLM, inference orchestration software, and storage vendor. “LLM workloads recompute the same context endlessly. That wastes GPU time and inflates time-to-first-token,” explained Junchen Jiang, Tensormesh CEO. “Tensormesh Platform addresses these costly problems, bringing production-grade KV cache reuse to enterprise on-prem inference stacks and making it operational at fleet scale.” Tensormesh Platform includes the management capabilities en
For best results when printing this announcement, please click on link below: Tensormesh Launches Tensormesh Platform to Cut LLM Inference Costs by Reusing KV Cache at Enterprise Scale Tensormesh ( ), the creators of LMCache, the world’s number one open-source KV caching technology, today announced Tensormesh Platform, the data management layer for AI inference. This multi-tier, scalable KV cache management software deploys across infrastructure with an enterprise Kubernetes deployment. With intelligent, AI-native data at its core, Tensormesh Platform enables enterprises to optimize the efficiency, cost, and quality of every inference they run.
This marks the arrival of the first enterprise-ready KV cache management offering that works with any LLM, inference orchestration software, and storage vendor. “LLM workloads recompute the same context endlessly. That wastes GPU time and inflates time-to-first-token,” explained Junchen Jiang, Tensormesh CEO.
” Tensormesh Platform includes the management capabilities enterprises need to run KV cache in production: Security: protects cached data across every tier Observability: provides real-time visibility into cache performance Resilience: high availability keeps inference workloads resilient through failures Policy Management: controls how cache data is stored, shared, and evicted Multi Tenancy: tenant management lets organizations serve multiple teams or customers from shared infrastructure, with each tenant's data isolated and governed "Tensormesh delivered strong results in our P2P KV cache transfer and resiliency benchmarks across multiple models.
We're excited to see them continue pushing the ecosystem forward," said Patrick Lundquist, senior software engineer at Nebius.
Enterprises deploying Tensormesh Platform also benefit from: Tiered Caching: KV Cache is managed across HBM, CPU, and storage layers automatically Prefix Caching: reuses the computed KV cache for the opening part of a prompt NonPrefix Caching (CacheBlend): reuses computed blocks anywhere in the prompt, not just at the start, making Tensormesh Platform ideal for RAG Peer-to-peer KV transfer: fetches KV cache directly from any peer in the fleet Advanced cache controls: allows users to pin, look up, move, compress, and clear entries via controller APIs, per workload or fleet-wide.
“Customers are under tremendous pressure both to drive efficiency through AI and to manage the costs of AI infrastructure that powers such capabilities,” said Rhys Oxenham, general manager of AI at SUSE. ” "AI has moved beyond a model challenge. It is now an infrastructure and application-development challenge," said Paul Nashawaty, principal analyst with theCUBE and ECI Research. "Our AppDev research shows nearly two-thirds of enterprises deploy applications daily or multiple times a day, and 66% of developers report frustration when AI solutions are almost right but need more iteration and debugging.
That translates directly into wasted compute, developer friction, and rising enterprise costs. GPU utilization, not GPU ownership, determines the economics of AI. " The Tensormesh Platform is available today.
Additional Resources Introducing Tensormesh Platform Tensormesh and AMD Collaborate to Empower Fewer GPUs to Serve More Models Why LLM Inference is a Data Problem, Not Just a Compute Problem Fixing AI’s Most Expensive Problem with Junchen Jiang, Tensormesh CEO LMCache: Build The Foundation of AI Memory Tensor with KV Cache Infrastructure Persistent KV Cache: Own Your Context Caching Lifecycle About Tensormesh Tensormesh is a technology company specializing in AI memory infrastructure and enterprise-grade software, and also known as the leader in smart AI-native data management and caching-accelerated inference optimization for enterprise AI. D.
researchers, and alumni from the University of Chicago, UC Berkeley, and Carnegie Mellon, and led by Junchen Jiang, University of Chicago faculty member and co-creator of LMCache, Tensormesh builds on years of academic research in distributed systems and AI infrastructure. 5 million in total funding and is backed by Valley Capital Partners, NVentures, AMD Ventures, CoreWeave, Laude Ventures, and more. ai)