Rafay Adds Token Router Capabilities to Token Factory
The new routing layer directs inference requests based on model, priority, health and cost. The body is truncated.
The new routing layer directs inference requests based on model, priority, health and cost. The body is truncated.