METASTONE Introduces TokenFleet for Efficient LLM Inference Scheduling Across Clusters
Compute operator METASTONE has launched TokenFleet, a new scheduling framework designed for large language model (LLM) inference clusters. According to the company's presentation, the implementation of TokenFleet has resulted in a 32.4% reduction in P95 time-to-first-token and a 29.4% increase in on...