Majestic Labs, an AI hardware startup, is addressing the memory limitations of large language models (LLMs) with its upcoming server, Prometheus, set to launch in 2027. This innovative server will feature up to 128 terabytes of memory, significantly surpassing the capabilities of Nvidia’s current offerings. Co-founder Sha Rabii emphasizes that this substantial memory increase will enhance performance and efficiency, particularly as models grow larger.
Prometheus employs a unique DRAM-centric architecture, utilizing LPDDR6 memory and a proprietary memory interface with miniature copper cables that allow for greater memory placement flexibility. This design aims to overcome the “memory wall” that hampers LLM performance, providing a memory bandwidth of up to 25.6 terabytes per second.
To complement its memory capabilities, Prometheus will incorporate the Ignite AI processing unit, which combines ARM application cores with RISC-V vector and tensor cores on a single chip. This integration allows for seamless handling of LLM inference tasks without the need for processor handoffs.
Majestic Labs is also focused on ensuring compatibility with existing AI frameworks like PyTorch and OpenAI’s Triton, allowing customers to run their models without modifications. The server, designed in compliance with the Open Compute Project, will be modular, enabling future memory upgrades.
Despite the advanced technology, Majestic Labs aims to offer competitive pricing by leveraging DRAM instead of more expensive high-bandwidth memory. Rabii claims that this approach could reduce customer capital expenditures and power consumption significantly, potentially by 10 to 50 times, depending on the workload.
Leave a comment