The traditional model of running large-scale Artificial Intelligence applications typically relies on powerful yet expensive hardware. This creates a barrier to entry for individuals and smaller organizations who often struggle to afford high-end GPU’s to run extensive parameter models. The democratization and accessibility of advanced AI technologies also suffer as a result.
Several possible solutions are available to try and alleviate this problem. Cloud-based services provide access to high-performance hardware for a fee. While this can be obstructively costly over time, it also leaves users subject to the performance and policies of external providers. Alternatively, model optimization for execution on less powerful hardware is possible, but this can compromise performance levels and accuracy.
A new solution known as Cake aims to disrupt the status quo. Cake is a Rust-based framework specially designed to distribute the computational load of large AI models across networks of conventional consumer devices. This strategy allows the utilization of hardware that might otherwise be considered obsolete, essentially turning devices like smartphones, tablets, and laptops into a distributed computing cluster. This not only democratizes advanced AI by making it more accessible but also offers practical use for older tech, helping reduce electronic waste.
Cake distributes computational tasks by breaking them down into smaller more manageable pieces. Each device in the network processes a part of the model, with the results combined to produce the final output. This sharding process enables large models that might typically surpass the memory of a single GPU to be run across multiple devices. Batch tasks are also performed in Cake to reduce the latency induced by data transfer between devices and increase overall efficiency.
Cake’s effectiveness is evidenced by its wide-ranging device and operating system compatibility. Supporting Linux, Windows, macOS, Android, and iOS, and hardware acceleration types like CUDA and Metal, Cake provides a significantly flexible solution. This allows users to repurpose nearly any device to assist in the computing process. Testament to its potential, tests demonstrate that Cake can successfully run models with over 70 billion parameters, distributing this load across multiple devices.
To conclude, Cake presents a potentially game-changing solution to the issues of running large AI models. By distributing the computational load across various consumer devices, it smartly leverages otherwise obsolete technology. This presents a cost-effective and eco-friendly solution to advanced AI computations. While it remains experimental and subject to ongoing development, Cake represents a significant stride towards democratizing AI and increasing its accessibility to a broader audience.
