According to a report by The Information, Microsoft is developing a large scale language learning model (LLM) called MAI-1, featuring a staggering 500 billion parameters. If the claims hold, the model will be the largest that Microsoft has deployed, surpassing the company’s Phi-3 Mini family of small language models that range from 3.8B to 14B parameters.
The estimated size of MAI-1 positions Microsoft as a major contender against OpenAI’s GPT-4 and Google’s larger Gemini models. Recent rumors suggest that GPT-4 has 1.76 trillion parameters, but its Mixture of Experts (MoE) model leaves only around 280 billion parameters in play during the inference phase. If MAI-1 uses a dense model structure instead of MoE, its vastness will potentially gift it with unprecedented power. This puts Microsoft’s upcoming model not only in competition with OpenAI and Google but also with Meta’s anticipated Llama 3 model, which is projected to contain 400 billion parameters.
Behind the development of MAI-1 is Mustafa Suleyman, a former head of applied AI at DeepMind, who co-founded the AI startup Inflection in 2022. Microsoft later hired a significant portion of Inflection’s workforce and purchased the startup’s intellectual property rights for $650 million. Despite these circumstances, it is understood that the MAI-1 project is entirely Microsoft’s initiative, separate from any pre-existing efforts at Inflection.
Although the release date remains under wraps, Microsoft might provide a glimpse of the MAI-1 at its Build developer conference on May 16.
Nevertheless, this move by Microsoft has raised eyebrows due to its status as OpenAI’s largest investor, creating conjectures about potential competition or diversification strategies. Reacting to the speculation, Microsoft’s CTO Kevin Scott downplayed the news, stating openly on LinkedIn that Microsoft builds supercomputers to train AI models, collaborates with OpenAI to create models, and then combines their powers to deploy in products and services for the benefit of many.
However, with the debut of MAI-1, it’s difficult to ignore the possibility of Microsoft coming head-to-head with OpenAI, wherein they have invested billions. Additionally, uncertainties surround the potential upstaging by OpenAI’s projected GPT-5 release.
As all these companies are grappling to push boundaries in AI development, notable debate, excitement, and suspense are building up around these mega-sized models and their impacts in the near future.
