Microsoft Small Language Model (SML) series Phi-3.5 will get an even broader distribution as the company packages it up for an open-source launch this week.
The Phi-3.5-mini instruct, Phi-3.5-MoE instruct, and Phi-3.5-vision-instruct gain open-source support landing on Hugging Face. Developers with access to a Microsoft-branded MIT license will get to play around with the new SMLs trained in basic and powerful reasoning along with enhanced video and image analysis.
Microsoft’s lowly Phi-3.5-mini-instruct was trained on 3.82 billion parameters and is said to beat Meta’s Llama3.1 8B, Mistral 7B, and goes toe-to-toe with the more advance Mistral Nemo 12B according to a HuggingFace software engineer and programmer, Vaibhav Srivastav on Twitter.
The lightweight multi-tun conversational Phi-3.5 Mini-instruct is Microsoft’s answer to AI computing in constrained environments while still supporting 128k token context lengths. Developers should find utility in their ability to code generate, mathematical driven problem solving and basic logical reasoning.
Then there is Phi-3.5 MoE instruct model that leans on its 42 billion parameters of training and 128K token context length to deliver more robust mathematical problem solving than Mini-instruct as well as multilanguage understanding. Based on RepoQA benchmarking, Microsoft has managed to produce a SML that outpaces that of its partner OpenAI’s latest GPT-40 mini when it comes to Massive Multitask Language Understanding for items such as extrapolating hummanitarian studies, STEM or social services.
As fir the Phi-3.5 vision-instruct, the multimodal reasoning SLM shows a promising recognition of for general imaging, character recognition, table understanding and video summations. For the privacy and creative community, Microsoft claims vision instruct was trained on “publicly available datasets, focusing on high-quality reasoning dense data,” according to a Venture Beat article.
For any developer interested in playing around with Microsoft’s home-grown SLM’s, the company does warn that its licesne granted to access these instructs is provided “as is” which translates to ‘without warranty’ which will leave it to the open-source community to come together to support any potential issues going forward.
