Microsoft has recently introduced a new small language model called Phi-3 mini, which is optimized for modern smartphones and delivers performance comparable to OpenAI’s GPT-3.5. This updated version of Microsoft’s lightweight language model has been trained with 3.3 billion tokens from larger and more advanced datasets compared to its predecessor, Phi-2, which was trained with 1.4 billion tokens.
Phi-3 mini boasts 3.8 billion parameters and can fit on a modern smartphone, taking up approximately 1.8GB of memory and can be quantified to 4 bits. The researchers tested the model on an iPhone 14 with an A16 Bionic chip, where it ran natively and offline at a speed of over 12 tokens per second. The overall performance of this model is comparable to larger models like Mixtral 8x7B and GPT-3.5, making it a powerful tool for a variety of applications.
The transformer decoder architecture used by Phi-3 mini supports 4K text length and is based on a block structure similar to Meta’s Llama 2, supporting packages developed for Llama 2. This makes it ideal for conversational chat formats and aligns with Microsoft’s values of robustness and security. In addition to Phi-3 mini, Microsoft has also trained two other models in the same family: Phi-3 medium with 14 billion parameters and Phi-3 small with 7 billion parameters, both trained with 4.8 billion tokens.
Overall, Microsoft is focused on providing efficient and secure language models for a variety of applications, making Phi-3 mini an exciting development for the tech industry as a whole.
A group of teenagers with special medical needs and chronic illnesses attended a prom of…
A recent study published in the British Journal of Sports Medicine has revealed that individuals…
When it comes to reducing “bad” low-density lipoprotein (LDL) cholesterol levels, experts recommend aiming as…
During hot weather, children with phimosis are at a higher risk of experiencing complications such…
French company Danone has completed the sale of its Russian business to Vamin R, according…
In 2023, the city of Kyle, Texas attempted to break the world record for the…