Meta’s Llama 3.2 lightweight language models fit on device

Meta’s (NASDAQ:META) latest large language model, Llama 3.2, was released today with mini versions available, allowing them to fit on edge and mobile devices, the social media giant revealed today at Meta Connect 2024.

The Llama 3.2 1B and 3B models are text-only and support a context length of 128k tokens. They can perform on-device use cases such as summarizations, following instructions and rewriting tasks locally.

The models are enabled for Qualcomm (QCOM) and MediaTek (OTCPK:MDTTF) hardware and optimized for Arm (ARM) processors. Meta chose Arm because it provides the foundational compute platform for 99% of mobile devices.

Llama 3.2 also has small and medium-sized models, 11B and 90B, which include image and vision understanding.

The Llama 3.2 models are intended to be the most developer-focused models yet, and help developers build on-device agents.

“These models empower developers to build personalized, on-device agentic applications with strong privacy where data never leaves the device,” Meta said. “For example, such an application could help summarize the last 10 messages received, extract action items, and leverage tool calling to directly send calendar invites for follow-up meetings.”

The on-device models allow for quicker responses, since the processing is done locally, and helps ensure privacy, since information does not have to migrate to the cloud.

Meta is not the first to introduce on-device AI models. Google’s (GOOG)(GOOGL) new Pixel 9 Pro has on-device Gemini AI. Apple’s (AAPL) iPhone 16 models feature Apple Intelligence, which will offer on-device AI once iOS 18.1 is released next month.