The AI industry has mostly measured progress by size: more parameters, more data centres, more compute. That makes sense for a frontier lab. It makes less sense for a pair of glasses that has to answer a simple question: what am I looking at right now?
PrismML is building for the second case. The AI lab was founded by researchers from Caltech and counts UC Berkeley's Ion Stoica among its advisers. It has now produced a version of its compact language models for smart glasses that run on Qualcomm's Snapdragon chips.
From the data centre to the frame: what Qualcomm showed at Snapdragon Summit
Most of the talk about shrinking language models centers on 4-bit and 8-bit quantization. Ternarisation goes much further. It cuts every weight in a model down to one of three values: -1, 0, or +1.
That idea has been circulating in AI research for a while, most visibly through Microsoft Research's BitNet b1.58 work. It gets especially interesting at the scale of a model with around 1.85 billion parameters. A model that size is small enough to experiment with on modest hardware, but large enough to be useful for real tasks.
What ternarisation means
In a standard model, each weight is stored as a 16-bit or 32-bit floating-point number. Ternarisation replaces each weight with -1, 0, or +1, usually paired with a scaling factor that is shared across a group of weights or a whole layer.
Running a 35-billion-parameter model on a single consumer graphics card usually means compromises. A new open source project called FastLocalAI tries to keep those compromises small. It's a tuned setup for running Alibaba's Qwen3.6-35B-A3B on an NVIDIA GPU with just 16 GB of VRAM, paired with the OpenCode coding agent.
The project is available on GitHub at https://github.com/24high/FastLocalAI. It uses llama.cpp's llama-server and moves part of the model into regular system memory so the rest fits on the card.
Why the model doesn't fit and how it works anyway
Qwen3.6-35B-A3B is a Mixture-of-Experts model. The quantized Q4_K_M version from Unsloth, which the project uses by default, is about 22 GB. That is too large for a 16 GB card.
Open-weight models are improving quickly, and that progress is changing how organizations approach AI. A growing number of companies are no longer relying only on externally hosted services. Instead, they are running smaller models on their own infrastructure.
Privacy and Cost Drive the Shift
Two motivations stand out. The first is privacy. When a model runs on a company's own hardware, sensitive data stays inside its environment. The second is cost. For many workloads, a compact self-hosted model can be a more economical choice than paying for access to a larger external system.
CyrioX is financed by advertising. You can choose how you want to use this website:
With advertising: we load an advertising script from a third-party ad network. The ad network may set cookies, use your IP address and device information, and may process data outside the EU.
Ad-free for €0.99 per month: no advertising and no advertising tracking. Cancel at any time.
You can change your decision at any time via "Cookie Settings" at the bottom of every page.
