Version One: A Small Language Model on a 16 GB Mac mini
The first version of my local voice assistant ran a 4-billion-parameter model on a 16 GB Mac mini. Simple commands took about 2.7 seconds, and sometimes a reply took 20. Some of that wasn't the hardware's fault.