How to run an LLM on your laptop

The local LLM world used to have a high barrier to entry: In the early days, it was impossible to run anything useful without investing in pricey GPUs. But researchers have had so much success in shrinking down and speeding up models that anyone with a laptop, or even a smartphone, can now get in on the action. In this edition of What’s Next in Tech, find out how.

TODAY: AI agents are evolving fast, shifting from helpful co-pilots to systems that can plan, reason, and act on their own. So how far have we come, and where are we headed? Join MIT Technology Review editors on today at 12 p.m. EDT for a special LinkedIn Live as they discuss navigating the rise of AI agents. Register for free.

For people who are concerned about privacy, want to break free from the control of the big LLM companies, or just enjoy tinkering, local models offer a compelling alternative to ChatGPT and its web-based peers.

Local LLMs aren’t just for proficient coders. If you’re comfortable using your computer’s command-line interface, which allows you to browse files and run apps using text prompts, Ollama is a great option. Once you’ve installed the software, you can download and run any of the hundreds of models they offer with a single command. 

If you don’t want to touch anything that even looks like code, you might opt for LM Studio, a user-friendly app that takes a lot of the guesswork out of running local LLMs. You can browse models from Hugging Face from right within the app, which provides plenty of information to help you make the right choice. Some popular and widely used models are tagged as “Staff Picks,” and every model is labeled according to whether it can be run entirely on your machine’s speedy GPU, needs to be shared between your GPU and slower CPU, or is too big to fit onto your device at all. Once you’ve chosen a model, you can download it, load it up, and start interacting with it using the app’s chat interface.

As you experiment with different models, you’ll start to get a feel for what your machine can handle. According to Simon Willison, who writes a popular blog about local LLMs and software development, every billion model parameters require about one GB of RAM to run, and we found that approximation to be accurate: Our AI reporter’s own 16 GB laptop managed to run Alibaba’s Qwen3 14B as long as they quit almost every other app. If you run into issues with speed or usability, you can always go smaller—they got reasonable responses from Qwen3 8B as well.

And if you go really small, you can even run models on your cell phone. Our reporter’s beat-up iPhone 12 was able to run Meta’s Llama 3.2 1B using an app called LLM Farm. It’s not a particularly good model—it very quickly goes off into bizarre tangents and hallucinates constantly—but trying to coax something so chaotic toward usability can be entertaining. If you're ever on a plane sans Wi-Fi and desperate for a probably false answer to a trivia question, you now know where to look. 

Read the full story to learn more about how you can run an LLM on your laptop—and why you might want to.

⚡ What's Next in Tech Readers: Claim your special, FREE 30-day trial subscription today.

Get ahead with these related stories:

1. A major AI training data set contains millions of examples of personal data — Personally identifiable information has been found in DataComp CommonPool, one of the largest open-source data sets used to train image generation models.

2. This tool strips away anti-AI protections from digital art — The proof-of-concept technology removes a layer of digital “poison” meant to disrupt AI models that scrape online images for training.

3. Cloudflare will now, by default, block AI bots from crawling its clients’ websites — The company will also introduce a "pay-per-crawl" system to give users more fine-grained control over how AI companies can access their sites.

Be prepared for the impact AI will have on our world with our weekly newsletter, The Algorithm. Sign up today for in-depth stories and exclusive AI insights.  

Image: Stephanie Arnett/MIT Technology Review | Adobe Stock, Envato