Local AI models for sensitive work
Open models now run well on a good laptop or a small server. For some confidential tasks, that’s simpler than any contract.
A year or two ago, running a useful language model yourself needed serious hardware. Now open-weight models run on a recent laptop or a single-GPU server, through tools like Ollama or LM Studio, with no data leaving the machine.
They’re not as capable as the best cloud models. For many everyday tasks they don’t need to be: summarising internal documents, cleaning up text, drafting replies, searching your own files.
When local makes sense:
- Data you’re not allowed to send outside: health records, legal files, government work, some client data.
- Offline or air-gapped environments.
- High-volume simple tasks where API costs add up.
Things to watch: download models only from their official publishers, keep the tool updated, and don’t expose its API to the network without a login. Local isn’t automatically safe. It’s just under your control.