A writer swapped his cloud chatbot for a free AI that runs on his own laptop

A Tom's Guide writer has dropped his cloud AI chatbot and now runs a free AI assistant on his own laptop. His reasons are privacy and cost. The tool is called Hermes Agent. It is open source, which means its code is public, and it installs on Windows and Mac like any other app.

What an AI agent is

A standard chatbot waits for you to ask a question. An agent is given a goal, some tools and permission to work on its own. Tools can include access to your files or a web browser. The agent thinks, acts, checks its work and keeps going until the job is done.

How the setup works

The writer says Hermes is the easiest to move to from a cloud service like his Gemini Pro subscription. He notes that OpenClaw and LM Studio Bionic have fans too, and calls the choice personal taste. Setup is short:

• Download and install Hermes Agent.
• Answer the setup questions: your name, which app connectors to add, and a theme color.
• Open settings, choose provider, then select local.
• Go to model and download a local one to act as the agent's brain.

Fitting a big model on a laptop

Which model you pick depends on your laptop's memory. Large models are shrunk through a process called quantization. Think of turning a huge RAW photo into a JPEG. The model's very precise numbers are rounded down, so you lose a tiny bit of precision and the file gets much smaller.

The writer runs a 27-billion-parameter model that fits in 32GB of memory on his ASUS ProArt P14. His tip is to look for the tags "GGUF" and "Q" when downloading. He points to modelfit.io as a guide for choosing.

The trade-off

Local AI is slower. In his example, reading a 50-page PDF and checking it against his calendar takes three to four minutes. A cloud service does it in 30 to 60 seconds. His advice is to hand over long jobs and do something else while they run.

It also needs more from you. A local AI only knows what you give it, so you must supply things like sample invoices or emails to show your style. It takes instructions literally, so vague requests get thin results.

What to watch

The writer says new hardware, including Nvidia RTX Spark laptops he tested for two weeks, has made this practical. He also mentions a rumored Apple M6 MacBook Pro. That one is a rumor, not a confirmed product. The excerpt of the article provided here ends before all of his tips, so the full list is not covered.