How to run an AI agent 24/7 without a GPU

Published

Last updated

You don't need a GPU to run an AI agent 24/7. The model runs at your API provider; the machine you own only runs the harness, a normal program that needs a little RAM and CPU. Here's the whole setup with Gray.

What you need

  • —A machine that stays on: a small Linux VPS or a home server. Gray needs about [x] MB of RAM at idle, and the smallest box we've tested it on is [x].
  • —An API key for a model provider, such as OpenRouter, OpenAI, Anthropic or DeepSeek.
  • —About [x] minutes.

1. Install Gray

bash
$curl -fsSL https://gray.alignment.id/install.sh | sh

2. Connect a model

Start gray and type /connect to pick a provider and paste your key; /model switches models at any time. Keys are stored in ~/.gray/auth.json with 0600 permissions.

3. Give it a schedule

A cron job is a stored prompt with a schedule. Add one from the shell, or ask the agent to add it for you:

bash
$gray cron add "every 1h" "check the deploy and tell me if anything failed" --name deploy-watch
$gray cron list

4. Keep it running in the background

The gateway is a user service that ticks cron jobs every 60 seconds and keeps app connections alive with no terminal open. It is supervised by systemd or runit.

bash
$gray gateway install     # write and start the user service
$gray gateway status      # daemon, service and ticker health

5. Talk to it from Discord (optional)

Install the official Discord plugin, then run its setup:

bash
$gray plugin install discord
$gray gateway setup discord --field token=… --field owner_id=… --field channel_id=… --start

What about local models?

Local models are where a GPU starts to matter. Gray connects to Ollama, LM Studio or any OpenAI-compatible server, but the model then runs on your own hardware, and its speed depends on that hardware. For an always-on agent on a small server, a hosted model is the simpler choice.

What does it cost?

Gray is free and open source (MIT). You pay for the server ([x] per month for the box we tested) and for the model tokens your agent uses.

← All posts