AI agent harness benchmark ([x]): RAM, CPU and setup time compared

Last updated

On a [x] VPS, Gray idles at [x] MB vs OpenClaw's [x] MB and Hermes Agent's [x] MB. Every tool ran on the same box against the same model, so the numbers measure the harness, not the model.

This page measures what an always-on agent costs to keep running. For cold start and install size of the CLIs on a dev machine, see the CLI benchmarks.

Results

ToolVersionInstall sizeDependenciesTime to first replyIdle RSSIdle CPUActive RSSActive CPUConfig lines
● Gray[x][x] MB[x][x] s[x] MB[x]%[x] MB[x]%[x]
OpenClaw[x][x] MB[x][x] s[x] MB[x]%[x] MB[x]%[x]
Hermes Agent[x][x] MB[x][x] s[x] MB[x]%[x] MB[x]%[x]
ZeroClaw[x][x] MB[x][x] s[x] MB[x]%[x] MB[x]%[x]
nanobot[x][x] MB[x][x] s[x] MB[x]%[x] MB[x]%[x]
NanoClaw[x][x] MB[x][x] s[x] MB[x]%[x] MB[x]%[x]

Idle: mean over one hour · Active: during a fixed 10-task script · lower is better

How we measure

same box

Every tool runs on the same machine: [x], plus a second run on [x]. Each starts from a clean OS image.

same model

Every tool talks to the same model through the same provider ([x]), so the numbers measure the harness, not the model.

install

Install size on disk and the number of dependencies after following each project's own install instructions, plus the wall-clock time from a clean box to the first agent reply.

idle

Resident memory and CPU sampled once a minute for one hour with the agent running and waiting for work.

pidstat -r -u -p $PID 60 60      # native installs
docker stats --no-stream $NAME    # container installs

active

Peak resident memory and mean CPU while each tool runs the same fixed script of 10 tasks.

config

Lines of configuration needed for one basic agent: "check X every hour and message me".

Raw CSV and the scripts to reproduce every number: [x]. We re-run the benchmark every quarter, or when one of these tools ships a major version.

Changelog

  • [x]: first run.

Rankings by use case are on best lightweight AI agent harnesses.