openclaw/docs/help/debugging.md

summary, read_when, title

summary	read_when	title
Debugging tools: watch mode, raw model streams, and tracing reasoning leakage	You need to inspect raw model output for reasoning leakage You want to run the Gateway in watch mode while iterating You need a repeatable debugging workflow	Debugging

This page covers debugging helpers for streaming output, especially when a provider mixes reasoning into normal text.

Runtime debug overrides

Use /debug in chat to set runtime-only config overrides (memory, not disk). /debug is disabled by default; enable with commands.debug: true. This is handy when you need to toggle obscure settings without editing openclaw.json.

Examples:

/debug show
/debug set messages.responsePrefix="[openclaw]"
/debug unset messages.responsePrefix
/debug reset

/debug reset clears all overrides and returns to the on-disk config.

Session trace output

Use /trace when you want to see plugin-owned trace/debug lines in one session without turning on full verbose mode.

Examples:

/trace
/trace on
/trace off

Use /trace for plugin diagnostics such as Active Memory debug summaries. Keep using /verbose for normal verbose status/tool output, and keep using /debug for runtime-only config overrides.

Temporary CLI debug timing

OpenClaw keeps src/cli/debug-timing.ts as a small helper for local investigation. It is intentionally not wired into CLI startup, command routing, or any command by default. Use it only while debugging a slow command, then remove the import and spans before landing the behavior change.

Use this when a command is slow and you need a quick phase breakdown before deciding whether to use a CPU profiler or fix a specific subsystem.

Add temporary spans

Add the helper near the code you are investigating. For example, while debugging openclaw models list, a temporary patch in src/commands/models/list.list-command.ts might look like this:

// Temporary debugging only. Remove before landing.
import { createCliDebugTiming } from "../../cli/debug-timing.js";

const timing = createCliDebugTiming({ command: "models list" });

const authStore = timing.time("debug:models:list:auth_store", () => ensureAuthProfileStore());

const loaded = await timing.timeAsync(
  "debug:models:list:registry",
  () => loadListModelRegistry(cfg, { sourceConfig }),
  (result) => ({
    models: result.models.length,
    discoveredKeys: result.discoveredKeys.size,
  }),
);

Guidelines:

• Prefix temporary phase names with debug:.
• Add only a few spans around suspected slow sections.
• Prefer broad phases such as registry, auth_store, or rows over helper names.
• Use time() for synchronous work and timeAsync() for promises.
• Keep stdout clean. The helper writes to stderr, so command JSON output stays parseable.
• Remove temporary imports and spans before opening the final fix PR.
• Include the timing output or a short summary in the issue or PR that explains the optimization.

Run with readable output

Readable mode is best for live debugging:

OPENCLAW_DEBUG_TIMING=1 pnpm openclaw models list --all --provider moonshot

Example output from a temporary models list investigation:

OpenClaw CLI debug timing: models list
     0ms     +0ms start all=true json=false local=false plain=false provider="moonshot"
     2ms     +2ms debug:models:list:import_runtime duration=2ms
    17ms    +14ms debug:models:list:load_config duration=14ms sourceConfig=true
  20.3s  +20.3s debug:models:list:auth_store duration=20.3s
  20.3s     +0ms debug:models:list:resolve_agent_dir duration=0ms agentDir=true
  20.3s     +0ms debug:models:list:resolve_provider_filter duration=0ms
  25.3s   +5.0s debug:models:list:ensure_models_json duration=5.0s
  31.2s   +5.9s debug:models:list:load_model_registry duration=5.9s models=869 availableKeys=38 discoveredKeys=868 availabilityError=false
  31.2s     +0ms debug:models:list:resolve_configured_entries duration=0ms entries=1
  31.2s     +0ms debug:models:list:build_configured_lookup duration=0ms entries=1
  33.6s   +2.4s debug:models:list:read_registry_models duration=2.4s models=871
  35.2s   +1.5s debug:models:list:append_discovered_rows duration=1.5s seenKeys=0 rows=0
  36.9s   +1.7s debug:models:list:append_catalog_supplement_rows duration=1.7s seenKeys=5 rows=5

Model                                      Input       Ctx   Local Auth  Tags
moonshot/kimi-k2-thinking                  text        256k  no    no
moonshot/kimi-k2-thinking-turbo            text        256k  no    no
moonshot/kimi-k2-turbo                     text        250k  no    no
moonshot/kimi-k2.5                         text+image  256k  no    no
moonshot/kimi-k2.6                         text+image  256k  no    no

  36.9s     +0ms debug:models:list:print_model_table duration=0ms rows=5
  36.9s     +0ms complete rows=5

Findings from this output:

Phase	Time	What it means
debug:models:list:auth_store	20.3s	The auth-profile store load is the largest cost and should be investigated first.
debug:models:list:ensure_models_json	5.0s	Syncing models.json is expensive enough to inspect for caching or skip conditions.
debug:models:list:load_model_registry	5.9s	Registry construction and provider availability work are also meaningful costs.
debug:models:list:read_registry_models	2.4s	Reading all registry models is not free and may matter for --all.
row append phases	3.2s total	Building five displayed rows still takes several seconds, so the filtering path deserves a closer look.
debug:models:list:print_model_table	0ms	Rendering is not the bottleneck.

Those findings are enough to guide the next patch without keeping timing code in production paths.

Run with JSON output

Use JSON mode when you want to save or compare timing data:

OPENCLAW_DEBUG_TIMING=json pnpm openclaw models list --all --provider moonshot \
  2> .artifacts/models-list-timing.jsonl

Each stderr line is one JSON object:

{
  "command": "models list",
  "phase": "debug:models:list:registry",
  "elapsedMs": 31200,
  "deltaMs": 5900,
  "durationMs": 5900,
  "models": 869,
  "discoveredKeys": 868
}

Clean up before landing

Before opening the final PR:

rg 'createCliDebugTiming|debug:[a-z0-9_-]+:' src/commands src/cli \
  --glob '!src/cli/debug-timing.*' \
  --glob '!*.test.ts'

The command should return no temporary instrumentation call sites unless the PR is explicitly adding a permanent diagnostics surface. For normal performance fixes, keep only the behavior change, tests, and a short note with the timing evidence.

For deeper CPU hotspots, use Node profiling (--cpu-prof) or an external profiler instead of adding more timing wrappers.

Gateway watch mode

For fast iteration, run the gateway under the file watcher:

pnpm gateway:watch

This maps to:

node scripts/watch-node.mjs gateway --force

The watcher restarts on build-relevant files under src/, extension source files, extension package.json and openclaw.plugin.json metadata, tsconfig.json, package.json, and tsdown.config.ts. Extension metadata changes restart the gateway without forcing a tsdown rebuild; source and config changes still rebuild dist first.

Add any gateway CLI flags after gateway:watch and they will be passed through on each restart. Re-running the same watch command for the same repo/flag set now replaces the older watcher instead of leaving duplicate watcher parents behind.

Dev profile + dev gateway (--dev)

Use the dev profile to isolate state and spin up a safe, disposable setup for debugging. There are two --dev flags:

• Global --dev (profile): isolates state under ~/.openclaw-dev and defaults the gateway port to 19001 (derived ports shift with it).
• gateway --dev: tells the Gateway to auto-create a default config + workspace when missing (and skip BOOTSTRAP.md).

Recommended flow (dev profile + dev bootstrap):

pnpm gateway:dev
OPENCLAW_PROFILE=dev openclaw tui

If you don’t have a global install yet, run the CLI via pnpm openclaw ....

What this does:

1. Profile isolation (global --dev)

   • OPENCLAW_PROFILE=dev
   • OPENCLAW_STATE_DIR=~/.openclaw-dev
   • OPENCLAW_CONFIG_PATH=~/.openclaw-dev/openclaw.json
   • OPENCLAW_GATEWAY_PORT=19001 (browser/canvas shift accordingly)

2. Dev bootstrap (gateway --dev)

   • Writes a minimal config if missing (gateway.mode=local, bind loopback).
   • Sets agent.workspace to the dev workspace.
   • Sets agent.skipBootstrap=true (no BOOTSTRAP.md).
   • Seeds the workspace files if missing: AGENTS.md, SOUL.md, TOOLS.md, IDENTITY.md, USER.md, HEARTBEAT.md.
   • Default identity: C3‑PO (protocol droid).
   • Skips channel providers in dev mode (OPENCLAW_SKIP_CHANNELS=1).

Reset flow (fresh start):

pnpm gateway:dev:reset

Note: --dev is a global profile flag and gets eaten by some runners. If you need to spell it out, use the env var form:

OPENCLAW_PROFILE=dev openclaw gateway --dev --reset

--reset wipes config, credentials, sessions, and the dev workspace (using trash, not rm), then recreates the default dev setup.

Tip: if a non‑dev gateway is already running (launchd/systemd), stop it first:

openclaw gateway stop

Raw stream logging (OpenClaw)

OpenClaw can log the raw assistant stream before any filtering/formatting. This is the best way to see whether reasoning is arriving as plain text deltas (or as separate thinking blocks).

Enable it via CLI:

pnpm gateway:watch --raw-stream

Optional path override:

pnpm gateway:watch --raw-stream --raw-stream-path ~/.openclaw/logs/raw-stream.jsonl

Equivalent env vars:

OPENCLAW_RAW_STREAM=1
OPENCLAW_RAW_STREAM_PATH=~/.openclaw/logs/raw-stream.jsonl

Default file:

~/.openclaw/logs/raw-stream.jsonl

Raw chunk logging (pi-mono)

To capture raw OpenAI-compat chunks before they are parsed into blocks, pi-mono exposes a separate logger:

PI_RAW_STREAM=1

Optional path:

PI_RAW_STREAM_PATH=~/.pi-mono/logs/raw-openai-completions.jsonl

Default file:

~/.pi-mono/logs/raw-openai-completions.jsonl

Note: this is only emitted by processes using pi-mono’s openai-completions provider.

Safety notes

• Raw stream logs can include full prompts, tool output, and user data.
• Keep logs local and delete them after debugging.
• If you share logs, scrub secrets and PII first.