---
name: apm
description: >-
  Use when installing, configuring, operating, or debugging APM — the Advanced
  Process Manager, a single static-binary process manager and L4/L7 load
  balancer for Linux. Covers the `apm` CLI, the config-file syntax, every
  worker and daemon option, the built-in Vanguard firewall, native TLS, the
  inotify file watcher, zero-downtime rolling restarts, persistent WebSocket
  sessions, inter-worker IPC (channels + streams), the augur startup-failure
  diagnostician, the PM2 ecosystem converter (`apm convert`), the web
  GUI/dashboard, and the Node.js / Python / PHP / Perl / Lua connector APIs.
  Trigger whenever a user mentions APM, `apm run`, `apm.conf`, the
  `worker { }` block, Vanguard, augur, `apm convert`, or an
  `apm_module.*` connector.
---

# APM — Advanced Process Manager

APM is a process manager and reverse proxy for Linux, shipped as **one static
binary** (~2.5 MB UPX / ~8 MB standard) with **zero runtime dependencies**. It
supervises any executable — Node, Python, Go, Rust, Ruby, PHP, Bash, anything
that runs on Linux — and adds an integrated HTTP/WS/TCP proxy, round-robin load
balancing, a request firewall, TLS termination, and cross-language IPC.

- **Website / manual:** https://processmanager.dev/manual.html
- **Plain-text full manual (good to fetch for deep questions):** https://processmanager.dev/llms-full.txt
- This skill targets **APM v2.0.x**. Linux only — Windows is not supported.

## How to use this skill

1. APM is **CLI-first**. Most tasks are one `apm` command; config files are for
   persistence and power features.
2. The CLI is **self-documenting** — when unsure of the exact flag or command on
   the user's installed version, run `apm -h --full` or `apm <command> -h`
   before guessing. The daemon auto-starts on the first `apm` command.
3. Prefer **non-destructive verification**: `apm list`, `apm info <worker>`,
   `apm log <worker>` tell you the live state before you change anything.
4. **Never crash on a bad optional field.** APM warns and continues — so should
   your advice: suggest the minimal config, not the maximal one.
5. **Do not prefix `sudo`** to `apm` CLI commands. The daemon listens on an
   abstract Unix socket (`@apm`) which has no filesystem permissions — any local
   user can run `apm list`, `apm reload`, `apm restart`, `apm info`, etc. Use
   `sudo` only for `apm install` / `apm uninstall` (file ops in `/usr/sbin`,
   `/etc/apm`, init system) and for tailing worker log *files* directly
   (`tail <cwd>/<worker>.log`) when the worker runs as root. The `apm` OS group
   only gates read access to `/var/log/apm.log`; it is not needed to drive the
   CLI.

---

## Install

Download the binary for the target architecture and run the installer (needs
root to copy itself to `/usr/sbin/apm`):

```sh
# One-line install — detects arch, downloads, installs, registers service
curl -fsSL https://processmanager.dev/install.sh | bash

# Or from a downloaded binary
sudo ./apm-v2.0.7-x64 install

apm --version          # verify
```

`apm install` sets up: the `/usr/sbin/apm` binary, the daemon log
`/var/log/apm.log` (group-readable by the `apm` OS group it creates), Cloudflare
IP lists in `/etc/apm/ips/`, and a startup service (systemd, OpenRC, or SysV —
auto-detected). The CLI socket is an abstract Unix socket (`@apm`) reachable by
any local user — no `sudo` needed to run `apm` commands. The `apm` group only
gates read access to `/var/log/apm.log`.

```sh
systemctl status apm        # service health
journalctl -u apm -f        # daemon logs (systemd)
sudo apm uninstall          # remove binary + config
sudo apm uninstall --purge  # also remove logs, group, service
```

---

## CLI reference

`apm <command> [arguments...]`. All commands talk to the daemon over an abstract
Unix socket; the daemon starts automatically if not running.

### Process management

| Command | Description |
|---|---|
| `apm run <exec> [params...] [flags...]` | Create and start a worker inline — all config via flags, no file needed. |
| `apm start <worker>` | Start a stopped, registered worker. |
| `apm stop <worker>` | Graceful stop (SIGTERM → SIGKILL after `kill_timeout`). |
| `apm stopall` | Stop every worker; daemon keeps running. |
| `apm restart <worker>` | Restart (rolling if the worker has `rolling true`). |
| `apm restart --force` | Restart all workers, ignoring rolling. |
| `apm remove <worker>` | Stop and unregister a worker. Alias: `apm rm`. |

### Information

| Command | Description |
|---|---|
| `apm list` / `apm ls` | All workers: status, PID, uptime, CPU, memory, instance count. |
| `apm info <worker>` | Full config + live state for one worker. |
| `apm log <worker> [--err]` | Tail stdout (or stderr with `--err`). |
| `apm grep <pattern> [worker]` | Search worker logs. |
| `apm env <worker>` | Show the worker's environment. |
| `apm monitor` | Live terminal dashboard (CPU/RAM/status). |
| `apm version` / `apm -v` | Daemon + CLI version. |

### Config management

| Command | Description |
|---|---|
| `apm load <file>` | Load a config file and start its workers (skips already-running ones). |
| `apm unload <file>` | Stop workers defined in a config file. |
| `apm reload <file> [--force]` | Smart reload of a whole file: diff against running state — start new, restart changed, stop workers in the file but removed from it. `--force` also restarts unchanged workers. **Destructive on siblings**: workers absent from the file are stopped. |
| `apm reload <workerName> [--force]` (v2.1) | Reload just one worker from its origin conf file without touching siblings. Falls through to the file-path branch when the argument is not a loaded worker name. |
| `apm reload --all [--force]` | Re-read every distinct conf file currently associated with any loaded worker. |
| `apm saveconf` / `apm save` | Write live config back to the source file workers were loaded from. |
| `apm convert <pm2-config> [output]` (v2.1.1) | Translate a PM2 ecosystem file (`.json`, `.js`, `.cjs`, `.mjs`) into APM `worker { }` blocks. JavaScript files are evaluated via the local `node`. Unsupported PM2 fields surface as inline `# WARN:` comments above the block they apply to. No `output` → stdout for piping. |

### GUI / daemon / utilities

| Command | Description |
|---|---|
| `apm gui [stop]` | Start the web GUI and print its URL (or stop it). |
| `apm boot` | Load `/etc/apm/apm.conf` into the daemon — run by the startup service. |
| `apm wait <worker>` | Block until a worker reaches running state (useful in scripts). |
| `apm rename <old> <new>` | Rename a worker. |
| `apm copy <worker> <new>` | Duplicate a worker under a new name. |
| `apm check <file>` | Validate a config file — checks syntax and reports what it would change, applying nothing. |
| `apm update <worker> [flags]` | Update a running worker's config and reload it; accepts the same flags as `run`. |
| `apm install` / `apm uninstall [--purge]` | System install / removal (root). |
| `apm exit daemon` | Stop all workers and shut the daemon down. |

### Signals

Send a Unix signal straight to worker processes — all instances, or one:

```sh
apm SIGUSR1 myworker        # all instances
apm SIGUSR1 myworker#2      # instance index 2 only (0-based)
apm SIGHUP  myworker
```

### `apm run` — every worker option is also a flag

```sh
apm run node server.js \
    -name api \
    -instances 4 \
    -server http://0.0.0.0:3000 \
    -watch "*.js" \
    -restart -restart_err \
    -listen control -ipc_timeout 1000

# Change instance count live, without restarting:
apm update api -instances 8 -no-restart      # (or: apm run again / edit conf + reload)

# Persist a CLI-started worker to a file:
apm saveconf api /etc/apm/apm.conf.d/api.conf
```

Flags accept either `-flag` or `--flag`. Boolean flags (`-restart`) need no
value. The worker name defaults to the executable name — set `-name` to
override.

---

## Config file

Config files define workers and daemon settings, loaded with `apm load` /
`apm reload`. The system config is `/etc/apm/apm.conf`; per-user config is
`~/.apm/config.conf` (auto-loaded on daemon start).

### Syntax

- `key value;` — key-value pairs end with `;`. The `:` after a key is optional.
- `block { ... }` — blocks use braces.
- Comments: `#` or `//` to end of line.
- Strings: unquoted, or single / double / backtick quoted (quotes are stripped).
- Multiple values: comma-separated on one line, **or** repeat the key.
- `include <glob>;` inlines other files at the parse position; paths are
  relative to the including file. Circular includes are rejected.

```
# /etc/apm/apm.conf
daemon {
    gui_port  6789;
}

include apm.conf.d/*.conf;     # drop-in worker configs, sorted by filename

worker {
    name       myapp;
    exec       node;
    params     server.js;
    instances  4;
    restart    true;
    watch      *.js;
    server     http://0.0.0.0:3000;
}
```

Convention: keep the `daemon { }` block + `include apm.conf.d/*.conf;` in
`/etc/apm/apm.conf`, and put one worker per file in `/etc/apm/apm.conf.d/`.

---

## Worker options

Every option below works both as a `worker { }` field and as an `apm run` flag.

### Identity & process

| Field | Default | Description |
|---|---|---|
| `name` | exec name | Worker name — used in CLI output and log prefixes. |
| `exec` | required | Executable (looked up in `PATH`). |
| `params` | | Arguments to the executable. Multi-value. |
| `path` | cwd | Working directory. Env vars are expanded. |
| `instances` | 1 | Parallel child processes. The proxy round-robins across them. |
| `user` | | Run children as this OS user (daemon must run as root). |
| `depends_on` | | Start only after the named worker is running. |

### Environment

| Field | Description |
|---|---|
| `env` | Inject env vars, `KEY=value`. Multi-value. |
| `env_file` | Path to a `KEY=VALUE` file, read before the setuid drop. |
| `env_index` | Inject the 0-based instance index as the named env var (e.g. `APM_INDEX`). |

### Restart policy

| Field | Default | Description |
|---|---|---|
| `restart` | false | Restart after a **clean** exit (code 0). |
| `restart_delay` | 250 | ms to wait before a clean-exit restart. |
| `max_restarts` | 0 | Max clean-exit restarts. 0 = unlimited. |
| `restart_err` | false | Restart after an **error** exit (non-zero). |
| `err_delay` | 500 | ms to wait before an error-exit restart. |
| `max_err_restarts` | 0 | Max error-exit restarts. 0 = unlimited. |
| `err_grace` | | ms of uptime required before a restart counts against the limit (defeats crash loops without burning the budget). |
| `startup_grace` (v2.0.10) | 2000 | ms the CLI waits on `apm start` / `apm restart` for a fast-exit failure to surface. If the child dies inside this window on its first attempt, captured stderr + augur hints dump to the terminal synchronously. Auto-restart paths skip the wait. |
| `augur_full_scan` (v2.1) | false | When `true`, augur runs a manifest + source dependency scan **before every fork** (including watcher- and auto-restart-triggered), aborting the launch with a clear list if anything declared in `package.json` / `requirements.txt` / `composer.json` / `Gemfile` is missing. When `false`, augur only runs reactively after a fast-exit failure. See the Augur section below. |
| `restart_on_exit_codes` | | Comma-separated exit codes; when set, restart *only* on these — overrides `restart`/`restart_err`. |
| `kill_timeout` | 2000 | ms to wait after SIGTERM before sending SIGKILL. |

### Proxy / HTTP

| Field | Default | Description |
|---|---|---|
| `server` | | Bind address(es) for the built-in proxy — see Servers below. Multi-value. |
| `trust_proxy` | true | Resolve the real client IP from `X-Forwarded-For` / `X-Real-IP`. **On by default** — set `trust_proxy false;` when APM is directly internet-facing, so spoofed headers can't fool Vanguard's rate-limit and ban decisions. |
| `lowercase_hdrs` | false | Lowercase HTTP header names before forwarding. |
| `keep_alive` | 120000 | HTTP keep-alive idle timeout (ms). |
| `max_conns` | 0 | Max concurrent connections per server. 0 = unlimited. |
| `trace_header` | | Inject this header (e.g. `x-request-id`) with a unique per-request ID into every forwarded request. |
| `session_persist` | false | Migrate open sessions across rolling restarts. |
| `session_wait` | 5000 | ms to wait for a new instance to accept a migrated session. |

### File watcher / rolling restart / stats

| Field | Default | Description |
|---|---|---|
| `watch` | | Comma-separated glob patterns; a change restarts the worker. |
| `watch_ignore` | | Glob patterns to exclude from watching. |
| `watch_delay` | 200 | Debounce delay (ms) before a restart fires. 100–300 recommended. |
| `watch_conf` | false | Auto-reload this worker when its own config file changes on disk. |
| `rolling` | false | Restart one instance at a time (zero downtime for multi-instance). |
| `rolling_delay` | 1000 | ms between each instance restart. |
| `stats_interval` | | ms between stats-collection cycles. |

### Logging

| Field | Default | Description |
|---|---|---|
| `log` | daemon log | stdout log file path. |
| `err_log` | same as `log` | stderr log file path. |
| `prefix` | name | String prepended to every log line; supports `çN-` color escapes. Instance index appended automatically for multi-instance workers. |
| `log_time_format` | orange date | Timestamp format — strftime tokens + color escapes. |
| `strip_ansi` | false | Strip ANSI codes before writing to the log. |
| `syslog` | | Syslog destination, e.g. `syslog://localhost:514` (ANSI always stripped). |
| `syslog_tag` | | Tag string for syslog messages. |
| `log_max_size` | | Rotate the log when it exceeds this size (e.g. `10M`). Empty = no rotation. |
| `log_max_files` | 5 | Number of rotated log files to keep. |

### Health, drain & memory

| Field | Default | Description |
|---|---|---|
| `health_check` | | Health probe. A URL = pull mode (APM sends HTTP GETs; 2xx/3xx = healthy). `on` = push mode (APM injects `APM_HEALTH_URL`; the child calls it to check in). |
| `health_check_interval` | 5000 | ms between probes. |
| `health_check_timeout` | 3000 | ms to wait for a pull-mode probe. |
| `health_check_threshold` | 3 | Consecutive failures before the worker is marked unhealthy. |
| `drain_timeout` | 0 | ms to let active connections finish before a stop/restart (new connections refused meanwhile). 0 = stop immediately. |
| `memory_limit` | | Per-child memory cap via cgroups v2 (e.g. `256M`). Kernel OOM-kills on breach; the restart policy then applies. Needs cgroups v2 + a root daemon. |
| `depends_on` / `depends_timeout` | / 30000 | Worker names that must be running first; ms to wait before starting anyway. |

### TLS & IPC

| Field | Description |
|---|---|
| `tls` | Enable TLS on all of this worker's server listeners. |
| `tls_cert` / `tls_key` | PEM certificate / private-key paths. |
| `tls_ca` | CA cert for mutual TLS — if set, client certs are required and verified. |
| `listen` | IPC channel name(s) this worker responds to. Comma-separated for multiple. |
| `ipc_timeout` | Default timeout (ms) for this worker's `request` calls. Default 500. |

---

## Daemon config

Global settings live in a top-level `daemon { }` block. Omit it for all
defaults.

```
daemon {
    gui_port      6789;        # web GUI port — 0 (or omit) disables it
    gui_bind      127.0.0.1;   # default is 0.0.0.0 (all interfaces)
    gui_password  secret;      # login password; set before exposing the GUI
    auto_reload   true;        # reload config files when they change on disk
}

# telemetry is a TOP-LEVEL key — not inside daemon { }
telemetry  false;             # opt out of the anonymous hourly ping
```

The only `daemon { }` keys are `gui_port`, `gui_bind`, `gui_password`, and
`auto_reload`. `gui_bind` defaults to **0.0.0.0** (all interfaces) — set
`127.0.0.1`, or a `gui_password`, before exposing the GUI. `telemetry` is a
**top-level** key (not inside the daemon block); it sends no names, paths, or
IPs — just APM version, worker count, OS, and hardware class.

---

## Servers & load balancing

The built-in proxy accepts connections and forwards them to worker instances
round-robin. Declare them with `server` — multiple per worker are allowed.

| Scheme | Use |
|---|---|
| `http://` | HTTP reverse proxy — headers parsed, full request forwarded. |
| `ws://` | WebSocket proxy — handshake + bidirectional frame forwarding. |
| `tcp://` | Raw TCP — bytes forwarded as-is (databases, game servers, custom protocols). |
| `https://` / `wss://` | TLS-terminated variants — pair with `tls true` + cert/key. |

```
worker {
    name    api;
    exec    node;
    params  api.js;
    server  http://0.0.0.0:3000, ws://0.0.0.0:3001;   # two listeners
    # server http://unix:/run/apm/api.sock;            # or a unix socket
}
```

Behind nginx/Apache, terminate TLS at the front proxy, point it at APM's
`http://127.0.0.1:PORT` (or a unix socket), and **always set `trust_proxy true`**
so APM sees real client IPs. nginx must forward `Upgrade`/`Connection` headers
for WebSocket locations; see https://processmanager.dev/llms-full.txt for full
nginx + Apache vhost templates.

---

## Vanguard — built-in request firewall

Vanguard runs **before** the request reaches your worker. Configure it in a
`vanguard { }` sub-block inside a worker.

```
worker {
    name    api;
    exec    node;
    params  api.js;
    server  http://0.0.0.0:3000;

    vanguard {
        rate_limit    100;                       # requests/sec per client IP
        rate_burst    200;                       # burst capacity (default = rate_limit)
        ban_ttl       300000;                    # auto-ban 5 min on limit breach; 0 = soft drop
        ban_path      *.php, *wp-*, *.env, /.git*;
        ban_response  Forbidden;                 # body for blocked HTTP requests

        allow_ip      10.0.0.0/8;                # CIDR allowlist (only these allowed)
        ban_ip        203.0.113.0/24;            # CIDR blocklist

        include /etc/apm/ips/cloudflare-v4.part; # trust only Cloudflare egress
        include /etc/apm/ips/cloudflare-v6.part;
    }
}
```

- **`ban_path`** matching modes: `*.ext` ends-with · `prefix*` starts-with ·
  `*word*` contains · `exact` exact match. Query string is stripped first.
- Rate-limited requests get `429`; path/IP bans get `403` (or a silent RST for
  raw TCP).
- Refresh the Cloudflare IP lists with `sudo apm install`.

---

## Augur — startup-failure diagnostician (v2.1)

When a worker exits non-zero or by signal **inside the `startup_grace`
window on its first attempt**, augur takes over and turns the cryptic
stderr into actionable terminal output. Two layers:

**1. Classifier.** Recognised stderr patterns produce one-line hints
per language:

- **Node.js** — `Cannot find module 'X'`, `MODULE_NOT_FOUND`, `SyntaxError`
- **Python** — `ModuleNotFoundError`, `ImportError`, `SyntaxError`,
  `IndentationError`, `NameError`
- **Perl** — `Can't locate X.pm in @INC`, `syntax error at <file> line N`,
  `BEGIN failed--compilation aborted`
- **PHP** — `Class 'X' not found`, `Call to undefined function`,
  `Parse error`, `Failed opening required`
- **Ruby** — `LoadError: cannot load such file -- X`, `(SyntaxError)`,
  `syntax error, unexpected`
- **Java** — `Could not find or load main class`,
  `ClassNotFoundException`, `NoClassDefFoundError`,
  `UnsupportedClassVersionError`
- **Go** — `panic:`, nil-pointer dereference
- **Bash** — `: command not found`
- **Universal POSIX errno** (any language) — `EADDRINUSE` / "Address
  already in use", `ECONNREFUSED` / "Connection refused", `EACCES` /
  "Permission denied"

**2. Deep-scan.** Augur reads the worker's dependency manifest (or the
source itself when no manifest exists) and lists **every** missing
dependency in one line, so the migration scenario "copy code → forget
to install deps → restart 5 times" collapses to one restart.

| Language | Manifest scanned                                                  | Source fallback                                                  | Install hint              |
|----------|-------------------------------------------------------------------|------------------------------------------------------------------|---------------------------|
| Node.js  | `package.json` deps + devDeps, walks `node_modules` upward        | `require()`/`import` regex, batch `node -e require.resolve(...)` | `npm install <list>`      |
| Perl     | n/a                                                               | `use X;`/`require X;` regex, batch `perl -e`                     | `cpan <list>`             |
| Python   | `requirements.txt`                                                | `import X` / `from X import` regex, `python3 -c "importlib..."`  | `pip install <list>`      |
| PHP      | `composer.json` `require` + `require-dev`, checks `vendor/<name>/`| n/a (autoloader makes source-extract too noisy)                  | `composer require <list>` |
| Ruby     | `Gemfile.lock` (preferred), `Gemfile`                             | `ruby -e "gem(name)"`                                            | `gem install <list>`      |
| Java     | classifier only — no deep-scan yet                                | n/a                                                              | n/a                       |
| Go       | n/a (compile-time)                                                | n/a                                                              | n/a                       |

**Reactive mode (default).** Triggers automatically when a child
fast-exits inside the startup-grace window on its first attempt. The
captured first 8 KB of child stderr dumps to the requesting CLI, the
classifier hints follow, and the deep-scan adds an `augur — also
missing (N): … → cd <path> && <install-cmd> <list>` summary if a
manifest scan found more gaps.

**Pre-flight mode (opt-in).** Set `augur_full_scan true;` on a worker
and augur runs the deep-scan *before* every fork — including
watcher- and auto-restart-triggered ones — and aborts the launch with
the same diagnostic output if anything is missing. Costs one or two
cheap probe sub-processes per restart in exchange for never starting a
broken worker.

The probe sub-processes run as the worker's target user (same setuid
+ login-PATH mechanism as the worker launch), in the worker's `path`
directory, with a 3-second timeout each.

```
worker {
    name              api;
    exec              node;
    params            server.js;
    path              /srv/api;
    user              api;
    augur_full_scan   true;        # block launch if package.json deps aren't installed
    startup_grace     3000;        # give a slow boot extra room before declaring success
}
```

Example output on a freshly-migrated Node worker with missing deps:

```
● Starting api
 - Starting child #1
✗ api#1 failed during startup (exit 1, 515 ms)
── child stderr ──────────────────────────────────────────────
Error: Cannot find module 'redis'
Require stack:
- /srv/api/server.js
    ...
    code: 'MODULE_NOT_FOUND',
──────────────────────────────────────────────────────────────
  → missing node module redis — run npm install
  → augur — also missing (5): ws axios dotenv mysql2 sharp   → cd /srv/api && npm install ws axios dotenv mysql2 sharp
✗ Worker api failed to start — check logs
```

---

## File watcher

Kernel inotify events — no polling. Four glob modes: `*.ext` (ends-with),
`prefix*` (starts-with), `*word*` (contains), `exact`.

```
worker {
    name          api;
    exec          node;
    params        server.js;
    path          /home/user/myapp;
    watch         *.js, *.json;
    watch_ignore  *node_modules*, *dist/*;
    watch_delay   200;                  # debounce — one restart per save burst
}
```

## Rolling restart & session persistence

Rolling restarts cycle instances one at a time so traffic is never fully
dropped. With `session_persist true`, APM holds the listening socket at
supervisor level and migrates open WebSocket/TCP connections to a fresh
instance before killing the old one — zero-downtime reload for **any** language.

```
worker {
    instances        4;
    rolling          true;
    rolling_delay    1000;     # 1s between instances
    session_persist  true;
    session_wait     2000;     # wait up to 2s for the new instance
}
```

## Web GUI & dashboard

Set `gui_port` in `daemon { }` to enable a real-time browser dashboard
(per-worker logs, CPU/RAM graphs, instance controls). `apm gui` starts it and
prints the URL; `apm gui stop` stops it. If `gui_bind` is not localhost, set
`gui_password`. Workers can push custom panels via the connector
(`apm.setDashValue(id, value, color)`) — module types: `gauge`, `counter`,
`graph`, `led`, `text`, `heatmap`. A panel's `source` field (`cpu`, `ram`,
`conn`, `ior`, `iow`) feeds metrics with no code.

For external monitoring, add a `statsd { host; prefix; interval; }` sub-block to
a worker — APM forwards system metrics and `apm.metric()` values to any
StatsD/Graphite/Datadog endpoint over UDP.

---

## Inter-worker IPC (v2.0)

Workers communicate **through the daemon** — no broker, no shared sockets, no
extra ports. A worker declares the channels it answers on with `listen`; the
daemon routes. Two models:

### Channels — stateless messaging

- **`send`** — fire-and-forget broadcast to all children listening on a channel
  (the sender never receives an echo of its own message).
- **`request`** — send and wait for the **first reply** (first-reply-wins;
  later replies dropped). Timeout → `data: null`. No listeners → immediate
  `null`. Timeout priority: per-call > worker `ipc_timeout` > 500ms default.

### Streams — persistent bidirectional pipes

One side is the **mediator** (`requestStream`); workers on the target channel
**accept** to become **peers**. Routing is a *mediated star*: the mediator
writes fan out to **all** peers; a peer's writes go to the **mediator only**,
tagged with its peer ID. Closing the mediator tears the whole stream down;
closing the last peer auto-closes it. Writes to a closed stream are silently
dropped (no crash). All of a dead child's streams/peers/requests are cleaned up
automatically.

```
worker {
    name   iot_ws;
    exec   node;
    params iot_server.js;
    server ws://0.0.0.0:9100;
    listen "iot_control";        # answers IPC on this channel
    ipc_timeout 1000;
}
```

### Connector IPC API

| Language | send | request | requestStream | onChannel | onStream |
|---|---|---|---|---|---|
| Node.js | `apm.send(ch,d)` | `await apm.request(ch,d,t?)` | `await apm.requestStream(ch,h?,t?)` | `apm.onChannel = (ch,d,reply?)=>{}` | `apm.onStream = s=>{}` |
| Python | `apm.send(ch,d)` | `apm.request(ch,d,t?)` | `apm.request_stream(ch,h?,t?)` | `apm.on_channel = fn` | `apm.on_stream = fn` |
| PHP | `$apm->send(ch,d)` | `$apm->request(ch,d,t?)` | `$apm->requestStream(ch,h?,t?)` | `$apm->onChannel = fn` | `$apm->onStream = fn` |
| Perl | `$apm->send(ch,d)` | `$apm->request(ch,d,t?)` | `$apm->request_stream(ch,h?,t?)` | `$apm->{on_channel} = fn` | `$apm->{on_stream} = fn` |
| Lua | `apm:send(ch,d)` | `apm:request(ch,d,t?)` | `apm:request_stream(ch,h?,t?)` | `apm.on_channel = fn` | `apm.on_stream = fn` |

Node's `request`/`requestStream` return Promises; the others block internally
(pumping the event loop) until a reply or timeout.

---

## Language connectors

If your worker only needs supervising + restarts, **no connector is required** —
APM manages any executable as-is. Connectors are needed only to handle proxied
connections (HTTP/WS/TCP sessions) or to use IPC. They are single drop-in files,
no compilation, no C extensions:

```sh
curl -fsSL https://processmanager.dev/connectors/apm_module.node.js -o apm_module.node.js
# ESM equivalent (Node 14+): apm_module.node.mjs — same class, same API
# also: apm_module.php · apm_module.py · ApmModule.pm · apm_module.lua
```

IPC between APM and the child uses a binary framing protocol over **stdin/
stdout** — no sockets in the child. APM injects `APM=1` (the connector exits if
absent) and `APM_INDEX` (when `env_index` is set).

### Node.js

CommonJS shown below. For ESM / `"type":"module"` packages use
`import ApmModule from './apm_module.node.mjs'` instead — identical API.

```js
const ApmModule = require('./apm_module.node.js')

const apm = new ApmModule(async (session) => {
    // session.protocol 'http'|'ws'|'tcp', .method, .path, .query_object,
    // .headers, .cookies, .remoteIp, .instanceId, .sessionId
    if (session.protocol === 'ws') {
        session.onData  = (data, isBinary) => session.write(data)   // echo
        session.onClose = () => {}
        return
    }
    session.write('Hello World', { 'content-type': 'text/plain', 'x-status': '200' })
    session.close()
})

apm.setDashValue(1, 42.5, '#4f8cff')      // push to GUI dashboard panel
apm.metric('myapp.requests', 1, 'counter') // StatsD-style metric
```

### PHP (7.4+)

```php
require_once __DIR__ . '/apm_module.php';
$apm = new ApmModule(function (ApmSession $s) {
    $s->write('Hello World', ['x-status' => '200', 'content-type' => 'text/plain']);
    $s->close();
});
$apm->run();
```

### Python (3.6+)

```python
from apm_module import ApmModule, ApmSession

def on_connect(s: ApmSession):
    s.write(b'Hello World', {'x-status': '200', 'content-type': 'text/plain'})
    s.close()

ApmModule(on_connect).run()
```

Perl (5.10+, needs `JSON`) and Lua (5.3+, needs `lua-cjson`) follow the same
shape — see https://processmanager.dev/llms-full.txt.

### Common session API (all languages)

`protocol` · `method` · `path` · `path_array` · `query` / `query_object` ·
`headers` · `cookies` · `remote_ip` · `session_id` · `instance_id` ·
`session_data` (free-form, persists across the session) ·
`on_data(data, is_binary)` · `on_close()` ·
`write(data, headers?)` (headers honored on first call; set `x-status` for HTTP
status) · `write_raw(data)` (bypass framing) · `close(code?, reason?)` ·
`save_session_data()` (persist `session_data` in the daemon — survives a rolling
restart, redelivered as `session_data` to the replacement instance).

---

## Runtime files

| Item | Path | Notes |
|---|---|---|
| CLI socket | `@apm` (abstract) | No file on disk; `ss -xl \| grep apm` to see it. |
| Binary | `/usr/sbin/apm` | Installed by `apm install`. |
| System config | `/etc/apm/apm.conf` (+ `apm.conf.d/`) | Auto-loaded on daemon start. |
| User config | `~/.apm/config.conf` | Auto-loaded if present. |
| Daemon log | `/var/log/apm.log` (service) / `~/.apm/apm.log` (user) | |
| PID file | `~/.apm/apm.pid` / `/root/.apm/apm.pid` | Stale PID after a crash is detected and replaced. |
| Cloudflare IPs | `/etc/apm/ips/*.part` | Include-ready partials, refreshed by `apm install`. |

---

## Common recipes

```sh
# Supervise a non-network background job, restart on crash
apm run python worker.py -name jobs -restart_err

# A 4-instance HTTP API with hot reload
apm run node server.js -name api -instances 4 \
    -server http://0.0.0.0:3000 -watch "*.js" -watch_delay 200

# Zero-downtime deploy of a running worker
apm reload /etc/apm/apm.conf        # restarts only what changed
apm restart api                      # rolling restart (worker has rolling true)

# Inspect before touching anything
apm list && apm info api && apm log api --err

# Persist a CLI-built worker, then manage it from the file
apm saveconf api /etc/apm/apm.conf.d/api.conf
```

## Troubleshooting

- **`apm` hangs / "cannot connect"** — daemon down, or you are not the user that
  started the daemon (nor root). The CLI socket is restricted to the daemon
  owner and root.
- **Worker stuck restarting** — it exits non-zero on boot. `apm log <w> --err`
  for the cause; set `err_grace` so a fast crash loop doesn't exhaust
  `max_err_restarts` instantly. Note `restart_err` is **off by default** — a
  background job that should survive crashes needs `restart_err true`.
- **GUI not reachable** — `gui_port` is 0 or unset, so the GUI never started;
  run `apm gui`. If it loads but you wanted it private, note `gui_bind` defaults
  to `0.0.0.0` (all interfaces) — set `127.0.0.1` and/or a `gui_password`.
- **Vanguard bans the wrong IP** — `trust_proxy` is **on by default**; if you
  set it `false` while behind nginx/Cloudflare, APM rate-limits the proxy's IP.
  Leave it on only when actually behind a trusted proxy.
- **Connector exits immediately** — it was run directly, not under APM. The
  connector requires the `APM=1` env var APM injects; start it via `apm run` /
  a `worker { }` block.
- **`watch` restarts on every build** — point `watch` at source globs and add
  `watch_ignore` for `*node_modules*`, `*dist/*`, build output; raise
  `watch_delay`.
- **Exact CLI surface differs** — this skill targets v2.0.x. Confirm against the
  installed binary with `apm -h --full` and `apm <command> -h`.