ExpressNode.jsTypeScript

Express Integration

Integrate ImmutableLog into any Express application with a middleware that automatically captures all HTTP requests. Uses Node.js 18+ native `fetch` API for non-blocking sending.

Installation

No extra dependencies beyond Express. The integration uses Node.js 18+ native `fetch`. For older versions, install `node-fetch`.

bash

npm install express
# Node.js < 18 only:
npm install node-fetch

Middleware Code

Save the code below in `src/middleware/immutablelog.ts`. The middleware uses `res.on('finish')` to capture the final response status after the handler completes, and `res.on('error')` to capture stream errors.

typescript

// src/middleware/immutablelog.ts
import crypto from 'crypto';
import { Request, Response, NextFunction } from 'express';

// Margem de seguranca: o limite real do servidor e 16KB (retorna 413 acima disso).
// Safety margin: the server hard limit is 16KB (returns 413 above that).
const MAX_PAYLOAD_BYTES = 12_000;

// Chaves cujos valores sao mascarados antes do envio (PII/segredos)
// Keys whose values are masked before sending (PII/secrets)
const SENSITIVE_KEYS = new Set([
  'password', 'pwd', 'passwd', 'pass', 'senha',
  'token', 'access_token', 'refresh_token', 'id_token', 'api_key', 'apikey',
  'secret', 'client_secret', 'authorization', 'auth',
  'credit_card', 'card_number', 'card', 'cvv', 'cvc',
  'pin', 'ssn', 'cpf', 'rg', 'otp', 'code',
]);
const REDACTED = '***REDACTED***';

function redact(value: unknown): unknown {
  if (Array.isArray(value)) return value.map(redact);
  if (value && typeof value === 'object') {
    return Object.fromEntries(
      Object.entries(value as Record<string, unknown>).map(([k, v]) =>
        SENSITIVE_KEYS.has(k.toLowerCase()) ? [k, REDACTED] : [k, redact(v)],
      ),
    );
  }
  return value;
}

// Normaliza o immutable_trail conforme as regras do servidor:
// trim, nao-vazio, max 256 chars, sem ':' (o servidor responde 400 se violado).
// Normalizes immutable_trail per server rules: trim, non-empty, max 256, no ':'.
function sanitizeTrail(value?: string | null): string | undefined {
  if (typeof value !== 'string') return undefined;
  let v = value.trim();
  if (!v) return undefined;
  if (v.includes(':')) v = v.replace(/:/g, '-');
  if (v.length > 256) v = v.slice(0, 256);
  return v;
}

function clampPayload(payload: object): string {
  let s = JSON.stringify(payload);
  if (Buffer.byteLength(s, 'utf8') <= MAX_PAYLOAD_BYTES) return s;
  const p = { ...(payload as Record<string, unknown>) };
  delete p.error;
  delete p.requestBody;
  s = JSON.stringify(p);
  if (Buffer.byteLength(s, 'utf8') <= MAX_PAYLOAD_BYTES) return s;
  return JSON.stringify({
    id: (payload as Record<string, unknown>).id,
    kind: (payload as Record<string, unknown>).kind,
    message: String((payload as Record<string, unknown>).message ?? 'event').slice(0, 500),
    timestamp: (payload as Record<string, unknown>).timestamp,
  });
}

function hashBody(body: string | Buffer): string {
  const buf = Buffer.isBuffer(body) ? body : Buffer.from(body);
  return crypto.createHash('sha256').update(buf).digest('hex');
}

function severityFrom(status: number | null, err: Error | null): string {
  if (err) return 'error';
  if (!status) return 'info';
  if (status >= 400) return 'error';
  if (status >= 300) return 'info';
  if (status >= 200) return 'success';
  return 'info';
}

// Usuario autenticado, se o seu app popular req.user (ex: passport, jwt middleware)
// Authenticated user, if your app populates req.user (e.g. passport, jwt middleware)
interface AuthUser {
  id?: string | number;
  email?: string;
  username?: string;
}

export interface ImmutableLogOptions {
  apiKey: string;
  apiUrl?: string;
  serviceName?: string;
  env?: string;
  skipPaths?: string[];
  // Timezone do cliente — usado pelo dashboard para exibir os timestamps
  // Client timezone — used by the dashboard to render timestamps
  clientTz?: string;
  clientOffsetMinutes?: number;
}

export function immutableLogMiddleware(opts: ImmutableLogOptions) {
  const {
    apiKey,
    apiUrl = 'https://api.immutablelog.com',
    serviceName = 'express-service',
    env = 'production',
    skipPaths = ['/health', '/healthz'],
    clientTz = 'America/Sao_Paulo',
    clientOffsetMinutes = -180,
  } = opts;
  const skip = new Set(skipPaths);

  return (req: Request, res: Response, next: NextFunction): void => {
    if (skip.has(req.path)) return next();

    const startedAt = Date.now();
    const requestId =
      (req.headers['x-request-id'] as string) || crypto.randomUUID();
    (req as Request & { requestId: string }).requestId = requestId;

    const rawBody: string =
      (req as Request & { rawBody?: string }).rawBody ?? '';

    const emit = (err?: Error): void => {
      const latencyMs = Date.now() - startedAt;
      const statusCode = err ? 500 : res.statusCode;
      const kind = severityFrom(statusCode, err ?? null);
      const eventName =
        (req as Request & { imtblEventName?: string }).imtblEventName ??
        `http.${req.method}.${(req.route?.path as string | undefined) ?? req.path}`;

      const user = (req as Request & { user?: AuthUser }).user;

      const payload: Record<string, unknown> = {
        id: crypto.randomUUID(),
        kind,
        message: err
          ? `${req.method} ${req.path} failed: ${err.constructor.name}`
          : kind === 'success'
          ? `${req.method} ${req.path} completed successfully`
          : `${req.method} ${req.path} processed`,
        timestamp: new Date().toISOString(),
        context: {
          ip: req.ip ?? req.socket?.remoteAddress ?? 'unknown',
          userAgent: req.headers['user-agent'] ?? 'unknown',
          // Enriquece com o usuario autenticado quando disponivel
          // Enrich with authenticated user when available
          ...(user && {
            userId: user.id,
            email: user.email,
            username: user.username,
          }),
        },
        request: {
          requestId,
          method: req.method,
          path: req.path,
          // query params passam por redact() — nunca enviar segredos em claro
          // query params go through redact() — never send secrets in the clear
          queryParams: Object.keys(req.query).length ? redact(req.query) : null,
        },
        metrics: { latencyMs, statusCode },
        severity: kind === 'error' ? 'high' : 'low',
      };

      if (kind === 'error') {
        payload.error = {
          statusCode,
          retryable: [408, 429, 500, 502, 503, 504].includes(statusCode),
          ...(err && {
            exception: err.constructor.name,
            exceptionMessage: err.message,
          }),
          ...(rawBody && { requestBodyHash: hashBody(rawBody) }),
        };
      } else if (kind === 'success') {
        payload.success = {
          statusCode,
          result: statusCode === 200 ? 'ok' : 'processed',
        };
      } else {
        payload.info = { statusCode, action: req.method.toLowerCase() };
      }

      // Trilha imutavel (immutable_trail). Prioridade / priority:
      //   1. req.imtblTrail (definido por uma rota, ex: `flow-${flow.id}`)
      //   2. header X-Imtbl-Trail propagado pelo cliente
      //   3. fallback por usuario autenticado: user-<username>
      // Omitido se nenhum se aplicar — nao invente um trail generico.
      const trail =
        sanitizeTrail((req as Request & { imtblTrail?: string }).imtblTrail) ??
        sanitizeTrail(req.headers['x-imtbl-trail'] as string | undefined) ??
        sanitizeTrail(user?.username ? `user-${user.username}` : undefined);

      const event = {
        payload: clampPayload(payload),
        // IMPORTANTE: todos os valores de meta precisam ser strings.
        // IMPORTANT: every meta value must be a string.
        meta: {
          type: kind,
          event_name: eventName,
          service: serviceName,
          request_id: requestId,
          env,
          ...(trail && { immutable_trail: trail }),
        },
      };

      // fire-and-forget — nunca bloqueia a resposta / never blocks the response
      fetch(`${apiUrl}/v1/events`, {
        method: 'POST',
        headers: {
          Authorization: `Bearer ${apiKey}`,
          'Content-Type': 'application/json',
          // Idempotency-Key e OBRIGATORIO (sem ele a API responde 400)
          // Idempotency-Key is REQUIRED (without it the API returns 400)
          'Idempotency-Key': `${eventName}-${requestId}`,
          'Request-Id': requestId,
          // Timezone do cliente — alimenta meta.client_tz / client_offset_minutes
          'X-Client-TZ': clientTz,
          'X-Client-Offset-Minutes': String(clientOffsetMinutes),
          ...(trail && { 'X-Imtbl-Trail': trail }),
        },
        body: JSON.stringify(event),
        signal: AbortSignal.timeout(5000),
      }).catch((e: Error) =>
        console.warn('[ImmutableLog] emit failed:', e.message),
      );

      res.setHeader('x-request-id', requestId);
    };

    res.on('finish', () => emit());
    res.on('error', (err: Error) => emit(err));

    next();
  };
}

Registration and Configuration

Register the middleware globally with `app.use()` before your routes. Configure via environment variables to keep credentials out of the code.

typescript

// src/app.ts
import express from 'express';
import { immutableLogMiddleware } from './middleware/immutablelog';

const app = express();
app.use(express.json());

// Registrar antes das rotas / Register before routes
app.use(
  immutableLogMiddleware({
    apiKey: process.env.IMTBL_API_KEY!,
    apiUrl: process.env.IMTBL_URL ?? 'https://api.immutablelog.com',
    serviceName: process.env.IMTBL_SERVICE_NAME ?? 'express-service',
    env: process.env.IMTBL_ENV ?? 'production',
    skipPaths: ['/health', '/healthz', '/metrics'],
  }),
);

app.get('/users', (req, res) => res.json({ users: [] }));
app.listen(3000);

Never pass the token directly in code. Use environment variables (.env) and ensure the `.env` file is in `.gitignore`.

How it works

The middleware records the start timestamp on entry, listens to `res.on('finish')` to capture the final status and calculate latency. Sending to ImmutableLog is done via fire-and-forget `fetch` — it never blocks the response to the client.

entry

Timestamp recorded, request_id generated or inherited from header

res.on('finish')

Latency calculated, event emitted via fire-and-forget fetch

res.on('error')

Stream errors captured and emitted as error event

The x-request-id is injected into the response for end-to-end traceability across systems.

Custom event name

Assign `req.imtblEventName` in any handler or preceding middleware to override the automatically generated event name.

typescript

app.post('/checkout', (req, res) => {
  // Sobrescreve o nome do evento / Override the event name
  (req as any).imtblEventName = 'payment.checkout.initiated';

  // ... logica de negocio / business logic ...
  res.json({ status: 'ok' });
});

Auto default: http.METHOD.route_path (e.g., http.POST./checkout).

Path exclusion

Pass `skipPaths` in the options to ignore specific routes such as health checks and metrics.

typescript

app.use(
  immutableLogMiddleware({
    apiKey: process.env.IMTBL_API_KEY!,
    apiUrl: 'https://api.immutablelog.com',
    skipPaths: ['/health', '/healthz', '/metrics', '/readyz'],
  }),
);

Error enrichment

On error events (4xx/5xx), the middleware includes the SHA-256 hash of the request body, `retryable` flag, and when available, exception details.

typescript

// Error handler do Express — capturado automaticamente pelo middleware
// Express error handler — automatically captured by the middleware
app.use((err: Error, req: Request, res: Response, next: NextFunction) => {
  res.status(500).json({ error: err.message });
});

error.requestBodyHash

SHA-256 of body (audit without exposing data)

error.retryable

true for: 408, 429, 500, 502, 503, 504

error.exception

Error class name (if available)

error.exceptionMessage

Error message (if available)

Immutable trail (immutable_trail)

A trail groups related events into a single auditable timeline (e.g., `flow-123`, `user-maria`). The middleware resolves the trail by priority: `req.imtblTrail` set on the route → `X-Imtbl-Trail` header propagated by the client → `user-<username>` fallback from the authenticated user. The value goes through `sanitizeTrail` before sending.

typescript

// 1. Definir a trilha em uma rota especifica / Set the trail on a specific route
app.post('/flows/:id/run', (req, res) => {
  (req as any).imtblTrail = `flow-${req.params.id}`;
  res.json({ status: 'ok' });
});

// 2. Ou propagar entre servicos via header / Or propagate across services via header
//    X-Imtbl-Trail: flow-123
// 3. Sem nada disso, cai no fallback user-<username> (se houver req.user)

Server rules: the trail cannot be empty, exceed 256 characters, or contain `:`. Violations return `400 invalid_immutable_trail`. That is why `sanitizeTrail` fixes the value (replaces `:` with `-` and truncates at 256) before sending.

Response and status codes

The ingestion API is asynchronous. Success returns `202 Accepted` (event queued) or `200 OK` when the Idempotency-Key already exists (`duplicate: true`). The body contains `{ ok, tx_id, payload_hash, status, duplicate, request_id }`. Since the middleware is fire-and-forget, these codes matter mostly if you send events manually.

Status	Meaning
202	Event accepted and queued (new)
200	Idempotency-Key already exists — duplicate: true
400	Missing/empty Idempotency-Key or invalid_immutable_trail
403	Inactive/expired subscription, scope or retention
413	payload_too_large (server limit: 16KB)
429	monthly_limit_exceeded — do not retry
503	mempool_full — transient, retryable

meta fields

Beyond the payload, the meta object carries the fields the SIEM uses to normalize (ECS) and enrich the event. All meta values are strings.

Field in meta	What it does in the SIEM
`event_name`	The event label (e.g. `http.GET.auth-me-user`, `Payment error`) — the identifier shown in the listing. Becomes `event.action` and derives `event.category` (e.g. a name with `login` → `authentication`).
`client_ip`	Becomes `source.ip` with maximum precedence and provenance `client_asserted`. See the section below.
`immutable_trail`	Becomes `imtbl.immutable_trail` — groups related events for investigation (e.g. one order, one user).
`type`	Event severity: `error` / `warning` / `info` / `success`. Colors the badge in the listing. (`event_type` is accepted as a synonym.)
`service, env, request_id`	Metadata: indexed and queryable in the dashboard (search/filters).

client_ip — end-user IP

The problem it solves

Between the browser and the core there are proxies/ALB. The IP the core sees on the connection is the previous hop (the client's backend or an AWS proxy), not the user's. Without action, source.ip would be the proxy IP — useless for geo, threat and IP-based detection. (Real example: XFF arrived as 44.192.13.3 (AWS) while the user was 179.110.4.205.)

The solution

The client's backend is the only one that sees the browser's real IP — so it forwards that IP in meta.client_ip when calling POST /v1/events.

How the SIEM handles it

Validates it is an IP (v4/v6). A non-IP value is ignored — it does not drop the event.
Writes to source.ip with maximum precedence (see order below).
Stamps imtbl.source_ip_origin = "client_asserted" — the UI shows the “User IP” badge.
Feeds GeoIP, threat intel and IP-based detection rules.

source.ip precedence:client_ipX-Forwarded-For (1º hop)X-Real-IPconnection IP

Provenance and trust

client_ip is asserted by the tenant — the client's backend claims the IP, and it is spoofable by whoever controls that backend. However it is self-contained to the tenant (it never crosses another tenant's boundary). A threat/geo hit on a client_asserted value is a claim, not an edge observation.

How to obtain the IP in your backend

Take the first IP from X-Forwarded-For (closest to the user); as a fallback, use the connection IP.

typescript

function getClientIp(req: Request): string | undefined {
  const xff = req.headers["x-forwarded-for"];
  if (typeof xff === "string" && xff.length) return xff.split(",")[0].trim(); // primeiro hop
  return req.socket.remoteAddress ?? undefined;
}

const meta = { event_name: "user.login", client_ip: getClientIp(req) };

Payload example

json

{
  "payload": "{\"user\":\"bob\",\"action\":\"login\"}",
  "meta": {
    "event_name": "user.login",
    "client_ip": "179.110.4.205",
    "immutable_trail": "user-bob"
  }
}

PII note: client_ip is PII and gets sealed into the immutable block (permanent). Decide consciously between forensic evidence and “right to be forgotten” before sending it.

This documentation reflects the current integration behavior. For questions or advanced integrations, contact the support team.