How-to: Measure Duration and Collect Metrics with Middleware

1) Decide which metrics you want (start minimal)

A good minimal set:

1. Counters

• operations_total{stage, message}

• operations_cancelled_total{stage, message}

2. Latency

• operation_duration_ms{stage, message} (histogram/summary)

3. Optional

• operations_success_total{stage, message}

• operations_error_total{stage, message} (only if your middleware can observe errors in your integration)

Keep labels/tags controlled:

• stage

• message name/type (stable string) Avoid high-cardinality labels:

• message id (do not use as a metric label)

• user id, tenant id, request ids

2) Pick a stable “message label”

Choose a stable string to identify message categories in metrics, for example:

• message class name

• message “type” identifier used in your project

• a curated mapping (“Command:CreateUser”, “Query:GetUser”)

Goal:

• stable dashboards over time

• predictable cardinality

3) Scope metrics collection with canHandle

Decide where metrics is useful:

• all stages, or only Execute/Callback

• only domain messages (exclude infrastructure/internal messages if applicable)

Then implement filtering in canHandle:

• stage-first

• then message subset

4) Record “attempts” in before

In before:

• increment operations_total

• capture start time for latency measurement

Important:

• If the operation is cancelled by a later middleware, you still want to count the attempt. Logging in before ensures that.

5) Record “success” and latency in after

In after:

• compute duration (now - start)

• record operation_duration_ms

• increment operations_success_total (if you maintain it)

Important constraints:

• after is only called on successful completion.

• after is not called on cancel, and may not be called on failures (depending on where the failure happens).

6) Count cancellations (without relying on after)

Cancellations happen in before of some middleware. That means a pure “timing middleware” may not see the cancellation unless:

• you also instrument cancellation at the point where cancellation is raised, or

• your system catches the cancellation error at the boundary and increments a cancellation metric there.

Practical options:

Option A (recommended): metrics middleware + boundary instrumentation

• metrics middleware measures attempts and successes

• cancellation counters are incremented where cancellations are handled (caller boundary)

Option B: place metrics middleware before all guards

• you still can’t get after on cancelled operations

• but you can at least log “attempts” and possibly “cancelled” if you also log/record at the cancellation boundary

7) Manage per-operation timing state safely

If you store start timestamps between before and after:

• key by message id (and optionally stage)

• keep the structure bounded to avoid leaks if after never runs

Examples of safe strategies:

• store start time only for stages you measure (often Execute)

• use a Map with periodic cleanup / TTL

• avoid storing anything for Publish if you only need counters

Minimal test checklist

1. Run an allowed Execute operation:

• operations_total increments

• operation_duration_ms records a value

• operations_success_total increments

2. Trigger a cancellation (guard middleware interrupts):

• operations_total increments (attempt seen)

• operations_cancelled_total increments at your cancellation handling boundary

• no duration is recorded via after for that cancelled attempt

3. Run a Publish operation (if instrumented):

• operations_total increments

• no latency metric (unless you intentionally measure it)

“Latency metric is missing for callbacks”

Cause:

• Callback completion is asynchronous and may not map to a simple before/after boundary depending on how your integration reports completion.

Fix:

• Decide what you measure for Callback:

  • request accepted latency (before → immediate)

  • end-to-end completion (requires hooking into callback completion signals)

• Document the choice and implement accordingly.

“Metrics cardinality exploded”

Cause:

• using high-cardinality labels (message ids, user ids, dynamic strings).

Fix:

• remove message id from labels

• use stable message name/type only

• bucket by role (Command/Query/Event/Executor) if needed

“Cancelled operations are not counted”

Cause:

• cancellation skips after and your cancellation counter is only in after.

Fix:

• increment cancellation metrics at the cancellation handling boundary (caller side)

• or ensure your guard middleware increments a cancellation counter before interrupting (if your design allows it)

“Metrics cause noticeable overhead”

Cause:

• heavy label construction

• expensive time sources

• synchronous export on hot path

Fix:

• precompute stable labels

• keep canHandle cheap and narrow

• use an async exporter / batching if available