Transport & Retries

The SDK's event delivery pipeline: how events are queued, batched, sent, and retried.

Overview

The transport layer sits between the SDK's event emitters and the Dicolytics API. It provides reliable, ordered delivery with bounded memory usage and automatic retry logic.

Event emitters (interactions, actions, gateway, custom)
    |
    v
  Enqueue (assign UUIDv7 id)
    |
    v
  Queue (bounded, max 5,000 events)
    |
    |-- Timer fires (every 5s) ----------> Drain loop
    |-- High-water mark (50 events) -----> Drain loop
    |-- Manual flush() ------------------> Drain loop
    |
    v
  Drain loop (single in-flight request)
    |
    v
  Build envelope + serialize JSON
    |
    |-- Body > 1 MiB? --> Split recursively
    |
    v
  HTTP POST /v1/events
    |
    v
  Handle response (retry/drop/quiet/disable)

HTTP Batch Envelope

Each HTTP request sends a JSON envelope containing a batch of events:

{
  "batchId": "0192a3b4-c5d6-7e8f-9a0b-1c2d3e4f5a6b",
  "sdk": {
    "name": "@dicolytics/discord.js",
    "version": "0.5.0"
  },
  "sentAt": "2025-01-15T12:00:05.123Z",
  "events": [
    {
      "id": "0192a3b4-c5d6-7e8f-9a0b-1c2d3e4f5a6c",
      "type": "interaction_slash",
      "ts": "2025-01-15T12:00:04.500Z",
      "guildId": "123456789012345678",
      "channelId": "234567890123456789",
      "userId": "345678901234567890",
      "data": {
        "commandName": "help",
        "interactionKind": "slash",
        "success": true,
        "latencyMs": 42,
        "shardId": 0
      }
    }
  ]
}

Envelope Fields

Field	Type	Description
batchId	UUIDv7	Fresh per HTTP attempt. Retries of the same batch get a new batchId; event id values remain stable.
sdk.name	string	SDK package name: @dicolytics/discord.js or dicolytics-discord.py
sdk.version	string	SDK version (e.g., 0.5.0 for JS, 0.1.0 for Python)
sentAt	ISO-8601	UTC timestamp of when the batch was sent
events	array	Array of event objects

Event Fields

Field	Type	Description
id	UUIDv7	Unique event identifier. Assigned at enqueue time, stable across retries. Used for server-side deduplication.
type	string	Event type from the closed taxonomy (see Event Types)
ts	ISO-8601	UTC timestamp of when the event occurred
guildId	string?	Server snowflake ID, when applicable
channelId	string?	Channel snowflake ID, when applicable
userId	string?	User snowflake ID, when applicable
data	object	Type-specific payload with guarded serialization

Request Headers

Header	Value
Content-Type	application/json
Authorization	Bearer {apiKey}
User-Agent	@dicolytics/discord.js/{version}

Batch Limits

Limit	Value
Max events per batch	500
Max batch body size	1 MiB (1,048,576 bytes)

If a batch exceeds 1 MiB when serialized, the SDK recursively splits it in half. Each split gets a fresh batchId, but event id values never change. If a single event exceeds 1 MiB, it is dropped.

Retry Policy

HTTP Status	Behavior
2xx	Accepted. Batch delivery complete.
400	Dropped permanently. Server rejected the batch as malformed.
401	Transport permanently disabled. API key is invalid. A single console.warn is emitted.
403	Retryable. May be a transient CDN/WAF issue (IP reputation, challenge page, geo-block).
429	Quiet period activated. See Rate Limiting below.
5xx	Retryable with exponential backoff.
Network error / timeout	Retryable with exponential backoff.

Exponential Backoff

For retryable failures (5xx, 403, network errors):

Parameter	Value
Max attempts	5 (including the initial attempt)
Backoff strategy	Full-jitter exponential
Base delay	1 second
Max delay	30 seconds
Jitter	Uniform random in [0, min(30s, 1s * 2^(attempt-1)))

After 5 failed attempts, the batch is dropped with a debug log. Event IDs remain stable across all retry attempts, enabling server-side deduplication if the server received but did not acknowledge a previous attempt.

UUIDv7 deduplication

Event IDs are UUIDv7 values assigned at enqueue time. They never change across retries. If a request succeeds on the server but the client receives a network error, the retry sends the same event IDs. The server deduplicates by event ID, turning at-least-once delivery into effectively exactly-once.

Rate Limiting

When the server returns 429 Too Many Requests:

1. The SDK reads the Retry-After header (delta-seconds or HTTP-date format).
2. If absent, a default quiet period of 300 seconds (5 minutes) is used.
3. During the quiet period, most events are held in the queue without being sent.
4. Exempt events: heartbeat and guild_snapshot continue flowing during the quiet period. This ensures uptime tracking remains accurate.
5. After the quiet period expires, normal flushing resumes.

INFO

The 429 quiet period does not drop events -- they remain in the queue (subject to the 5,000-event buffer cap). If the quiet period is long and event volume is high, the buffer may fill and oldest events will be dropped.

Buffer Management

Parameter	Value
Max buffered events	5,000
Overflow policy	Oldest events dropped first (FIFO eviction)
Overflow warning	console.warn emitted once on first overflow

The buffer is a simple array. When enqueue() pushes the length past 5,000, the excess oldest events are spliced from the front. This bounds memory usage regardless of network failures or quiet periods.

Flush Triggers

Trigger	Condition
Timer	Every flushIntervalMs (default 5,000ms). Timer uses setInterval with unref().
High-water mark	When the queue reaches 50 events after an enqueue().
Manual	analytics.flush() call.
Shutdown	analytics.shutdown() or process signal (SIGINT/SIGTERM/beforeExit).

Only one drain loop runs at a time. Concurrent flush calls coalesce onto the existing drain promise. This ensures order-preserving delivery without parallel requests.

Shutdown Flush

When the process receives SIGINT or SIGTERM:

1. The SDK's signal handler fires.
2. A final flush runs with a 3-second timeout.
3. After the flush completes (or times out), the signal is re-raised so the default termination handler proceeds.
4. If other listeners exist for the signal, the SDK does not re-raise (it defers to the application's handler).

The beforeExit event also triggers a 3-second flush, catching clean exits where no signal was involved.

INFO

All SDK timers use unref(), so they never prevent the Node.js process from exiting. The shutdown flush is a best-effort attempt to deliver remaining events.

Redirect Handling

The SDK sets redirect: 'manual' on all fetch requests. If the server returns a 301 or 302:

1. The transport is permanently disabled.
2. A console.warn is emitted explaining that the endpoint should be set to the final URL.

This prevents a common misconfiguration where an HTTP endpoint redirects to HTTPS, silently converting POST to GET and losing all event data.