HTTP Sink
The HTTP sink posts records from a KurrentDB stream to an external API over HTTP or HTTPS. The connector converts each record’s data to JSON and sends it as the request body to the URL you specify. It sends records individually as it consumes them from the stream, without batching. Record metadata can travel as HTTP headers. The connector also supports Basic Authentication and Bearer Token Authentication. See Authentication.
Prerequisites
Section titled “Prerequisites”Before using the HTTP sink connector, ensure you have:
- A data protection token configured in your KurrentDB instance (required to encrypt sensitive fields like passwords and bearer tokens)
- A target HTTP/HTTPS endpoint that can receive POST requests
- Appropriate authentication credentials if the endpoint requires authentication
Quickstart
Section titled “Quickstart”Create the HTTP sink with client.Connectors. Replace example-http with your own
connector id:
Dictionary<string, string> settings = new() { ["instanceTypeName"] = "http-sink", ["url"] = "https://api.example.com/", ["subscription:filter:scope"] = "stream", ["subscription:filter:filterType"] = "streamId", ["subscription:filter:expression"] = "example-stream"};
await client.Connectors.Create("example-http", settings);await client.Connectors.Start("example-http");Once started, the connector posts each record appended to example-stream to the URL.
Managing Connectors covers the rest of the lifecycle.
Settings
Section titled “Settings”Adjust these settings to control how the HTTP sink connector behaves and how it interacts with KurrentDB.
| Name | Details |
|---|---|
url |
required Description: The URL or endpoint that the request or message is sent to. See Template Parameters for advanced settings. Default: "" |
method |
Description: The method or operation to use for the request or message. Default: "POST" |
defaultHeaders |
Description: Custom headers to include in the request. See Specifying Custom Headers. Default: "Accept-Encoding:*" |
pooledConnectionLifetime |
Description: Maximum time a connection can stay in the pool before it is no longer reusable. Default: "00:05:00" (5 minutes) |
Authentication
Section titled “Authentication”The HTTP sink connector supports both Basic and Bearer authentication methods.
| Name | Details |
|---|---|
authentication:method |
Description: The authentication method to use. Default: "None"Accepted Values: "None","Basic", or "Bearer" |
Basic authentication
Section titled “Basic authentication”| Name | Details |
|---|---|
authentication:basic:username |
Description: The username for basic authentication. Default: "" |
authentication:basic:password |
Description: The password for basic authentication. Default: "" |
Bearer authentication
Section titled “Bearer authentication”| Name | Details |
|---|---|
authentication:bearer:token |
Description: The token for bearer authentication. Default: "" |
Resilience
Section titled “Resilience”The HTTP sink uses the shared
resilience settings, including
resilience:enabled and the three delay bounds. It adds these keys on top of them.
| Name | Details |
|---|---|
resilience:requestTimeoutMs |
Description: The timeout in milliseconds for a single HTTP request. Default: "3000" |
resilience:maxRetries |
Description: The maximum number of retry attempts. "-1" means retry without a limit.Default: "-1" |
resilience:transientErrorDelay |
Description: The delay before retrying a transient error. Default: "00:00:00" |
resilience:retryOnHttpCodes |
Description: A comma-separated list of status codes to retry on. A code can be exact, such as 408, or a wildcard for a class, such as 5xx. Empty means retry on any non-2xx code.Default: "" |
Headers
Section titled “Headers”Besides the default headers included with every request, the HTTP sink connector adds the following headers to the HTTP request:
| Header | Description |
|---|---|
esdb-request-id |
A unique identifier for the request |
esdb-request-date |
The timestamp when the request was processed |
content-type |
The media type of the resource |
accept-encoding |
The encoding types the client can handle. Default value is * |
Specifying custom headers
Section titled “Specifying custom headers”The HTTP sink connector lets you include custom headers in the HTTP requests it
sends to your specified URL. To add custom headers, use the defaultHeaders
setting in your connector configuration. Specify each custom header with the
prefix defaultHeaders: followed by the header name.
Example:
{ "defaultHeaders:X-API-Key": "your-api-key-here", "defaultHeaders:X-Tenant-ID": "production-tenant", "defaultHeaders:X-Source-System": "KurrentDB"}With this configuration, the connector includes the custom headers and their values in every HTTP request. It sends them alongside the default headers that its plugin adds automatically.
Template parameters
Section titled “Template parameters”The HTTP sink supports template parameters in the URL, so the connector builds the request URL from record data. Customize the destination URL for each record to integrate with APIs that require specific URL structures.
The following template parameters are available for use in the URL:
| Parameter | Description |
|---|---|
{schema-subject} |
The record’s schema subject, converted to lowercase with hyphens |
{event-type} |
Alias for {schema-subject} |
{stream} |
The KurrentDB stream ID |
Usage
To use template parameters, include them in the Url option of your HTTP sink configuration. The connector replaces
each parameter with its corresponding value for every record.
Example:
https://api.example.com/{schema-subject}For a record with schema subject “TestEvent”, the URL becomes:
https://api.example.com/test-eventTutorial
Section titled “Tutorial”Learn how to configure and use a connector using the HTTP Sink in KurrentDB through a tutorial.