Docs / Connectors / Confluent

Confluent Connector

The ASAPIO Integration Add-on connects SAP S/4HANA and SAP ECC directly to Confluent Kafka clusters. Two connector types are available: the REST-based Kafka connector communicates through the Confluent REST Proxy, while the Native Kafka connector communicates directly using the Kafka binary protocol via a sidecar daemon running outside SAP. Both support real-time event streaming for SAP business objects.

Overview

ComponentType
ASAPIO Integration Add-on – FrameworkBase component (required)
ASAPIO Integration Add-on – Connector for ConfluentAdditional package
Connector TypeCloud TypeHandler ClassProtocol
REST KafkaKAFKA/ASADEV/CL_ACI_KAFKA_HANDLERConfluent REST Proxy (HTTP)
Native KafkaS4KAFKA/ASADEV/CL_S4_KAFKA_HANDLERNative Kafka binary protocol

Prerequisites

For the REST connector: A Confluent Kafka cluster with the REST Proxy endpoint reachable from the SAP application server. The REST Proxy must be accessible over HTTPS.

For the Native Kafka connector: The ASAPIO Kafka daemon must be running on a host reachable from SAP. The daemon handles the native Kafka protocol so SAP can publish using standard RFC HTTP calls.

SAP Connection Setup (in SAP Gui) (REST Connector)

The connection setup in SAP GUI is a required first step for all configuration paths. This creates the RFC destination, imports the SSL certificate, activates the BC-Set, and registers the connection instance that both Event Studio and SAP GUI-based event configuration rely on.

Step 1 — Create RFC Destination (SM59)

Create an HTTP connection of type G to the Confluent REST Proxy endpoint. On the Logon & Security tab, enable SSL and enter the Basic Auth credentials (API key and secret) if required.

SM59 RFC destination for Confluent REST Proxy
RFC destination authentication settings

Step 2 — Import SSL Certificate (STRUST)

In transaction STRUST, import the Confluent TLS certificate and add it to the Certificate List used by the RFC destination.

STRUST certificate import for Confluent

Step 3 — Import BC-Set (SCPR20)

Import the ASAPIO Kafka framework BC-Set to activate the cloud adapter configuration, codepage definitions, and IDoc segment definitions:

SCPR20 BC-Set import for Confluent

Step 4 — Configure Cloud Adapter

In SPRO → ASAPIO Cloud Integrator → Maintain Cloud Adapter, set:

Confluent cloud adapter configuration in SPRO

Step 5 — Create Connection Instance

In transaction /ASADEV/ACI_SETTINGS, create a connection instance pointing to the Confluent REST Proxy RFC destination. Set the following default connection values:

Default AttributeValue
KAFKA_ACCEPTapplication/vnd.kafka.v2+json
KAFKA_CALL_METHODPOST
KAFKA_CONTENT_TYPEapplication/vnd.kafka.json.v2+json (or application/vnd.kafka.jsonschema.v2+json for schema registry)
Confluent connection instance and default values

Create an outbound interface (Event Studio)

Prerequisite: The connection instance must already be set up in SAP GUI (see SAP Connection Setup above) before Event Studio can be used to configure events.

Event Studio is the recommended web-based interface for creating and managing events. To configure a Confluent/Kafka event:

  1. Open Event Studio and select the backend SAP system.
  2. Go to the Data Catalog to find the data object to stream.
  3. In the Available Standard Events table, click Deploy, or use Create to define a new event.
  4. Select the Confluent connection from the Connection dropdown — the Header Fields table auto-populates with the connection's default attributes.
  5. Set KAFKA_TOPIC and any additional header attributes (see below).
  6. Save and activate the event.

After creating the interface in Event Studio, see Header Attributes for message configuration, Batch Job Method for scheduled delivery, Dead Letter Queue for error handling, or Native Kafka Connector for direct protocol integration.

Header Attributes

Configure the following header attributes on each outbound object. Header attributes apply equally when configuring via Event Studio or SAP GUI.

Header AttributeValue / DescriptionRequired
KAFKA_TOPICTarget Kafka topic nameYes
KAFKA_KEY_FIELDSemicolon-separated list of SAP fields to use as the Kafka message keyOptional
KAFKA_SCHEMA_IDSchema Registry schema ID for the message value (when using schema registry)Optional
KAFKA_KEY_SCHEMA_IDSchema Registry schema ID for the message keyOptional
Confluent header attributes on outbound object

Create an outbound interface (SAP Gui)

MethodExtraction FMFormatting FMResponse FM
Simple Notify/ASADEV/ACI_GEN_NOTIFY_KAFKA/ASADEV/ACI_KAFKA_RESP_HANDLER
Message Builder/ASADEV/ACI_GEN_VIEW_EXTRACTOR/ASADEV/ACI_GEN_VIEWFRM_KAFKA/ASADEV/ACI_KAFKA_RESP_HANDLER
Packed Load/ASADEV/ACI_GEN_VIEW_EXT_PACK/ASADEV/ACI_GEN_VIEWFRM_KAFKA

Create Outbound Object

In transaction /ASADEV/68000202, select the connection instance and add an entry under Outbound Objects. Set the Extraction FM, Message Type, Load Type, and Response Function as shown above. Then add the header attributes under the Header Attributes tab.

Confluent outbound object configuration
Confluent outbound object detail

Event Linkage (SWE2)

Register the event trigger in transaction SWE2:

SWE2 event linkage for Confluent

Packed Load — Header Attributes

AttributeDescriptionExample
ACI_PACK_TABLETable for key fields (typically the header table)MARA
ACI_PACK_SIZERecords per delivery package500
ACI_PACK_KEY_LENGTHTotal length of key fields13
ACI_PACK_WHERE_CONDOptional WHERE condition on ACI_PACK_TABLE
ACI_PACK_RETRY_TIMESeconds before retrying server group resource acquisition300
ACI_PACK_BDCP_COMMITGenerate change pointers per entry (~50% performance impact)X

Batch Job Method

The Batch Job method allows scheduling periodic full or delta loads to Kafka without relying on SAP event linkage. This is useful for initial loads or when real-time event triggers are not feasible.

  1. In transaction /ASADEV/ACI, open the replication object and uncheck the Sync.On checkbox.
  2. Create a variant for the report.
  3. Schedule the job in SM36 using report /ASADEV/AMR_REPLICATOR with the saved variant.

The extraction FM for batch jobs is /ASADEV/ACI_SYNC.

Dead Letter Queue (DLQ)

The DLQ feature routes failed messages to a dedicated Kafka topic instead of discarding them. This allows downstream systems to inspect, reroute, or replay failed events without losing data.

Configure the following header attributes on the outbound object to enable DLQ:

AttributeValue / Description
DLQ_TOPICKafka topic name to route failed messages to
DLQ_ERROR_CODESComma-separated HTTP error codes that trigger DLQ routing
DLQ_RETRIESNumber of immediate retries before routing to DLQ
Confluent DLQ header attribute settings

Native Kafka Connector

The Native Kafka connector uses the binary Kafka protocol directly, bypassing the REST Proxy. A lightweight daemon process handles the native protocol connection outside SAP. SAP communicates with the daemon via a standard HTTP RFC destination.

Cloud Adapter Configuration

Daemon Default Parameters

Configure the following default values on the Native Kafka connection instance:

ParameterDefault ValueDescription
DAEMON_BUSY_TICK_MS50Polling interval (ms) when the daemon is actively processing
DAEMON_IDLE_SLEEP_MS300Sleep time (ms) when the daemon has no pending messages
DAEMON_MAX_ITEM_BYTES524288Maximum size per message item in bytes (512 KB)
DAEMON_QUEUE_MAX_BYTES209715200Maximum total queue size in bytes (200 MB)
DAEMON_QUEUE_MAX_ITEMS10000Maximum number of items in the daemon queue
DAEMON_QUEUE_TTL_SEC0Time-to-live for queued items (0 = no expiry)
DAEMON_SEND_TIMEOUT_MS60000Kafka send timeout in milliseconds
DAEMON_TIMEOUT_CHECK_MS1000Interval (ms) for checking message timeouts
KAFKA_USERNAMESASL username for Kafka cluster authentication

Store the Kafka SASL password in the SAP Secure Store (transaction /ASADEV/SCI_TPW).

Inbound (Kafka to SAP)

The Confluent connector supports inbound message delivery from Kafka topics to SAP via IDoc. Configure the following attributes on the inbound object:

AttributeDescriptionRequired
KAFKA_CONTENT_TYPEContent type of the Kafka message (e.g., application/vnd.kafka.json.v2+json)Yes
KAFKA_GROUPNAMEConsumer group nameYes
KAFKA_INSTANCE_NAMEKafka consumer instance name (must be unique per consumer group)Yes
KAFKA_DOWNLOAD_TOPICKafka topic to consume fromYes
KAFKA_DOWNLOAD_ACCEPTAccept header value for consumer requestsOptional
KAFKA_MAX_BYTESMaximum bytes to fetch per poll requestOptional
KAFKA_TIMEOUTConsumer poll timeout in millisecondsOptional
KAFKA_CONSUMER_FORMATMessage format: binary, avro, json, jsonschema, or protobufOptional
KAFKA_CONSUMER_OFFSET_RESETOffset reset policy: earliest or latestOptional
KAFKA_CONSUMER_AUTO_COMMITEnable automatic offset commitOptional

Inbound Function Modules

FMPurpose
/ASADEV/ACI_SAMPLE_IDOC_JSONSample inbound handler — creates an IDoc from a JSON Kafka message
/ASADEV/ACI_SAMPLE_IDOC_JSON2Alternative sample handler with extended JSON mapping
/ASADEV/ACI_JSON_TO_IDOCGeneric JSON-to-IDoc converter for custom inbound mappings
Confluent inbound configuration