Skip to main content
Confluent Cloud is a fully managed Apache Kafka service. The Relevance AI integration lets your agents connect to Confluent Cloud clusters to manage topics, produce records, and monitor consumer groups — enabling event-driven automation and real-time data workflows.

​
Connect the integration

Connecting Confluent Cloud requires four values: an API key and secret scoped to your cluster, the cluster’s REST endpoint URL, and the cluster ID.
Your API key must be cluster-scoped. Organization-level or Cloud API keys target the control plane and will return 401 errors when calling the Kafka REST API. Create a cluster-scoped API key from the Confluent Cloud console under Cluster Overview → API Keys.

​
API key credentials

​
Cluster details

​
Steps to connect

  1. Go to the Integrations & API Keys page in the sidebar of your Relevance AI dashboard.
  2. Click on Confluent Cloud from the available integrations.
  3. Click Add Integration.
  4. Enter your API Key and API Secret.
  5. Enter your REST Endpoint URL and Cluster ID.
  6. Click Save to complete the connection.

​
Getting your credentials from Confluent Cloud

1

Find the REST endpoint URL

In the Confluent Cloud console, open your cluster and go to Cluster Overview → Cluster Settings. The REST endpoint is listed under Endpoints. Copy the full URL including the port (:443).
2

Find the cluster ID

On the same Cluster Settings page, the cluster ID is displayed at the top. It starts with lkc-.
3

Create a cluster-scoped API key

Go to Cluster Overview → API Keys and click Create key. Select Granular access and scope the key to the cluster (not to the organization or cloud). Download or copy the key and secret — the secret is only shown once.

​
Available tool steps

The Confluent Cloud integration provides the following tool steps for use in your agents and tools.

Generic Confluent Cloud API call

Makes any authenticated request to the Confluent Cloud Kafka REST API. Automatically substitutes the cluster ID into path placeholders, so you can write {cluster_id} in the path without hardcoding the value. Use this for operations not covered by the purpose-built steps below.

List topics

Retrieves all topics on the connected cluster, including their configuration and partition details.

Create topic

Creates a new topic on the cluster. You can specify the topic name, number of partitions, and replication factor.

Delete topic

Deletes a topic from the cluster by name. This action is irreversible — unconsumed records in the topic are lost.

Produce a record to a topic

Publishes a record to a specified topic. You can set the record key, value, and optional headers.

List consumer groups

Lists all consumer groups on the cluster along with their state, making it possible to monitor consumer lag and group activity.

​
Example use cases

  • Trigger agent actions in response to Kafka topic data, or produce records that drive downstream systems.
  • Automate topic creation and cleanup as part of environment provisioning workflows.
  • Use agents to produce enriched or transformed records to Kafka topics from external data sources.
  • Regularly list consumer groups and alert when groups fall behind, enabling proactive incident response.

​
Frequently asked questions (FAQs)

The most common cause is using an organization-level or Cloud API key instead of a cluster-scoped key. Organization and Cloud API keys target the Confluent control plane, not the Kafka REST API. Create a new cluster-scoped API key from Cluster Overview → API Keys and update your integration credentials.
In the Confluent Cloud console, go to your cluster and open Cluster Overview → Cluster Settings. The REST endpoint is listed under Endpoints. It is specific to your cluster — do not use the control plane URL (api.confluent.cloud).
The generic API call step is scoped to the single cluster configured in your integration credentials. To work with multiple clusters, set up a separate Confluent Cloud integration for each cluster.
Yes. Relevance AI encrypts all integration credentials at rest. The API secret is never returned in plaintext after the initial save.