KSQL

Overview

KSQL is a powerful, SQL-like streaming engine for Apache Kafka that enables real-time data processing and transformation. Integrating KSQL with Condense will enable users to perform complex data transformations and aggregations on Kafka topics directly from the platform.

Key Features

Real-Time Data Processing

  1. Execute SQL-like queries on streaming data for immediate insights.

  2. Supports filtering, joins, and aggregations in real time.

Stream and Table Management

  • Create, update, and delete streams and tables from Kafka topics.

  • Streams for unbounded event sequences

  • Tables for stateful data

Transient Queries

  • Run queries that process data on the fly without storing results.

  • Ideal for real-time monitoring and alerting

User-Friendly Interface

  • Integrated directly into the Condense UI for seamless query execution.

  • Visualise query results in real time.

Role-Based Access Control

  • Admin/Developer: Full access to create and manage queries.

  • Viewer: Read-only access (no KSQL permissions)

How to Use KSQL

Prerequisites

  • Admin or Developer role in Condense

  • Access to Kafka topics with read/write permissions

  • Basic knowledge of SQL syntax

Step 1: Access KSQL Interface

  1. Log in to Condense with your credentials

  2. Navigate to KSQL from the main menu

  3. Select New Query to open the editor

Step 2: Create a Stream

  1. For processing real-time event data, in the query editor, enter your query

  2. Click Execute to create the stream

Step 3: Create a Table

  1. For stateful data, in the query editor, enter your query

  2. Click Execute to create the table

Step 4: Run queries

  1. Execute transient queries for real-time analysis

  2. Click Execute to execute and view real-time results

Step 5: Monitor & Troubleshoot

Review Error Logs:

  1. Access logs below the query editor

  2. Common issues: topic permissions, schema mismatches

Frequently Asked Questions (FAQs)

Q1: What is RBAC in KSQL?

RBAC (Role-Based Access Control) restricts access based on predefined user roles:

  • Admin: Full create/edit/delete permissions

  • Developer: Create/edit permissions

  • Viewer: Read-only access (no KSQL access in initial release)

This information may be updated or revised in the future

Q2: Can we allow KSQL to read from some topics but not others?

  1. Grant the user "Developer" or "Admin" role in RBAC

  2. Ensure ACLs allow topic creation permissions

Q3: How to allow KSQL to create new streams and tables?

Yes, ACLs can be configured at the topic level to grant or restrict access to specific topics.

Last updated

Was this helpful?