Eventstream and the new KQL Processing

Eventstream has many differences in relation to the technologies it proposes to replace. Event Hub, Stream Analytics, Streaming Dataflows and more. We can compare these technologies, but EventStream in Microsoft Fabric has some specific differences from all of them.

One of the differences is how the transformation of the input data is linked to the target of the data.

On previous technologies, we could receive the data stream, define transformations for it, and send the result of the transformations to multiple targets. The features for the transformation were the same.

On EventStream, this works differently. The transformations are linked directly to the target of the data. Lakehouse, as a target, has a specific set of transformations possible.

On November, Fabric announced the transformation features in the KQL target. This is a considerable advance. KQL target is usually the Speed Layer in a lambda architecture, while the lakehouse is the Batch Layer.

A diagram of a process

Description automatically generated


The possibility of making transformations on the Batch Layer and not on the Speed Layer was a problem. Now we can make the transformations on both.

KQL Event Processing in EventStream

When configuring the target, we can choose between two options:

  • Direct Ingestion
  • Event Processing before ingestion

A screenshot of a computer

Description automatically generated

In my opinion, the most common scenario for a Speed Layer is to transform and summarize the data, while the Batch Layer may ingest the data as is.

A screenshot of a computer

Description automatically generated

However, you may face many different scenarios and challenges.


We have some simple transformations:

  • Manage Fields
  • Filter
  • Expand

These are some transformations for grouping data:

  • Group By
  • Aggregate

This is a transformation using multiple sources:

  • Union

A screenshot of a computer

Description automatically generated


This feature is absolutely needed. In my opinion, it’s strange to have the transformations linked to the target. However, we have features to make transformations for the Speed Layer and for the Batch layer, allowing us to build a complete lambda architecture starting with a single EventStream.

The EventStream is different, but it seems more powerful than all previous features it replaces.