Eventstream has many differences in relation to the technologies it proposes to replace. Event Hub, Stream Analytics, Streaming Dataflows and more. We can compare these technologies, but EventStream in Microsoft Fabric has some specific differences from all of them.

One of the differences is how the transformation of the input data is linked to the target of the data.

On previous technologies, we could receive the data stream, define transformations for it, and send the result of the transformations to multiple targets. The features for the transformation were the same.

On EventStream, this works differently. The transformations are linked directly to the target of the data. Lakehouse, as a target, has a specific set of transformations possible.

On November, Fabric announced the transformation features in the KQL target. This is a considerable advance. KQL target is usually the Speed Layer in a lambda architecture, while the lakehouse is the Batch Layer.

The possibility of making transformations on the Batch Layer and not on the Speed Layer was a problem. Now we can make the transformations on both.

KQL Event Processing in EventStream

When configuring the target, we can choose between two options:

• Direct Ingestion
• Event Processing before ingestion

In my opinion, the most common scenario for a Speed Layer is to transform and summarize the data, while the Batch Layer may ingest the data as is.

However, you may face many different scenarios and challenges.

Transformations

We have some simple transformations:

• Manage Fields
• Filter
• Expand

These are some transformations for grouping data:

• Group By
• Aggregate

This is a transformation using multiple sources:

• Union

Summary

This feature is absolutely needed. In my opinion, it’s strange to have the transformations linked to the target. However, we have features to make transformations for the Speed Layer and for the Batch layer, allowing us to build a complete lambda architecture starting with a single EventStream.

The EventStream is different, but it seems more powerful than all previous features it replaces.