{"id":102938,"date":"2024-07-03T21:24:45","date_gmt":"2024-07-03T21:24:45","guid":{"rendered":"https:\/\/www.red-gate.com\/simple-talk\/?p=102938"},"modified":"2024-09-03T20:04:44","modified_gmt":"2024-09-03T20:04:44","slug":"real-time-intelligence-more-than-a-name-change","status":"publish","type":"post","link":"https:\/\/www.red-gate.com\/simple-talk\/blogs\/real-time-intelligence-more-than-a-name-change\/","title":{"rendered":"Real-Time Intelligence: More than a name change"},"content":{"rendered":"<p>When everyone starts to announce Real-Time Intelligence in Microsoft Fabric as something new, I need to double check what&#8217;s happening: Am I crazy or is everyone else? Wasn&#8217;t this already there?<\/p>\n<p>Finally, I realize that Real-Time Intelligence is a new name for Real-Time Analytics, and they are doing this so fast we don&#8217;t even have time to notice the difference.<\/p>\n<p>What&#8217;s Real-Time Intelligence and what&#8217;s the difference from Real-Time Analytics?<\/p>\n<h2><strong>What&#8217;s Real-Time Intelligence<\/strong><\/h2>\n<p>The difference is very simple. Real-Time Intelligence is the Real-Time Analytics but:<\/p>\n<ul>\n<li>It has additional features<\/li>\n<li>It has features we were long waiting for<\/li>\n<li>It has a new UI which levels up our work with real-time data<\/li>\n<\/ul>\n<p>Microsoft is making a habit of announcing new features as &#8220;available&#8221; while we can&#8217;t see them in our tenants yet.<\/p>\n<p>These are the promised features for Real-Time Intelligence:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1568\" height=\"301\" class=\"wp-image-102939\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-1.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>These are the features we can see now in our tenants:<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"929\" height=\"226\" class=\"wp-image-102940\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-2.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<h2><strong>Real-Time Hub<\/strong><\/h2>\n<p>The new UI comes in the form of the Real-Time Hub. It appears on the left-side bar, and it brings a completely new UI to work with real-time data.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"118\" height=\"137\" class=\"wp-image-102941\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-3.png\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1379\" height=\"369\" class=\"wp-image-102942\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-4.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<h2><strong>Real-Time Intelligence Features<\/strong><\/h2>\n<p>It seems like everything is getting a new name. It&#8217;s important to map the new names of the features with our current knowledge:<\/p>\n<h3><strong>Event Ingestion<\/strong><\/h3>\n<p>This is the eventstream. What&#8217;s new?<\/p>\n<p>The evolved UI in Real-Time Hub allow us to start the process in an easier way using the <strong>Get Event<\/strong> button.<\/p>\n<p>We also can visualize all our eventstreams in the environment, check if they are endorsed and choose to use them.<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1441\" height=\"719\" class=\"wp-image-102943\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-5.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>One of the very interesting promises for the EventStream is the capability to process the data independent of the destination of the data.<\/p>\n<p>Today, we need to choose a destination and the processing depends on the destination. However, we will be able to process first and sending the processed data to multiple destinations. <img loading=\"lazy\" decoding=\"async\" width=\"1379\" height=\"514\" class=\"wp-image-102944\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-6.png\" srcset=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-6.png 1379w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-6-300x112.png 300w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-6-1024x382.png 1024w, https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-6-768x286.png 768w\" sizes=\"auto, (max-width: 1379px) 100vw, 1379px\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>\u00b4This is a new preview UI for the EventStream. We have to methods to reach it: Using the Get Event in the Real-Time Hub or creating a new event stream and selecting to use the preview UI<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"530\" height=\"439\" class=\"wp-image-102945\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-7.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>I&#8217;m proud that last year, when delivering a great session about real-time in Fabric during the Data Saturday Sofia, I pointed out that the fact the transformations were dependent on the target was a flaw.<\/p>\n<p>That&#8217;s it: They aren&#8217;t any more. Should I say, &#8220;I told you!\u201d?<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1304\" height=\"520\" class=\"wp-image-102946\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-8.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<p>You may would like to watch <a href=\"https:\/\/www.youtube.com\/watch?v=syZHAC7QkQ8\">Fabric Monday 39: Enhanced Eventstream<\/a><\/p>\n<p>&nbsp;<\/p>\n<h3><strong>Real Time Analytics<\/strong><\/h3>\n<p>This feature mainly involves our Kusto database, the place for our data and analysis. Of course, we don&#8217;t call it a Kusto database anymore.<\/p>\n<p>We create an EventHouse. An EventHouse is made to contain multiple Kusto databases. When the EventHouse create it already contains one.<\/p>\n<p>The existing Kusto databases are being converted to EventHouses, we can&#8217;t create a Kusto Database anymore.<\/p>\n<h3><strong>Real-Time AI<\/strong><\/h3>\n<p>Let&#8217;s simplify and summarize this to what it really is: It&#8217;s a co-pilot to generate Kusto queries from your natural language questions about your data in the EventHouse.<\/p>\n<p>It&#8217;s great, but are we really expecting managers and other people who need to query with natural language to reach this point?<\/p>\n<p>Last year I would say no. Today, I would say maybe. Everything is evolving so fast that maybe we will need this kind of tool as well.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"542\" height=\"161\" class=\"wp-image-102947\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-9.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<h3><strong>Real-Time Dashboards<\/strong><\/h3>\n<p>This was another critical pending point in real-time analytics: The regular Power BI dashboards were not capable to support this data and refresh the data in real-time. A new kind of dashboard was needed.<\/p>\n<p>KQL Querysets got a new button &#8220;Pin to Dashboard&#8221;. Like what we can do in a Power BI report, but in this case, it can be done directly from the queryset.<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"999\" height=\"572\" class=\"wp-image-102948\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-10.png\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"709\" height=\"532\" class=\"wp-image-102949\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-11.png\" \/><\/p>\n<p>&nbsp;<\/p>\n<h2><strong>Real-Time Triggers<\/strong><\/h2>\n<p>Real-Time Triggers is like a subset of what was Data Activator. Is Data Activator dying?<\/p>\n<p>I wouldn&#8217;t say it&#8217;s dying, I would say it&#8217;s evolving in such a way it will be integrated to the UI in many different ways. In a not so distant future, we may not recognize anymore when we are using Data Activator.\u00a0<\/p>\n<p>We should be able to generate triggers from many different places, but at the moment I could only locate one: In the Real-Time Hub, we open an EvenStream and we use the button Set Alert<\/p>\n<p>&nbsp;<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"1379\" height=\"415\" class=\"wp-image-102950\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-12.png\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"340\" height=\"902\" class=\"wp-image-102951\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-13.png\" \/><\/p>\n<p>One important new feature related to the triggers is the capability to run a Fabric Item, which can be a pipeline or a notebook.<\/p>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"324\" height=\"345\" class=\"wp-image-102952\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-14.png\" \/><\/p>\n<p>You may would like to watch <a href=\"https:\/\/www.youtube.com\/watch?v=qs-YZfMX_58\">Fabric Monday 42: Taking Actions from your data<\/a><\/p>\n<h3><strong>Event Driven Actions<\/strong><\/h3>\n<p>Last, but not least, we have the Event Driven Actions.<\/p>\n<p>In other words, we can trigger actions based on events, such as a file created in a storage.<\/p>\n<p>In the way it is now, I think it&#8217;s quite incomplete. The most important item we would like to see is not there yet: Trigger an action based on a file added to the Files area of a lakehouse.<\/p>\n<p>This would be great, because the lakehouse could be the drop zone and automatically trigger transformations on the data.<\/p>\n<p>However, at the moment, there are only two types of events available:<\/p>\n<ul>\n<li>Blob Storage Items<\/li>\n<li>Fabric Workspace Items<\/li>\n<\/ul>\n<p><img loading=\"lazy\" decoding=\"async\" width=\"676\" height=\"411\" class=\"wp-image-102953\" src=\"https:\/\/www.red-gate.com\/simple-talk\/wp-content\/uploads\/2024\/07\/word-image-102938-15.png\" \/><\/p>\n<p>Both options trigger our imagination, but don&#8217;t reach the main goal.<\/p>\n<p>The blob storage events are only for Azure blob storage, not for files in Fabric. In this way, our drop zone would still need to be in Azure.<\/p>\n<p>The Fabric workspace item doesn&#8217;t include Files in a lakehouse. In this way, it has mostly an administrative purpose. Do you see other purposes, besides the administrative one? Let&#8217;s talk on the chat.<\/p>\n<p>The UI and the response for these events is quite the same as the triggers.<\/p>\n<h2><strong>Summary<\/strong><\/h2>\n<p>The news is great, but it&#8217;s in preview. Only to give us the taste of what&#8217;s coming.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>When everyone starts to announce Real-Time Intelligence in Microsoft Fabric as something new, I need to double check what&#8217;s happening: Am I crazy or is everyone else? Wasn&#8217;t this already there? Finally, I realize that Real-Time Intelligence is a new name for Real-Time Analytics, and they are doing this so fast we don&#8217;t even have&#8230;&hellip;<\/p>\n","protected":false},"author":50808,"featured_media":102944,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[2,159164],"tags":[159113,159056,159112,158997,159114,159111],"coauthors":[6810],"class_list":["post-102938","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-blogs","category-microsoft-fabric","tag-eventhouse","tag-eventstream","tag-kql-database","tag-microsoft-fabric","tag-real-time-dashboard","tag-real-time-intelligence"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/102938","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/users\/50808"}],"replies":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/comments?post=102938"}],"version-history":[{"count":4,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/102938\/revisions"}],"predecessor-version":[{"id":102957,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/posts\/102938\/revisions\/102957"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media\/102944"}],"wp:attachment":[{"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/media?parent=102938"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/categories?post=102938"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/tags?post=102938"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.red-gate.com\/simple-talk\/wp-json\/wp\/v2\/coauthors?post=102938"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}