Register Log in

Shortcuts, Capacities and Costs : Some Considerations

Articles tagged lakehouse

Blogs
in Blogs

Shortcuts, Capacities and Costs : Some Considerations

Let’s start with a question: Question: Where is the cost charted when one or more shortcuts in a lakehouse points to a table located in...

3 min read
in Blogs

PySpark: The flexibility of a loop

Pyspark has many flexible syntaxes which are not so common to other languages. One of these syntaxes is the loop format. Loops in PySpark can...

3 min read
Blogs
in Blogs

Microsoft Fabric: Differences Between %Run and Notebook.Run

When developing notebooks in Microsoft Fabric, we have two different methods to call one notebook from another: %Run and MSSparkUtils.notebook.run We need to decide which...

2 min read
Blogs
in Blogs

Microsoft Fabric: Ingesting from API’s and the JSON result

Ingesting from API’s is a common task to build a Data Intelligence repository. When making this ingestion in a Fabric Lakehouse, there are many tricks...

7 min read
Data Lakehouse
in BI

Choosing Between the Lakehouse and Warehouse in Microsoft Fabric

Microsoft Fabric is a new centralized, AI-powered cloud data platform hosted by Microsoft. It combines several services of the existing Azure Data Platform – such...

20 min read
Blogs
in Blogs

Dynamic Partitioning and a Simple Incremental Load

Let’s consider a simple statement for partitioning and save a table in a lakehouse: df.write.mode("overwrite").format("delta").partitionBy("Year","Month","Day").save("Tables/" + table_name) Let’s consider we load the data daily, with...

2 min read
Blogs
in Blogs

Creating a Single JSON File for Configuration using Notebooks

On the blog Fabric Notebook and Deployment Pipelines I explained a technique to keep notebooks configuration values in JSON files on lakehouses, a good solution...

2 min read
Blogs
in Blogs

5 Secrets about Dataflows Gen 2

Dataflows Gen 2 are the new version of Power BI dataflows. There are so many changes in relation to the previous version they are considered...

5 min read
Blogs
in Blogs

Semantic Model: More than a Simple Name Change

The animation on the top of this article tries to track the evolution of the enterprise architecture since SQL Server 7.0 introduced tools for ETL,...

3 min read
Blogs
in Blogs

Microsoft Fabric: Using VS Code to Develop Notebooks

The possibility to use Visual Studio Code (or VS Code) to develop your Microsoft Fabric notebooks seems very interesting. It may bring many benefits for...

9 min read
BI
in BI

SQL Endpoints and Lakehouse UI Differences and Access Secrets

There are some differences and secrets between the UI of a SQL Endpoint and the UI of a Lakehouse. I believe the lakehouse UI was...

7 min read
Blogs
in Blogs

Fabric: My Content compilation

Microsoft Fabric was first announced during Microsoft BUILD. Since then, I’m publishing content about Microsoft Fabric, creating an interesting sequence of content. In this blog,...

3 min read
Blogs
in Blogs

Microsoft Fabric and the Delta Tables Secrets

Microsoft Fabric storage uses OneLake and Delta Tables, the core storage of all Fabric objects, as explained in my introduction to Microsoft Fabric. Either if...

10 min read

Microsoft Fabric: Checking and Fixing Tables V-Order Optimization

Download the notebook used on this blog I explained in a previous article how the Tables in a lakehouse are V-Order optimized. We noticed this...

5 min read

Microsoft Fabric: using Notebooks and Table Partitioning to Convert Files to Tables

When Microsoft Fabric was born, the only method to convert files to tables was using notebooks. Nowadays we have an easy-to-use UI feature for the...

5 min read
Blogs
in Blogs

Microsoft Fabric: Loading On-Premises and External Data

Loading On-Premises and external data is a fundamental need for a Data Platform such as Microsoft Fabric. However, during this preview stage, Microsoft Fabric brings...

9 min read
Blogs
in Blogs

Fabric Lakehouse: Convert to Table feature and Workspace Level Spark Configuration

I have been working as a no-code data engineer: Focused on Data Factory ETL and visual tools. In fact, I prefer to use visual resources...

2 min read
Blogs
in Blogs

Discover the Microsoft Fabric Data Wrangler

The Data Wrangler is as interesting as hidden inside Microsoft Fabric. It’s not easy to find and activate it, but it is worth the trouble....

6 min read
Blogs
in Blogs

Microsoft Fabric: Enabling your Free Trial

After the great announcement of Microsoft Fabric this week, it’s time to get a trial and start playing with this great product. Microsoft Fabric is...

3 min read