(Un)limited Lakehouse Power! – Get the most out of your capacity

Along with the release of Microsoft Fabric, many users struggle with choosing the right capacity size for their workloads. On the one hand, the measurement of workloads in capacity units is not trivial and on the other hand, users exceed their capacity limits frequently, especially when working with smaller capacity sizes (e.g. F2 or F4). This sessions aims to unravel the complex topic of Fabric capacities in two steps.

In the first part of the session, the theoretical groundwork is laid by thoroughly explaining Fabric capacities along with what they cost and provide. This part includes the necessary explanation of terms like CUs, spark nodes and vCores, how these relate to the different capacity sizes and whether to buy one large or many small capacities for company workloads. Additionally, methods of monitoring your capacity are demonstrated as well.

The second part covers a practical demo in which common challenges for users with small capacities are demonstrated along with solutions without increasing capacity size. The challenges are shown on a sample lakehouse project and include topics like limitations of data pipelines, defining and tuning DAGs in notebooks, creating custom Spark pools for collaboration, automated pausing and resuming of Fabric capacity and more.

The goal of the session is to show that with the right steps, Fabric is a highly competitive product in terms of functionality as well as costs for compute and storage.

 

Share this on...