-
Upgrading Enterprise Power BI Architecture
This course is designed for users who want to learn to enhance Power BI models for enterprise deployment, creating sharing strategies, and ensuring security. In this course, you will learn valuable tools and skills to create, configure, and manage enterprise-grade Power BI models and take your organization's data analytics to the next level.
First, the session will review how to architect models for enterprise. It will look at configuring tables, columns, hierarchies, and measures using tools such as Tabular Editor. It covers advanced features such as incremental refresh and
Next, we’ll look at the deployment of Power BI models and how to manage the content effectively. It covers version control, DevOps, and build and deployment pipelines. We’ll look at a range of tools such as Tabular Editor, Visual Studio, Azure DevOps, or GitHub. We’ll cover all terminology at a introductory level so you won’t get left behind. You will learn publication options, how to configure datasets, refresh and monitor datasets, very large datasets, configure workspaces, and control workspace access.
We’ll cover different sharing strategies for sharing distributing content and how to collaborate with workspaces and compare different sharing techniques. You will learn how to configure role-based access control (RBAC), implement RLS and OLS, and apply security labels. We’ll learn some common data leak pitfalls and how to avoid them.
Participants should have a basic understanding of Power BI and data modeling concepts, including tables, relationships, and measures. Prior experience with Tabular Editor and DevOps is not required, but it may be helpful.Chopin Chapel (Dijlezaal)Mon 09:00 - 17:00 -
Practical Agile Data Product Design: From Strategy to User Story
Understanding organisational strategies can be difficult, translating those organisational strategies into a supporting data strategy and tangible deliverables is even more so.
In this course on Practical Agile Data Product Design, we’ll use a fictional company to understand its organisational strategy, how data can support it, and how to distil and deliver value – taking the high-level of what an organisation wants to achieve and creating tangible data products that support it.
From understanding the organisational strategy, identifying domains, understanding domain activities and prioritising them against the strategy, identifying and mapping the events, through to building a backlog to ensure that the first iteration of a data product delivers value immediately.
Our comprehensive agenda includes the following highlights:
- Relevance of Data Modelling: Uncover why data modelling remains essential in the era of big data.
- Data Product Thinking: Understand what a data product is and how Product Thinking applies to it
- Agile Principles: Dive into the principles and flavours of Agile methodologies, including the Data Ops Manifesto.
- Innovative SunBeam: Explore the ground breaking concept of SunBeam, from its history to the product design process and documentation.
- Hands-on-Experience with Agile and SunBeam: Participate in a series of hands-on group activities using a fictional company and business process as the backdrop, to fully learn how Agile and SunBeam can be used together to design data products.
- Delivery Planning: Using the design of your data product, understand how you can create effective data stories and user questions to facilitate delivery.Mozart Mansion (Scala)Mon 09:00 - 17:00 -
Build a Microsoft Fabric Proof of Concept in a Day
Are you itching to get hands-on building solutions in Microsoft Fabric, but aren't quite sure where to start? Maybe you're coming from the world of Power BI and want to explore more of the tech behind the scenes? Or is it finally time to move away from the on-premises Microsoft BI stack to fully embrace the latest and greatest of analytical tools in the cloud? If so, this workshop is for you!
In just one day, you will learn everything you need to build a Microsoft Fabric Proof of Concept.
We will start with an overview of Microsoft Fabric and its experiences, common architectures, and the most important factors that will help you decide which parts of Microsoft Fabric are appropriate for your project. Lakehouse or Warehouse? Pipelines or Notebooks? Low-code or code-first? Microsoft Fabric can do it all! When you have a plan and a few rough diagrams, it's time to dig into the details.
After you've set up your own workspace, we will go through options for ingesting, transforming and storing data. By discussing your proposed solutions with each other, you will also gain valuable insight and learn from each other’s experiences.
Once the data is ready for consumption, we will look at how to create beautiful visualizations and tell stories with our data using Power BI, how to empower business analysts to take action based on changing data, and how to roll out a fully governed solution in your organization.
Throughout the day, we will cover security, CI/CD, the pricing model, best practices for development, and lessons learned from real-world projects.
Don't miss this opportunity to experience Microsoft Fabric's game-changing potential firsthand and learn how it can revolutionize your analytical workflow. Bring your laptop and get ready for a day full of hands-on experience and engaging discussions!Bach Ballroom (Alcazar)Mon 09:00 - 17:00 -
Build A Fabric Real-time Intelligence Solution in One Day
Real-Time Intelligence in Fabric is designed to enable organisations to bring their streaming, high-granularity, time-sensitive event data into Fabric and build various analytical, visual and action oriented data applications and experiences with it. In this session, you will learn about what Fabric Real-time Intelligence is meant for and will then build an end-to-end solution using streaming data.
This session will get you hands-on with Real-Time Hub, Eventstream, Eventhouse, Data Activator, Real-time Dashboards, Power BI, and Copilot with combination of Microsoft MVPs and Fabric product group members to guide you along the way.
The end-to-end solution we build together will encompass:
• Discovering and cataloging your data streams using Real-Time Hub
• Connecting to streaming data from various sources around the world
• Cleaning, normalizing, preparing data for superior query time experiences
• Leveraging Copilot to discover streaming data
• Low-code/no-code experiences for data exploration
• Developing dashboardsThe Orchestra (Auditorium)Mon 09:00 - 17:00 -
Writing T-SQL code for the engine, not for you! - The training day
A slow query on premises will still be a slow query in the cloud!
Moving to the cloud won't be a silver bullet for your slow queries, however, it seems easy to slide the bar to the right and get more firepower.
That won't solve the main problem and actually, it will just bring a new one which is much more expensive.
While everyone can throw a "SELECT" query to the engine. not everyone is aware of some pitfalls when writing T-SQL. These can make you doubt yourself or even make you doubt what you "have heard" to be the solution.
Writing queries from scratch with good enough performance may be tricky. Now imagine that you haven't written those queries but you found them because they were slow. What options do you have?
Indexes can help most of the time. However, even with the right indexes in place, you may only be able to get maximum performance from a query if you rewrite it.
In this full-day training session, we will see which tools we can use to identify slow queries, how to analyse their execution plans to understand why they are slow and ultimately follow my "decision tree" that will give suggestions on how we can try to re-write the T-SQL to make it faster!
Don't throw more hardware into the problem before attending this training!
NOTE: This training isn't about how to create indexes. Also, we won't work with InMemory or Columnstore technologies.Tchaikovsky Terrace (Herten Aas)Mon 09:00 - 17:00 -
Automation for DBAs and database developers
When I started as a developer and an accidental DBA, most of my administration routines and my deployments to production were done manually. Tests were more or less exclusively end-to-end-tests, performed by clicking in the UI.
Since then a lot of things happened. When DBAs used to take care a dozen SQL instances, they now have hundreds. Databases grew bigger and code is deployed to production much more often.
On this training day, I will share my experience with automation for DBAs and database developers. This includes:
- Automating SQL Server health checks using PowerShell, dbatools and dbachecks
- Database projects to enable CI/CD for SQL Server databases
- Azure pipelines for automation and scheduling
- Automated database tests using containers and database projects and tSQLt.Dvořák Dungeon (Begijnenzolder)Mon 09:00 - 17:00 -
Designing Data Warehouses For Modern Clouds
The move to cloud computing changes the ways you gather, organize, and query data. It is now possible to build solutions that were simply impossible 10-20 years ago. In this talk, Conor will go through the design process we use to develop Fabric Data Warehouse and how you can take advantage of the economics of cloud computing to solve your business problems more efficiently than ever.
The Orchestra (Auditorium)Tue 09:00 - 09:45 -
A Deep Dive into Azure Automation
Have you ever tried to schedule jobs against an Azure SQL Database? Or wanted to turn VMs on and off to save money? In this session you will learn about the ins and outs of the Azure Automation service. You will learn about how Automation works, how it can integrate with other applications like Logic Apps, and how you can execute code with VMs. In this demo heavy session, you will take home real-world code examples showing patterns for various patterns and problems.
Chopin Chapel (Dijlezaal)Tue 10:10 - 11:10 -
The Sixth Sense: Using Log Analytics to Efficiently Monitor Your Azure Environment
Log Analytics sees all, knows all, and is an ever-present help in any Azure environment. Azure Monitor logs and diagnostics can be collected from almost any Azure resource you could ever think of, and can help offer powerful insights on the performance metrics of your resources, keep track of changes and deployments in your environment and more!
This session will cover the benefits and capabilities of Log Analytics and the underlying Kusto engine, and how you can interactively analyse your logs and metrics using the Kusto Query Language (KQL), with demonstrations of how you can visualise resource usage over time, spot anomalies and trends and configure alerting that could be crucial to keeping your business-critical resources up and running. You'll have a better understanding of how you can optimise your Azure usage to manage and monitor your environment more efficiently.Mozart Mansion (Scala)Tue 10:10 - 11:10 -
Staying out of the datamess, unity catalog for data-mesh
In this session, we explore the integration of Databricks Unity Catalog within the Flemish government.
Let’s delve into how you can leverage Databricks Unity Catalog to effectively manage your data mesh and avoid the notorious data mess.
I’ll explain how we achieve flexibility while maintaining a unified management infrastructure across our data and analytics workloads by dividing our agency’s data into governed data domains.
These data domains, in turn, feed our data applications and AI models, providing valuable insights back into the core applications.
Furthermore, we’ll take a deeper look at how we utilize workspaces as the primary analytical unit within our agency. These workspaces serve as gateways that map to our data mesh domains, allowing us to break down data silos even across different government entities.Bach Ballroom (Alcazar)Tue 10:10 - 11:10 -
Fishing for Insights: Exploring Fabric OneLake and DirectLake Mode in Power BI
Microsoft's announcement last year of Fabric and its OneLake was a groundbreaking innovation, designed to transform the way organizations manage and analyze data.
But what exactly is the OneLake? And how can we use it from an analytical point of view? In this session, I will cover the basics of OneLake, explain how you can access it with the new DirectLake mode in Power BI. You will learn what the advantages are, but also when you probably should better stay with an imported data model!
Whether you're a business analyst, data scientist, or IT professional, this session will equip you with the knowledge to make the most out of Fabric OneLake and Power BI Direct Lake. It's paving the way for advanced data analytics and decision-making within your organization.The Orchestra (Auditorium)Tue 10:10 - 11:10 -
Embedding Power BI reports on your own portal
Power BI is one of the best reporting tools on the market, but can get expensive if you want to share it outside of your organisation with unlimited users. Power BI embedded reports are a solution for this problem.
In this session the ins and outs of Power BI embedded will be explained and shown. Everything from licensing to the API calls you need to write / code you need to provide.Tchaikovsky Terrace (Herten Aas)Tue 10:10 - 11:10 -
Saying no is OK
In the fast-paced and dynamic world of information technology, professionals often find themselves juggling numerous tasks, projects, and expectations. The common refrain is to always say "yes" to opportunities and requests, but this often leads to overwork, stress, and a diminished work-life balance. It's time to challenge this paradigm and recognize that "Saying No is OK."
This presentation explores the vital concept of setting boundaries in the IT industry. By providing real-world insights and practical strategies, it empowers IT professionals to assertively manage their workload, protect their well-being, and achieve greater success in their careers. The discussion will delve into the art of saying no, helping attendees to understand the significance of personal and professional priorities, and offering guidance on navigating work requests while maintaining professionalism. Through this engaging session, attendees will learn how to strike a balance between their work and personal lives, ultimately contributing to a more fulfilled and sustainable IT career.
Join me for an eye-opening journey that will change the way you approach your work in IT. Discover that Saying No is not just acceptable—it's a powerful tool for personal and professional growth.Dvořák Dungeon (Begijnenzolder)Tue 10:10 - 11:10 -
I have a real-time streaming pipeline, nice! What now?
Setting up real-time streaming pipelines has never been easier with Microsoft Fabric. Compared to other streaming solutions, it’s a breeze. You have developed and deployed your first streaming data pipeline but do not know how to handle the volume of real-time data? Eventhouse to the rescue! No more struggling with processing vast amounts of streaming data through your pipelines and storage.
This session will deep dive into efficiently using Eventhouse to manage and analyze the substantial amounts of streaming data you ingest. We’ll explore the possible ways of setting up a pipeline that captures event-based, IoT, and time series data, and analyzing the data using Eventhouse.Schubert Studio (Verloren Zoon)Tue 10:10 - 11:10 -
Become a meta data driven SQL Server DBA
I am a SQL Server DBA. I don't know how it happened, I just know I used to be a Windows developer and now I am a SQL Server DBA. This means I spent years and years learning how to use tools like SQL Server Agent and SQL-scripts to (sort of) automate tasks like index maintenance, backups, restore tests and more.
The same way I don't really know how or when I transitioned from developer to DBA, I'm not sure when it happened that I almost stopped using SQL Server Management Studio. Instead I find myself using PowerShell for just about everything SQL Server related.
In this session, I will show some examples of how to combine meta data repos in git, Azure Pipelines and self-service api's to manage a large number of databases in a large number of production and test environments. All of course driven by PowerShell and some fantastic modules developed by the PowerShell community.
The key is meta data. By documenting your servers and your SQL Server instances, automation and abstraction is possible. This gives you as an admin or DBA more freedom to move things around without ruining the day for your developers or users.Chopin Chapel (Dijlezaal)Tue 11:30 - 12:30 -
The importance of data testing in data engineering.
Unit testing and integration testing have become part and parcel of development processes the world over. So why don't we apply the same rigour to the data persistence layer?
We trust that our unit tests will catch bad data being sent to the data store, but what if they let us down? A great (or terrible, depending on your view) example of what can happen is the failed Horizon project at the UK Post Office.
I'll propose why we should, and how we might go about adding a robust testing mechanism for the data store which can help spot issues earlier in the SDLC before end users are involved and start reporting issues.Mozart Mansion (Scala)Tue 11:30 - 12:30 -
Effortless Query Optimization: Exploring Fabric SQL Engine Mechanics and Performance Monitoring
In this session, we will delve into the inner workings of the Fabric SQL engine to uncover what happens behind the scenes when a customer runs a SQL query. We'll explore the intricacies of query optimization and discuss how our solutions take the burden off your shoulders, ensuring optimized performance without the need for manual intervention. Additionally, we'll cover effective methods for monitoring query performance and identifying trends, empowering you to make data-driven decisions and maintain peak database efficiency.
Bach Ballroom (Alcazar)Tue 11:30 - 12:30 -
Multi-Layered Isolation Protecting Your Data in Databricks Serverless
Databricks Serverless helps you be confident that your sensitive data is protected. This talk will walk you through our security posture at every stage of a SQL query’s lifecycle, and how they pass through our Serverless Compute offering. We will discuss the various layers of data & access security, network security, and process security that Databricks employs in order to protect the Control Plane, isolate the Serverless Compute plane using defense-in-depth techniques, and secure your access to data. We’ll go deeper into compute pre-assignment, assignment, and runtime operations, so you get a glimpse into how we’ve managed to build a secure Serverless offering leveraging security best practices on top of existing cloud provider infrastructure, and are able to offer our customers a fast, efficient, and reliable system to run their toughest Data+AI workloads on Databricks.
The Orchestra (Auditorium)Tue 11:30 - 12:30 -
Level-Up your Power Query game with Custom Functions
Functions are one of the fundamental building blocks of Power Query. In addition to the native ones, we can create our own so that we don't have to write everything repeatedly. Custom functions can be one of your best friends for making your work easier in Power Query. But you need to be able to create them and set the input parameters, including whether they are mandatory or optional.
But once we create the function, it doesn't end there! To pass the function to someone or not forget what it does, it is good to know how to add documentation metadata that will be displayed in the preview of the function and used for quick understanding.
Start building custom functions and advance your Power Query capabilities.Tchaikovsky Terrace (Herten Aas)Tue 11:30 - 12:30 -
Talking about mental health for you and for them
Everyone experiences, or is adjacent to people who experience, challenges with mental health. In both the workplace and personal life, it can feel very difficult to talk with people about these challenges.
In this session, I will outline potential behaviours or changes in behaviour that may indicate mental health issues to help you to better identify them in yourself and in others.
I will also provide you with a set of tools to help navigate these uncomfortable conversations, making them easier for everyone involved. As a qualified and experienced "brain talker to" I cannot pass on all of my knowledge but I can help you to be able to better help those around you.
You will leave this session with some tools to be able to help yourself identify and deal with these challenges and a better understanding and knowledge of how to be most supportive for your family, friends and colleagues who are facing them.
You will also know that you are unlikely to make it worse !Dvořák Dungeon (Begijnenzolder)Tue 11:30 - 12:30 -
Design Thinking in the BI Product Lifecycle
There is no such thing like two exactly the same BI report or project. The requirements are always evolving, so does the business. Due to the knowledge and data democratisation trends, the stakeholder group is even more heterogenous now than ever before.
To address this problem, different approaches can be followed during product delivery, and Design Thinking method could easily establish a common ground where the users, the designers and the developers can collaborate to create data products beyond expectations.
During my session I would like to present:
• how Design Thinkin approach can be applied,
• when the wireframing method can be useful,
• where to start gathering feedbacks,
• which questions needs to be asked and interpreted,
• how establishing visual standards can support the process,
• how delivering simple products may have significant effects on data literacy,
• what are the different approaches to effectively train & onboard your users,
• how user adoptance can/should be tracked.Schubert Studio (Verloren Zoon)Tue 11:30 - 12:30 -
Another Brick in the Firewall: How to Secure your Azure Data Platform
These days, most people want their Azure Data Platforms to be deployed in a secure a network topology. As Data Engineers, we are often the ones that have the make this happen. The cloud has made it easy for us to deploy a virtual network here and a private endpoint there, but what does a good, networked data platform actually look like, and how does it work? Simple things become complex: how will my ADF Integration Runtime talk to my data sources? How do I securely access my resources to do development?
In this session we will look at some of the core network components which can be used to secure your data platform; what they are, and how to use them effectively. We will also look at some of the decisions that need to be made when moving your data platform inside a private network, which weren't a consideration previously. Some basic knowledge of what a virtual network is is required.
By the end of this session you should feel more confident in working with network components in Azure and how you can use them to secure your Data Platform.Chopin Chapel (Dijlezaal)Tue 13:30 - 14:30 -
Database DevOps...CJ/CD: Continuous Journey Continuous Disaster?
Transforming a team that was used to a SVN-based-big-single-repo work style and little to no automation was a bumpy ride.
Join me in this session where I share my experience in implementing Azure DevOps with Git, automated build and release pipelines and disposable personal databases.
In this session I will tell you about my days as team lead and the challenges I faced while introducing:
DevOps:
- The formal DevOps term and process
- Working with sprints (our 4th attempt)
Git:
- Git as version control to SVN / TFVC people
- Using branches instead of only the master
- Adapting a Git branching strategy
Pipelines:
- Adapting a Build & Release Workflow
- Implementing naming conventions for: repos, branches, builds and releases
- The Introduction (and success) of pipelines
- Manual vs Pull Request Release
Dedicated Environments:
- Working with a database IDP (internal development platform)
- With versioned personal databases
- Including stashing databases when priorities change
Migrating an existing team that is stuck in its ways, critical by nature and fed up with deadlines is an interesting journey you don't always hear about.
But in the end...it's a lot of patience, work and yes it's exhausting, but completely worth it.
Take-aways
1. Insights in a real world DevOps migration / transformation
2. Knowledge and experience sharing on the often forgotten DevOps part: the database
3. Beside all the talking: demonstration with tips you can directly make use ofMozart Mansion (Scala)Tue 13:30 - 14:30 -
Exploring the art of conditional formatting in Power BI
Join this session to explore Conditional Formatting in Power BI and see immediate improvements in how reports communicate data. I'll introduce effective techniques and uncover the strategic value behind them, enhancing your understanding of both the 'how' and the 'why.' You'll learn how to use this simple yet powerful feature to streamline complex information and make your reports not only visually appealing but also fundamentally more effective. Don't miss this opportunity to level up your data visualization skills!
Bach Ballroom (Alcazar)Tue 13:30 - 14:30 -
Fabric Warehouse Query Processor Deep Dive
Conor Cunningham will go through how query plans are created and executed in Fabric Data Warehouse, including where it overlaps with what you know from SQL Server and how it differs. Expect a fast-paced, demo rich session that will show you things you've not seen before.
The Orchestra (Auditorium)Tue 13:30 - 14:30 -
Knee-Deep In Tech Live @ DataMinds Connect
We'd like to record an episode of Knee-Deep in Tech live at DataMinds Connect! The episode is a 45 minute and recorded in front of a live audience. The audience will get to be part of the episode in several ways, and the hosts will take questions that will be discussed in the episode.
Tchaikovsky Terrace (Herten Aas)Tue 13:30 - 14:30 -
Multitasking for Mortals - How to Stop Multitasking and Start Getting Things Done
Only a monster has five heads. You don't.
Stop expecting yourself to multitask like one!
We often juggle multiple projects and challenges, trying to tackle everything at once. Unfortunately, this approach doesn't yield the desired results - simply because the human brain isn't built for true multitasking.
What can be done? You can either scale down, scale out or risk burning out.
By embracing better structure and practicing mindful, conscious context switching, you'll be able to maintain or even improve your focus across multiple areas. This is "multitasking" that works for the human brain.
An ounce of planning is worth a pound of result. Good planning reduces stress and increase your chances of achieving your goal. Let me show you how.Dvořák Dungeon (Begijnenzolder)Tue 13:30 - 14:30 -
Power BI CI/CD: Build your collaboration strategy!
Power BI CI/CD is here! The combination of Power BI Projects (PBIP), Tabular Model Definition Language (TMDL) and the Fabric Git Integration have enabled advanced collaboration and versioning for Power BI in all our developments. How do we now incorporate these separate features into our daily practice, how can you create a basic setup and which considerations do you need to make when implementing it for your organization?
The session focusses on the building and decision-taking that go hand in hand with defining and developing your collaboration strategy and how this compares to the building of a house. Each house starts at the foundations, for your Power BI CI/CD setup this is no different. Azure DevOps, your local Power BI Desktop and the Power BI workspace form the building blocks for your Power BI collaboration strategy. However, without a good plan the building blocks are nothing more than mere resources.
Therefore, we will be looking at how these tools come together to form the foundations of your Power BI CI/CD setup. An end-to-end demo will show you how you can go from your trusted local Power BI Desktop to the Power BI Workspace and on the way make use of Azure DevOps to improve your versioning, incorporate automated best practices checks and enable collaboration between your team members (without merge conflicts).
Lastly, I want to share different considerations you need to take into account when starting on the actual build of your collaboration strategy and setup. These practical considerations revolve around differential development lifecycles, restricted permissions, selective deployments and new possibilities for advanced Power BI automation.Schubert Studio (Verloren Zoon)Tue 13:30 - 14:30 -
SQL Server Infernals – A Beginner’s Guide to SQL Server Worst Practices
Are you faced with complaints from users, poor performing code from developers, and regular requests to build reports? Do you uncover installation and configuration issues on your SQL Server instances? Have you ever thought that in dire times avoiding Worst Practices could be a good starting point? If the answer is “yes”, then this session is for you: together we will discover how not to torture a SQL Server instance and we will see how to avoid making choices that turn out to be not so smart in the long run.
You are probably thinking: “Hey, wait, what about Best Practices?”. Sometimes Best Practices are not enough, especially for beginners, and it is not always clear what happens if we fail to follow them. Worst Practices can show the mistakes to avoid. I have made lots of mistakes throughout my career: come and learn from my mistakes!Chopin Chapel (Dijlezaal)Tue 14:50 - 15:50 -
A gentle introduction to Python
We will dive into the versatile world of Python programming. Discover the essentials of setting up virtual environments and harness the power of debugging tools to gain deeper insights into your code’s behavior. Explore the interactive landscape of Jupyter notebooks, and learn how to streamline your data analysis with powerful libraries such as pandas and matplotlib.
Mozart Mansion (Scala)Tue 14:50 - 15:50 -
Become the ADF pipeline guru in your organization
You have just begun your Azure journey and want to create your first pipeline and ETL-process in your new insights and analysis data platform.
This session will take you through how you can set up a basic pipeline and make it more sophisticated as your competence increases. I will walk you through static, dynamic and semi-dynamic pipelines making it easier for you to choose which is a better fit for your solution. And finally, I will show you how you can set up automated pipelines using scheduled triggers.
Let’s embark on your journey on becoming the ADF pipeline guru in your organization.Bach Ballroom (Alcazar)Tue 14:50 - 15:50 -
What's wrong with the Medallion Architecture?
In recent years, companies have seen an explosion in adopting lakehouses - with every analytics developer suddenly rebranding themselves as a Lakehouse Export... but time and time again, we hear from organisations that they regret the layering of their lake, and once it's in, its difficult to change!
Maybe the zones don't quite fit what they were trying to achieve, or no one in the company understands what "silver" vs. "gold" actually means, maybe they had to go back and tack in new layers so we expand into "Diamond", "Platinum" and..."Tin"? We need to tackle a key question: Is the Medallion Architecture right for most businesses - and how should you interpret the advice?
In this session, we'll break down the different stages of data curation and talk about how it works in reality, calling on practical examples from many, many real-world implementations. We'll talk about schema evolution, data cleansing, record validation, and traditional data modelling techniques, layering them on top of our lakehouse zones so we truly understand what happens where.
This session is ideal for data architects, engineers, and analysts looking to design the best platform possible, backed by nearly a decade of Lakehouse development, not a few months on a public previewThe Orchestra (Auditorium)Tue 14:50 - 15:50 -
Optimizing Power BI: Enhancing Performance Through Data Modeling
Do you want to enhance the performance of your dashboard? Experience shows that most performance issues in Power BI can be addressed with a few simple adjustments to the data model.
Therefore, this session will address the following questions:
- What constitutes a strong foundation for a data model?
- What are the advantages and disadvantages of different data models?
- How can you manipulate data to generate the right insights?
A robust data model can resolve many issues effortlessly, including:
- Even with limited expertise, you can boost performance with a few tips, resulting in faster loading times for your reports.
- Utilizing the appropriate data model makes it easier to craft DAX formulas, reducing complexity in the process.
By the end of this session, you'll have a clear understanding of what factors to consider. You can then apply this knowledge directly to your daily tasks.Tchaikovsky Terrace (Herten Aas)Tue 14:50 - 15:50 -
There is no talent shortage in Data & Analytics!
It is widely agreed that the talent shortage is one of the greatest challenges for businesses everywhere.
Data & Analytics teams especially struggle to find highly skilled professionals. With the need for more advanced analytics, new cloud technologies, and machine learning, demand has skyrocketed.
In this session, I will argue that the talent shortage might be a misconception and how a people-centric approach to recruiting and talent development can fix that.
We will also explore how every team member can avoid unintended harmful behavior, which can help create an environment that everybody is happy to work in.Dvořák Dungeon (Begijnenzolder)Tue 14:50 - 15:50 -
Effective Data Quality Checks and Monitoring with Databricks and Pandas
In today’s data-driven world, ensuring data quality is crucial for making informed business decisions. Poor data quality can lead to incorrect analyses, misguided strategies, and ultimately, financial losses. When data is not properly validated, it can contain errors, inconsistencies, and inaccuracies that undermine trust in analytics and reporting. This session will delve into strategies for assessing and maintaining high data quality using Databricks and Pandas, ensuring that your data remains a robust asset for your organization.
This session is designed for data engineers responsible for managing and maintaining data pipelines and ensuring data quality within their organizations. Attendees will benefit from practical insights and hands-on techniques for implementing data quality checks and monitoring strategies, all demonstrated through a detailed case study that serves as a red thread throughout the session. The session will cover essential aspects of data quality management, including data profiling upon initial ingestion, regression testing after software updates, and continuous monitoring. By focusing on tools like Databricks and Pandas, attendees will learn how to validate data types, perform statistical analysis, handle missing values and data duplication, enforce business rules, and set up effective monitoring systems.
Participants should have a working knowledge of Python and SQL, as the session will involve coding examples and practical demonstrations using these languages. The learning objectives include understanding how to perform data profiling upon initial ingestion, conducting regression testing after software updates to ensure data quality, and setting up continuous monitoring systems. Attendees will also learn to use visualization techniques like histograms, box plots, and scatter plots to detect anomalies, generate summary statistics and data quality reports, implement schema enforcement, adhere to naming conventions and regulatory requirements, and utilize tools like PyTest for automated testing and Grafana for real-time performance monitoring. By the end of the session, attendees will be able to maintain data quality and system performance effectively, ensuring their data is reliable and fit for business use.Schubert Studio (Verloren Zoon)Tue 14:50 - 15:50 -
Ground to Cloud: Your Migration Success Checklist
Cloud migration projects are often massive projects with sprawling scopes. Even with experienced project leadership it can be very difficult to know that all preliminary preparations are complete. Worse yet, with such a large and complicated project, it can be very difficult to know when the project has actually been successfully completed.
Don't you want some help sorting through this enormous pile of work? You need to zero in on the critical items that need to be prepared, executed, and completed to ensure a successful cloud migration project. Join an experienced cloud migration consultant for a targeted walkthrough of their cloud migration success checklist so you can make sure your project gets off to a good start and reaches a successful completion.Chopin Chapel (Dijlezaal)Tue 16:10 - 17:10 -
Get better (and actionable) alerts from your orchestration pipelines
Do you find the alert emails from Azure Data Factory (and related orchestration engines) too vague and unhelpful? Do you wish you could get more information and guidance on how to resolve the issues that trigger the alerts? If so, this session is for you.
You will learn how to customize and enhance the alerts from Data Factory with more context and even a call to action. You will see how to create alerts that are more informative and actionable.
When the session is over, you’ll have the skills to improve your own monitoring and alerts for data orchestration, no matter if you are working with data in Azure SQL, Databricks, Fabric or other systems.Mozart Mansion (Scala)Tue 16:10 - 17:10 -
EFSA implements a data mesh at scale with Databricks: practical insights
This presentation outlines the results of large, real-world implementation project undertaken by EFSA and delaware. In this project we focused on rolling out an elaborate governance model to enable different data engineering teams to develop, industrialize and share data products in a reliable manner.
Inspired by the data mesh principles, we established a decentralized governance model that gives the data engineering teams the autonomy to build further on each other's data products (centered around Delta Sharing for cross-team data sharing) while enforcing all teams to adhere to common policies and shared development best-practices (including the DevOps workflow).
This session caters to IT professionals, architects, and decision-makers looking for practical insights on how to realize a decentralized platform governance model in Databricks.Bach Ballroom (Alcazar)Tue 16:10 - 17:10 -
Optimizing Power BI Development: Unleashing the Potential of Developer Mode
Developer mode seamlessly integrates developer-centric features directly into Microsoft Power BI Desktop. This empowers users to implement source control for Power BI reports and semantic models, thereby boosting efficiency and enhancing the maintenance of Power BI projects. When combined with Azure DevOps and Fabric Git Integration, Developer Mode opens up unprecedented development-team collaboration possibilities, facilitating the establishment of robust Continuous Integration/Continuous Deployment (CI/CD) pipelines. These pipelines not only improve the reliability of Power BI projects but also enable scalability, ultimately elevating the development and collaboration experience to new heights. Join us to explore how these powerful tools can transform your Power BI development experience.
The Orchestra (Auditorium)Tue 16:10 - 17:10 -
From Spotify to Smarter Insights: Exploring Databricks' Intelligence Features
In this session, we will explore how Databricks' new Data Intelligence Platform can revolutionize data analytics and AI, illustrated through a use case that merges music and data engineering. As a data engineer by day and a musician by night, I will demonstrate how I created language-specific Spotify playlists using Databricks' AI capabilities to manage a vast song library without language metadata.
The session focuses on the “INTELLIGENCE” aspect of Databricks' rebranding from a data lakehouse to a data intelligence platform: Data Lakehouse + Generative AI = Data Intelligence Platform. Key features include Databricks Intelligent Search, AI Generated Documentation, Databricks Assistant, and AI Functions.
Using the Spotify example, I will guide you from idea to insights, demonstrating these features' practical applications. Additionally, I’ll provide a sneak peek into upcoming DatabricksIQ features from the latest Data + AI Summit.
I will also draw parallels to common business problems, such as analyzing large volumes of unstructured data to derive actionable insights. This illustrates how Databricks' AI capabilities can provide scalable solutions to real-world challenges.Tchaikovsky Terrace (Herten Aas)Tue 16:10 - 17:10 -
Gender, beyond 0/1
Between 0.5% a 5% of people does not identify with the gender they were assigned at birth. Chances are you work with someone, or will work with someone in the future, who is not cisgender, and is transgender and/or nonbinary.
Do these sentences already confuse you and use terms you do not know? Want to learn what all the letters in the LGBTQIA* acronym stand for? And how to create a welcome environment at your workplace for queer folk of all shapes and sizes? Come to this talk! Feel free to ask the questions you were always afraid to ask (as long as they're asked with an open mind and with respect).Dvořák Dungeon (Begijnenzolder)Tue 16:10 - 17:10 -
Fabric adoption roadmap: Napoleon's success story
Adopting fabric, Power BI or any other tool or way of working in your organisation is difficult, but the MS roadmap provides a great guide for it. It was even a success before it existed, Napoleon implemented it in his rise to power and dominance. But he also made mistakes, let's see how a 200 year old conqueror measures up to the roadmap and what we can learn.
Schubert Studio (Verloren Zoon)Tue 16:10 - 17:10 -
Technical Keynote - Azure Databricks
The keynote will cover the highlights from the Data and AI Summit, as seen through the eyes of four different personas — data engineer, data steward, data scientist, and data analyst—showcased via a compelling end-to-end technical demo.
Silviu, the trigger-happy data engineer, will make data ingestion and transformation easier through a simple live demo of automated jobs and serverless compute - demystifying orchestration and requiring little help from any additional cloud platform teams
Marius, the slightly paranoid data steward, will show the audience how to ensure adequate data governance controls are in place, without slowing down the other team members, via Attribute Based Access Controls(ABAC) in Unity Catalog
Athulya, the Gen AI enthusiast, will demonstrate how to integrate this enterprise data into the AI lifecycle using Mosaic AI, in a secure and cost-effective manner.
Liping, the newbie data analyst, will demonstrate a generative BI experience in Databricks using Genie and how to utilize AI functions in SQL.
After this keynote, the audience should have a better understanding of what the recently announced Databricks functionality means and how it could impact the day-to-day work of people within their organization.The Orchestra (Auditorium)Wed 09:00 - 09:45 -
How the length of character datatypes influences memory grants
How does a character datatype impact your query performance?
You could perfectly use a NVARCHAR(MAX) to store a bit value, but is this a wise decision?
Is there a relation between your column length and memory grants in SQL Server? How about CPU usage? I'll clearly show you the impact with a few examples. Furthermore I'll show you how the engine will (tries) to save you.Chopin Chapel (Dijlezaal)Wed 10:10 - 11:10 -
Tackling the Gaps & Islands Problem with T-SQL Window Functions
The gaps & islands problem is a common problem in data analysis. Some examples are:
* how long are active periods that my server was running? Or, how long are the periods that the server was down?
* on average, how long are the billable periods of my consultants?
* what was the longest stretch of days without rain?
In this session, you'll be introduced to the powerful concepts of window functions in T-SQL. With a practical use case, the gaps & islands problem is explained and solved. At the end of the session, you should feel confident in using window functions in your own data analyses.
A basic understanding of T-SQL querying concepts is required to follow this session.Mozart Mansion (Scala)Wed 10:10 - 11:10 -
Optimize your Azure Data & AI platform for peak performance: rethink security and stability
As businesses increasingly migrate to cloud platforms, many mistakenly view the process as a straightforward lift-and-shift operation. This common misconception often results in the unplanned transfer of data to cloud storage without adequate preparation, thereby neglecting essential aspects such as performance, stability, security, and continuous monitoring. Such oversights can lead to suboptimal cloud environments that are vulnerable to security breaches, performance bottlenecks, and instability.
In this session, participants will delve into the critical importance of thoughtful architecture and strategic planning for cloud migrations. The presentation will provide actionable insights into implementing comprehensive security measures, continuous monitoring, and performance optimization specifically within Microsoft's Data & AI Azure data platform. Through real-world examples and best practices, attendees will learn how to transform their cloud migration approach to ensure a robust, secure, and high-performing data platform.Bach Ballroom (Alcazar)Wed 10:10 - 11:10 -
What Fabric means for your Power BI semantic models
Microsoft Fabric brings many new features that will change the way that you build and use your Power BI semantic models (formerly datasets). To get the most from Power BI, you should know what these features are, as well as how and when to use them.
In this talk, I’ll discuss:
- What a Power BI semantic model is
- Why it's so important in Fabric
- Three example scenarios to illustrate how different teams choose and use the new Fabric features to build and use their semantic models to address pains and problems that they have in Power BI today. I'll also highlight the considerations these new features bring.
In this talk, it's assumed that you've already heard the basics of Fabric features like Direct Lake storage mode, semantic link in notebooks, or Git integration. I'll focus on the "how and why" of these features, over the "what".
By the end of this talk, you'll have a better understanding of how you might approach these new features for the semantic models you manage today, or will make in the future.The Orchestra (Auditorium)Wed 10:10 - 11:10 -
Can Generative AI make the world more Inclusive?
We hear about the dangers of Generative AI. Generative AI can exclude people. The system can generate bias and it's possible to strengthen the current stigma people from diverse backgrounds may have. But for this talk, let's reverse the question: Can Generative AI make the world more inclusive?
As a tech-optimist I would say yes. But you don't need to believe me. Instead of a blind believe in the yes, I give you some interesting cases where Generative AI limits bias and makes the world more inclusive.
- For people without formal education, Gen AI can explain theories;
- For people with dyslexia, Gen AI can write stuff in better wordings and grammar;
- For people with autism, Gen AI can help them to understand social situations better;
- To close the gender gap, Gen AI can help with writing or transforming text in a gender-neutral way;
- For people with a visual impairment, Gen AI can help with reading text;
- For people with migration background, Gen AI can help with translating text;
In this talk we are going to explore some use-cases how Gen AI can make the world more a more inclusive and better place for everyone.Tchaikovsky Terrace (Herten Aas)Wed 10:10 - 11:10 -
Help! My data product is invisible.
How do we apply the Scrum framework when our product is as elusive as data? What could be more captivating than showcasing a SQL query during your demo? Perhaps a new Excel export with 4 additional columns... the epitome for an increment of value!
We'll be sharing the ingenious solutions and ideas we've crafted for various events and artifacts to enhance the appreciation of the team's efforts and to captivate stakeholders' interest for a thorough inspection.
Enter the dynamic duo: a married couple, united in life but diverse in their professional domains. She, the adept Scrum Master steering a data team within a banking institution; he, the visionary chapter lead in the energy sector. Despite similar contexts, their distinct personalities offer complementary perspectives.Dvořák Dungeon (Begijnenzolder)Wed 10:10 - 11:10 -
Developing user focused BI-dashboards for business adoption
Many organizations have embraced data reporting to foster a more data-driven culture and democratize data. However, ensuring maximum impact and usability for the organization’s investments requires more than just focusing on the data. This talk will guide you through a comprehensive end-to-end process, addressing challenges in data reporting adoption. It covers not only the steps taken before development, such as empathizing with end users, but also often overlooked post-launch steps like training and support. These steps are vital to ensure your dashboard lands successfully within your organization.
Schubert Studio (Verloren Zoon)Wed 10:10 - 11:10 -
Monitoring Fabric, make do, buy or build
Monitoring Fabric, make do, buy or build
In the ever-evolving landscape of Fabric tenant management, obtaining a holistic overview is of paramount importance.
There are several ways you can monitor the content of your Fabric tenant and the activities of your users. Microsoft has some things out of the box such as the Monitoring Hub and the Capacity Metrics app. You can also buy ready-made solutions from different vendors, or you can build your own solution.
What you choose depends on many factors but most importantly what do you want to get out of the solution.
In the first half to this session, we will cover what the built-in capabilities cover, we will talk a little about why you might buy a ready-made solution (we will not compare different solutions on the market) and then we will talk about how you can build your own solution.
The second half of the session will focus on what you need to build your own solution and how you can take advantage of things the community has created. We will look at solutions build in Fabric Data Factory as well as coded solutions build in Fabric Spark.
At the end of the session the audience will understand how to be sure if the built-in capabilities are enough, if buying is right for them or if they should build their own solution. They will also know what it takes to build your own solution and how to jumpstart the process by using solutions the community has made publicly available for free.Chopin Chapel (Dijlezaal)Wed 11:30 - 12:30 -
Approximate functions: How do they work?
Sometimes, a close approximation is good enough. And sometimes, a close approximation is a lot faster. Microsoft has introduced “Approximate Query Processing” (the APPROX_COUNT_DISTINCT and APPROX_PERCENTILE functions) to give you exactly that benefit when you don't need exact answers.
But do you have a good response when you propose to use this function and your manager asks you to explain how they work first? Or is your only option to claim "black magic by smart Microsoft engineers"?
The algorithms used are not a secret. HyperLogLog and KLL Sketch. And now you most likely know exactly as much as you already knew before. And when you google for those terms ... you end up with a headache.
Time to join me for a session where I explain the black magic in the simplest possible terms, so that you can then explain it to your manager!Mozart Mansion (Scala)Wed 11:30 - 12:30 -
Cost control and cross charge on Databricks using Power BI template
Prior to the introduction of system tables, the cost management and cross charge of Databricks platform can be complicated. This session will cover the structure and content of different system tables and how they can used to make cost management efficient and provide audience with an example visualization of Databricks cost management and cross charging in the form of a Power BI template.
The session is aimed at Databricks admins and budget holders. This is a beginner to intermediate session with no prerequisite. Audience will walk away with a good understanding of the data model of Databricks system tables and a Power BI template to visualize cost and cross charge of Databricks spending with ease.Bach Ballroom (Alcazar)Wed 11:30 - 12:30 -
Getting more from your data with Power BI Copilot
During this session you will learn all about what's available and up and coming in Power BI with Copilot! Learn how you can harness the power of AI to get more from your data. You'll learn about a variety of Copilot experiences, including creating a page, getting answers from your semantic model, using Copilot to help with DAX and more!
The Orchestra (Auditorium)Wed 11:30 - 12:30 -
Data Modelling Showdown: Unravelling the Mysteries of Different Data Modelling Approaches
Some people and organisations have a preference for a data modelling approach, whether that's a Kimball Star Schema, Data Vault, One Big Table, or something else. Lots of people and organisations don't know where to begin with data modelling, let along have a preferred approach.
In the world of data management and analytics, choosing the right data modelling approach is a critical decision that significantly impacts the efficiency and effectiveness of data-driven projects. Join us for an exhilarating "Data Modelling Showdown: Unravelling the Mysteries of Different Data Modelling Approaches!" as we explore the diverse landscape of data modelling techniques.
In this dynamic session, we will embark on a thrilling journey through various data modelling methodologies, including Dimensional, Data Vault, Graph, and more. This session will demystify the unique strengths and use cases of each approach, providing a comprehensive understanding of when to leverage them for optimal results.
Whether you are a seasoned data professional seeking to expand your modelling toolkit or a data enthusiast eager to unravel the mysteries of data modelling, this session promises to be an eye-opening and empowering experience. Don't miss the chance to join us for this exhilarating showdown, and discover the best data modelling approach for your next data-driven endeavour!Tchaikovsky Terrace (Herten Aas)Wed 11:30 - 12:30 -
Why Am I Here? Dealing with Imposter Syndrome
Have you ever questioned your abilities? Think that everyone else is so much better than you? Consider that your current role or successes are down to luck or something you did not do. You are not alone; it is estimated that 70% of the population struggles with these feelings. Maybe you have felt this way sometimes ;
You attribute success to luck or external factors rather than your skills and abilities.
Excessive focus on mistakes and shortcomings while minimizing or discounting achievements
Feeling like you don't belong in your current role or that others are more qualified
Avoiding challenges or opportunities due to fear of failure or being exposed as a fraud
A persistent sense of fear and anxiety related to your work or personal endeavors
The Imposter Phenomenon is something that I deal with every day. This session is about what I have discovered about the Imposter Phenomenon. What strategies and techniques do I use to deal with all the feelings that come with my Imposter Phenomenon?
This session is not about a magic solution or a cure. I will share techniques, ideas, suggestions, and thoughts on the Imposter Phenomenon. Please share some of my struggles and experiences. If you want to know that it's not just you, come along and listen. You may leave with a different way of looking at things.Dvořák Dungeon (Begijnenzolder)Wed 11:30 - 12:30 -
A journey from classic data warehousing to data engineering based on a customer story
The technology of the data landscape is changing fast. Concepts stay the same. Or are concepts changing too? How does a classic data warehouse team with BI, ETL & SQL adapt to Python and modern data engineering? And how does your team adapt and how do you stay relevant to the business?
This session will explore the journey that the internal data warehouse team at Boskalis, a Dutch and “leading global dredging and offshore contractor and maritime services provider” underwent, turning towards data warehousing AND data engineering.
We will focus in this session on the team aspects like upskilling and leadership, organization dynamics and the complexity of handling changing technology stacks, while still delivering business value.
This session does cover parts of the technology stack used (Azure Data Platform & Azure Databricks), but the focus is on organizational aspects and business value.
When you walk away after this session you will know:
* How Boskalis went from a classic data warehouse team to a team that includes data engineering
* What challenges there were, and which solutions were applied
* Practical learnings that will give you handholds to grow to data engineering for you and your team
* How the changing technology stack was handledSchubert Studio (Verloren Zoon)Wed 11:30 - 12:30 -
With great power comes a great bill: optimise, govern and monitor your Azure costs
Azure allows us to easily create an enormous amount of powerful resources and features. However, that presents its own problems. You might unknowingly create a resource, or enable a feature that shows up on the top of your costs reports very quickly, or grows uncontrollably over time.
The first goal of this presentation is to provide guidelines to prevent a costs crisis in your organisation.
The second goal is to better understand the cause of your currently growing costs, to monitor and to reduce them.
We will cover resources that are very commonly used such as Data Factory, Databricks, SQL Databases, Synapse, Fabric, Log Analytics, Data Lake, Virtual Machines etc.Chopin Chapel (Dijlezaal)Wed 13:30 - 14:30 -
What makes a good commit? Good Practices for Data Professionals
Committing code is an integral part of the development lifecycle. However, not all commits
or commit messages are created equal. As more data professionals are required or expected to use source control within a continuous integration or deployment solution, a lack of good information is reducing the efficacy and usefulness of commit messages.
In this session we will explore the source control system git, what makes a commit and what a commit message is for.
We will also learn what makes a good, and a bad, commit message and how to automatically link commit messages to User Stories and release notes.
You will leave this session with a better understanding of commits and commit messages and useful guidance to make them most effective for yourself and for your team.Mozart Mansion (Scala)Wed 13:30 - 14:30 -
Real-Time Hub: Starting Point of Real-Time Intelligence in Fabric
If you ask a user how often they need data their initial answer is often "in real time", right? Once you solve for getting them that data in real-time, then where would you store it? Once it was stored, what tool or tools would you use to visualize it? Once that ecosystem is set up and stable, how does it interact with the data warehouses and data lakes in your estate?
The Real-Time Hub is your starting point for building real-time applications in Fabric. It's a kind of action center for bringing real-time events from a variety of sources (and clouds) into Fabric, learning how to analyze - and act upon - that data, storing it, and then visualizing it in real-time dashboards. In this session, we will explore what you can do with Real-Time hub and what real world scenarios you can unlock based on our real world experiences. Following on from there, we'll be using some real-time sample data sources provided within the hub to show you how to easy it is to pull in some real-time data and see all the capabilities that Real-Time Intelligence has to offer you.Bach Ballroom (Alcazar)Wed 13:30 - 14:30 -
The Power of Paginated Reports
Power BI reports are intended for interactive use. But what if you need to produce a nice PDF, Word or printed report? That is where paginated reports come in!
These reports can have a dynamic number of pages, have better options to nest visuals inside one another, and support way more exporting options than regular Power BI reports, and the cost of being less interactive.
Come and see for yourself in this demo-rich session if paginated reports could help you build the right reports for the right audience.The Orchestra (Auditorium)Wed 13:30 - 14:30 -
Goodbye visual shite, hello vega-lite
Break free from the restrictions of Power BI core visuals and enter a world where you have complete control of every element of your data visualisations.
With Vega-Lite and Deneb, you will enter a world of possibilities, walk down the path towards date visualisation awesomeness and discover a world free of the restrictions of clicky clicky draggy droppy datavis.
But be warned, to enter the world of Vega-Lite and Deneb is to learn the possibilities which cannot be unlearned, and will be your first steps to becoming a data visualisation pilkunnussija.Tchaikovsky Terrace (Herten Aas)Wed 13:30 - 14:30 -
Data Strategy: Your First-Class Ticket to Data Nirvana
A well-designed and well-managed data platform can significantly amplify an organization's ability to transform data into business value. However, substantial technological investments are no magic solution for bureaucratic data processes, an inefficient data team structure, or the lack of data knowledge and appetite within your organization. This is why even the most advanced data platforms are often underutilized.
This talk explores how you can break this pattern and shift towards a "Data Nirvana" where people, processes, and data tools live in harmony. The secret to getting there? A robust data strategy! I will introduce a data strategy framework that can empower individuals within your organization (the "who") to become more data literate by aligning your company’s goals (the "why") with "the what"—whether it be technological components like artificial intelligence or data platforms, as well as process and people-driven data management initiatives.
But describing and defining your "Data Nirvana" is only the beginning. You also need to convey your data strategy to organization-wide stakeholders, ensuring broad understanding and support. I will show how storytelling techniques can accelerate the adoption of your data strategy.Dvořák Dungeon (Begijnenzolder)Wed 13:30 - 14:30 -
dbt and Fabric - a match made in heaven?
Not too long ago, data build tool (dbt) has taken the data world by storm. With official support for Fabric, is it the right time to also get to grips with dbt?
In this session you will learn what dbt is - the good, the bad and the ugly. You will see what adopting this part of the modern data stack can offer you. After attending the session you will be able to start using data build tool right away, whilst also having the knowledge to combat data quality and governance issues all within Fabric and your favorite code editor.
Prerequisites: basic Fabric and basic SQL knowledge are recommended.Schubert Studio (Verloren Zoon)Wed 13:30 - 14:30 -
Disaster Recovery Strategies for SQL Server
"Hope is not a strategy" - Google SRE Team motto
Being able to recover our data in the event of an outage is a (if not the most) critical aspect of managing SQL Server. But how do we plan for the worst? What technologies are available to us? And how do we implement a disaster recovery strategy when the worst has just occurred?
Join Microsoft Data Platform MVP, Andrew Pruski, in this session to delve into SQL Server disaster recovery strategies.
We'll start by defining the difference between high availability and disaster recovery. They are often combined together but they are different, more like two sides of the same coin and that affects our choices when picking a technology to implement.
But before we start to implement any DR technologies we need to define a few things: -
- What is the maximum amount of data that we can afford to lose? (RPO)
- How long do we have to recover our SQL Server instances? (RTO)
- Where is our DR site located? Is it in a remote data centre or in the Cloud?
Then we'll look at the different technologies available to us in SQL Server, how can they help us define our disaster recovery strategy? Each option has a different implementation and we'll look at: -
- Backup/Restore strategies
- Log Shipping
- Mirroring
- Always On Availability Groups
After we have defined the technology we'll look at how to test our disaster recovery plan. We'll cover: -
- Alerting, when do we need to push the big red button?
- Enactment procedure, who will initiate the DR fail over?
- Fail back, how do we get back to our main production environment?
This session is for anyone who works with data and wants to learn more about how to implement a resilient disaster recovery strategy.Chopin Chapel (Dijlezaal)Wed 14:50 - 15:50 -
X-Raying Schema Operations: Adding and Removing Columns
As database developers and/or administrators, we often encounter the need to make schema changes to tables. These changes involve various operations, such as adding or removing columns.
Understanding the inner workings of these operations may be crucial. Why are some changes quick and seamless, while others can significantly impact system performance?
In this session, we'll delve into SQL Server internals to explore the processes involved in adding and removing columns. We'll examine the intricacies of these operations, including their utilization of the transaction log, the locking system, and space management. This exploration will unravel their impact (or lack thereof) on system resources and performance.
Is adding a new column always a slow and a blocking operation? And what about dropping a column? Will this single action immediately reclaim the used space? Let's explore these and other questions together.
By gaining insight into these specific examples, you will be better equipped to manage similar schema changes effectively in your database environments.Mozart Mansion (Scala)Wed 14:50 - 15:50 -
Medallion for Data Mesh: Exploring Workspace, Capacity, and Domain Design in Microsoft Fabric
Designing a medallion architecture is more than just setting up a few Lakehouses and Data Warehouses. In this session, data architects and engineers learn how to design a medallion architecture with Fabric Workspaces, Capacities, and Domains. We’ll cover the segregation of responsibilities, granular access control, and how to pick and choose the right Capacities for every workload. By the end of this talk, you’ll be ready to design a scalable and secure data platform with Microsoft Fabric.
Bach Ballroom (Alcazar)Wed 14:50 - 15:50 -
Power BI refreshes - reinvented!
Have you always been scheduling your Power BI refresh in the service? Struggling to get your data available in your model in a short amount of time? Then this session is for you!
During this session, you will learn advanced patterns to optimally configure and refresh your Power BI data models in the service. We will not only discuss incremental refresh, but also how you can get your data available in your model in (near) real time.
We will discuss various patterns from high frequency refresh operations, single table or partition refreshes, detecting data changes in archived partitions and more advanced patters.
After this session you will be able to;
- Configure incremental refresh.
- Refresh individual tables and partitions in a published Power BI model.
- Connect your Power BI refresh to other processes and orchestrate centrally end-to-end.The Orchestra (Auditorium)Wed 14:50 - 15:50 -
Start 2 MLOps: From the lab to production
Welcome to the world of MLOps!
In this session we will start from a black-box AI model. Together we will explore how run this model in production in a sustainable and reliable way. The different ideas we uncover will be cross-checked with the industry standard: The CRISP-ML(Q) framework. Once we know what to do and where we stand in the model life cycle we are ready to get to the real work. Based on the different ideas we will look at the commonly used tools for the trade and start building simple MLOps recipes. These recipes will be your solution accelerators for all ML projects to come.
By the end of this session you are ready to embark on your first steps as a collaborative MLOps superhero. You know what MLOps entails and how to get a basic setup going.
All code and documentation will be shared on the Plainsight GitHub page.Tchaikovsky Terrace (Herten Aas)Wed 14:50 - 15:50 -
Flight Planning - How Governance Can Save You a Fortune
Why is it that cloud projects often end up way more expensive than they should? The answer might surprise you: a lack of governance.
It is easy to get stuck on the technical aspects of a cloud project and hence try to find equally technical ways of saving money. There is no such thing as a free lunch, and seldom is that clearer than in an environment where you pay for every second of compute consumed. Making a query run faster, optimizing a pipeline, or storing a bit less in a data lake, can decrease your overall consumption costs, but there might be even bigger savings lurking just out of view.
The solution for saving the big bucks is spelled "governance". Most people see governance as something tedious, boring and ultimately futile, as users will inevitably find a way around the governance limitations and romp all over the environment. But if we look at governance as a tool for enablement, a way to make work easier for the end user and the administrator alike, as an opportunity optimize spend - then the whole dynamic shifts. Wouldn't it be amazing if we could combine useful guardrails, increased productivity AND optimized cloud spend? Let me show you how!
This session will look at unexpected drivers of cost, discuss ways to surface (and mitigate) some of them, and try to convince you why governance is really the best way to save money in the long run. You might even be able to take some of the learnings from this session and apply it to your on-prem environment as well!Dvořák Dungeon (Begijnenzolder)Wed 14:50 - 15:50 -
Beautiful and Smart: A User-Centric Approach to Data Visualization.
1. General Do’s and Don’ts in Visuals.
2. Learn how to go from a user story to choosing the right visual type.
3. Show the evolution of a visual from a standard Power BI default visual, to a visual that:
- Tells the intended message.
- Brings actionable insights
- Corresponds to the user story.
- Satisfies all rules of best practices visuals.
- Has the correct visual type.Schubert Studio (Verloren Zoon)Wed 14:50 - 15:50 -
The Renaissance of Microsoft Purview
In this session we discuss the latest and greatest of Microsoft Purview. In the spring of 2024, new features were released that make Microsoft's Data Cataloging solution a more complete and business-focused product. In the forest of data catalog solutions, can Purview finally be the big tree that stands tall above all others? Join us to find out!
Chopin Chapel (Dijlezaal)Wed 16:10 - 17:10 -
Protecting and working with sensitive data in Azure
When it comes to working with sensitive data inside any data platform environment you need to make sure you have your security and privacy measures in order. This is especially true when you move sensitive data into a cloud platform like Azure where the geographic location and the access to your data changes.
Thankfully Azure provides many options that we can use to secure, monitor and protect our sensitive data. In this session we are going to explore a number of these options, ranging from authentication and authorization to row-level security and dynamic data masking. Next to these various technical measures, we are also going to explore more policy-based ones, like data sharing agreements and data governance.
So, if you are already working with sensitive data in Azure - or planning to - this session will help you decide which options best fit your situation, how to get started with their implementation and measure their impact on your use-cases.Mozart Mansion (Scala)Wed 16:10 - 17:10 -
Microsoft Fabric: Building a Data Ingestion and Processing framework to Drive Efficiency
Data pipelines are essential for moving and transforming data between different systems. However, managing a large number of data pipelines can be challenging and time-consuming. How can you ensure that your data pipelines are efficient, reliable, and consistent?
In this session, you will learn how to use a metadata-driven approach to manage your data pipelines and Notebook in Microsoft Fabric. Metadata is data about data, such as source, destination, schema and format.
By using metadata to define and control your data pipelines, you can achieve the following benefits:
1. Simplify and automate the creation and execution of data pipelines
2. Optimize the performance and scalability of data pipelines
3. Monitor and troubleshoot data pipelines
We will show you how to implement a Data Ingestion and Processing framework based on the Medallion Lakehouse architecture. We will also share the key learnings, best practices, and patterns that we have discovered from applying this framework in our own work.
All code used during the demo will be shared afterwards, so you can start building a framework directly after the sessionBach Ballroom (Alcazar)Wed 16:10 - 17:10 -
DAX fundamentals: what is filter context?
This session aims to demystify the concept of filter context, illustrating how it interacts with other contexts such as row context. We will explore how DAX processes and modifies filter context through various functions and expressions, demonstrating with how it influences the behavior of measures.
This will be a theoretical session, looking to understand the specific theory and how filter context works under the surface.
Attendees will learn how to effectively manipulate filter context to build more intuitive and responsive data models. The session is designed for intermediate users looking to deepen their understanding of DAX and enhance their skills.The Orchestra (Auditorium)Wed 16:10 - 17:10 -
The Sidekick's Guide to Supercharging Power BI Heroes
Step into "The Sidekick's Guide to Supercharging Power BI Heroes," a transformative session crafted for those who operate behind the scenes, making Power BI heroes shine. This engaging workshop is dedicated to unveiling the synergistic powers of C# scripting and Calculation Groups, equipping you with the capabilities to automate, optimize, and revolutionize Power BI environments. As the unsung hero, you'll learn the ins and outs of scripting from scratch, enabling dynamic and efficient data model adjustments that respond in real-time to the shifting landscapes of business intelligence.
We will navigate the path of creating calculation, crafting measures that encapsulate complex logic with simplicity, and enhancing reports with rich, customizable formatting. Each step is designed to arm you with the tools necessary to support Power BI power users, turning intricate data challenges into straightforward narratives of success.
Beyond mere technical skills, this session delves into the philosophy of empowerment, teaching you how to foster an environment where every user feels equipped to explore data with confidence and creativity. By mastering the art of support, you not only elevate the capabilities of your team but also contribute to a culture of innovation and insight.
Ready to transition from the background to the forefront of BI success? Join us and redefine what it means to empower and engineer in the world of Power BI.Tchaikovsky Terrace (Herten Aas)Wed 16:10 - 17:10 -
A Picture is Worth a Thousand Words
Working in technology we're used to architecture pictures, that can be an efficient means of communication. You might have also come across one where you have to follow a jumble of lines trying to make sense how things fit together! So what makes a picture a good way of communication?
In this session we will dive into how to use pictures as a means of communicating a technical topic. We will look at the different contexts of presenting on a topic from discussing a topic with a colleague to making an architecture picture for documentation. Through concrete samples we will start to gain understanding of what makes a picture really worth a thousand words! Join this session to elevate your skills of creating better images to convey technical knowledge, whether using a tool or drawing by hand.Dvořák Dungeon (Begijnenzolder)Wed 16:10 - 17:10 -
Leverage Power Query to get data from Rest APIs
Unlock the full potential of Power BI by mastering the art of leveraging Power Query to seamlessly connect and retrieve data from REST APIs. In this session, we will dive into the intricacies of using the Power BI API to empower your data analysis and visualization endeavors.
1. Introduction to Power Query: Gain a solid foundation as we explore the fundamentals of Power Query, understanding how it forms the backbone of data transformation within Power BI.
2. Unveiling the Power BI API: Discover the capabilities of the Power BI API and its role in obtaining dynamic data from REST APIs. Learn how to harness its power to enhance your data connectivity.
3. Step-by-Step Guidance: Follow along as we provide comprehensive, step-by-step guidance on integrating the Power BI API into your Power BI workflows using Power Query. Acquire the skills needed to effortlessly retrieve, transform, and load data from diverse sources.
4. Practical Examples and Use Cases: Illustrate your newfound knowledge through practical examples and real-world use cases. Understand how to navigate common challenges and optimize your data extraction process.
5. Best Practices and Optimization Tips: Unearth best practices and optimization tips to ensure efficient and streamlined data retrieval processes. Learn how to troubleshoot common issues and enhance the performance of your Power BI queries.
At the end of this session we want you to be able to takeaway:
1. Proficiency in utilizing Power Query with REST APIs.
2. Practical insights into leveraging the Power BI API.
3. Enhanced skills in transforming and loading data seamlessly within Power BI using APIs.Schubert Studio (Verloren Zoon)Wed 16:10 - 17:10