Microsoft Fabric and Data Analytics

In this blog post we’ll explore Microsoft’s data platform and its capabilities and provide insight into how Softcat has been supporting customers with Fabric.

What is Microsoft Fabric?

Microsoft describes Fabric as…

'An end-to-end analytics and data platform designed for enterprises that require a unified solution. It encompasses data movement, processing, ingestion, transformation, real-time event routing, and report building. It offers a comprehensive suite of services including Data Engineering, Data Factory, Data Science, Real-Time Analytics, Data Warehouse, and Databases.'

Microsoft Fabric at Softcat

Since Microsoft Fabric’s exciting public preview announcement in May 2023, Softcat has worked to gain the expert knowledge needed to educate, inform, and provide consultation to our customers.

Many customers shared the same excitement and enthusiasm as Softcat, but naturally there were some reservations. Some of the most common initial questions around Fabric were:

· How will this affect our current Azure Data Platform?

· Will Microsoft continue to support existing Synapse and Data Factory resources?

· Where will my data be stored and what are the security risks?

· Which experience should I use? How should I architect my solution?

·  How much will it cost per month?

​​​​By having an open dialogue with our customers around their reservations, Softcat has gained a better understanding of their pain points and how we can help solve them. By arming our experts with knowledge of what’s on Fabric the roadmap, we’ve been able to provide customers with the assurance they need to take advantage of Microsoft Fabric.

Customer engagements

As well as providing consultations, demonstrations and Q&A sessions, Softcat has worked on many Fabric engagements, covering the following subject areas:

· Utilising the Fabric (F64) free trial for customers to create proof-of-concepts / pilots to enable them to gain firsthand experience of Fabric’s capabilities.

· Transitioning the PoC / Pilots into Production and providing Capacity recommendations and purchase.

· Providing consultancy and choosing the appropriate Fabric experience based on requirements and other related aspects such as customer skillset.

· Creating solutions that adhere to industry architecture, such as Data Domains, Medallion, etc.

· Set-up and administration of Microsoft Fabric, configuring Domains, Sub-Domains, experiences such as Data Factory, Lakehouse and Warehouses, and Items.

· Advising on Workspace Design and Security best practices including Row-level security for both Data Warehouses and semantic models.

· Installing Fabric’s Capacity App and providing an overview of other Admin reports such as features usage and an adoption report.

· Implementing source control, git integration and ALM (Application Lifecycle Management) deployment pipelines.

· Data Governance, sensitivity labels, certifying and endorsing data.

· Scheduling and monitoring Data Factory pipelines in Monitoring Hub.

· Creating business-ready semantic models.

· Training, documentation and handover.

Fabric experiences

Fabric consists of several components known as experiences. Each experience is tailored to a distinct role and serves a specific task. Fabric allows these experiences and workloads to work seamlessly together, providing deeply integrated analytics across the entire organisation. Each experience is easy to learn, accessible and reusable.

The diagram below shows the different experiences and illustrates how, in unison, they build the Intelligent data foundation with OneLake.

​​​​​​

Data factory

· Provides a modern data integration experience to ingest, prepare and transform data from data sources through Data Pipelines and Data Flows.

· Data Pipelines are like Azure Data Factory and Azure Synapse pipelines that allow users to move PB size data, orchestrate end-to-end ETL activities and run add-code first activities for Spark Notebooks, SQL Scripts, and SQL Server stored procedures.

· Data Factory components can be scheduled to run hourly, daily, weekly, or as required.

Data Engineering

· Utilises the Spark platform to allow notebooks to read, analyse and transform data in both PySpark and SparkSQL.

· Suits the creation of a Synapse Data Lakehouse, which allows organisations to store and manage structured and unstructured data in a single location.

Data Warehouse

· Allows for a data warehouse to be created, which supports structured data and allows data manipulation through SQL.

Data Science

· Enables Data Scientists to build, deploy and operationalise machine learning models.

· Allows integration with Azure Machine Learning studio and provides the ability to integrate the machine learning prediction models into Power BI reports.

Real-Time Intelligence  

· Provides an end-to-end solution for event-driven scenarios, streaming data, and data logs.

Power BI

· Power BI allows users to connect data sources and Fabric semantic models to visualise data through its visualisation capabilities.

· Allows AI to be integrated into the reports, allowing businesses to make much more informed decisions.

Data Activator

· Allows actions such as emails, or Power Automate workflows when specific patterns or conditions are met in Power BI reports or Event streams.

OneLake

· OneLake is built on Azure ADLS Gen2 and is the unified location for storing all workloads for the entire Tenant, including all Microsoft Fabric experiences.

Fabric’s evolution

Microsoft Fabric has come a long way over the last year and therefore it has been important to stay up to date with the new monthly feature releases and Microsoft Fabric Roadmap. For example, initial concerns of data being publicly accessible are no longer a problem due to private endpoint support. Or just new upgrades in general, such as Data Factory providing fast copy support for on-premises SQL Server databases and now the introduction of the new Real-time hub. Microsoft Fabric is always advancing.

Softcat is always keeping up to date with new features and we continually anticipate how these features could change future consultations and customer engagements. If you’d like to find out more, please reach out to your Softcat Account Manager or get in touch with our Sales team.