Microsoft Fabric for Data Management

5 min. read

Academics, researchers, and technologists have defined what a modern data technology architecture requires: ingestion, storage, engineering, governance, and reporting. Microsoft Fabric matches this definition, enhanced and streamlined by AI.

In our previous exploration of the essentials of an AI Estate and its transformative innovation powered by artificial intelligence, we outlined what academics and industry have defined as a comprehensive data repository, engineering platform, and business intelligence suite. This architecture must be able to “accommodate novel and edge-related data repositories that are consistent, protected against nefarious actors, work in real-time, and integrate the ever-growing ecosystem of external and third-party data,” according to Forrester who refer to this architecture as the “data fabric”.

The data architecture must include data ingestion, unified storage, analytics and engineering capabilities, in-built governance and information control, reporting dashboards and insights, all powered in part or in whole by generative or machine learning AI to automate these processes and streamline data transformation and discovery.

In our opinion, Microsoft Fabric running on Microsoft Azure enables data teams – data analysts, scientists, engineers, and other business users to achieve business outcomes using a unified user experience – fulfilling the industry specification for a modern data fabric. Microsoft Fabric is an intelligent cloud platform that unifies databases, analytics, governance, and AI.

Microsoft Fabric addresses the data fabric architecture: it collates and simplifies data from a variety of sources and integrates them for analysis and manipulation using a single unified platform. Each user can work on data in real-time across analytics engines and code bases or languages. This data “lake” (known as OneLake) can be enabled for collaboration and real-time decision making. The AI Estate in this instance accelerates decision making and ensures greater accuracy.

Microsoft Fabric also automatically indexes data for easy discovery, sharing, governance, and compliance using Microsoft Purview software. It reduces or eliminates data duplication, which enhances data risk management.

Users of all stripes can then create custom analyses and applications using natural language prompts using Microsoft’s CoPilot generative AI, embedded into Fabric.

According to Microsoft, Microsoft Fabric:

“Combines the best of Power BI, Azure Synapse Analytics, and Azure Data Factory to create a single, unified software as a service (SaaS) platform with seven core workloads—each purpose-built for specific personas and specific tasks. By creating a single platform with tools for every data professional in a unified experience and architecture, Fabric can reduce the typical cost and effort of integrating analytics services and help simplify your data estate. Fabric’s unified architecture simplifies billing by providing a single pool of capacity and storage that is used for every workload.”

Features of Microsoft Fabric

We’ll tackle each discrete yet unified component of Microsoft Fabric.

The first plank is data ingestion and unified storage. Microsoft OneLake empowers organisations to virtualise data across clouds, accounts, and domains without overwriting or changing ownership. It is a complete data collation and collaboration solution weaved into Microsoft Fabric. Shortcuts and mirroring data features eliminate the need to move data and consequently, reduce coding. Combine that with AI-powered queries and every department of your organisation can use data insights at their command.

Microsoft Synapse, powered by machine learning, enables data scientists and engineers to manipulate and perceive data as they wish using SQL, Azure, and natural language queries. This covers the analytics and data science/engineering facet of curating data for AI. Harnessing data from your OneLake, data scientists and engineers can visualise data while taking advantage of intelligent workload management, workload isolation, and limitless concurrency. Data scientists are empowered to create analytics dashboards or proofs of concept in minutes without extensive coding. Synapse also integrates into “traditional” Microsoft 365 (Office) suite products.

Information governance and control is handled by Microsoft Purview and security settings for identity, authorisation, classification, content, and monitoring, amongst others. Purview governs system-wide information security governance through domains and Data Loss Prevention, powered by AI labels and classification systems. Sensitivity labels can be automatically applied to new data, and data inputted or retrieved can be audited at the user level. AI can detect sensitive data uploads or downloads and alert administrators or data owners, preventing breaches or inappropriate use. Identity and access management, monitoring and auditing, and AI model content filters provide the controls for a safe, secure, and trusted information environment.

Automation and insights powered by AI is also part of Microsoft Fabric, using Microsoft’s powerful generative AI platform Copilot. Copilot weaves all the capabilities of the Microsoft Fabric data estate and enables users of any skill level to create reports, analyses, insights, and virtually any type of visualisation or data transformation using semantic (language) prompts, powered by Microsoft’s own generative AI. As it is generative AI, it can complete code, automate routine tasks, summarise complex data sets or reports, and nearly anything else a layperson or data engineer would need from their data lake and analysis platform. It can be programmed to take on different permitted personas and view data from those perspectives. It reduces costs in time and labour through executing instructions in natural language instead of waiting for specific code to be written, debugged, and deployed. Copilot also integrates into all other Microsoft products including 365, Edge, Windows, and Azure.

Microsoft Fabric matches and exceeds the expectations of a modern, AI-powered and information-led intelligent data platform for business and can be designed, implemented, and deployed within the organisation and tested against specific use cases.

Though a transformative platform, many features will already be familiar to users of Microsoft products – with greater efficiencies and enhancements granted using natural language prompts and generative AI instead of coding or niche query language knowledge.

How we can help

Irada assists growth-minded organisations apply practical, aligned, and robust AI solutions in business operations.

Links to external websites were correct at the time of publishing. Articles may be behind a paywall. Irada is not responsible for the content of external websites.

The information in this article is general in nature. Your circumstances and needs may vary.

This work is licensed under CC BY-NC-SA 4.0