Data mesh vs data fabric: What's the difference? – Analytics India Magazine

Data mesh is a highly decentralised data architecture equipped to address challenges including lack of ownership of data, lack of quality data and scaling bottlenecks. The goal of data mesh is to treat data as a product, with each source having a data product owner who could be part of the cross-functional team of data engineers. Data mesh — introduced by Zhamak Dehghani of Thoughtworks in May 2019– overcomes the problems of traditional data lakes and data warehouses.
Data fabric is an all-in-one integrated architectural layer that connects data and analytical processes. It leverages existing metadata assets to support the design, deployment, and proper data utilisation across all environments and platforms. Data fabric aims to accelerate inference from data through automated processes and provide real-time insights. It integrates data, analytics, and dashboarding into one and serves as a management solution, allowing frictionless access in a distributed environment.
According to Gartner, data fabric is a design concept. The approach leverages continuous analytics over existing, discoverable and inferences metadata assets to enable the design, deployment and utilisation of integrated and reusable data across all environments.
Data mesh approaches data from a people-and process-centric view and treats data as a product. Data fabric leverages human and machine capabilities to access data in place or support its consolidation where appropriate. It combines technologies that connect sources of data, types and locations with different methods for accessing the data. Gartner used the analogy of a self-driving car to explain the concept: Data fabric monitors the data pipelines as a passive observer and then suggests more productive alternatives. When both the data “driver” and the machine learning are comfortable with repeated scenarios, they complement each other by automating improvisational tasks while leaving the leadership free to focus on innovation. 
Data fabric continuously identifies, connects, and enriches real-time data from different applications to discover relationships between data points. It does so by building a graph storing interlinked data descriptions that algorithms can use for business analytics.
In data mesh, the data is stored decentrally within its domains inside a company. Each node has local storage and computation power, and no single point of control is necessary for operation. Essentially, original data remains within domains and copies of datasets are generated for specific use cases.
In data fabric, the data access is centralised with high-speed server clusters for network and high-performance resource sharing in the data fabric. 
According to Thoughtworks, the data mesh paradigm is a strong candidate to supersede the data lake as the dominant architectural pattern in data and analytics. Data mesh introduces an organisational perspective, independent of specific technologies. Its architecture follows a domain-driven design and product thinking to overcome challenges related to data. Data mesh culture is about connecting people and creating a federated responsibilities structure.
While data fabric leverages metadata to drive recommendations, data mesh collaborates with subject-matter experts to oversee domains. These domains are independently deployable clusters of microservices that communicate with users. It consists of codes, workflows, teams and a technical environment. 
Data fabrics work with and are mostly compatible with technical, business and operational data. Visualisation tools make the technical infrastructure easy to interpret and help organisations manage their storage costs, performance, security and efficiency. In addition, companies can deploy a singular data fabric virtually over various data repositories to manage disparate data sources and downstream consumers.
In data mesh, data is made available via controlled datasets. First, the information is copied from the department data store to a shared location. 
In data fabric, data is made available via objective-based APIs. The data is copied into specific datasets for specific use-cases, and the business unit that owns the data is in control.
Data mesh is ideal for hybrid cloud networks. Data fabric enables single-point data access, address data quality and storage issues and handling of security threats.
It is critical to note that data mesh and data fabric are not mutually exclusive concepts. Organisations can leverage both approaches across different use cases. According to James Serra, Data & AI Solution Architect at Microsoft, the difference between the two concepts lies in how users access data. Data fabric and data mesh provide architecture to access data across multiple technologies and platforms, he said. “But a data fabric is technology-centric, while a data mesh focuses on organisational change. [A] data mesh is more about people and process than architecture, while a data fabric is an architectural approach that tackles the complexity of data and metadata in a smart way that works well together,” he added.
Webinar
Speed up deep learning inference
13th May
Conference, in-person (Bangalore)
MachineCon 2022
24th Jun
Conference, Virtual
Deep Learning DevCon 2022
30th Jul
Conference, in-person (Bangalore)
Cypher 2022
21-23rd Sep
Stay Connected with a larger ecosystem of data science and ML Professionals
Discover special offers, top stories, upcoming events, and more.
Google is also set to unveil Health Connect by Android, a new set of APIs designed to help you reach more Android users with less work.
Analytics India Magazine caught up with Srinjoy Ganguly, senior data scientist at Fractal, to understand the scope and future of quantum computing.
In 2021, Marcus published an article on Nautilus, titled ‘Deep Learning Is Hitting a Wall’.
“Machine learning in production can be done. It shouldn’t hurt so much.”
The top military leadership in India have regularly spoken about stronger AI integration in the forces.
The scope for AI-focused innovation is tremendous, given India’s status as one of the fastest-growing economies with the second-largest population globally.
As a digital-first company, we constantly endeavour to push the boundaries of what tech can do to create a highly differentiated user experience in fitness and wellness.
The vendors can prevent cloud squatting by preventing IP address reuse.
A graph attention network can be explained as leveraging the attention mechanism in the graph neural networks so that we can address some of the shortcomings of the graph neural networks.
A deterministic approach is a simple and comprehensible compared to stochastic approach.
Stay up to date with our latest news, receive exclusive deals, and more.
© Analytics India Magazine Pvt Ltd 2022
Terms of use
Privacy Policy
Copyright

source
Connect with Chris Hood, a digital strategist that can help you with AI.

Leave a Reply

Your email address will not be published.

© 2022 AI Caosuo - Proudly powered by theme Octo