Conf42 Platform Engineering 2025 - Online

- premiere 5PM GMT

Building Resilient Data Platforms: Implementing Blockchain-Based Master Data Management at Enterprise Scale

Video size:

Abstract

Production blockchain data platform that actually works, Discover battle-tested distributed ledger architecture solving real platform engineering challenges. Learn deployment strategies, security patterns & hard-won lessons from enterprise systems. Get proven blueprints & working solutions today

Summary

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hello. Hi everyone. Good afternoon. it's pleasure to be here today. My session is about very exiting area, using the blockchain technology to reinvent master data management. Where does this matter? Because master data is like a patient identities in healthcare or customer, and the product data in enterprise systems is a backbone of all the digital operations. If this foundation is insecure and consistent and reliable, everything build on top of it becomes fragile. In my role at LabCorp, I have worked hands-on with these challenges and I will show how we solve them by combining the blockchain with the enterprise MBM. By the end, you will see how the distributed, immutable, and encrypted platforms can transform enterprise data management. What are the traditional MDM chart data consistency issues with the traditional MDM and the security vulnerabilities and performance bottleneck in compliance complexity. Traditional MDM started simple, mostly as a database Consolidation are data consolidation. But modern enterprises operates across multiple geographies and multiple clouds and dges of the applications. That creates four big pain points, data consistency. The if the two systems update, the customer recorded the same time, you often get the conflicts are drift. Second is security. Centralized. MDM is a huge target, one breach. under the attacker controls all the master data. Then the third one is, which is performance. As a data grows exponentially, systems slows down. And the fourth one, final one is compliance competencies. regulated industries like healthcare and finance, requests, detailed audit trails and lineage. Most legacy systems will these futures on or afterwards. So she makes them fragile. These challenges show us why we need new foundation. Blockchain is for mainly four categories, immutability and consensus driven updates, and built in cryptographic security and transparency. Blockchain directly addresses these four pain points. First one is immutability means that once, once data is started, it can be tampered with. That is automatic. Second one is consensus means that no single system decides multiple nodes must agree before the data changes, which removes the single points of the failure. The third one is built in security through cryptography, ensures authenticity and confidentiality. And final one is transparency. Every participant can see and verify the state of the data. In short, blockchains DNA is natural fit for the problems of MDM faces today. In traditional systems, this means keeping the logs and backups and complex access monitoring. but in the blockchain, immutability is built in. Once the transaction is written, it's mathematically locked into the chain. If anyone tries to change it, the system will reject it because the cryptographic ash no longer matches these turns audit trails from the complex manual process into something automatic under tramp proof consensus is about the agreement. In centralized system. The database administrator or single system can approve the changes. That's a convenient but risky act. With the blockchain, multiple independent nodes must agree on the change. We use a practical Biogen time fault tolerance mechanism, which can tolerate even third, one. Third of the nodes being compromised, still remain reliable. This critical for. Resilience. It means the platform can withstand failures and cyber attacks are insider threats and skill. Deep data, trustworthy. Every transaction is signed with a digital signature, sadly, who performed it. Data is encrypted both in the rest and in the transit. So that's much stronger and simple than managing thousands of individual keys in short securities over into the fabric of the system. Transparency is about the trust in many industries and disputes arise because each party maintains their own version of the truth. For example, in supply chains, one partner might claim and update. Was made while another says that it wasn't. With the blockchain, everyone who has access to see sees the same verified ledger that reduces the disputes, increases the collaboration, and builds confidence across teams and organizations. Transparency doesn't mean loss of privacy. Data is encrypted, but the fact that the transaction is happened is visual to authorize parties. Hyperledger Fabric is the one of the thing, and attribution based encryption is another one. IPFS is for large storage, large data storage, and work to J-W-T-T-L-S 1.3 for the modern security. So we build our MDM platform only carefully selected. Stack at the core is a Hyperledger FA fabric, a permissioned blockchain. Ideal for enterprise use for fine-grained access. We use attribute waste encryption, which enforces the policy, set the data level for storage, large data objects, going to IPFS and a PPP two P file system. And while the blockchain only keeps the cryptographic fingerprints. of those files keeping the ledger efficient. Finally, we secure everything with modern protocols or the two for authentication. The WTS for session management, TLS 1.34, encrypted communication. Together, these technologies give us both power and practicality. Our architecture is layered, for the clarity and the scalability at the top. Presentation layer exposes the rest and drop graph ql APAs Enterprises can integrate easily. The next one is application layer. It contains the business logic in the form of smart contracts, what Hyperledger calls chain code. And the con layer, con sense layer ensures the agreement using the PFBT and the data layer combines the blockchain metadata, IPFF, IP IPFS payloads and the ready caches for the for fast retrieval. And at the base infrastructure layer runs. On the Kubernetes clusters across the availability dunes for the failover and the scaling, each layer specialized, but together they form a resilient hole. We didn't jump straight into production in the phase one, we built a proof of concept, which is basically crowd operations, consensus testing and performance benchmarking that validated the blockchain potential. In phase two, we. Piloted with the real workloads and developing smart contracts and the optimizing throughput, we also deployed the attribute based encryption. Finally in the phase three phase of production, building infrastructure as a code, and setting up monitoring with the different tools and designing disaster recovery processes. And this strategy changed the journey, minimized the risks, and ensure the enterprise readiness. 3090 9.9 percentage, 60 percentage. The results are enter, enterprise grade. Our platform processes, or the 3000 transactions per second delivers 99.9 percentage of time and includes the data access by 60 percentage. But the performance only the part of the inventory compliance is automated and smart contracts enforce retention policies and every change is logged immutably. In short, we didn't just match the traditional MDM. We exceeded in this, in scale and reliability and trustworthiness. Security implementation. How, security implemented in defense, in depth a S 2 56 at rest and the TLS 1.3 in transit. We protect the data at two levels, a s 2 56 encryption for storage and TLS 1.3 for all the communication. This ensures both the stored and the transmitted data for secure. And the second one is digitals. Every transaction is digitally signed and so we know what exactly who performed it and can be prevent further remediation and zero knowledge proofs. And for highly institute operations we use. Zero knowledge proofs. This lets someone to prove they are authorized without revealing the actual data. It's a security without flexible or, exposure. Instead of static rules, we use the attributes based encryption. For example, only doctors with oncology specialization and active license can access the oncology data policies drive access, not the manual permissions. The second one is dynamic policy evaluation. Access is evaluated in, if someone's status is changed, their access changes instantly. No manual updates are needed. Immutable, all access events are logged immutably on the blockchain preventing that, providing the regulator Reg Regulat with the tamper proof Oddity system. Third one is to treat me medication. DDOS protection. We use multi-level rare limiting and failover to observe the of the service attempts without downtime. Insider threat prevention. The second one is in insider threat prevention. No single administrator can override the system because the blockchain requests distributed consensus. even insider attacks are prevented. The third one is data leakage prevention. We integrated enterprise gate DLP tools to stop the unauthorized data extraction, adding it another layer of the safety. Data retention policies is the automated compliance framework. Within that data retention policies, the smart contracts automatically enforce the retention rules. No manual cleanups, no mis deadlines. Then the second one is comprehensive access, logging, even accesses logged mutable on the blockchain, creating a reliable audit trail. And the third one is complete data lineage. We can trace the data across, its entire life cycle. Who created it, who modified it where it is used? Suppose the right to be forgotten with the cryptographic deletion, then HIPAA compliance and enhance it. Protection for the sensitive health information such compliance, building suppression of the duties, ensuring financial accountabilities. Blue green deployments. We run two identical production in environment side by side. When the updates are needed, we switch the traffic from blue to green, ensuring the zero downtime, and then cany. Religious new smart contracts are deployed fast to the small subset of the no. If issues appears, we can roll back automatically before the problem spreads. Then future flags. Third one is future flags. We can turn the futures on or off selectively in production. This lets us test the real world behavior without putting the entire system at risk monitoring and observability the CPU and memory and disc, and the network access for all. No. And predictor Analytics helps us spot problems before they cause the outages and application metrics. The second one is application metrics. We measure the transaction throughput, latency, error rates, and business KPAs. Then the third one is block saying basic metrics, and we monitor the consensus participation and blockchain propagation times and chain growth rates to ensure the ledger itself. Is running smoothly. Incident response, how the incidents are tracked and how the incidents responses are captured. Automated detection, machine learning and anatomy detection, smart detection spot, usually, activity escalation procedures. We have clear escalation paths and automated notifications. The right people are elected quickly. Support motion buses post process. Every incident is thoroughly analyzed and lessons are added to our runbooks and making the systems smarter and stronger over the time. And to conclude the future data management is distributed, encrypted, and immutable. Blockchain based MDM is no longer a theory. It's running in production today, solving real challenges. It gives the enterprises safe. Platform that is faster, more secure, more compliant than the legacy systems at LabCorp. This journey has just begin, but the results already show the tremendous value. I hope today's session give you very clear picture that way the blockchain D matters, and how it can help your organizations build res resilient future ready data platforms. Thank you.
...

Chandra Adapa

Principal Enterprise Architect @ LabCorp

Chandra Adapa's LinkedIn account



Join the community!

Learn for free, join the best tech learning community

Newsletter
$ 0 /mo

Event notifications, weekly newsletter

Access to all content