Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hello everyone.
I am Numan.
Today I will be speaking on a big data and machine learning in healthcare
and how these technologies are driving improvements in diagnostic
accuracy, personalized treatment planning, and operational optimization.
We will explore the data landscape and scalable processing architectures
and the role of machine learning in analyzing patient's data at scale.
Healthcare data is growing exponentially by 2025.
It's projected to reach 2,500 exabytes growing faster than any other
industry, including finance and media.
Here's the kicker, 80% of the data is unstructured.
Things like imaging, clinical nodes, and sensor data from variables.
The opportunity healthcare providers using big data have
reported cost reduction of to 20%.
And improved patients outcome by 30%.
But to get there, we need systems that can handle this data reliable and securely.
And at scale source of healthcare data, let's take a look closer.
Look at where this data comes from.
EHRs includes patient's history, medications, labs, and physician notes.
Medical imaging makes up the bulk of healthcare data by volume.
A single radiology department may generate.
A hundred thousand images daily.
Genomic data sequencing costs have dropped from 10,002,000 USD, but each
genome still adds 200 plus GB of data.
Variables collect 10 to 20 biometric points per second, generating
five to 15 MB daily per person.
That's a massive stream of heterogeneous data.
Right for EML, but challenging for traditional infrastructures.
Three core challenges define this space, data heterogeneity, and volume
and velocity and data quality issues.
Let's go one by one and explore this data heterogeneity structure, the
databases, free text notes, time series, vitals, and images, all coexist.
About 80% of the unstructured and meaning SQL alone won't cut it.
A single 300 bed hospital might generate 50 terabytes per year from EHRs.
ICU.
Monitor can emit 2000 plus readings per second.
Data quality issues.
For instance, smoking status is missing in 60% of records.
Diagnosis codes vary between hospitals.
We need smart pipelines.
Fault tolerant systems and flexible schemas to make sense of the chaos.
So here's where scalable data frameworks enter Apache Spark provides
in-memory processing, idle for large scale, batch and steam processing.
One provider reduce training time for clinical models by 88% after moving
from disc based system to spark.
A W SEF supports petabytes, scale queries and cost effective storage.
Your hospital in Germany reported 40 percentage of infrastructure
cost savings after migration.
OOP handle semi-structured and unstructured data.
Your public health network used it to integrate radiology scans and
clinical nodes from 10 plus hospital enabling ML based triad support.
These platforms are the foundations of model healthcare analytics.
Cloud-based healthcare data lakes, the traditional warehouses
force early schema decisions.
But in healthcare where data types are dynamic, we need schema on reflexibility.
Cloud-based Data Lakes solves that the consulted structure and
unstructured sources supporting more data formats than legacy systems.
One US network built a data lake that combined imaging EHS and real telemetry.
Result.
The analytic pipelines that once took days now run in another an hour
and dashboards refresh in real time.
The ETL process in healthcare are in just about moving data.
They are about preserving clinical meaning steps in encode extraction
from different systems, normalization, using snowed CT loin and RX nom.
Data quality validation to catch testing and inconsistent entries.
Transformation for models, readiness and incremental loading.
To support the steaming data efficient ETL.
Reduce pipeline latency from 2.5 days to 11 of us at one hospital, resulting in
realtime validating for critical care.
Is reshaping how clean clinicians are making decisions.
A declining model for 30 day readmission predicted achieved A A UC of 0.87
outperforming traditional models.
Disease risk models predicted diabetes at 94.3% and heart disease at over
90% and various cancers with up to 96% accuracy for deterioration.
Prediction.
LSTM networks trained on ICU time series data flagged.
Patients decline six to 10 hours earlier than manual methods hospital.
Using these tools are seeing fewer readmissions, better outcomes,
and improved resource allocations.
Medical imaging is a primary area for ml CNN.
Detect lung nodules on x-rays shows sensitivity and specificity comparable
to the expert radiologist unit.
Architectures enable on our typical segmentation, reducing
manual processing time by 60%.
In one pilot, a assistor radiology system help clinicians detect
pneumonia and chest scans with a 9% boost in diagnostic accuracy.
These tool accelerate diagnosis improvement precisions, and help
radiologists manage increasing workloads.
NLP helps us tap into rich but messy free text data.
NLP engines extract structured insights from discharge summaries.
Pathology reports and clinical notes.
Clinical trial matching platforms.
Now automate patient enrollment using NLP, reducing switch times
by weeks, adverse event reduction.
Using NLP saw recall rates jump up 28% in one hospital.
One model achieved F1 score, so over 0.90.
For extracting medication dosage and side effect information.
This enables real time text analytics that complement structure S, but
it's not solve smooth sailing.
There are some limited implementation challenges for being.
Security and privacy.
Over 60 of healthcare breaches stem from access control.
Misconfigurations, HIPAA demands, multi-layer protection, encryption,
role-based access and audit logs.
Workflow integrations.
Doctors already spend two to three hours on EHRs per hour for patient care.
Any tool must integrate seamlessly into their workflows, ideally,
within existing systems.
Ethical oversight, we need transparency.
If a model denies care.
Our gives poor recommendation.
We must know why.
This means explainable AI and clearly defined accountability structures.
Engineering AI for healthcare means building for trust, transparency,
and clinical usability, so to control big data and mission learning or
transforming healthcare, but the success on robust label and responsible systems.
And we are uniquely potent to enable this future.
Ensuring performance at scale, enforcing security, and keeping
patients outcome at the center.
Thanks for the time.