Transcript
This transcript was autogenerated. To make changes, submit a PR.
Hello everyone.
My name is meu and I'm a software engineer working with Microsoft.
Today we are going to talk about building resilient, smart city platforms
and how engineering distributed system going to help at urban scale.
So by 2030, about 60% of global population is projected to live in cities and
providing an infrastructure which supports mobility, energy, safety, and governance.
Is going to be digitally backbone by distributed systems, and it's going
to support the millions of residents.
So let's deep dive into it.
The engineering challenges.
Now with the conventional enterprise platform, smart city systems must
operate in real time, but that doesn't happen with the conventional enterprise
platform because we do want cities.
To have their data process in real time, and this requires distributed system
architecture that can support massive concurrency fault tolerance, and high
availability while upholding security, compliance and sustainability with the
monolithic system, it is not possible to provide this, not the distributed
system as the urban operating system.
So it comprises with the four different pillars, which includes
decentralization, elastic scalability, fall tolerance and redundancy, and
data-driven resource optimization.
Smart city platform functions as an urban operating system coordinating multiple
systems such as transports, utilities, emergency responses, and citizen services.
At this scale, monolithic architecture collapses under the weight of complexity.
And that's when distributed system comes to provide the aid.
What the distributed system has it, as we discussed, that it
provides the decentralization.
Critical services cannot depend on a single control point.
Distributed system ensures that there is autonomously.
These decisions can be made a autonomously and even in the
central hub if it's offline.
Elastic.
Scalability platforms must handle unpredictable spikes.
For example, during festivals, emergencies, or any peak time.
The auto-scaling based on the workload for casting, ensures performance, scalability.
Fall tolerance and redundancy.
Redundant services across multiple regions ensures resilience against outages.
Active congregation minimizes downtime, so we do need a lot of fall tolerance and
redundancy to support the realtime system.
Data-driven resource optimization.
Distributed architectures have pro improved operational efficiency by 67.8%,
and the infrastructure cost has been reduced by 34.2% in real-time deployments.
This is all thanks to the dynamic resource and schedule dynamic resource
scheduling and predictive analytics.
Now let's talk a little bit more about intelligent transport system
for the intelligent transport system.
We have seen with the smart cities, which are driven by the distributed
network, they have about 91.7 data consistency, 91.7% data consistency,
which is distributed across nodes leading to synchronized signaling,
routing, and congestion management.
And we have also seen that about 76.2% commuters benefit with this structure.
They get direct benefits from improved trans transit efficiency through
adoptive scheduling, multi-model integration, and predictive rerouting.
And again, with this transportation system, there are engineering
strategies which we follow to it.
So there, so the basically are the message oriented middleware in this structure.
We need a way that all these messagings are getting transferred
or through real time pipeline.
So for example, Kafka and Puls, they ensure the reliable
real time data streaming.
From traffic sensors, ZPS devices, transit apps, or maybe from incident
reports by customers, service smash.
This is something which helps to decouple the microservices, which helps
in routing, ticketing, scheduling, while also maintaining observability.
We'll talk about observability little bit more detail in the next slides.
Now predictive AI models, this strategy really helps the machine
through machine learning models.
We can forecast the congestion and we can adopt the traffic signals
accordingly and dynamically.
That definitely helps reduce the travel time, the congestion duration.
It helps us reducing that.
These engineering strategies work together to create a cohesive,
responsive transit system that adopts to changing condition in real time.
Now, reliability through AI powered incidents response with the traditional
I it incident management model, it fails to scale at the city level.
The integration with AI-driven incidences responses into the micro
city microservices architecture has transformed the reliability
management system in a huge way.
We have seen the, there has a 20, a 42.6% reduction in MTTR which is nothing but
predictive monitoring identifies anomalies before they escalate into outages.
Automated remediation, self-healing workloads a workflow offers
automatically remote work workloads reroute workloads, restart failing
parts or provision, addition provision additional compute notes.
So with this automation way where the nodes are getting readjusted or the
resource are getting allocated or rerouted of the information, that has helped a lot.
And we have seen with this the distributed network, we have seen that about 63.8%
improved resource allocation where the AI allocates commute bandwidth
or transit resources dynamically based on evolving city conditions.
Now let's just talk about AI incidents response like a,
with a real world example.
In this case study, we will talk about a large metropolitan smart city development.
An AI driven observability platform predicted and prevented a potential
blackout by detecting anomaly pattern in good telemetry, isolating
failing systems and auto reallocating power loads before outage cascaded.
This example demonstrated how AI powered systems cannot only detect problems,
but take autonomous action to prevent cascading failures that could affect
thousands or millions of residents.
With this internet of Things telemetry pipeline, we know that it process data
from about 5,000 plus IOT sensors.
The smart cities in general need to rely on the information of energy usage,
traffic, air quality, water system, and security, and these in info And this,
these information needs to process and through the sensors which we have
placed in, in the smart cities, the da if you talk about the data volume,
the terabytes of telemetry flow daily from the heterogeneous devices there's
a huge, that's a huge chunk of data.
We also have seen that the low latency in this process, delays can com
compromise safety critical functions like traffic lights and emergency responses.
That's why we need low latency of the system.
The Edge computing has helped us processing the data at the sensor of the
gateway, which reduces network strain and enables local decision making.
And we have seen with this architecture and with this the overall strategies,
that's about 78.4% anomaly.
Detection has been improved with these AI models, which detects popula
pollution spikes, structural traces, and bus bridges, or unusual water
pressures are the few examples, which helps us to detect like which which
are needed to be detect quickly.
Now with this all AI driven, machine learning driven and the distributed
system, it has always been a concern of privacy, compliance and how we are
going to do the governments at scale, smart city platform, manage sensitive
personal and behavioral data of 82 point.
82% of residents about without robust privacy net frameworks,
trust and adoption collapses without doing getting into all that.
We have also seen with this, the data minimization, secure multi-tenancy
and the consent frameworks.
This all is something which provides with the whole flow.
Let's just talk about the rev The, the privacy and governance
result with the whole system.
We have seen the improvement in data protection.
It has about 34.8% improvement across deployments, implementing privacy by
design principles, higher public trust.
We have seen about 67% of higher public trust in the city that deployed governance
dashboards with transference auditing.
So when we have so much transparency in the system, it really helps to build
the trust and if you have dashboards, all the information that, hey, this
is the situation, or this is what is happening, these are the incidences,
or this is the current scenario.
All these information being transparent to the public has increased the
trust and higher service integration.
Success rates as privacy concern is no longer block adoptation of
new civic services because the service integration has become easy.
It's like 45% with all different services.
This metrics is demonstrate that privacy is not just an ethical concentration,
but a practical requirement for successful smart city implementations.
Now let's just talk a little bit more about the observability
and deployment pattern.
Observability is nothing but looking into the system through dashboards or knowing
that what's happening with the centralized dashboard, ingest telemetry from diverse
platform, it has helped us to provide a one off the stack in the observability way
because this is something which is more important to have a centralized dashboard.
We also need like a proactive alerting with AI correlation across
multiple services, end to end tracing for a complex transactions.
Yes, this is something which is part of our observability stack and it
helps to build nothing but trust start and make an efficient system.
Now, the deployment patterns, the blue green deployment,
safe rollouts without downtime.
This is something which is.
Requirement as part of DevOps can, it releases testing
with a small set of users.
It's definitely needed, if you have a system which you are placing you, if you
do a testing done with a small set of users is definitely helps to know the
issues in the end and see the feedback from the users and make more decisions if
we need to change something in the system.
Kubernetes orchestration, centralized workloads for elastic scaling.
So with this as for an example, Kubernetes orchestration has
helped that you can containerized workload for elastic scaling.
If you want to scale based on the traffic p cars or different cities,
different scenarios, this can help.
With this distributed system, let's talk about the sustainability and
carbon footprint, which it's making.
Smart cities are not just judged by not only by the service delivery,
but also their environmental impact.
We have seen the carbon footprint reduction of about 28.9%.
We are compute task shifts to data centers with renewable energy availability.
Resource utilization has seen improvement about 35% by adopting sev serverless
patents and adaptive orchestration.
Circular data practicing.
This is one of the thing which we use in the system.
It definitely helps reduce the carbon footprint and increase the sustainability
where reusing telemetry data across departments reduces duplication,
cutting both cost and emissions.
Now, let's see, of how a practical framework look like.
For platform engineers designing next generation urban system, a
structured approach is essential.
So we, here are the building blocks to form a smart city network, foundational
layer, distributed service, mass identity and access management, data governance.
You need a form, a foundation layer, which is nothing but a distributed service mash.
You have a data access and data management data governance, that's one of the
foundation layer, which we really need.
The data layer.
The data layer helps in the real time ingestion which helps distribute the data
lakes and the ML interface engine engines.
Those are the part of the data layers.
Now once you have the data layer with the pipeline, you want the application
layer, which is nothing but the microservices for Transport, energy,
safety, and Civic Services or any other where you would use this data to, to
do a prediction or analyze the data.
Now the, another building block is observability, layout
layer, unified telemetry.
Observability, unified Telemetry, ai, power monitoring
and automated remediation.
This is something which provides through observability layer that
you have, monitoring dashboards and everything, not the governance layer.
With all this data and information, you need a governance layer
where the privacy is by design.
Compliance automation, citizen engagement dashboard, all these things, if you
make your system transplants, going to help design an urban system and a
sustainability layer where you are working on how you can reduce and make people
or the system carbon aware and help with the green infrastructure utilization.
This is something which we need to all these, like all these building blocks
are very much important when we are thinking as part of platform designers.
When we wanna create a smart city networks.
Now measurable outcomes.
Resilient smart city platform.
Em embodied the convergence of distributed systems, engineering, civic
responsibility and sustainability.
The measurable outcomes are compelling about 1.2 million concurrent operations
we have seen with about 99.7 uptime.
We've also seen 67.8% operational efficiency gains and 34.2 cost
reduction with the system.
91.7% data consistency, which has held about 76.2 commuters
in the transport system.
42.6% reduction in MTTR and 78.4.
Better anomaly detection.
Now we are at the part of our conclusion of this.
By leveraging microservices, event-driven architecture, AI powered
incidences, sponsors, and privacy by design frameworks, it is can achieve
a balance of reliability, performance, public trust for platform engineers.
These figures demonstrate that resilient, smart cities performs are
not aspirational, they're achievable.
The future of urban living depends on how we scale, govern, and sustain
this distributed infrastructure to serve millions responsibly.
The smart city is not something in future, it's here now.
The only question is how are we going to adopt it?
So let's ask these questions to ourselves when we are thinking to
design a smart city infrastructure.
Thank you.