Conf42 Observability 2025 - Online

- premiere 5PM GMT

Dashboards, Not Drama: How Fixed-Cost Clouds Made Observability Fun Again

Video size:

Abstract

Observability shouldn’t feel like defusing a bomb every billing cycle. Let’s explore how adopting a fixed-cost mindset and a simpler monitoring model can make dashboards joyful again, without surprise costs or hidden complexity.

Summary

Transcript

This transcript was autogenerated. To make changes, submit a PR.
Hey everyone. Thanks so much for joining today. Before we go further, a little bit about me, I am Dennis Tomer de, and welcome to my talk. And my talk is dashboards are not dramas anymore. So let's start. Have you ever been right in the middle of debugging a critical production issue and when suddenly you get a message? From someone in finance saying, Hey, our cloud bill just increased by $5,000. What happened? Yeah, that syncing feeling isn't pleasant, right? All of a sudden you are not just trying to fix the problem. You are also stressed about how much monitoring tools you are using might be cost while you are solving it. Now in theory, monitoring our system should be straightforward. Ideally, it'll give us clear visibility to catch issues before users do help us quickly develop problems when they occur, give us insight into our system performance, and also help us plan capacity needs, and sometimes source stakeholders that everything's running smoothly. Sound reasonable, doesn't it? But the reality in most cloud environment is quite different. We often end up in what I called the monitoring fear cycle, as you can see in this slide, which looks something like we deploy a new service and set up basic monitoring, then an incident happened. When we realize our visibility isn't enough, so we add more metrics, logs, tracing, but then cloud bill spike unexpectedly. So someone from finance team ask us to cut costs. So we scale back monitoring. But again, another incident happened and be back to the Esquire one. This cycle repeats again and this in justly. I personally seen this happen. A team I work with built great dashboards for their service, the tracking everything from response time and error rates to database transformation. But as their service grew from 10 to 50, their monitoring cost increased dramatically. Eventually they had to delete many dashboards and sample their metrics at only 10% to manage costs. Yeah, so there are some points of practical impact. First one is self sensor engineering don't create the dashboards they need because they're afraid of costs. Second one is delayed detentions. Go unnoticed longer because we are not collecting the right data we want. Third one is blind debugging. Sometime when problem occur, we lack the visibility to solve the them quickly. Fourth one, crisis monitoring. We only turn on the full monitoring during emergencies, which is exactly when we don't need extra stress. Fifth one is knowledge gap. New team member never learned to use observability tool because it seems as a scare resources. Let me show you how this play out in a typical incident response. For example, consider how this play out in a typical incident. As you can see in this slide at 10:00 AM users report slowness. But there are no early volume. By 10 15, the team start looking at basic metrics, but they have no idea, and then they request permission to enable detail details monitoring, which only happen at after 30 minutes. After waiting another 30 minutes for data, they finally pin pinpoint the database bottleneck at 11:15 AM So resolve the issue by. After 15 minute, but the next day monitoring is scaled back down again to save cost, like nearly two hours of impact. Mostly just wasting for visibility. It's not just incident every day development suffer to, I have seen many developers avoid adding essential custom metrics because it, its needs management approvals. Team often settle for generic dashboards that don't match their needs. Historical data is quickly deleted, making them exporting trends Im impossible. Logging levels are set so high so they miss valuable debugging information. Also, all because of cost concern. That's not right. I also once was a senior engineer. Spent three hours optimizing monitoring just to save $200 per month. A clear misuse of valuable time and worst off, all these costs are unpredictable. Sometime it's 200, sometime is 300, sometime's 500. These are unpredictables. Traffic spikes or bug unexpectedly. Inhale your monitoring bills overnight. So exactly what, why, exactly. Why is monitoring so expensive? It boils down to how cloud and platform observability typically is worse. First agent, collect your data from servers, then send it off to the central collection services. Data is stored, indexed, and run curries, though the dashboard alert, this is, and so this architecture create four separate cost drivers. First one is data transfer. Moving data from your instance to monitoring service, which is cost money, especially across different regions, all to the different third party tools. Second one is ingest processing. Passing, transforming and indexing monitoring data, required more compute resources. And third one is storage. Keeping your data available for queries, especially at higher resolutions. And last one is query cost. Sometime providers even charge for your own, accessing your own data, which is not correct. This cost structure for us to build compromised monitoring system. As you can see in this slide, how much code we have to write to get some little bit idea and also how much it is complex. And another one you can see is this setup, alerting. This is also too much complex to just set up an alert, a simple task. So what's the alternative? The key is making observability a built-in feature for your in infrastructure, but not expensive. Add-on in a fixed cloud model. Fixed first cloud model monitoring become part of the platform mat logs and T traces are collected automatically without additional charge. There are no data transfer fees, retention periods are predictable and included and curing your data doesn't extra charge. So that's the good way. Yeah. So let me show you what this look like with Seamless, a fixed cost cloud platform. I have working with, as you can see in the dashboard slide, which you create a server. Dashboards covering CPU memories, disc io and network performance and more With thir 30 days of high resolution historical data, there are no setup, no additional cost, and no complicated pricing that look good. Yeah. Also look logs work similarly, again, without not worrying about the charges. Here's the difference between make in the real world incident management at 10 and 10:00 AM monitoring catches issu immediately. Then firing an early alert within minutes. Detailed metrics are accessible and allow team to quickly identify a database connection issue by after 10 minutes, the issue is fully resolved immediately after that Word. Afterward the team easily create an additional data board to track this scenario better. All without cost worries. This means the issue is resolved in 25 minutes instead of nearly two hours waiting for, to get some ideas reducing. And this co this will also reduce stress and improving outcomes. Ultimately observability should tend to be something we fear. It should be something we rely on every day. Whether you are exploring, like platform or rethinking your current setup, the goal should be clear. After all, you can't fix what you can't see, and you'll never look if you are scared of the bills. So thanks so much. Thank you so much as this and this is a prerecorded session, so if you have any questions or any doubt regarding this talk or want to chat about new tech or developer problems, feel free to ping me up on the LinkedIn or tutor and these are my handles, so bye-bye. Thank you everyone.
...

Denish Tomar

Developer Relations Engineer @ Qumulus

Denish Tomar's LinkedIn account Denish Tomar's twitter account



Join the community!

Learn for free, join the best tech learning community for a price of a pumpkin latte.

Annual
Monthly
Newsletter
$ 0 /mo

Event notifications, weekly newsletter

Delayed access to all content

Immediate access to Keynotes & Panels

Community
$ 8.34 /mo

Immediate access to all content

Courses, quizes & certificates

Community chats

Join the community (7 day free trial)