Hello everyone. My name is Swap Neil, and the topic of today's discussion is going to be to cover one of the critical use cases in conversational ai which is AI summarization. Generative AI is the process which ingests large amount of data. Mostly for the scope of this discussion, we are going to cover actual data. And AI summarization is a technique used to generate some insights from large amount of text data to summarize certain topics that are covered as part of any communication. The textual data can be of variety of formats and a variety of applications generate textual data. But for the scope of this. Discussion. We will deep dive into utilization of generative AI for business use cases. So when we talk about business communications any communication that the business handles is between the consumers of the business and then let's say the sales, marketing or customer service departments of that business in addition. For the business to run smoothly. The business representatives and employees would also be interacting with their coworkers for variety of different business processes. So we can think of it as an internal conversation between the business as well as an external conversation that the business has with its customers. So we can think of customer trying to reach customer service representative for a business using a chat bot that is offered on their website, or they write an email to the customer service department. Or sometimes there might be a shopping assistant that is there on the website and the person might be interacting with the shopping assistant. And then asking for product recommendations. And eventually when they're ready to make a payment, they might have a question and they might reach out to an expert to seek opinion. So they might write an email or they might continue chatting with a customer service agent. So a variety of different business use cases. Generate business conversations. And the data is mostly text based. So for the scope of this presentation, we will cover some of the architectural challenges in developing AI systems for summarizing message and email channel content across different business communications. So to get started, we'll explore some of the critical hurdles in building an AI summarization system. Some of those hurdles I've listed here are model hallucination, inherent biases, data distribution shifts, information preservation, and then contextual understanding and contextual awareness. So we will present certain architectural approaches and methodological frameworks that you can. Adopt to address these business challenges. So the focus would be on developing robust AI summarization system that can operate for a single thread communication, multi-threaded communication between multiple people or even cover different departments of a business. Let's look at how the digital communication have evolved for business interactions and what areas they affect for a business. So for a typical day a customer service agent who is interacting with business customers might be interacting on. Social channels, they might be interacting with customers for email or they might be having a live chat with the customers. So on an average they spend about five and a half hours out of a eight hour workday on active communication. So it is a significant portion of the day at a contact center for a customer service department spending. This much amount of time managing digital communications drafting documents, generating invoices, exchanging emails with the customer, sometimes escalating things to their coworkers or business partners, and then doing a wrap up of their interaction with the customer to give them a summary. So a variety of different use cases. Can be solved with AI summarization and introduction of instant messaging through Facebook Messenger, WhatsApp, iMessages, et cetera. Just increases the kind of applications that are generating text for businesses. Why it is important for mediums size businesses. To focus on digital communication channels is because it is crucial not only to have a good customer service interaction for loyalty purposes, but also making sure the customer needs are resolved at the end of the day. As well as having the right tools at hand definitely leverages certain improvements in the business. That can be driven by ai. If AI summarization is introduced as part of the business, the use case become even more complex when we talk about multi-threaded emails. So imagine you reaching out to a customer service representative regarding your concern, exchanging emails with them, attaching proofs or your concerns, and then having transferred over to another. Customer service represe, and then having them to go through all of that over again and then having you to repeat all the information it's an frustrating customer experience, and this can easily be solved by utilizing AI summarization for emails, especially multi-threaded emails where multiple people are involved. There's a long trail of emails and then some key information is being exchanged in the email. Customers are looking for a speedy resolution without having to repeat themselves. So what's the impact of the AI assisted Communication management systems in the businesses today? So it has been proven that businesses have reduced the time spent on routine communication tasks significantly. Which allows more focused time for the business representatives to attend other strategic initiatives for the business and run the business for essentially driving it forward. Whereas before more time would be spent on active communication. So this can be even leveraged further with AI summarization. The key challenge here is maintaining the right balance between. So summarization efficiency and information retention so that the context gets carried from. Now let's take a look at how we can achieve and highly efficient AI summarization system. So I would like to break it down into three major layers for data processing. Firstly, at the bottom we have a strong foundation. Of text processing. So this is crucial for any ization system to have a very strong text processing pipeline. It involves multiple activities such as text, normalization, text organization, and filtering. So let's look at what text normalization is. So it involves a variety of techniques such as converting uppercase to lowercase, removing punctuation marks. Expanding on certain abbreviations that might be present in the conversation. Converting numbers, currencies, and dates into words so that it is relevant for text processing. Normalization is crucial for a variety of applications like search engines or text-based workflows and machine learning. And this is. At the high level an overview of the text normalization, and the main goal of this is to reduce ambiguity and improve the overall model performance. Now let's talk about text tokenization. So tokenization kind of simplifies complex text by dividing it into smaller, manageable components, which allows easier processing of the natural language. So there are different types of text tokenization. We can split individual words into characters. We can split a sentence into words. We can split a paragraph into sentences. So a variety of different tokenization techniques can be utilized depending on the use case that you're trying to solve. So the next layer about that is semantic analysis. So semantic analysis analyzes the grammatical format of the sentences, including arrangement of words, phrases, and clauses. And the main goal behind semantic analysis is to establish relationships between the independent terms identified as part of text processing. So this is something you can think of it as identifying the actors. The system who are actively participating in the conversation. And then what is the relation between those actors whether your text is semantically correct. So all that is crucial. Before we can attempt generating a summary based on the text at the very top we have the summary generation aspect. So this is largely dependent on the efficiency. Success of the bottom two layers. So if we take this heretical approach and if we combine this text processing with certain optimization techniques like data caching, reducing latencies with multithreaded systems, and those four significant advantages or mono architectures. While maintaining superior accuracy. So we'll cover this in more detail in the upcoming slides. So one of the key considerations for a highly efficient AI summarization system is that the analysis of a large scale messaging data require certain data flow architectures to handle complex. Threading patterns while maintaining the data consistency. So we need certain components as part of architecture, and those would be scalable message processing systems or message use, essentially that can be utilized to effectively manage concurrent processes. And this is like a divide and conquer strategy where. Complex tasks can be broken down into multiple threads, which can be executed in parallel, and then we can have a governing thread that can stitch together all the insights driven by the different threads, and eventually present complete picture for data integrity. Now let's take a look at the next. Key challenges. So here I'm covering top three considerations. One is hallucination detection, other one is quality metrics, and then performance accuracy. And what are the trade offs? So let's take a look at hallucination first. So hallucination as the word indicates is something that is not factual. In the world of generative AI hallucination is an important problem to solve. And having the right systems in place to detect hallucination is also critical. So just to give you an example, if we want generative AI to be very creative, let's take an example of using chat GPT for composing a poem. From certain text that you have. So for such a use case, we require high level of creativity by the generative ai. So we would allow hallucination in that use case. However, when it comes to business conversations and business communications, we want to minimize any risks of hallucination because if there is hallucination it would mean that. It is not factual for the business, and then it can lead to significant consequences for the business, which is why we have to have hallucination detection techniques as part of the architecture. Now, regarding quality metrics, evaluation of any national language processing system for that matter, requires like comprehensive frameworks that can effectively assess. Multiple aspects of the system performance. So we need some sort of structured evaluation techniques that can accurately measure both the accuracy of the AI summary that is generated, as well as it can gauge the practical usage or practical usability of the AI summarization system. So while reviewing the quality metrics, it would be apparent to review any hallucination. Scenarios and then flag them so that it can be solved and avoid hallucination for the future implementations. So one of the considerations you should also think of while doing the ity analysis is introducing automated regressions so that let's say you are fine tuned AI summary. To be generated for a particular use case and you want to update the large language model prompt. You don't necessarily want to break anything that you have built. So it is important for you to have automated digressions so that you can catch any new issues that may arise by prompting. And in addition, it is also important to have. A human in the loop for doing any manual reviews because you cannot totally rely on automated regressions if you are doing prompting. Now let's take a look at the performance accuracy and trade offs. If you can think of improving the summary accuracy, you would think. What if I try to scale horizontally and just add more capacity and processing power so that my AI summary quality is improved? However contradictory to that understanding research demonstrates that the lean, the relationship between the computational resources spent for AI summary generation versus the output quality is not linear. So what that means is. Even if you spend additional dollars scaling horizontally, adding more computational capacity, it does not solve your s quality issues. So there has to be a robust, functional component as part of your architecture to review the air summary quality and then have different quality checks as you are tuning your prompts. Another key challenge to solve the AI summary like fairly is generating the output such as such that it is relevant to the business use case, but also it shouldn't be skewed introducing bias against one of the metrics. So let's take an example. If there's a business use case, which is about offering different pricing models for iPhones, and depending on the color of your iPhone, you would be paying a different price. So if we give this task for, to ai some AI models it is likely that based on the data that. Is fed to the large language model. It can be heavily biased towards one gender versus another. And then the pricing, if it is set based on AI algorithms, can be biased by the gender. So this is just one of the example of bias, but there can be a variety of different attributes that can generate biases for the AI summarization systems. So how do we solve these kind of problems? How do we solve the bias in the A SMR and the model outputs? So one of the key techniques is ha carefully selecting your data. So when you're training your model, your data has to be neutral. It shouldn't be skewed in favor of one trait versus another. That is one way of achieving unbiased output. The next is having sufficient data cleaning so that any noise in the data doesn't introduce any inherent biases. In addition, pre-processing the data to neutralize the dataset. So those are some of the techniques. Now let's take a look at one of the most critical challenges, which is context management. So having the right kind of context for any communication is critical, whether you're talking to a robot or you're talking to a human. So with AI summarization, what is critical is whatever context is there for the communication, it is preserved. After you summarize the text. So identifying the right actors who are participating in the communication and then effectively parsing the data based on the actors, establishing the core relationships, and then identifying the hierarchy of the communication based on the timestamps, based on the actors, based on their roles for the particular business use case is very crucial. For achieving a successful summary for complex use cases such as summarizing multi-threaded emails. So one of the key challenges is having logical output. So as you are building your AI summarization architecture, you can have different logical checkpoints in the system necessary for optimizing the context so that. The context is relevant for your communication. So for instance, if we come, if there is a con conversation about an upcoming flight reservation and a customer is reaching customer service on a date prior to their departure the context is about an upcoming flight. However, in the same conversation, if the member sends a follow up email a day after. The flight reservation, most likely they're contacting about a flight that they missed or they had to cancel or they lost baggage or something related to the flight after the fact. So with this contextual awareness, AI summarization systems should modulate, and the way this can be achieved is if you have logical checkpoints. As part of your architecture, you will be able to achieve that. So when I say logical checkpoint, it means introducing certain metadata as part of your text. So when you're processing your text, you could introduce certain checkpoints based on the timestamps in the conversation or based on who the actor is in the conversation. Now, if we compare. The performance of how these AI summarization systems perform for different use cases if it is a single threaded topic. For instance, I send an email to a customer service department inquiring about a refund for something I did not receive. It is a single thread where I have a clear intention about inquiring about something. Then the AI summarization system detect it as one actor and then it efficiently summarizes the conversation, whereas for the same email, if the customer service respond with some information and I send a follow up response, challenging the outcome of the discussion, and then the customer service agent escalates. The conversation to a supervisor. Now we have three actors essentially playing a role in this conversation. So for these multi topic threads it is getting more complex to process the text and identifying the actors and establishing relationships. So you can see that when the complexity increases, the accuracy and contextual awareness dips slightly. Now let's continue the same example and think about this use case where the supervisor is not able to solve the customer problem and the refer the customer to another department. So in this scenario, there is another fourth factor that is introduced. In addition, there's also a different department that gets introduced in the mix of the conversation. So it is important. For it to factor in the right kind of expectation for the accuracy when there is multi-threaded conversations that span not only across different actors, but also across different departments. So you can see the accuracy dips even further for cross department threats. And then lastly, for time critical threats. So whenever there, there are time critical conversations, like messages for instance, those tend to have slightly less amount of text to be processed. So whenever the delta between conversations is less, that would mean that the model would have more contextual awareness and then higher accuracy, and it can finish processing in. Now let's take a look at what drives the success for AI summarization systems? So having a good deployment strategy and pushing for operational excellence is very critical in achieving success. So the system deployment strategies can be having A-C-I-C-D pipeline. Having checkpointing system, having automated regressions, having inherent bias detection techniques, having hallucination detection techniques, so all this can be streamlined as part of your architecture as a data pipeline. And it has proven that organizations that invest in coming up with a comprehensive strategy. Planning for systematic implementation and approaching the use case as it fits the organization has proven like higher success for the summarization use case. In terms of performance optimization, we did talk about it briefly earlier. With multi-threaded architectures, we can reduce the execution times and. Wherever needed, we can have a trade off in the storage between caching versus long-term storage. So there is no single strategy that suits all use cases. So depending on the use cases definitely have the flexibility to implement a hybrid architecture where you can have the best of both worlds higher performance with. Faster memory access with caching versus long term storage and reducing the processing times. Lastly, security and scalability. So developing a scalable security framework is very crucial for maintaining the system integrity while supporting the organized growth. For more use cases, AI related use cases for generative ai, and as the business is evolving, it is crucial to come up with some AI governance strategies so that the security of the conversations remains intact as well as there is a, that there are, there is a railroad established for your use case. So that it follows a certain pattern, and then you've identified all the security constraints for your risk case. Lastly to conclude, if you want to address hallucination bias mitigation, dynamic data adaptation and context preservation, then you need. Data pipeline for it, which factors in this as part of architecture and as businesses rely more and more on digital communications, these summarization systems are going to play a very crucial role in enhancing the overall productivity of the organization as well as how the information is managed across the organization and how data is exchanged. So by implementing these strategies and frameworks that we discussed today. I hope you have the tidbits needed for creating a summarization system that can effectively balance accuracy and efficiency as well as it is contextual aware to understand the customer needs to solve the business problem. And since this is an evolving area, definitely have a robust AI governance process. Where your security of the system does not get compromised. Thank you very much.