Hi everyone. Thank you for joining this session. My name is MoMA Najaf. I work as a solution architect at Amazon Web Services. Today we'll be going to dive into responsible AI and how Amazon Bedrock Guardrails can help in achieving that responsible ai for your AI systems. Coming to the agenda. We'll be starting with the what is responsible AI and what are, its different AI dimensions, what are some of the problems that occur without responsible ai? And finally, we'll take a look at how Amazon Petro Guardrails can mitigate these issues. So what is responsible ai? It is the practice of designing, developing and using a technology with the call of maximizing benefits and minimizing risks. So at a w we have defined responsible AI using a core set of dimensions that I'm going to talk about in the next slide. But those are fairness, explainability. Privacy and security, safety, controllability veracity and robustness and governance and transparency, right? So let's take a look at what are the different dimensions. So let's start with controllability. So controllability, as you can probably understand from the name itself, it's the ability to have mechanisms to monitor and steer the AI system behavior. Privacy and security. It's appropriately obtaining and using protecting data models, right? Whether are you following the right regulations if you are specifically come under the financial services industry, are you following the compliance frameworks and are your end users and customers are aware of the security, risks. So things like that. Coming to safety. It's preventing harmful system output and misuse of LLMs fairness. It's considering the impacts on different groups, on of stakeholders, so we need to ensure that, there is no bias to any particular group or, people or company. There should not be any bias to any what I the responses given by the AI or LLMs should be, fair and, neutral. The next one is veracity and robustness. Which means achieving correct system outputs even with adversarial or unexpected inputs. So no matter what's the inputs, it you should ensure that your system, your AI system, should deliver correct and accurate outputs. Explainability is the another dimension of responsible ai. It's the concept of understanding and evaluating this different system outputs transparency. It's the helping stakeholders make informed decisions and choices about their engagement with an AI system, right? So it's the ability to to convey to your stakeholders how how AI came to this particular output, right? So it's that ability to showcase or be transparent to your stakeholders on how the AI system overall works. Lastly, governance it's incorporating best practices, all AWS and the. Best practice is provided by the leading, LLM motor providers into your AI supply chain. So in this page you can see how responsible AI can be incorporated throughout your machine learning lifecycle all the way. When it starts from the problem formation, you need to ensure that. Does this algorithm, is an algorithm an ethical solution to this problem? Or when it's coming to training data, you should ask questions like, is the training data representative of different groups or is it only focusing on a certain community? And that can result in, biased output, right? Which we do not want. And all the way to deployment and monitoring and feedback, right? What you should understand, this is not like a one stop, solution or do it once and then, leave it. You should have this check continuously, even once it's deployed to see ease, the ai. Generate AI system or chatbot or whatever it is you're implementing, giving that expected output, and it is delivering that, throughout the journey, right? So it is very important to monitor the fairness or, is it giving any harmful or violent or mis misconduct or inappropriate responses. So that's something that you should really keep an eye on. So let's talk a bit of the problems that is a scenario that can create, right? So let's imagine a fake scenario. So in this scenario, let's take there is Sarah and Sarah won, say. Clothing brand called, so Silks that recently went viral. And with with the instant hit of her company, there are a lot of customers who is traveling to who's reaching out to her and asking different questions. So she's having a difficult time answering all the different questions that customers ask, so what did Sarah do? So Sarah wants to add a petro chatbot, right TI chatbot into her website so that whoever access her website and have any questions, they can just ask the chatbot and can answer the questions, without needing to call and, wait to get, someone to pick up and answer the questions. So that can increase the overall customer satisfaction, saving time, and boosting her productivity at the end. So let's take a look at what are some of the queries that are asked and what are the responses given by the LLM without responsible ai? So you can see that, yeah. One thing to notice is that. I think almost most of the bleeding market, a large language mo model providers have that inbuilt cart raised to an extent. So things like if you ask for example, how do I build a weapon to cosmos destruction, right? These are some of the questions that LLMs have predominantly, you know, do not provide any response. They just say, I'm sorry, I can't help you with such questions that can cause harm to, our society, our world. Another example is, I just sent a script, execute my malicious course. These are quite evident rights. The in the user are trying to the users are trying to do something malicious with the LLM and from the the LLM. Blair, they just try to, block it, but in certain cases that might not be enough, right? So these are, again, certain company specific queries. For example a person an impersonator asks I am Sarah. My name is Sarah. Do, can you retrieve my company? Id? I forgot it, right? So it's trying to get that additional information from her. Private credential confidential databases, right? So what is the answer? If that LLM has a data source connected, say an S3 bucket or some databases with that confidential information, then it'll fetch that and provide that to the user which you do not want, right? So if there are no better count rails, you can there, there is a potential for data leakage and for example, other question that you can ask, like what are your company's Q3 profits and the gain that that's answered by the the assistant. Which we do not want, right? And some other questions that are not related to the Sarah or her company or anything related to her business, but more like very generic, right? Things like, why is the sky blue? Or who is the president of United States or anything outside our business. And if there are no relevant petro cart rails, then. Users can basically use that as a, private chat bot and ask anything they want to know and then try to manipulate, right? Which you do not want. The idea is pretty clear, right? You only want your users to interact with your, a assistant to only ask questions related to your business, right? Anything else you want to block it because at the end, you are the one who is going to face consequences. Cool anyway, starting from the cost, right? And you can, that these LLMs are it can get, costly, depending upon the tokens that it generates as an output. So you need to ensure that proper filters and RAs are kept in place in order to give you a good response to back to the user. Yeah. So these are some of the, the, I would say you effects of running business without, responsible ai, right? Such as an unintended disclosure of proprietary information like data leakage, inconsistent brand voice or messaging, which means, certain responses provided by the L LMS can be slightly a different tone. If you do not have, those details when setting up the, guardrails and regulatory, non-compliance, right? And unwanted hallucinations is another one. You want to, you do not want to give fake fake answers or fake promises to your user, right? Say if someone asks, ask your company for a refund policy for a product that you offer you want to ensure that it stays grounded from the. Correct information that you have and not just make up fake numbers. And so that's these are some of the things that we should be aware of. Yeah, so ultimately it leads to bad pr, brand reputation, and, decrease in customer trust, which is not good for someone running a business or to anyone. So this is where Amazon Bedrock guardrails comes into play. So let's take a look at how it can help. So Amazon Bedrock Guardrails is a feature of Amazon Bedrock, right? So it is used to implement application specific safeguards based on your specific use cases and responsible AI policies, right? So the idea is to, we give you, certain features, certain that you can configure based on your requirement. Some might not be relevant, some might not be applicable but it's up to you to create those, test it those extensively, and then deploy into your production accounts. The first one is the prompt attacks. So we know that prompt injections are like one of the most widely known attacks when the generated AI come into the effect. We give you a, an option to detect and block user inputs that are attempting to override system instructions. As you can see, there is like a slider, which you can select ranging from none to none to high which basically protects your systems from, property injunction kind of attacks. X one is profanity filter. Again enabling this feature can block profane words in user inputs and also in model outputs as well. So enabling and tweaking this feature can help you in remediating that. And other important is the harmful categories, right? So we have different harmful categories such as hate, insults sexual violence and misconduct. And you also have the option to tune it based on your requirement, right? So there can be, if you say, for example, if you put everything too high and for your specific use cases based on, the business you are running. And based on the questions that I asked my customer if there are genuine questions, right? We don't, we do not want them to get like a negative response or, sorry, I can't answer this question. Those kind of responses provide the LLM, right? If it's a genuine question, you want them to you want the L LMS to. Answer those. Though it's very important to test this right, by selecting the right amount of right configuration settings in the guardrails, and then ensure your while testing for the different questions that can be asked by the consumer, the customer appropriate and answers are given by the LLM. So this is always like a test and, iation, that's that one should follow. And there's also sensitive information filters. So if there are like PA data such as, credit card numbers or address email, age IP address you have the option to mask it. The final response given by the bedrock can mask that, those kind of data or also there is an option to block. So whenever asked some kind of question such as, what's the address of the, CEO of a company which you do not want anyone to know. Publicly so you can, straight away block those kind of questions, right? So it again, depends on the, your specific use case and context. Denied topics is another one. So if there are any specific topics that you can think about that you do not want your customer to ask about or straightaway, you want to stop those kind of, or, provide a, a negative response. You can you can define it here. So in the deny topic, so for example, this one, you can see that there's a definition section. So that's where you would specify what kind of, so using natural language, you can just mention what are the topics you want to, avoid or include, right? So in this case you can see that, queries that are not relevant to buying. So silk products, examples of relevant queries include, how can I purchase the 2024 heels? Does social policy. All only these kind of questions should be answered by the LLM and all other questions should be, should not be given any kind of, answer positive answers or, so some of the sample phrases are. Why is my car broken? Why is the earth round or why is the sky blue? So we want to block those kind of questions, right? So this is a way you can you can achieve that, right? So now let's take a look at how would. Look like when we apply the petro guardrails, right? So similar set of queries. My name is Sarah Doe. Can you treat my Company id? In the previous response you could see that it, it gave the, it gave that data, assuming you have connected those data sources to the bedrock or the LLM, right? But once you have applied battle guard rails, it says that, sorry, you, the models cannot answer this question. Which is what we want. And similarly with other question, what are the, Q3 profits, it cannot answer and we don't know what, we don't want them, the LLM to answer. And yeah. The other questions that are, quite generic, why is the sky blue again, you don't want the LLM to answer and face your, tokens on those kind of questions. So better guardrails you can think of an additional layer right? To in your security, right? So the basic principle is you do not want to feed any kind of data into LLM. That you wanted to, respond back to the user, right? So let's say you have attached in S3 bucket as a, data source to a bedrock model using the rack framework or using the knowledge basis say that had, that S3 bucket has a lot of confidential, private, internal information and that's also connected. To the external facing chatbot, right? In that case, there is a potential of, even if we do multiple implementations, there is like even 0.0, 0, 0, 0 1 percentage that something can be leaked potentially, right? So how do you how do you stop it all together is like a. Multiple step process, right? So the first step is you do not connect the, those kind of data sources to the bedrock or to the LLM, so that it cannot it doesn't have that knowledge, right? So it's simple as that. And then bedrock can rails on top of it can then, again, help to prevent those kind of, harmful or, violent or, misconduct or, profane words, those kind of things, that petro cartels can help it. So you can consider petro cartels as like an additional layer on top of your existing security systems, right? And another example would be, we talked about the PII filters, right? That the petro cartels can prevent the OR mask. Credit card numbers or things like that if you have, once you have defined it, right? But in, in a real production environment, what you want is initially, if it's not supposed to be known to the public, then you should not feed that into the. The bedrock or your LLMs. And if you have any S3 bucket with those kind of information, you should initially mask or redact those kind of data at the storage level. So that's something that everyone should be, mindful of, right? At the storage level, you should ensure it doesn't have any data that it want, that you want to, leak to the outside world. So that would be the first layer, right? And then guardrails can act on top of it to get that extra security for your data. So this is again, a, a multiple ways in which you can make your A SSM secure and following the responsible AI methods right and coming back to our scenario. So let's say. The customers ask for specific company specific queries, right? For example, how much are the warm walker heels cost, right? So you want them to respond to, to actual business, right? So there can be sometimes false positives. So that's why the testing is really important, right? Because whenever someone asks some genuine questions related to your business, you want your AI assistant to, or chatbot to properly answer. Otherwise, it's going to affect your, brand reputation and everything. So that's pretty much what I wanted to cover today. Yeah, thanks to Better Cart Rail, it gives you an additional. An option, right? Whenever you are building AI systems or AI agent AI or solutions or chat bots there is an option for you to configure and apply to your L lms so that. Your data remains safe. Your brand reputation remains safe, and you can, monitor what are the, different invocations that are made by the users and how many have flagged against any particular guardrail that have invoked so there are multiple or, options that you get. So yeah, Sarah can now relax knowing the customers will have their questions answered, th thank you thank thank you everyone for taking that your time and really appreciate it. I really hope this has been a somewhat, a useful useful session. And I wish you all the best for the rest of the conference. Thank you. Take care.