Hey, everyone. Welcome to the CONF42 Prompt Engineering 2024 conference. I'm going to be talking about the topic AI in product management and how the different AI tools and all the AI changes are going to be impacting the product management process. My name is Sabrinath Bijisalbam and I'm a product management professional. I work as part of the leadership team at dd. dev and I've worked across like different geographies across several hyper growth startups. Now, as a product manager, there are so many things that you tend to do, typically in your job and there are so many hats to wear. Think about the day when you had, very limited time and probably you're going to be launching some important product, but at the same time you had to sift through so much of user feedback. That was required for some of the project. Think about the scenario where you had so less time, but you had to, wait for your, analyst to come back with some insights by processing a lot of data. These are some scenarios which happens in the day to day life of any product manager. So what I'm going to be discussing is how using AI, we can actually build a very efficient product management. and that would actually give you ample amount of time, save you a lot of time, which you can then utilize for your thought leadership or any of the other more interesting aspects of product management. I want to start with a bold claim by saying that product management is dead or is it? When I say product management is dead, what I mean is three years down the line or five years down the line, product management is going to look very different from what it is today. We are already entering what I call the age of the product alchemist. What is a product alchemist or who's a product alchemist and how is the product management role going to change? there are so many exciting changes that are coming, coming the way of product management. So that's what we'll, dive into. So before we get into the nitty gritties, we need to understand like what product management looks like today. So today that is typically an, UX team or a design team, then there's an engineering team. There's also. of the business team and product management, the product management leader or the product manager tends to orchestrate things between these different teams. They also set the product mission goals, try to talk to users and so on. So these are the different responsibilities of a product manager. So be it, setting the strategy and vision at a slightly higher, or a senior level. conducting user research, be it through focus group discussions. One on one interviews with your customers, or even running surveys and processing that information. So once you have certain initiatives that are decided, the product manager tends to create the specifications, in such a manner such that it is clear to engineering on why and what they are working on and also linking all the necessary designs and making sure that is not a lot of ambiguity. Data analysis, also like one, major aspect of product management. As a product manager, you need to go through a lot of, data on a day to day basis, especially in the B2C, side where, you know, sometimes you're very reliant on your product analyst or the data science team and, waiting for them to process the data and give you the top insights. Then there's the communication aspect, the software aspect, where you're working towards aligning your stakeholders, sharing, what the current, Code map, status looks like, and what are the roadblocks, et cetera. there's all kinds of, communication you're doing, vertical communication, horizontal communication, and so on. So that's also the wireframing, reviews, part of the design reviews. Typically a part of where you're interacting with the design team, brainstorming and coming up with ideas, around what exactly needs to be the concept design, what exactly needs to be the visual design, how the information architecture needs to flow, so on and so forth, right? Then there's also the QA and testing part where, which, comes up when the product is getting shipped. Before that, there's, while there may be a QA, a dedicated QA team, as a product manager, you are expected to test all the different flows and so on. In the middle of all this, you're also probably like running meetings, sending out updates and so on. you're definitely responsible for the roadmap of whatever charter you're running. And finally, the go to market. making sure that the different teams are aware of how exactly the product is going to be launched in different markets. This can completely vary depending on whether it's a B2B context or a B2C context for sure. But however, like as a product manager, you need to be involved in that go to market, like discussing, the launch dates, the launch checklist, and all of these things with the relevant teams. Now I'm classifying these results. responsibilities into three buckets. One is a ideation bucket, which is, typically the time phase when you're thinking and we don't when you're figuring out like what exactly needs to be built. And once this is done, you enter the execution phase and once everything has been decided as part of the execution, that's when, you're also like barely aligning a lot of names. Some alignment does happen even before the execution starts that is between the ideation and the execution phase as well. What I've done is, I've referenced several resources on the internet and come up with what is the different task category and how, what is going to be the AI impact level. So for example, when you think about a strategy and vision, it's probably going to be very highly impacted because, when you write certain prompts to the AI tools like ChatGPT, they do come up with, what kind of like product strategy you should follow. Likewise, creating specifications, it is something where you spend a lot of time. A lot of, time on making sure that it's all like crystal clear, but that's something that AI can completely, do for you. while it may not really be reached the a hundred percent, of the, specification, the know polished document, for sure, it can reach maybe 75% level and the beyond, the last 25% of the last mile. is something you can definitely, polish and then send it to engineering. Likewise, in the user research, front, you're probably spending several hours together, sifting through a lot of user interviews, like drawing out insights. But once you have processed the initial data, if you can feed this to the different GPTs that are available, then they can definitely extract insights for you. And you can take the next, actions or, I try to identify like how to convert this into the feature roadmap. Likewise, the roadmap development is also something that, Chargibitty can come up with a lot of ideas, but you need to brainstorm with the different teams that are, available like the design team and, the engineering teams to finally like fine tune the final roadmap. Likewise in execution development, there are several areas where AI is going to be impacting the day to day process. the last one is with respect to leading with influence or the stakeholder alignment aspect. This is where, there's not going to be a lot of impact. It's mostly a moderate impact, be it in terms of internal communication, convincing people like meeting them and trying to build a team. case around like why you should build a product, so on and so forth. Now let's talk about what are the structural changes that are going to happen in a digital product management scenario based on all the things that we have just seen. So today there's a product trifecta, right? So where there's an engineer, there's a designer, then there's a product manager also supported by an analyst, for example. In the future, there's probably going to be a role, called the product alchemist, which would entail a single person running like multiple of these functions. There's going to be a lot of overlap by, leveraging the AI tools that are available in the market. this, role, is going to be. potentially a hands on role that completely overlaps, as I said, with design. There's one person who's probably like generating the wireframes and the mockups for a particular project and also in parallel, like writing and preparing, the code and launching products. So that's how like I visualize, the future role of the product alchemist. And this is completely possible while it might be difficult to imagine today, a world of three to five years down the line is going to look completely different. So there's also going to be a flattening of the GPM role. And that's going to be the growing significance of the senior, I see or the individual contributor. So if you see, this is how the product chart looks like today. There are, there's a VP at the helm, and then there are some product directors reporting into them. And these product director Also has some senior early PMs to whom some junior PMs report into. But how I visualize this pro, this, structure to change is that it'll become much leaner. So for example, here you can see that there are, close to 12 people, product org. But that's going to flatten because like there. The AI tool is going to do a lot, bunch of work, which frees up time, which means you need fewer people to give the same amount of output or the outcome. And that would be, I would claim that it's going to be at least like 40 to 50 percent, Reduction in the typical team size or, they're going to be leaner and more efficient teams. So we'll, in the next slides, we will look at how you can actually like leverage all these AI tools and become a more efficient PM or even a PM leader for the, for that matter. And actually drive a lot of, significant outcome. There is, this wonderful quote I came across. Online where, Abel Joseph, a founder of a startup called, , which got acquired. It's, very recently. So this person says, world class software can actually be built with less than four engineers and AI is going to further drive the efficient. So in the next section, we will talk about some of the prompt engineering techniques that are available. While we can go and ask charge GPT certain queries, as we, think about it. it may not fetch the best results, which is why using some of these prompt engineering techniques would help in structuring the prompt. And it will make sure that the GPT also understands the exact type of output that you're expecting and the search results that you get would be more relevant. So some of the common prompting techniques are one is you can assign a personality to a GPT. So for example, if let's say, you are a film, writer who's trying to, Write a script, but you want, you want the script like, as probably like a Christopher Nolan would narrate it. Then you can just say imagine you are Christopher Nolan and then try to help me with, how you would come up with a storyline or something like that. Then there's the zero short, prompting. Then there's a few short prompting where you would give the GPT some examples, meta prompting, where you can mention a series of certain rules or, specifications based on that. On which you want Chad GPT to think about a particular problem and then come up with the results. Then there's the chain of thought prompting where you give step by step instruction to the GPT and try to walk the GPT through how you are thinking about a particular problem. We look at some of the examples quickly, so this is a example of a few short prompting. So let's say you are trying to create content for a client or maybe even for a product that you're trying to. then you can give the, give the GPT some examples. So with an example, and you'll say what's the brief and what's the content. Likewise, you feed it a couple more, examples, and then you prompt the GPT to come up with something, for the latest brief. And then you would see that GPT is able to emulate the tone, the type of, structuring, et cetera, which is Quite useful, which will also like, help you in sticking to your brand guidelines and such rules that you're expecting. Then that's a chain of thought prompting. We have, two examples here. for example, the first input you are saying that Roger has five tennis balls. He buys two more cans, and that's a query, right? And you give worse answer as well. And then you are trying to give another question, but the output comes to be wrong, where you say okay, there are 23 apples, they use like 22. Make lunch and bought six more. How many apples do they have? It should ideally be like 23 minus 20 plus six, which is nine. But the model comes up with something like 27. The same thing how you would prompted using a chain of thought prompting is that you go with the step by step answers. For example, in the answer you're mentioning, rose started with five balls, two cans of three tennis balls. Each is six tennis balls. Therefore it is five plus two to three uses five plus six, which is 11. Now when you feed it, So now when you feed a similar question, the model comes up with the correct answer. So this is the power of using prompting techniques. In the next part, let's quickly do a couple of case studies and see how we are, what kind of results we are getting. We'll go for, the market research, market research that's that you need to pursue for building an app. I actually have quite a few samples here, for market research. what exactly we are expecting the GPT to do here is it needs to gather some publicly available data, develop a few buyer personas, identify competitors, and get some suggestions on how to fill existing market gaps. let's take an example. let's say, we'll tell the GPT to assume it is a market researcher or user researcher. And for the app building example, let's say we are trying to build something in the sports market and let's see what kind of results we get. This is just an example, an arbitrary example, trying to build a product, sports management canyon, publicly available data. So as a product manager, you need to be familiar with certain terms like, TAM, SAM, etc, so that you can give this feedback to the GPT. So you come back with, you get appropriate results. So in this case, I'm going to say, try to create a product. I'm going to come up with the time. Great. It's already given us an overview of like how the bucket has been growing, like what the user segments are. For example, it says like amateur sports clubs and associations, educational institutions, so on and so forth. And it's also estimating the market size. Great. This is amazing. So next, what we'll do is, let's look at, so it does give a very preliminary analysis of the market research. Of course, we need to vet the data points and see whether it's working or not and talk to users, et cetera. Let's ask it to come up with some features for one of the user segments. Okay. Let's take amateur sport clubs and associations. And come up with the primary goal is to streamline Administrative tasks, enhance communication numbers and improve the overall management. That's great. it says we can come up with member management event and schedule management, communication, notifications, managing rosters, so on and so forth. Let's see what else it's coming up with. Great. It's very comprehensive and definitely very useful as a starting point in the test of time, what I would do is I'd also feed it another, input, which is to try to prioritize some of these features. Can you prioritize use ICE or ICE is usually like Impact Confidence Effort Framework As a product manager, you would be familiar with this term and hence you just need to give this input Or Moscow is Moscow should have, could have, framework Great, it's already assigning values but this is where, as a product manager, you will still have a lot of work to do Which is, you need to actually talk to the users and Try to understand what is the potential impact from a user perception perspective as well as from a business perspective, and then try to come up with the high score. But as you can see, you can actually, once you gather that information, you can input that to the GPT and it's going to come up with a very, polished structure, which you can then use for prioritization. This is great. so I think we should just, switch to the next case study, which is around like a quick data analysis, but this should give you a good gist of like how across the product lifecycle, you tend to have multiple ways in which you can make the product management process way more efficient using GPT by giving it appropriate prompts and using different prompting techniques. In this section of the presentation, We will look at how we can use chat GPT to process the data points related to churn and the retention of the customers and see the what type of insights can we draw and how we can extend that to certain product features. But before we go into the actual data processing, I would like to, just touch upon why the retention information is very important as a product manager. As a product manager, you're measuring a lot of metrics, right? And retention metrics is one of the key metrics that you want to measure for. any of the products that you're launching. The reason being, retention is a measure of the product market fit of any product and hence you need to keep a close eye on it. Now, A better retention typically indicates a better product market fit. For most products, there are some exceptions though. now let's look at some sample data. This is publicly available, it's a publicly available data set of a fictitious fin, finance, company, and see, how we can, use that, use ChatGPT to process information, draw insights. So here we can see the customer ID, the credit score, some demographic data, then the balance of the user, how long they have been, using the product and whether or not they have a credit card, whether they are an active member, what's the estimated salary and so on and so forth. Now, let's ask dpt to help us with processing the information. Let me draw insights. customer churn data by pausing it and checking for correlations between different user parameters. It clearly mentions the approach and it says like we need data preparation, which is typically the data collection. And also exploratory data analysis is, the fundamental slicing and analyzing of data in order to give us. Baseline upon which we can further build insights. So there's also a feature correlation with churn, which is such as may not be applicable in this particular case and analyzing trends. That's definitely possible in our case. and an optional machine learning model. We wouldn't go into that in this particular example, but let's quickly, try to, upload this information and see, what ChargeBD can help us with. So this is that, file. Okay. It is analyzing. That's nice. I really like the way it has categorized information into demographic, financial, and account information. that's what, an analyst would start with, right? So by doing a quick slice and dice of what, what the situation of the data is. Oh, wow. It's also given us a correlation matrix where it's, drawing the correlation coefficient, between each of the factors. It's, it's again, a preliminary analysis. And it says that there are some direct correlations with churn, how the satisfaction score appears to have a negative correlation with exited. And also whether, that's correlation with, balance as well. Okay. It says the coalition is low. that's okay. So that's also something related to credit score. yes, let's, get some visualizations. So draw more insights by categorizing into segments. It does say geography, gender, and car type, but I'm not sure like why it is not taken up, let's say something like a salary or age which were present there. But yeah, let's give that as an input just after it finishes analyzing, this part. Oh, it's here. It's the tenure balance and satisfaction score. no problem. Can you run an analysis also, by including the age, salary, points? Let's see how these correlate to Interesting. I love the way it gets the fundamental blocks, right? Let's see, if we are able to, get some insights here. clearly it's, also, showing a little bit about, like, how, the senior, folks, they seem to be at a slightly higher, churn rate compared to the young ones. there doesn't seem to be a lot of, difference in the churn rate, based on the, salary segment or the point segment. So I think this is good. Now let's, let's think about this a little bit, why, this analysis is important. So in a typical product management setting, as a product manager, you would need to raise this, analysis request with the analytics team and depending on their bandwidth, they may already have some backlog items, which means this analysis might have taken maybe between a few days to maybe even a week or two, depending on the size of the org and the priorities. When you leverage GPT, you can typically do this preliminary analysis yourself, and you can actually request your analytics team just to build on top of this. So that's a humongous time saving that you can gain by, prompting GPT with the specific ones. So you just need to know that you need to run the retention analysis and see like how you can What are the factors based on which you need to segment? and what are the different parameters that might correlate? So for example, in this case, it didn't call out the age in the beginning, but then like I did prompt it, with, the age. And then then we were able to draw some insights. Now let's just do one last bit, which is to see if we can ask chat GPT to suggest like what kind of features we need to build to address this, to create. It has given us a series of, potential features that we can build, which is quite interesting. And Nice. Yeah, this is great. So this is a good input and we can further enhance it. But let's move on to the next part of the presentation. This is just a very quick example. So I hope the micro case studies that we mentioned are quite, were quite helpful. And if you have any questions, feel free to reach out to me on LinkedIn. And I want to conclude by saying that there's just so much excitement around AI, obviously. And there's just so many use cases using which we can leverage AI in product management. And Make our processes way more efficient, build leaner teams, and, and be able to deliver much better impact. Thanks for listening.