Decoding AI Co-Pilots with Amaresh Tripathy

Decoding AI co-pilots represents a fascinating frontier in artificial intelligence and technology. These innovative AI systems are designed to work alongside human decision-makers, enhancing their capabilities and decision-making processes. They have garnered increasing attention for their potential to improve various industries, but questions arise regarding the underlying technology, data privacy, human-AI collaboration, and the future impact on businesses.

In this week’s CDO Insights we had Amaresh Tripathy, Managing Partner at AuxoAI. Amaresh is a seasoned expert in decision making and AI. With a career marked by founding, growing, and scaling AI-powered decision-making solutions, he has consistently driven double-digit growth for top management consultancies. An advocate for AI collaboration, Amaresh speaks at industry events and contributes to whitepapers on enterprise AI and evidence-based decision-making. As Managing Partner at AuxoAI since June 2023 and Senior Advisor at Genpact, he focuses on strategic M&A and Generative AI. In his previous role at Genpact, he led a $1 billion business, overseeing a global team of 15,000 analytics experts and significantly scaling the business while enhancing data fluency across the organization.

AIM: In today’s tech-driven world, the term ‘AI copilot’ has become a buzzword, with everyone eager to develop their version of this innovative concept. To gain a deeper understanding of what an AI copilot is and how artificial intelligence empowers it, let’s delve into the definition of AI pilot or co-pilot and explore how AI plays a pivotal role in making it a reality.

Amaresh Tripathy: I’ll just start with my definition of how I think about co-pilot and in what context. So the easiest definition of a co-pilot is thinking about what is the optimal man machine configuration to do a task or to a job. And there’s a difference between those two. Which is to let machines do what machines are good at and humans do what humans are good at. And that marriage of doing the work is essentially what co-pilot enables. So it’s essentially an AI companion to the work. There are many types of copilot and I think the definition is still emerging. There are co-pilots in the context of a task, you are writing a copy or you are trying to produce music or you’re trying to kind of write a letter or email. So there are some specific tasks that you’re doing, that would be co-pilots for that or what our perspective is and that’s going to be everywhere. That’s the task of co-pilots. And then they are thinking about the job, you are doing a job, a set of tasks woven together and you could have a co-pilot at a job level.

If you’re a teacher, you do multiple things, you do multiple tasks in your job. So you could have a copilot at a job level. So basically, it is an AI. I mean, I don’t like the word assistant because it means that the machine is subservient to you, but it’s probably more of a companion.

AIM: Why do you prefer not to refer to it as an assistant, despite it essentially performing the tasks of an assistant? Is there a specific reason for this distinction?

Amaresh Tripathy: If you are doing a reasoning or if someone is generating ideas, it’s almost a colleague at that point in time. If you and I are chatting and you are generating ideas or I’m saying, “Hey Kashyap, give me a few starting points around something like this.” I’m treating you as a colleague, you’re not an assistant doing it, or I’m not an assistant to you at that point in time. I’m a companion to you. So it’s a little bit more of an equal setting and a lot of the knowledge workers need companions, not assistants. Assistants kind of give you flavor of what RPA used to do which is like okay what’s going to work for you? It’s not that it’s a little bit more than that. It’s the reasoning Aspect to it. There’s a reasoning aspect to it, a brain perception. So basically there are perception agents that are like the brain or which are kind of the reasoning agents. And then there are kind of almost like action agents. And all three of them come together.

AIM: To set the context, could you clarify whether you consider something like a ChatGPT, and for instance, a ticketing system with a chatbot, to be a co-pilot or not?”

Amaresh Tripathy: You can use any of them in a co-pilot form. Which is why I think co-pilot is not only a technology. It’s a technology of how you actually wrap around, how you would use it. So ChatGPT if you’re asking a question if it’s used as a search engine. Yes, I mean it’s kind of a co-pilot in the sense but if you are using ChatGPT to say, “Hey listen, make a travel itinerary or give me a starting idea then you are using it as a co-pilot.” Co-pilot is almost like how you are also using it rather than what you’re asking it to do.

[Chatbots] In that case it is the interaction. It’s any of the bots. I mean, it’s more of an assistant at that point in time, it’s an assistant to kind of go and do that. That’s the notion between an assistant and a companion that I try to kind of call out, which is where you are actually going and using it to appear versus your asking it to do things.

AIM: Recent conversations and market research reports have highlighted the significance of high-impact, quickly deployable projects in the market. Could you provide insights into some of these high-impact, low development time projects that are already making waves? Perhaps, you could also share examples from your work at AuxoAI, where you’ve initiated projects that offer substantial impact in a relatively short time frame.

Amaresh Tripathy: There’s a broader framework that I see of broader generative AI impact that is there, which to kind of jobs requires work, which is this whole WINS framework. If you think of a two by two and say which are the tasks that have worse images, number, sounds wins and on the X axis on the Y Axis think about which of the work has been digitized. How much digitization has already happened. 

if you industries or if you put functions, let’s see if we want to do that, two by two, you certainly start getting a very interesting picture. So let’s say, it’s just industries, what you would see is very high winds, work, and very high digitisation, education, media, professional services, financial services. Which will be on the top right, which is kind of where you see a lot more disruption and think about, it’s not an accident that most professional services companies are pledging huge amounts of dollars, taking behind this effort. It’s because it’s a disruption. Same thing we have seen in education also.

On the completely other side. If you’re like, janitor, if you’re working with hands basically. Less digitisation, less wins, carpenter plumber, less disruption. So that’s one way. And you can still put the same thing on a function level. Sales, customer service, or in a hospital setting like an emergency room nurse probably it’s not of this thing. But if you’re a nurse who’s doing a lot of prior authorization and kind of doing primary care, it will fall into different things. So that’s kind of a one framework to kind of start thinking about where the disruption can happen and how quickly it could happen and where’s the value.

AIM: Let’s explore some high-impact use cases with significant digitalization. Can you take our audience through the process of how these projects operate? Could you select a specific use case and provide a detailed explanation of how it functions?

Amaresh Tripathy: We are working with a couple of clients on similar kinds of things. I’m going to blend it so that it’s to protect client confidentiality almost. So let’s talk about two use cases. One is the sales use case. And the company essentially is thinking of it as they provide services that help companies manage their ESG goals. So they kind of have a way to reduce the carbon footprint. So their whole thing is, they have a set of salespeople who are going after those in the Fortune 1000 companies and they have a couple of industrial manufacturing that they do, they have locations. And they want to essentially leverage, they want to reach out to companies which are close by to their locations and those companies should have prioritized ESG goals. And they care about ESG and then the service offering makes sense. So what we’re doing is, we’re doing sales co-pilot. And the way the sales promotion works, what we do is for all the companies in their ecosystem. We’re essentially creating almost a demand single generator. We ended up creating an ESG GPT. Almost, for all those thousand companies, we are collecting all data. They have a sustainability report, have they gone and talked about in the 10K and 10Q reports around Net Carbon Zero. And, have they talked about any programs around sustainability. On the other hand, the EPA has done fines there. Are there other regulatory things from a sustainability perspective that they have to kind of follow. Are there any jobs that they’re looking at, on LinkedIn jobs. We kind of collect all that signal and we have created this basically, an NLP framework to score their ESG rating in some ways. And if the new things come in, we are always going to continuously looking for it. So think of that demand signal representry. And what happens is now you take that and you very basically quickly say, “Okay, Let’s say this beverage company or this is this manufacturing or a drug manufacturing company and I kind of go and create based on all the signals, a score. And then that’s when the score changes, it actually triggers the salesperson. “Hey, listen, you need to go and have a conversation. It doesn’t only do that. So, that’s kind of, okay, who do you want to talk to? But it also says, because they are talking about water, landfills. Here is the particular offering you should be talking to them about from your current situation. And by the way, here are the three case studies that we have done which are similar that you should talk about.” That’s kind of the second piece of it and third is, “Hey, by the way, if you don’t want to initiate the contact, here is a personalized template email that you can just send.” That entire process happens automatically with the salesperson and the salesperson can say, by the way, I actually met these guys here. This might not be relevant. Let’s update that email or hear the three things you should incorporate. Then the salesperson can bring his or her knowledge and customize it and kind of send a note out and the same thing before the meeting he goes in, prep around how it actually happened. So think about, who do you contact? What do you want to talk about? And what’s the most relevant conversation to have? And the materials to send. That’s a sales co-pilot. And then you kind of do it again, and then you basically see, okay, you contacted What actually happened? There’s a measurement and framework aspect of it. Did they respond? Did the model get it right? If not you will get that feedback back and then the model becomes better. So That’s an example of a sales copilot that we’re working on.

AIM: Typically, when a person interacts with a copilot, is the format of their interaction primarily text-based, where they provide inputs through text, or can it take on different forms?

Amaresh Tripathy: It could be many many different ways. I mean, text is one way, so more and more, we are getting into a format where it’s normally a Q&A. But the Q&A is not even putting in text. The Q&A could be, “Here are the charts. Hey, This doesn’t make sense to me” or you could click through and say, “Okay explain this” and that explanation could be text or explanation could be another chart. So I think there is an interface that you need to do. The most common one is obviously a chat, but again, what we find is it’s normally not the most effective all the time. Because most jobs actually, you want to narrow the experience around a few questions. Like the salesperson they only care about literally five six things. What happened? When did it happen? What is the code thing that you do? So as you guide that experience through, you will create elements or user interface elements which are beyond chat. It could be a lot more simpler conversations. It could be more like buttons. It could be placed that here are the three potential questions that you should ask. And so that way you can go and do that and if you need to kind of in our world, you go to do a freeform text at the last resort. That’s not where you start because you’re guiding them through a set of processes that you’re trying to do. 

AIM: When designing a copilot, particularly in terms of the user interface and user experience (UI/UX), what unique challenges do you face, considering the various engagement methods it offers? And, as you navigate the early stages of this technology, what are some of the key learnings and feedback you’ve received regarding its interface and user experience?

Amaresh Tripathy: So, if you think about it, I mean, you use the website, anything you click, it’s a deterministic output. Here, anything you click it’s actually probabilistic output. I mean, at that point mentally, that’s the big difference. How do you design interfaces, when the outputs are probabilistic? Number one, if you want to obviously define those probabilistic outputs to a narrow range, which are hallucinations and other things you can probably talk about but fundamentally, you also need to start thinking about the interface in that potential case. You use it to an advantage. So for instance, there’s a sales assist tool that we have kind of created for Medicare when you’re selling Medicare sales that we have done and instead of basically saying “Okay for instance you’re a patient and say I want to actually shop for a health plan and is MRI covered in my health plan? It’s a simple question. The answer is yes, MRI is covered in your health plan. That could be an answer. That’s an interface and that’s right. I say, “Yes, MRI is in health plan but you have 150 dollars out of pocket and by the way in five days of these conditions, you might need a prior authorization.” That could be a much more complete answer. In the sales process you want to give a complete answer rather than just a factual answer to what has been asked. And then what you do is you can just that person answer, but the right user interface that what you want to decide is not only that but say “by the way here are the three conditions in which you would want prior authorization, do you want to know? And then you actually predict the three questions they’ll likely ask. How do you actually get the prior authorization done? How do you calculate Co pay? How do you actually go and think about “Am I already in the Co pay limit? Those are the most likely next questions they’re going to ask. Let’s say If they ask it then suddenly it’s a click, not a text thing and the agent will have more time, they will be able to have more time to engage. So your thing is what you want to engage in. What is the sales agent very good at?  Having a conversation, understanding your requirements and trying to kind of guide them to the right thing. The less it’s actually interacting with the system with that text more with clicks. It’s easier because then you’re paying attention a lot more. It’s an example. But you just don’t know where the thing is going to go. And then the whole thing is probabilistic. The answer to the problem is probabilistic and the discussion is going to be probabilistic. So you design a system that is evolving the next set of questions that are going to be asked depending on what is the starting point? It is very powerful, a probabilistic interface could be very powerful but you have to be also very careful around how you decide it. That’s an example of that.

AIM: Beyond talent acquisition, which can be challenging given the novelty of the technology, what other major hurdles or technological problems do you encounter in developing a copilot?

Amaresh Tripathy: So a few things. This notion of copilot and especially from a technology perspective, I’ll bucket in three broad bases. One is I would say this infrastructure which is not only generative AI. There’s a set of generative AI challenges. But there is a broadly AI stack challenge because none of the co-pilot uses just Gen AI. It uses a lot of machine learning, it uses a lot of other similar techniques, decision trees and any of these techniques. So as you start thinking about that, how do you take in I would say the ML and Gen AI concepts. And now you think about managing models and MLOps across both of those kinds of tool sets. So one thing is, how do you actually think of an architecture that actually is more comprehensive. We call it Hybrid AI architecture rather than just a Gen AI architecture, that’s kind of number one.

Second is within the Generative AI, that stuff changes literally on a weekly basis right now. And there are things that will give you speed. You can do an API based kind of a call with Open AI, it’ll be faster but if you end up doing hundreds of thousands of calls of a GPT4, it’s going to be very expensive. Versus so I do open source if I do LLama 2, that I have to figure out, how do I get a GPU? Where do I kind of get the GPU which I can’t get? So in the short term challenge around it or by the way I need to spend a lot more to find units. So, do I fine tune it or do I do better prompt engineering? And if I fine tune it, how do I actually organize the Q&A data set and how much data set do you have to do that? These are a bunch of questions that you need to answer at the model level. There’s a set of questions you need to answer, “Hey, how do I manage all my prompts from a prompt layer perspective? What do I think about my governance layer? If the two questions, the two people in the organization are asking, Is there a mechanism I can actually check before sending it to an API? Has this been answered for. So how do I actually think about the governance of that thing? And then what is the guardrail that the answer that came out, can I check it with maybe another Llm and do this thing? And errors are there: what is the feedback loop? So the whole art design and architecture is just the application of a lot of decisions to be made. It’s not like challenges. There’s a whole series of decisions you need to start thinking about and those decisions could optimize for speed in the beginning. But then when they optimize for scale, which is most of where it hasn’t come, whether it was enterprise or not, there you have to start thinking, it’s likely a different way. So those are the Gen AI’s kind of set up questions around that.

The third is the whole bias and the understanding that we talked a little bit about the governance around it, the hallucination. How do you measure it, and how do you fix it? And what is the right approach and a governance framework around it. For instance banks have been doing model validation for many many years. What can we learn from them to figure out model validation in this kind of a context? That’s another technical set of challenges. We talked about the UI/UX, which I think is a major one that you will do. And then the final one is integration of any of these co-pilots and everything, it’s kind of useless if it’s standalone. It has to work through a current workflow, think about the entire integration layer that you need to start thinking about all of that and that’s just the technical side of it. You talked about the people’s side, which is the talent but also the adoption piece of it. These are completely separate things, but these are the four five, broad technical challenges and the issues you need to work through.

AIM: Are there still concerns or resistance regarding the adoption of co-pilots or generative AI due to data governance issues, especially in cases where open-source models are being used for development?

Amaresh Tripathy: I just look at the world from an enterprise Fortune 1000 view of it. In Fortune 1000 we are in the experimentation, early stages. So I think there’s curiosity, that curiosity is turning into, “Let’s see what can I do and is there any value? I think these are the set of questions I think as an industry we are answering right now. If you are able to say, there is value then it’s like what are the risks associated with them? You have to understand the risks and everything and how do I mitigate the risk? So I think there’s a natural curve and evolution that is happening there. If you were to ask my point of view. Yes, I think for a lot of the large enterprises, the data security is in even smaller ones, Data security is definitely an issue. But how do you actually protect the data security in a very democratized tool like what we have right now is its own set of challenges. So there’s a bunch of things being figured out on kind of access and what the policies are and then there’s a whole set of things that you have to figure out in terms of do I do long term driven, API based, do I do open source in my own environment and see the pros and cons that we talked about, that you have to kind of go and deal with it in that piece of it. So I think those are coming. There’s a discussion happening right now and there’s absolutely worry about it but those worries will lead to solutions as we prove the value and start scaling.

AIM: Do you believe that as we move towards co-pilots and more intelligent automation, there might be some jobs at risk, or do you think that co-pilots will always augment human beings? What is your opinion on the potential impact on employment in this context?

Amaresh Tripathy: I’m a general believer of half glass half full and especially on data analytics and AI for a very long time. I think it is about amplifying human potential. But having said that, it does not mean jobs will not change. I think the key point is that the job will change. It also won’t go away. And exactly how each and every job will change some jobs will change more than the others without a question like a  customer care job. Jobs will definitely change but more than the others, but what it would look like could be very, very interesting. I mean, for instance, I had a great conversation yesterday with one of the Chinese entrepreneurs in AI. And his point was, they have been offering human agents in financial services, wealth managers, human agents and by task they also offer an option for an AI agent in Bangalore. People can choose either. Many times what they find is there are certain tasks like, preparing a will, for instance, where people want to start with an AI agent and then go to the humans and in some tasks around Hey, help me think through my asset allocation, they will start with the human agent and then go to AI. So, there are different paths. So the point is, I think the jobs will evolve and I think people’s skill sets have to evolve all along that. It’s not something that you take lightly. But overall neck neck, it says much more amplification of the work rather than replacement.

It doesn’t make a very good media story, which is why in the world that values clicks and likes, robots taking away jobs is a much more fun story.

AIM: What technological trends and innovations do you believe will define the copilot industry? Do you foresee chips in the brain or smart goggles as potential co-pilots? How is the copilot industry expected to evolve in the near future and over the next decade? 

Amaresh Tripathy: Most of the immediate future stuff I think the bigger challenge is going to be more human than technological. Kind of just working through the adoption side of it.

What it means doing that. I think going to goggles and chips and everything will probably hurt the case a lot more than help. I mean, I could argue the augmented stuff is already for a lot of field sales technicians, augmented reality and kind of what you’re doing. Those kinds of use cases, that we tried for probably 10 years and that’s what’s there. I really have seen at least some prototypes around it. So I think that’s just another interface in my mind at that point. There will be innovations on interfaces, innovations on the courtship end of the computer aspect of it, then innovations all around the stack, which is, I think, fairly normal. And the core model which is actually increasing at a crazy pace. Basically right now, large language models are learning faster than a human.

To think about what a human learns in 12 months, a large language model is learning faster. So there’s a ton of innovation already happening and I think it will continue to happen. But the core thing, I think what’s going to be important is how do you go for the value and how do you bridge the gap? So that people are comfortable using it and leveraging it the right way. And there’s a ton of technology innovation that will potentially help and kind of enhance, but if you don’t do that, it doesn’t really matter.