LLM Economics – A Guide to Generative AI Implementation Cost with Neeraj Pratap

In this week’s CDO insights we have with us Neeraj Pratap who is the CEO of Hansa Cequity, India’s leading CX service provider. He’s a problem solver at the intersection of marketing and technology, focusing on top-notch solutions for businesses. With experience in marketing, brand building, and digital marketing, Neeraj has worked on projects for major companies like Mahindra & Mahindra, HDFC, Raymond, and more. He also runs CXM Lab, a platform for sharing insights and tools for marketers and tech leaders. Neeraj holds an MBA from UCLA Anderson School of Management and is passionate about Behavioral Science. He’s certified by the Institute of Data & Marketing, London, and the Rotman School of Management, Toronto.

Neeraj delves into the critical topic of Generative AI which while it presents numerous opportunities, its implementation is far from straightforward. Understanding the balance between costs and benefits is vital for achieving optimal ROI. The interview discusses various cost components including infrastructure, data acquisition, talent, and maintenance, providing valuable insights for organizations considering AI adoption.

AIM: What made you specifically want to collaborate on this report, which scrutinizes the cost aspects of implementing generative AI in business infrastructures?
Neeraj Pratap: Hansa Cequity has always been at the forefront of driving innovation and application of new technologies. After having served some of the biggest brands in BFSI, Auto, Retail, Home building and Pharma we have also understood the relevance of differentiating hype from relevance. We have built a customer-centric, tech agnostic, integrated services company that takes pride in providing the most relevant, the latest, and cost-effective solutions. In this context it was only natural for us to understand better and help our clients take more informed decisions in the context of Generative AI.

AIM: How can tech companies use the reports and insights to guide their strategic planning and decision-making, especially in optimizing costs associated with generative AI adoption?
Neeraj Pratap: This report, in collaboration with AIM Research has been painstakingly put together with deep and extensive conversations with industry experts and C-suite executives. It throws light on the evolving landscape of generative AI, offers critical insights into the complexities and variables that influence implementation costs in today’s dynamic economic climate. The Report provides a roadmap for implementation of Generative AI. It showcases a case study approach to Generative AI in MarTech lifecycle management, cost Analysis and cost reduction. All of this put together will help organisations in optimizing costs associated with its adoption.

AIM: How can data science service providers better educate customers about generative AI and support them throughout the implementation process?
Neeraj Pratap: As is the case with any new technology, before implementing generative AI solutions based on a cost-benefit analysis, it is also important to conduct the feasibility, viability, and desirability of the project. Overall, the applications of generative AI are vast and varied, and it has the potential to transform many different industries. Data Science service providers will be an important filter and sounding board in helping organizations build relevant, cost- effective, implementable, and measurable solutions in this space. The initial phase of any new shiny object is always of euphoria and FOMO but once things settle down and pertinent business questions go unanswered, that’s when organisations start struggling. Data Science service providers live this reality on a daily basis, and they are best equipped to not only support but also help their clients navigate this space in a much more structured and outcome driven approach.

AIM: Given the report’s indication that the marketing sector is set to experience a significant impact from generative AI sooner, could you identify some areas within this sector where companies might boost efficiency through the integration of generative AI?
Neeraj Pratap: In today’s world a marketer is expected to understand shifting customer behaviour, media consumption habits, data analytics – transactional and behavioural, data privacy laws, marketing technology, new disruptive technologies and platforms, customer service and customer care and more. They are also now being held accountable for Marketing ROI. It’s a tough and yet very interesting space to be in. Some of the ways that marketers can use generative AI are – Personalising customer experiences, streamlining operations and efficiency, enhancing decision-making, preserving privacy and security, fraud detection and cybersecurity. Having said that, most organisations are encountering challenges when implementing generative AI in their systems. Understanding the costs involved and developing sustainable solutions is crucial for organisations looking to leverage generative AI effectively. There will be a steep learning curve and many curveballs along the way!

AIM: Can you discuss the strategies Hansa Cequity has adopted to upskill its employees in
preparation for the advancements in the generative AI landscape?
Neeraj Pratap: Over the last 15 years Hansa Cequity has emerged as not only one of the top Data Science and Martech service providers but also as a university and training ground for Data scientists and Martech professionals dealing in customer and marketing analytics and technologies. We are committed to helping our clients provide connected customer experiences to drive business growth. To deliver this we are continuously upskilling our employees in new technologies not only through enabling them take up relevant online/offline training programs but also through experimentative case building and designing experiments. We also endeavor to ensure that across levels people with deep skills and certifications are leading these engagements.

AIM: Can you discuss any recommendations the report makes for companies with different
priorities and budgets aiming to implement generative AI?
Neeraj Pratap: In this report, we have outlined a roadmap to financially sustainable implementations of
text-based Large Language Models (LLMs). In fact, we have also created a custom-built LLM cost calculator that will help professionals build estimates. Focusing on long-term cost- effectiveness, we have explored strategies for optimizing expenses and resource allocation, ensuring a balanced budget while maintaining optimal performance. The report outlines a strategic framework for adopting and integrating language models into various workflows efficiently and sustainably. Emphasizing long-term value and scalability, the report starts with a thorough assessment of current needs and future aspirations, ensuring the model aligns with the organization’s goals. There is no one-stop solution to all use cases. It is better to experiment with a mix of both API and self-hosted models. For use cases that need to be shipped early to measure an early impact, implementation can be done via API, while those involving crucial organizational data can be implemented through different levels of experimentation in terms of data readiness, prompt engineering, fine-tuning, and hardware. Moreover, as we find the industry at a crossroads, with the present structures showing signs of strain under the monopoly of a few large corporations, the discussion on cost reduction gains even more relevance.
Furthermore, considering the promising efficiency of narrower AI models, the guidance provided in this report could assist stakeholders in navigating the complex dynamics of AI implementation, aiding them in making informed decisions that balance both technological advancements and economic viability, thus facilitating a more sustainable and inclusive growth trajectory.

Check out the full report here