Data and Analytics 2.0 with Shashank Garg

In this week’s CDO insights we wanted to capture how the next phase of analytics is going to look like. In this age of unprecedented technological advancements and rapidly evolving business landscapes, the role of data and analytics is undergoing a transformative shift. Data and Analytics 2.0 represents the next era in harnessing the power of data to gain deeper insights, drive informed decision-making, and unlock new possibilities across various industries. 

To give us more insights into this, we had the chance to talk to Shashank Garg, Founder & CEO at Infocepts. As the CEO of InfoCepts, a global Data & Analytics solutions firm, Shashank is passionate about helping organizations become data-driven and stay modern by building reusable solutions for the most challenging data and analytics problems. Beyond his professional work, he mentors early-stage startups through TIE and leads the NGO, Orange City Runners, motivating over 10,000 people to embrace running for fitness. A dedicated father to two teenagers, he draws inspiration from tech leaders who’ve turned to philanthropy and is eager to connect with fellow Data and Tech leaders, CEOs, and Change Makers for the exchange of transformative ideas and experiences.

AIM: Why did your team choose to focus on the subject of Data and Analytics 2.0, especially in the midst of discussions about other topics like Gen AI, and what makes this subject particularly intriguing or relevant to your team?

Shashank Garg: Let me just briefly introduce our firm. We’ve been in business for about 20 years, founded by experienced data analytics professionals. And if you look broadly, the industry we play and people use the term business analytics in one phrase. And after having spent many years in this field, having worked with several large organizations across the industries. I want to tell you and everybody who’s listening that “The words of business and analytics continue to be further away than most people think they are.”

So, for example, if you were to ask CDO what they thought the business value of everything that they’re doing is, they would probably rate themselves, very high, maybe 90%. You ask the same question to a business user, like the actual person on the ground. What’s the value of the whole data analytics ecosystem? They will probably rate themselves 30-35%. So this gap is very real. And even after two decades or three decades of tech revolutions, it continues to be a very real gap. And that’s the gap that we are here to solve. By resolving the most common and complex challenges that come in the way to make data driven decisions, which is the backbone of AI and unlock the hidden value that lives in the data to advance the business forward. We are an end-to-end data analytics solutions provider with an increasing focus on AI. And we’ve taken the approach or what I call productized solutions. It is essentially packaging, processes, expertise proprietary technologies, open source all packages together to give predictable outcomes to our clients. So that’s sort of a quick background of where we are.

AIM: How Has the Landscape of Data and Analytics Evolved Over the Last Two Decades, and What Impact Has This Evolution Had on the Discrepancy Between Expectations and Reality in the Industry?

Shashank Garg: What I have seen evolve in this market and I’ll talk about the market in general. Data used to be fundamentally the domain of the IT departments of large organizations. So the CIO or the CDO, all anchored in central themes. They used to own the data, make all the decisions, which platform to buy, how to build up the teams, how to service, what process. That is starting to shift. We are starting to shift more and more. Business leaders realize that if they need to be data driven, if they need to drive their business using data, they need to have a far more active involvement in all the decisions that happen relating to how data is collected, processed, where it is and how it is serving up the new analytics that they want to use their business to drive. So there’s a change in buying patterns. From a very IT centric buying to a very business centric buying. That change is very real and that’s reflected in how vendors are now shaping their approaches, the global market approaches.

The industry is not that recent actually. It started with client server technologies, then web came. So all of that Onprem tooling, what you may call, now referred to as Legacy Tech. Still has a good 50, 60, 70 in some organizations, 100% of their data. The analysis and processing of data is on a set of technologies, which I would call not modern. But in the last 10 years, as you were saying, 2013 onwards, this emergence of this term is called modern data stack. As the Hyperscaler’s built up the cloud services, it became more and more easy to create new technologies. And you saw a surge of technology in data analytics. 15 years ago, the number of technologies that you had to choose from was less than 40. Today’s 3000. Everybody went into this mad rush to modernize. We have to modernize our tech stacks. And I call it sort of the race to modernize. And today, we are in a scenario where several organizations who are sort of the early adopters of this modern technology happen to have data spread across three clouds. About eight or nine data pipeline technologies, a few lakes and about four visualization tools, which serve up the analytics in addition to the analyst and data scientists who use Excel. It’s become very complex, it’s becoming far more complex that it needs to serve the same need which was to give me the data at the right time with the right analytics so I can predict where I’m going for my business. The need hasn’t changed in the last 20 years, but look at the transformation on the tech side and what we’re doing to that. So I’ll call it the race to modernize and they need to simplify now. They’re still companies who have not moved in some industries. Some industries tend to stay ahead than others, Data is universal. So there are still companies who need to modernize but there are absolutely organizations who need to focus on simplification of their data and analytics ecosystems. Third thing, obviously, everybody talks about AI in general, especially generative AI will have a very real impact on everything data. It is here to change many things. So, if you look at the last 15 years of AI and bucket it has largely been based on Supervised learning where supervised learning can sort of predict, people use people call it data labeling. So it’s very good at labeling stuff. Self-driving cars are in application, quality control applications.You’re labeling defective components. But now Generative AI, organizations are going to rush into processing far more unstructured data than ever before. So, the field of analytics was mainly about structured data, that’s about to change with Generative AI.

The problem of governance and ensuring that you have a single version of truth is already a challenge. Imagine what happens when a lot more unstructured data starts coming into the ecosystem. So they need better architecture, standards, governance and security. And especially ensuring fairness of the algorithms, responsible AI, that becomes very, very important. And lastly, all of us have consumed analytics either through Excel reports or tools or SAS tools or dashboards. That is definitely going to change. We call it sort of disruption in how we consume insights. The way you look at a report, there is a visualization there at the top where you consume data. And an enterprise user is very used to that but I think that’s going forward to be far more conversational. The way we are used to conversing with each other right now.

AIM: In the evolution from viewing data as the property of data owners to today’s cutting-edge applications with intelligent co-pilots for decision-making and process optimization, what were some of the key challenges we encountered during this transformation, and what challenges do you anticipate as we progress from data to intelligent decision-making in the future?

Shashank Garg: It’s kind of unfortunate, but I’m gonna say the challenges remain the same. 

We have two decades of immense progress in tech especially. Number one challenge it’s sort of the race to modernize. I see people out there just working really, really hard. And they’re sort of stuck in this perpetual modernization cycle. We used to be here. Clients have gone to the SAS model cloud. And now we have this tool and now we’re going into this tool and I think people need to slow down and look at “Is it really needed?” And instead of focusing on perpetual modernization, what can we do to really focus on the value? So this sort of stuck in perpetual modernization, I think is a challenge.

I do think organizations struggle really to build great data and AI teams. That is not easy. And it has become a lot worse in the last three years, especially post covid when the whole digital wave, really took a step forward. There is no company of significance you will find, which is not hiring for data and AI talent. The Big Tech, Fortune 500, Large consulting, Boutique consulting, Digital natives you call it. It takes a while to produce the data and AI talent. You just can’t take somebody. So they need data skills, math skills , they need understanding of UX visualization. How humans perceive data. And they need all the business context which cannot just be taught in universities. You need to go through the rigor to understand the processes and what data you’re collecting and what you can really use to predict. So that’s all the functional knowledge, which just takes a while. So from my perspective, the demand for this talent grew sort of 3x in two years. But you just can’t produce enough people. And you act with the complexity of how people need to collaborate between businesses and IT, you need a certain level of maturity and you need strong leadership. So it just continues to be very hard to build Data and AI teams. I’m not just talents, I’m just saying teams. 

Lastly, in all the spend we do on data or tech, it’s all to produce a value for the business. It has always been hard to pinpoint the number and I’m a math guy. I’m a numbers guy. So if  I am investing, I am asking ROI, if I am investing X am I going to get 5x, revenue left, cost decrease or reduction of risk. That’s how I measure ROI and many leaders struggle to define that upfront when they invest in Data and AI. It’s a much easier case to make that Hey, we should do this and in future and call out all the intangibles. But the moment we start challenging ourselves and say, Hey guys, let’s just put tangible numbers against it. It becomes very hard. Of course, because most of it is based on the future. Having the discipline to say something, track it, measure it, is just something that I don’t see organizations doing a very good job at.

AIM: In light of the persistent challenges such as data quality, talent availability, and understanding contextual business problems, why do we see a shift towards Data and Analytics 2.0? Should we transition to 2.0 if we haven’t fundamentally addressed the issues from Data and Analytics 1.0?

Shashank Garg: There are organizations who have solved these challenges. It’s just that there are many more who haven’t. Not that these challenges cannot be solved. I think a lot of them have to do with leadership and culture. So this is fundamentally a field where the tech team cannot do anything by themselves, the business teams cannot do anything just by themselves, until they come together. It just sits at the intersection of many things, and it’s the collaboration and it all time comes down to leadership and culture. So there are excellent examples of companies getting this right. But I’m just saying there’s many more examples who haven’t gotten this right and it’s a work in progress. Coming to your sort of DNA 2.0, or what’s happening today and a large part of that has to do with the field of AI.

And especially Generative AI. If you take the last 10, 15 years it was all about supervised learning and our ability to predict and label certain data. So we’re able to classify and hence, do certain things faster or be able to predict things that we couldn’t do before. I think generative AI just makes that supervised learning process and our ability to predict way better. So

I think if I just apply what’s the impact of Generative AI to the field of Data analytics, a few things come out. First of all, Generative AI is good at content generation. It can help in content consumption. And it can cause a serious acceleration in tech enabled innovation. If you look at it broadly, “What can Gen AI do for DNA?” If that’s the question. But it is very important to also know that conceptually, fundamentally, Gen AI apps are far better suited for where analysis is required, pattern recognition is required and based on all that you come up with certain outcomes, which are more probabilistic a little bit on the creative side and inductive. So, Gen AI outputs, generally you should apply in scenarios where precision is not important. And there are no definitive answers. For example, categorizing content and responding to Broad inquiries or translating languages. But when it comes to reporting, and being specific you have to be very careful in using Gen AI. In the short run though all of this will play out. There are a lot of things that haven’t been figured out in this whole field. But people will continue to work on it and figure this out. In general, if you say in the next 12 months what we expect. In the next 12-15 months I absolutely expect application of Gen AI or LLM within the DNA field to primarily improve end user experience, simplifying data operations, assisting engineering themes, and the whole co-pilot use case. However in the long run, I expect Gen AI to be used far more on things like classification of data. And In the field of data quality, classifying data is a huge problem in many fields. And right now it’s all rule-based. I think Gen AI can do much better in those areas in the future. I absolutely expect that. So what we’ve seen in the next 15 months is going to be significantly different from what we see let’s say two years hence and  I’m very hopeful there.

AIM: What advice would you offer to CDOs worldwide to proactively stay ahead of the curve in this rapidly evolving environment, where intelligent technology is empowered by data, analytics, and insights, while these technologies, in turn, rely on data and analytics to enable their capabilities?

Shashank Garg: I’d say one or two things. I would say, approach the new problems and even the old problems with an AI first mindset. If you look at how these leaders grew and look at the sequence, they all grew with a certain set of technology and then AI came. And we need to think in that approach and that approach leads to incremental measures. I think it’s time to flip and say “AI First”. If I were to solve this problem that I have solved for 15 years in a certain way can I flip it and say I have none of that? Can I just use it with an AI first mindset and how would that look and what can I get to. And where I can get the accuracy, I’ll solve for that. We are at the point where we can shift our mindsets to AI first and human second. 

Secondly, look for end to end solutions. When you look at a business problem and solve it end to end with the state of data and AI you absolutely have an opportunity. Don’t solve parts of it. On the people’s side, If I were a CDO I would be training my  data analyst and business analyst to be story writers. Right now they do the jobs of doing the analysis, putting it all together and serving up for people to consume. That’s where Gen AI will take away a lot of that. But with Gen AI, you run the risk as with social media you only hear what you want to hear. If you ask a specific question it gives a specific answer. So train them to be story writers, train them to be data journalists if required because they can still bring in the context with. If i don’t ask the prompt it will not answer. But a journalist will insert the context that is required to build the right story. 

And lastly, focus on building your business values story. That’s an absolute must.

AIM: What do these three points directly mean for a data professional working on a day-to-day basis? What can he or she do to ensure that the data and analytics technology being built today is responsible and provides equitable opportunities for society? And, looking forward, where do we go from here?

Shashank Garg: If I am an aspiring data scientist or a data professional, this may vary but I think that I should focus on asking the “Why?” and really getting into the root of the problem I am trying to solve and becoming business oriented, that is an absolute must. 

I worked in times where getting the business context was so very hard, but with some of what is happening in the public domain, Gen AI, maybe it’s not hard. Use it as a learning on the business side because sometimes it’s very hard for them to get time from the stakeholders. But you can. So just becoming business oriented to be able to articulate the value of the problem you’re trying to solve is absolutely important.

As for you, it depends on where you are in your journey. We use a term internally called “Cross skill”. It’s very important that you don’t just become a data engineer or an analyst or a visualisation engineer or a cloud engineer. You can develop deep expertise in one or two areas but you have to know because we are shrinking teams as we talk. We no longer have 20 people to do a data project. We have a 3-4 person team. You have to know deep enough on the conceptual side but on the tool side you have to know more than one.

Lastly, really jumping ahead and becoming AI fluent. Being able to understand the basics, the math behind the AI,  using them for your advantage but in the business side you’re using co-pilot and ensuring you understand at a root level how these foundational models or general purpose AI can be used to solve a variety of problems. Because as they say “AI is not going to replace but a human who knows far better certainly will.”