Evolution of Decision Making in GenAI Era

Every new technology invention has led to certain benefits and posed some challenges. As various technologies progressed, human behaviour has evolved around the technology, from paranoia to acceptance to close integration, where it complements our lives.

So I came across a post on Medium in which the author mentions about challenges of using the GenAI tools.

In this article the author discusses the challenge of defining value in generative AI (GenAI) for organizations. She argues that unlike traditional AI with one right answer, GenAI produces multiple valid outputs, making traditional ROI metrics insufficient. The article emphasizes the crucial role of leadership in defining success criteria and developing appropriate measurement strategies for GenAI's multifaceted results.

While I have been using GenAI to get answers about existential crisis various things, I get all sorts of answers, some satisfactory, while some need further querying.

Let’s take the case of image generation. Most of the image generation tools that I have used come up with 4 possible options to chose from. While what design choices led to the tools giving only four options, not more, not less, is something to be thought about.

With this background in place, each time I use the GenAI tools for text generation, it takes me back to school time when my mom used to make us study.

Mom: Asks a question…

Me: Give some random answer

Mom: Looks at the write answer in the textbook and visibly miffed, asks the question again

Me: Comes up with beat-around-the-bush answer

Mom: (Now having lost the patience, goes and comes back with a 12’ wooden ruler and keep beside me. Asks the question again

Me: (Sweating profusely) finally manage to come up with answer that matches the one in textbook

Result: Mom happy, me happy (for not getting myself thrashed)

Whenever I use the GenAI tool, I get a somewhat similar feeling.

Me: Ask a question

GenAI tool: Gives an answer

Me: Knowing, it is a probable answer, out of many possible answers, refine the question, this time pressing the keys on the keyboard harder and bringing a glass of water for the GenAI tool

GenAI tool: (Sweating profusely) comes up with some text

Now the challenge is unlike in the school curriculum, the questions we have don’t have a unique answer. There might be ‘n’ number of possible answers.

Like it happens in Strategy, Marketing or other creative fields. Rory Sutherland, in one of his podcasts mentioned, the opposite of a good idea is another good idea.

Everything is probabilistic in nature, nothing is deterministic. And this very fundamental change in mental model I experienced when I went to B-school.

Two things almost every professor used to say, whenever asked a question

1)      ‘It depends’… so much so that it became a reason for huge round of laughter

2)      There is no right or wrong, everything is context dependent

Coming from an engineering background and having worked in product design, there was a strict adherence to follow standards and guidelines, so that the structural integrity of product is maintained throughout its life.

However, the world doesn’t operate like that, and everything is nebulous.

Now coming back to GenAI and its application in enterprises.

How will the decision making evolve based on the output generated from GenAI tools?

Will we need more subject matter expertise than before to verify whatever comes out of the tool is safe to act upon?

In stats there is a concept of Confidence Interval.

What Is Confidence Interval? A confidence interval shows the probability that a parameter will fall between a pair of values around the mean. Confidence intervals show the degree of uncertainty or certainty in a sampling method. They are constructed using confidence levels of 95% or 99%.

The GenAI tools are sophisticated versions of statistical models.

While some tools provide a response along with a few reference links, so that one has an option to verify, will a numerical value ‘confidence level’ be displayed against the answer in future?

Somehow our brains work better with numbers.

Another aspect that needs to be thought about is term AI, as it is just another (though sophisticated) form of technology.

Let’s look from the perspective of customers. Customers pay for getting their problems solved. The least they care about is how it is getting solved. Either by humans, or technology, or the new technology GenAI. Are they looking at GenAI from the lens of efficiency gains or is it just the Shiny Object Syndrome overpowering the definition of true value created by using the new technology?

Moreover, as the GenAI technology provides intelligence, a lot of it depends on the data it has been trained on and data it will be used to provide responses to the questions asked. A lot will depend on the data maturity of the organizations to derive true benefits from the technology.

In almost all competitive exams that we attempted, we were asked to select one right answer from the four possible choices.

While selecting one image out of the four possible options might help doing a non-critical job, how will the new technology and decision making of humans evolve while choosing possible answers from almost infinite possible options, is something to be watched out for!

P.S.: No human or LLM was harmed while writing the post, :P. Neither was I harmed while studying :D