Gartner Talk on AI Transformation & Innovation

Generative AI
Turning hype into reality!
Philip Basford
© 2024 Cognizant
November 2024
Senior Director (Data & AI CTO)
Technology Consulting UKI

By 2026, more than 80% of enterprises are expected to have used or deployed
Gen AI-enabled applications in production environments1
2
Market Trends
Market by the numbers
$1T
Projected annual value added to global
economy by Gen AI by 2032
76.8%
Projected 5-year Gen AI adoption
CAGR2
Sources: Gartner1
, International Data Corporation2
, Everest Group3
90% of jobs could be disrupted in some way by Gen AI
© 2024 Cognizant | Private
However, right now 80+% of AI use cases fail to deliver on their full potential

4
The realized impact of generative AI on the labor market
will depend heavily on business adoption rates
* Adoption rates vary by industry and by firm size (small vs larger). The charts below reflect the national average adoption rate.
Time profile of adoption rates
High
Adoption could ultimately
reach 100% of all
companies
Central
Low
15% may not adopt in
our 30-year timeframe
Year
Adoption rate
13%
2023 - 2026
Adoption will start off slowly,
with 13% of businesses
implementing by 2026
31%
2026 - 2030
Adoption will then spike
to just close to a third
of businesses.
46%
2030 - 2032
Nearly half of businesses
will adopt gen AI by the end
of the decade.
Source: Cognizant and Oxford Economics
© 2024 Cognizant | Confidential &
Proprietary
Wave
1
Wave
2
Wave
3

5
Three waves of change in focus
Wave 1
Low-risk
experimentation
Gradual
adoption
Caution and
uncertainty
Business adoption
(“high” scenario)
13%
Wave 2
Regulatory
clarity
Lower barriers
to entry
Reshaped business
and operating models
Business adoption
31%
Wave 3
Widespread
use cases
Proven
reliability
Meaningful role in decision
making and strategic insights
Business adoption
46%
© 2024 Cognizant | Confidential &
Proprietary

6
Customer Success Stories

Input:
Business objectives
Start Small and Prove Value Quickly
Proven Approach for Successful AI Implementations
Enabled by accelerators
to deliver speed-to-business value
Rapid delivery of business value

Automating audio transcription at
scale
Provider of business and administrative support services,
working with global clients in complex and highly regulated
environments, including the legal and financial services
industries.
Challenge
• Reduce the time spent transcribing audio files for clients
• Improve transcription accuracy vs existing automation
tools
Approach
• Created an Intelligent Document Processing pipeline
• Custom ML models developed in Amazon SageMaker
• Generative AI (Amazon Bedrock and Claude LLM) to
convert dictation commands into formatting instructions
• Built a bespoke user interface to allow for human review of
transcriptions, ensuring accuracy and improving model
performance over time
Business outcomes
Improved experience for customers
and employees
Accuracy rate of 92.5%
Increased speed – processing 8 hours
of audio in less than 30 minutes

PREDICTIVE AFTERSALES TO DRIVE SUCCESS
CASE STUDIES
 Customer-specific measures are calculated for
Loyalty, Propensity for Upsell and Pricing
 Customers engaged in the pilot with personalised
reminders or offers from Salesforce
Approach
 Data Ingested from Dealers in Dubai, Abu Dhabi, Kuwait,
& Saudi Arabia
 ML Models that forecast current mileage and propensity
for needing a standard service or brakes service
17% yearly growth in
visits to Dealers
12% average quarterly
growth in after-sales
Response rates 73% for
standard and 56% for
brakes services emails

Gartner Talk on AI Transformation & Innovation

11

UK Insurance due to the demographics of their client base, the
pensions and annuities business is very paper based. The
complexity comes with dealing with correspondents in their standard
forms (semi-structured), letters & emails from layers and financial
advisors (unstructured). Therefore, the forms must be digitalised and
re-keyed before processing as traditional OCR is not adaptable
enough
Challenge
• The client has ab incumbent GSI for IT with little knowledge of
problem domain / Gen AI
• AWS Professional Services built Phase 1 but needed a trusted
partner to handle Phases 2 & 3 including a Managed Service
Capability.
• Data is PPI and PCI, & highly sensitive with no access to real
data for training
Approach
• Work jointly with AWS Professional Services to move the solution
into a Managed Service
• Performing regular FM OPS and deploying fine-tuned small LLM
using synthetic data
• Developing new features using Generative AI Models (SageMaker)
with few-shot prompts for new LoB
Insurance: Pensions and Annuities

CASE STUDY
The Customer:
AI in Action: Transports and Logistics
The Sector: Logistics
 4m address and time-slot predictions per day embedded into web and
mobile channels, modernising the entire customer experience
 Improved transit-time prediction for global logistics and
increased accuracy in shipping-date predictions by 74%.
 Improved contact centre efficiency, eliminating 40% of
inbound customer calls related to shipments.
The Solution:
The Requirement: Customer Service Transformation
Aramex wanted to enhance the efficiency of their customer contact
centre operations.
Ø Designed and built the integration of numerous AWS services
Ø Configured a scalable data ingestion process
Ø Deployed over 100 ML models in production, covering 10 use cases
and processing 1.5m rows of data every 15 minutes
Ø Delivered a secure enterprise-grade platform in just 8 weeks
The Result:

CASE STUDY
The Customer:
AI in Action: Demand Prediction
The Sector:
The Result:
The Solution:
The Requirement:
 Multi ML model approach for each region/state to handle social,
economic and environmental trends that impact which model of car is
bought, i.e., hatchback vs MPV, normal drive vs 4x4, electric vs
combustion
 An ML model per car model is then used to predict the extras that will
be ordered so that the correct configurations are built, i.e., sunroof,
colour, engine size
 Guard rails and weighting are built to allow for business adjustments,
such as selling a new or more profitable model of car
 Reduction in the amount of time analysts are required to look at sales
data and calculate the order for the next month. The analysts now look
at outliers and exceptions
 Successful results in initial target regions has led to full commissioning
in every region and full productionisation to be complete in Q4 2021
Top Global Car
Manufacturer
Automotive
Ground Stock levels are low due to supply chain
issues, therefore predicting the cars and configurations
to be built next is vital to fulfilling next quarter’s orders

15
Gen AI Architecture

APPLICATIONS THAT LEVERAGE LLMs AND OTHER FMs
TOOLS TO BUILD WITH LLMs AND OTHER FMs
INFRASTRUCTURE FOR FM TRAINING AND INFERENCE
GPUs Inferentia
Trainium SageMaker
EC2 Capacity Blocks Neuron
UltraClusters EFA Nitro
Amazon Bedrock
Guardrails Agents Customization Capabilities
Amazon Q Amazon Q in
Amazon QuickSight
Amazon Q in
Connect
Amazon
CodeWhisperer
AWS Generative AI Stack
© 2024 Cognizant
16

17
Foundational models on AWS
AWS provides secure access to the widest range of FMs
Custom Model
Open Sources
LLMS
Use cases &
capabilities
Amazon
SageMaker
Flagship service
• A full service for
Machine Learning
• API or batch consumption
• Pay per min/hour pricing
• SageMaker has access to latest
hardware, including Inf2 and Trn1
• Cognizant has access to a wide
range of FMs (proprietary and open
source)
• Cognizant has worked with
AWS at becoming specialists
in distributed training, initially
using Hugging Face
Amazon
Bedrock
New service
• Managed service for
proprietary FMs
• Proprietary FMs require
EULA with FM author

• FMs can be fine-tuned on
your own data without
sharing your data with
everyone
• Agents, Guardrails & Model
Eval

• API + Batch based
consumption
• Now GA in limited
Regions
• Pricing per token
Proprietary LLMS

Secure Generative Architectures
18

19
Gen AI seems easy but…
Source: Eduardo Ordax – Generative Lead @ AWS
Starting playing with LLM model with quick
demo/prototypes is rather “easy”
However, moving towards a Production
grade Gen AI solution is a different story…
Þ LLM models are not NEW to us -
We have started leveraging them
for 3 years already.
Þ And have been able to implement
production grade solution
embedding LLM models

20
Enterprise Knowledge Navigator
“Please give me the current share prices for 10 best
performing FinTech companies in the past 5 years
and summarize their performance”
Advance search/QA
The ability to search inside private documents, images,
or websites to find related content and then return that
content
Retrieval-augmented generation
Integrations with live systems to augment the results
with up-to-date information or perform actions that
may be required
Security and privacy
Private FMs are not like internet SaaS products; your
data is not shared and is kept securely

21 © 2024 Cognizant | Private
Bringing Gen AI to life
Example with our Frankie AI agent/demo leveraging Cloud Secure Generative Architecture
• FrankieAI is a standard React / NodeJS
application.
• All files are uploaded and stored securely in
Amazon S3.
• Extracting the contents of files is done with
Amazon Textract.
• Langchain JS is used to store and retrieve
extracted document information to and from
a vector store.
• Langchain is also used to call AWS Bedrock
LLM models (especially Claude V2 + Titan
Embeddings)
• All information is held in standard secure
data stores. Amazon Bedrock does not
store prompts or use information passed for
training purposes.

22
Enterprise Knowledge Navigator: Data lakes
The ability to help the business user to interact with their data lakes and produce insights
Benefits
• Quick access of data to explore key insights or generate new insights from the data lake; no SQL expertise needed in writing a good SQL
• ~60%–70% productivity gain; ask question in natural language and let generative AI (FMs) do the rest of work in generating insights for you
Conversational interface
Providing a simple interface that allows the
business users to speak/chat in plain English
using domain-specific phrases
Code and domain understanding
Creating domain-specific code to retrieve
information contained within data products
within a data mesh
Outcome playback
Generation of reports or a playback, containing
generated graphics and text summarizing the
result

Enterprise Knowledge Navigator: Data lakes
23

24
Responsible AI and Regulation

25
Challenges our clients are facing with AI :
“My Lines of Business have developed & deployed models &
AI solution with no oversight or consistency”
“I operate in a regulated market with specific needs and I need
to know if I can use (and prove that I can) use my AI solution”
“I need to be able to operate AI with different risk profiles in
various geographical markets with different and a mesh of
regulations”
“How can my organization transform with AI, where do I start
and how can I do it in a managed & responsible way ”

https://www.gov.uk/government/consultations/ai-regulation-a-pro-innovation-approach-policy-proposals/outcome/a-pro-innovatio
n-approach-to-ai-regulation-government-response#a-regulatory-framework-to-keep-pace-with-a-rapidly-advancing-technology
Safety, Security & Robustness
Appropriate Transparency and Explainability
Regulators
• Competition and Markets Authority (CMA)
• The Information Commissioner’s Office (ICO)
UK AI Regulation
A pro-innovation approach to AI regulation
Accountability & Governance
Contestability & Redress Legal Aspects
• Consumer Duty
• Equality Act 2010
• Computer Misuse Act 1990
• Data Protection Act 2018
• The Copyright, Designs and Patents Act 1988
Fairness

AI Regulations
The EU AI act is a proposed set of regulations designed:
“To ensure a human-centric and ethical development of Artificial Intelligence (AI) in Europe, MEPs endorsed
new transparency and risk-management rules for AI systems.”
The degree of regulation applied by first a classification of risks ranging from no actions required,
strict requirements and obligations, to outright prohibition:
• Nearly all AI usages use data therefore they are also subject to GDPR and other data privacy regulations
Any breaches of the AIA Act will be fined by the regulator; however, additional legislation is coming to
allow for class or individual prosecution of a company and their use of AI
Unacceptable:
• Real-time biometric systems (exceptions in law
enforcement)
• Social scoring algorithms that evaluate individuals based
on personal characteristics.
• Manipulative systems that exploit the vulnerabilities of
specific individuals to distort their behaviour
High Risk:
• Biometric identification and categorisation of natural
persons
• Management and operation of critical infrastructures
• Education and vocational training
• Employment and worker management
• Access to essential services (this aspect applies particularly
to AI systems used in the financial services sector)
• Law enforcement
• Border control management
• Administration of justice and democratic processes
Low Risk:
• Include systems that neither use personal data nor make
predictions that are likely to affect any individual directly or
indirectly
Limited Risk:
• Limited risk refers to AI systems with specific transparency
obligations. When using AI systems such as chatbots, users
should be aware that they are interacting with a machine so
they can take an informed decision to continue or step back.
Al System
(Use
case)
Risk-
level?
Unacceptable
High Risk
Limited Risk
Low Risk
Use case may be permitted
but must comply with strict
rules concerning risk
management, data quality,
and technical
documentation
Use case may be permitted
but must state clearly in
terms that AI is being used
and how
No Action
Use Case prohibtted
27

28
US State AI Governance Provision Types
Assurance
• Registration
• Third-party review
Governance
• AI governance program & documentation
• Assessments
• Training
• Responsible individual (owner)
Individual Rights
• Opt out/appeal
• Non-discrimination:
Trustworthy AI
Guiding
Principles
Data Privacy *
Transparency
• General notice:
• Explanation/incident reporting
• Labelling / notification
• Provider documentation
Blueprint for an AI Bill of Rights
Executive Order on Safe, Secure and Trustworthy AI
USA regulations considered:

Responsible AI
Our Responsible AI principles address these by promoting respect for people and society and to drive trust
across the AI lifecycle:
Our principles are upheld by two foundational layers – technology and governance:
Fair &
Inclusive
• Engage diverse perspectives
• Define fairness and inclusivity
goals
• Minimise biases
• Promote inclusivity and
accessibility
Safe, Secure &
Privacy-enhanced
• Design for safety, security &
privacy
• Identify potential threats and
develop an approach to combat
threats to the AI system
• Collect and use personal data in
a lawful and ethical manner
Transparent &
Explainable
• Promote open communication
• Design for transparency
• Design the model to be
interpretable
• Encourage continuous
monitoring and feedback.
Accountable
• Establish an end-to-end AI
lifecycle strategy.
• Keep a human in the loop
• Define roles & responsibilities
• Document, monitor and update
the AI systems
Technology
• Resilient to failure and able to recover
• Reliable and accurate outcomes
• Robust and performant
• Supported by documentation
Governance
• Be comprehensive and integrated
• Manage AI-related obligations and risks
• Measure and manage impacts
Environment &
Sustainability
Economic &
Social Impact
Ownership,
Copyright &
Authorship
Ethics &
Regulation
We help organisations innovate with AI in full consideration of wider contextual impact
Roles & Responsibilities
Policy, Procedures &
Standards
Operating Model
Data Governance
Risk and Compliance
Software Development
Life Cycle
Model Approval Tooling
Monitoring & Evaluation
Data Management
Access Controls
Logging and Auditing

The role of AI Governance
 AI Governance is needed to increase & speed up AI
innovation, adoption & maturity by providing a path from
concept to realisation.
 AI Regulations (and existing Data Regulations) are a mesh of
requirements that require most uses of AI to be governed.
 AI strategies must focus on increment iterations, validating
Business Value, Understanding the Cost of Adoption,
Managing AI risks, and embracing Responsible AI.
 AI Governance is a collective responsibility, not just ML Ops
or a Centre of Excellence task.
 AI Governance requires a cultural shift across the entire
organization.
 Managing AI requires updating or creating rules (policies,
guidelines and standards) and roles for managing an
organisation's use of AI.

AI Governance & Risks
Comprehensive and integrated
31
four core functions:
• Govern: Establishing policies and procedures to manage AI risks.
• Measure: Assessing and analyzing AI risks and their potential impacts.
• Map: Identifying and understanding the context, scope, and nature of AI risks.
• Manage: Implementing strategies to mitigate and monitor AI risk
with 72 AI Actors and Tasks covering:
• Affected Individuals and Communities • Fairness and Bias • Operation and Monitoring
• AI Deployment • Governance and Oversight • Procurement
• AI Design • Human Factors • TEVV (Test, Evaluation, Verification, & Validation)
• AI Development • Domain Experts • Third-party entities
• AI Impact Assessment • End-Users
mapped access the AI Lifecycle:
0
Applicati
on
Context
Plan
and
Design
TEVV
includes
audit and
impact
assessm
ent
Data &
Input
Collect
and
Process
Data
TEVV
includes
internal
and
external
validation
AL
Model
Build
and Use
Model
TEVV
includes
model
testing
AI
Model
Verify
and
Validate
TEVV
includes
model
testing
Task &
Output
Deploy
and Use
TEVV
includes
integratio
n,
complian
ce testing
&
validation
Applicati
on
Context
Operate
and
Monitor
TEVV
includes
audit and
impact
assessm
ent
People
and
Planet
Use or
Impacte
d by
TEVV
includes
audit and
impact
assessm
ent
TEVV
AI
Life
Cycle
Key
Dimensions
Other Frameworks:
NIST Artificial Intelligence Risk
Management Framework
(AI RMF 1.0):
= 170+ Controls (Processed & Tools) with clear accountability

CASE STUDY
The Customer:
AI in Action: AI Governance and Risks in action
The Result:
The Solution:
The Requirement:
 Definition of Principle / Control Domains
 Definition of AI Lifecycle Stages based on NIST AI RMF
 Comprehensive framework of actionable AI risk mitigation and technical
controls based on NIST RMF
 Alignment of controls to current risk management processes
 Prioritisation and definition of technical enhancements for initial use caset
 Recommend NIST AI RMF 1.0 mapped industry, national & global
regulations + standard
 Identified 14 Risk Process owners over the AI RMF 1.0 AI Life Cycle
 Mapped 72 AI actions and tasks in AI RMF 1.0 to internal TEVV
controls (both process & technical )
 Supported the go-live of first use case with initial 117 required controls
The Sector:
Global Insurer
The organisation was attempting to productionise both
ML and Gen AI use cases in regulated markets. The
organisation needed to make sure its current
governance and risk management is updated to handle
AI challenges and regulations
Insurance

33
Let's Talk about AI
Transformation!
Thank you

Gartner Talk on AI Transformation & Innovation

More Related Content

What's hot (20)

Similar to Gartner Talk on AI Transformation & Innovation (20)

More from PhilipBasford (17)

Recently uploaded (20)

Gartner Talk on AI Transformation & Innovation

Editor's Notes