Assessing the ROI of AI takes time. Not only does it require some level of research and consideration on behalf of the project leader (an internal AI champion in the enterprise, or an external AI consultant), but a project leader also often needs time and participation from various stakeholders to estimate costs, think through challenges, etc.
For this reason, it behooves AI project leaders to ensure that they only take the time to assess AI opportunities and challenges that warrant an ROI analysis. In the sections below, we’ll cover project selection, problem framing, and the importance of having multiple project options for reaching strategic goals.
Selecting the Right AI Opportunity
What we want to think about off the bat—before we start thinking through the ROI of AI—is if we’re selecting the right problem.
We want to ask ourselves: Is this what we want to work on?
It takes a lot of effort to structure an ROI case: to think about examples and to find representative use cases.
There’s a lot that goes into it. So, we ask the question: Is this where we should put our focus?
To begin determining whether a project is worth focusing on, think first about client priorities.
Clients have near-term issues, and they have longer-term strategic anchors. Here’s an example that shows the relationship between making the AI business case and those strategic anchors:
Source: (Emerj Plus) How to Succeed with AI Projects – Lead with Strategy
These are things that clients might care about, such as three- to five-year goals. So, we want to consider these and ask the question, do we have the right problem identified?
For example, clients might come in and say, “Hey, we need a chatbot.”
Then when you talk to them about their near-term issues and talk about their longer-term strategic goals, it’s pretty self-evident that maybe that chatbot solution isn’t the best fit for them. Next up, we might want to look at their opportunities in the data itself.
So given the data that they have on hand, and considering:
- what they’re working with
- the state of their data
It’s possible to do a very quick data audit. We might ask ourselves, where are the actual opportunities?
Then, well, there’s project viability to think about.
So, finding the right AI opportunity is based on how realistic the project is, and the AI maturity of the client.
How far along are they?
Are they talking about some kind of insane, far-out, hyper-futuristic AI use case when they’re still writing things on yellow pads?
In that case, we’re probably not looking at a very viable match.
Is this a project that even makes sense?
Is the project realistically possible given their level of maturity?
Then lastly, here, we can look at precedents of use. For most clients, this is a very informed view of AI, even if some people find it pessimistic.
Project Selection: The Value of Being Realistic
Let’s say we’re going to use computer vision for this very specific manufacturing problem where no computer vision system has ever been used before. Most clients are not ready for screaming-edge AI-related innovations.
But, some will be. The key is to realistically look at the expertise and resources on-hand and make the decision that’s right for your client or organization. Some clients will be massively funded with huge R&D budgets. Compared to their peers, they’ll be very digitally savvy and they’ll have the in-house talent needed to thrive with a big AI project.
Only under those circumstances, when we’re talking about market-leading companies with large R&D budgets, with great in-house staff, and with the right kind of digital maturity to start with, only then should we realistically consider screaming-edge, unheard-AI projects.
Most companies are not going to be served by doing something like going to the moon. They should instead be focusing on much more realistic objectives. If we’re going to a project selection phase, we want to find some kind of precedents of use.
If nobody in the world has done it, or anything like it, there’s a good chance that it’s way outside of your ballpark. When we talk about initial projects, we should begin with projects that are accessible in a way that’s going to be realistic for a client or company.
So precedents of use become very important. Clients don’t always think that way, but it’s often important to encourage them to consider this way of thinking because we want the highest level of success with our early projects.
Framing the Problem or Opportunity
When we want to frame the problem and opportunity, we want to be able to bound it in a controlled way and understand what it is that we’re studying.
For AI, this is often much more elusive than it initially seems, but thinking through bounding the problem can provide tremendous business value. Let’s consider a couple of examples.
Reducing Fuel Consumption in Airplanes
Let’s say we’re talking about reducing fuel for our airplanes. Let’s say I’m a big airline company like American Airlines or Lufthansa.
If I want to reduce the fuel use in my airplanes, we’ve got to figure out where AI fits into this goal. We don’t just throw AI in the fuel tank and then wait until it tells us where we could save money on fuel. That’s not how it works.
We need to think about how we’re measuring fuel, the places and parts of our journey, and of our processes where fuel might be wasted. We need to consider where AI might be a realistic fit.
For example, we are talking about wasting fuel on all of our flights. In other words, is there some way to apply AI to every flight, or are we talking about just longer international flights where there’s more potential to consider drivers for fuel use such as how the engines fight against the weather?
Making Customer Service More Efficient
Let’s say we’re routing call center calls or chat conversations, and that’s the use case we want to consider.
We need to consider first, why?
If we know AI gets somebody on the phone with the “right agent,” are we looking to improve customer satisfaction? If so, how?
Are we looking to reduce the time it takes for customers to reach an agent? Are we seeking a solution that can reduce the time it takes for them to get their issue resolved?
Preventative Maintenance of Manufacturing Equipment
Below is an example of how a general business priority might be distilled down to a bounded problem or opportunity for AI:
Initial business priority: Preventing catastrophic breakdowns of manufacturing equipment, reducing cost (in time and money) of ongoing maintenance.
- Framed problem or opportunity:
- Data used: Manufacturing machine data. Production data. Telemetry sensor data (heat, vibration, etc.).
- AI application: Train algorithms on historical and present data streams to find correlations with previous breakdowns and issues.
- Workflow changes: The addition of sensor upkeep and sensor-testing processes.
- Desired result: Predict future machine breakdowns with multiple historical data streams, use less FTE time, and have fewer errors per hour of machine operation.
By breaking down a business problem into a set of components, teams can get aligned about what the AI application involves, how it might be measured, what it might cost, etc.
From AI Idea to Well-Framed Opportunity
Boiling down a general AI idea (or loose aspiration) to a well-framed problem or opportunity often requires ideas from data scientists, subject-matter experts, and business leadership.
The more members of the project team who are familiar with the range of AI use-cases, the better – as existing precedents of use are often the best jumping-off point for new project ideas.
There are many ways that we could think about that, but boiling down the problem is going to help us determine how we’re going to measure ROI. Measuring ROI for reducing fuel, for example, is just not good enough.
We need to consider:
- The junctures of the process where artificial intelligence would fit in
- The data it would use
- What kind of judgment call it would make in that process
- How we could use that judgment call
- The bounded space where AI can find its fit in a business process
When we answer these questions, we can start thinking about AI ROI.
A Portfolio Approach to AI Projects
When we consider a portfolio approach to AI, it’s useful to have more than one AI project on your list.
Think about your AI transformation vision. What do we want to change or transform? How does AI fit into that vision?
When you consider this leveled-up digital transformation vision, think about the capabilities we need to build and a suite of potential projects that fit into that.
We want a portfolio of ideas. We want to generate as many ideas as we can and consider them before we decide on which ones to propose to the client, execute, and invest our initial dollars in.
Before we ever talk to the economic buyer, we want to think through a lot of projects. We want to boil down those that will offer the highest impact. When we talk to the economic buyer, we’re going to present more than one potential project, depending on how big of an initiative this is.
A smart AI maturity roadmap is going to involve several projects and we may not be able to accomplish all of them simultaneously, especially if we’re a small company.
Maybe these projects have to be done in succession, as different kinds of tests and experiments. Often, we’ll go to the economic buyer with a handful of projects—that represent variety. As a portfolio, these projects might be done together or in succession, but they all will contribute to the same transformation.
With AI, we won’t always know what’s going to work and what won’t. However, we’ll know the capabilities that we need to build, the strategic value we’re striking at, and the transformation vision we’re targeting. We will need projects that are in alignment.
A portfolio approach to AI can present a lot of ideas that we can consider and then bring to the economic buyer.
We have to have this mindset from the start. If we start to narrow our scope to a specific project for which we’re going to measure ROI, then, we don’t have the right focus. None of us have crystal balls, we don’t know which projects are going to work out and which ones won’t.
Fractal Analytics’ Chief Practice Officer, Sankar Narayanan, put it this way in a recent blog post:
The approach the company took was to give every initiative a six-week time to show progress. What it allowed the company to do is become more rapid, not just in defining problems but also in defining the quantifiable measurement of success and progress. So, in 12 months, 30-40 different initiatives were executed to achieve Minimum Viable Outputs (‘MVO’) out of which 5 to 6 specific initiatives were taken to organizational scale.
(Interested readers may want to listen to our full interview with Sankar titled: How to Measure the ROI of AI, which was our most downloaded podcast episode of 2019.)
Moving Forward: Assessing the Upside of AI Projects
While your organization might not take an aggressive approach, such as adopting a massive focus on experimentation and a top-down alignment that is beyond the scope of possibilities for most enterprises, it’s important to note that we need this kind of focus on testing viable and reasonable AI project ideas to find pockets of value and build enterprise-level confidence in the technology.
It takes time to assess the ROI or AI. Constructing a portfolio of potentially valuable ideas can help us plot a course so we can move ahead. While it takes time, resources, and effort to properly calculate our costs and benefits, this is a critical step that we must complete before we move forward to the next phase of the process.
Check back next week when we’ll explore how to apply Emerj’s AI ROI Model when we assess the upside of AI Projects.
