Discover the importance of asking the right questions in machine learning models. Avoid pitfalls, maximize success, and align with business strategies for optimal outcomes.
The machine learning market is expected to reach $96.7 billion by 2025, for a growth of 43.8% from 2019 to 2025. The growth is attributed to increased data generation and demand for forecasting future outcomes. The most popular use cases are for revenue growth and cost reduction; however, combating fraud is becoming a dominant use case, with almost 50% of enterprises looking to implement a solution.
Despite the projected growth, only a small percentage of machine learning (ML) projects deliver successful business outcomes. According to Gartner, 85% of machine learning projects fail. Only 20% of those deployed result in analytic insights that deliver business outcomes. Yet, businesses continue to invest in the technology. A McKinsey report found that:
Companies, for example, are still spending disproportionate time cleaning and integrating data, not following standard protocols to build AI tools, or running “shiny object” analyses not tied to business value.
While focusing on data and protocols can increase the chances of a successful implementation, only tying a project to a business strategy can ensure a positive outcome. Failing to ask the right questions at the start of a project can lead to
- Solving the wrong problem
- Creating a deployment gap
- Forgetting about culture
Let’s look at how asking the wrong questions can impact the results of your machine learning models.
Solving the Wrong Problem
Nothing dooms a project like a vague or unrealistic goal. A general objective such as reducing operating costs or increasing revenue doesn’t help much when defining project objectives. There are numerous ways to accomplish either one. Without a clear directive, a project team may focus on areas that management did not intend. As a result, the final product doesn’t meet corporate objectives and is considered a failure.
Asking an organization what they want from machine learning models can also lead to poor outcomes. Suppose a company wants to improve its sales forecasting with an ML model. It’s clear what the company wants and how it wants to accomplish it. What isn’t clear is why. Without a clear why, team members assume that the company wants to increase revenue. The problem is how the modelling effort ties to a business strategy.
For companies using volume sales, forecasting sales require different criteria than a company that depends on relationship sales. For example, companies that depend on relationships usually have long sales cycles. At the same time, customers may purchase add-on products or recurring services that impact revenue. If a traditional forecast model is used, recurring services and possibly some add-on purchases could be overlooked.
What the company needed was a model of its revenue streams. The problem wasn’t having a better handle on new sales but assessing how well it maximized its existing customer base. Unless the right questions are asked, a project can solve the wrong problem.
Creating a Deployment Gap
Asking the wrong question means that essential stakeholders may be excluded. For example, a company’s customer support department has difficulty scheduling the correct number of agents for the workload. Some days there aren’t enough agents, which increases wait time; other times, agents wait for calls to come in. The company needs a way to forecast the number of agents to schedule.
The process seems straightforward, so management tasks the IT department with developing a model to forecast and schedule agents. IT has started to build a model to answer the question of how to schedule employees to eliminate idle agents. The developers look at historical data to determine peak call times and the average length of a support call. From that basis, the data scientists build algorithms to create a forecast model. After months of developing and testing, the model is placed into production.
The company is pleased with the initial results. Wait times are virtually eliminated, and agents are rarely idle, but forecasting isn’t running smoothly within months of deployment. Complaints come into IT. The technical support staff tries to identify the problem, but they’re struggling. Why? No one asked the question of how the model would be supported.
The modelers carefully included algorithms to address changes in the number and length of support calls. They diligently tested every use case, but they didn’t involve the people who would support the model once it was in production. The developers were laser-focused on delivering the forecast model.
What if the project started with a why question rather than a how or what? Instead of asking for a forecast model for scheduling, the company needed to start with why they wanted the model. The business value may seem obvious — reduce operational costs — but for this company, the business value had caveats. The model needed to help reduce operational costs without impacting customer satisfaction ratings.
If the right question had been asked, the project team would have included IT support. Instead, the project created a deployment gap. A knowledgeable support staff is essential to working technology. Without it, a company struggles to deliver on its crucial business strategy.
Forgetting About Culture
Underlying any project is the company’s culture. Not every business is open to technology, while some can’t wait for the next “shiny object.” Most companies fall somewhere on the spectrum of technology acceptance. Even organizations that view tech positively admit it can be disruptive. ML projects have an added constraint in that not everyone trusts the results. That’s why at the start of any ML project, a question regarding employee acceptance needs to be asked.
Ensuring a technical project’s success requires asking questions far beyond the technical scope. If employees resist technology, the model’s usefulness is curtailed. Developers can create the model, deploy it successfully, and deliver valid results, but what happens if no one trusts the results? When a model produces outcomes that go against what employees perceive, they may continue to operate as if the model did not exist.
For example, most companies use sales forecast models in some form. They may be on spreadsheets or in databases, but they exist. If sales don’t trust the technology, they will continue to use their trusted methods, negating the value of the ML model. Asking the right question can reduce resistance. If sales participated in the project, data scientists could explain how the results were produced, making them more willing to use the model.
Asking the Right Question
The right questions can make a difference in machine learning models. They can align technology and business strategies to deliver value. They can identify the key performance indicators (KPIs) to use when evaluating a model’s success. Whether creating a deployment gap or solving the wrong problem, asking the wrong question can lead to unusable results. That’s why organizations must work with experienced professionals who understand the right questions to ask.
Contact F33 today to discuss how to operationalize machine learning models in your organization and start with the right approach.
