Day 306 – Accuracy Costs

Every time you come up with a question to ask an organization, the price tag associated with the answer scales with the accuracy you want.

This seems like an obvious observation; however, I have years of firsthand experience that would prove otherwise. People in executive positions love to ask their organization questions. When they get an answer, they inevitably probe into its accuracy. The next step is to ask for the accuracy to improve.

This is when the costs start to rise. What is interesting to me is that often, the executive does not know or understand just how much that question is costing them. Rarely will they be delivered a proposal along with their question to let them know how much money the accuracy will cost. They will sometimes be told in terms of timeframe that the accurate answer will take some time to determine, but even then, rarely will that be associated with costs.

I will provide an example. I was on a forecasting call in which the executive team would ask sales leadership what the forecast was for a given quarter. During that call, one of the sales leaders explained that their forecast would be difficult because they did not have enough information on booked business that was waiting to be shipped by the manufacturer. This was referred to as backlog, which is to say, a backlog of orders that could be billed once a product had been shipped.

Sales leaders always cited this backlog as an “excuse” for why the forecasted numbers were not accurate. One day, in response, an executive asked the leader of a particular operations team if he could get an answer as to what items in backlog would be shipped in the current quarter. The operations team, who valued their jobs, said yes, that was possible. A week later, on the same forecasting call, the operations team showed the report. The report was not completely accurate. Many shipping dates needed to be included or corrected or had all been set to some mythical date in the future.

In response, the executive asked if they could get a more accurate report on the backlog with completed and verified shipping dates. The operations team, once again, said yes. A week later, they produced a report that had all the dates filled in, verified, and seemed to be relatively accurate. The executive was very pleased, and he was so happy that he asked them to show this same report every week. This would clear up the confusion and eliminate using “backlog” as an excuse for not knowing the accuracy of a forecast.

What the executive should have seen, and was never shown, was the cost associated with this accurate backlog report. This backlog was gathered from order details, which frequently had missing or incomplete shipping details. There was no easy way to gain this information from all vendors, so the operations team resolved this by scheduling calls with sales leaders to ask about each order and get the correct information. Another call was planned before the weekly forecasting call to report this to the operations management team. 

The sales leaders needed the answers so that they would ask their midlevel management the same questions. This led to more calls being scheduled before the operations call occurred so that the sales leaders could complete this detail. The midlevel management needed this information to schedule a call with the team responsible for order placement. This call was planned before the midlevel management call with sales leadership. Finally, the team responsible for the orders also needed this information, so they would start sending emails and making phone calls directly to manufacturers to collect the details.

This one question: What orders in the backlog will come in this quarter? turned into at least four more meetings for each sales team and countless emails and phone calls to the manufacturers. We had the most accurate report possible for shipping backlog, but at an extraordinary cost. There might have been 15 sales teams, so we are adding 60 hours weekly in team calls between various people.

I know what you are thinking, but Guy, AI can handle this type of thing. True enough, but at what cost? You certainly cannot say free. That would be just as naive as an executive who wonders what his team is doing all day when he keeps finding out questions that no one has an answer to. It is amazing to me how often we are doing work within modern companies with only one purpose, and that is to answer questions from the CEO.

I work with a company that specializes in AI implementations. We will be the first to tell you that it is possible to answer a natural language query like this and provide a fast and accurate response using AI tools. However, we cannot promise to deliver this at no cost. Accuracy always costs; it just matters where you want to spend money to achieve the level of accuracy you want.

I have been told that getting to an accuracy level of 80% is relatively simple; however, the last 20% of accuracy requires the bulk of the work. This is a rule of thumb to remember if you start asking an organization seemingly simple questions and asking for more accuracy. If you are looking for ways automation and AI could help your business, look no further than the questions you are demanding your team to answer.

Subscribe
Notify of
guest
0 Comments
Inline Feedbacks
View all comments
Share the Post:

Recent Blogs

Day 310 – I Actually Love Opera

The author explores the reasons why people might hesitate to reveal certain aspects of themselves, using their love for opera as an example. They differentiate between privacy and concealment, arguing that while privacy is a healthy boundary, concealment stems from a fear of judgment and can be a burden. The piece concludes by advocating for the freedom found in being less concerned with labels and more comfortable with revealing one’s true complexity.

Read More

Day 309 – The Price Tag of Automation

The author recounts his frustrating experience with home automation after Google abandoned its Nest architecture, leading him to build his own system. He highlights the significant time and effort required for maintenance and the eventual abandonment of his automation efforts due to the high ‘economic life’ cost. However, with the advent of AI, the cost and effort of automation have decreased, making it more feasible and renewing his interest in deploying future automations.

Read More

Day 308 – Speed is The Champion

This article explores the concept that speed often leads to success in various fields, from business to technology and sports. It argues that systems capable of rapid action, feedback, and adjustment tend to outperform those focused on avoiding errors. The author highlights the importance of fast feedback loops, citing John Boyd’s OODA loop and its application in AI systems. The piece also discusses the speed-accuracy tradeoff, noting that while speed can lead to more mistakes, these are often recoverable, and the compounding effect of rapid learning ultimately drives better results.

Read More

Day 307 – Iterative Could be Insufficient

This article explores the contrast between iterative and linear thinking, highlighting their respective strengths and weaknesses. It argues that while iterative thinking is beneficial in many fields, it can sometimes be an excuse for insufficient upfront planning, and proposes a hybrid approach for more robust decision-making.

Read More
0
Would love your thoughts, please comment.x
()
x