It has become popular to manage operations with data. New tools to collect and analyze data are continually appearing. But things can still go badly wrong with data. Dashboards and analytics apps rely on multiple assumptions, and if external factors change, models don’t work anymore. COVID-19 activities in many countries have been good examples of this. When you don’t know all the elements, individual numbers can be misleading. You can never manage only by the numbers you have; you must have insight, understand the environment, and be ready to look for changes outside the numbers.
Some years back, US Defense Secretary Donald Rumsfeld made the famous statement about known unknowns and unknown unknowns. Many people laughed at that statement, but it is quite an excellent way to describe reality, not only in war and foreign politics but also in business environments. Many companies focus on things they know, but they are not prepared to handle external factors they are unaware of.
Many countries specified data-based recommendations and rules during the current pandemic, e.g. tier systems to close shops, services, restaurants, or varied travel restrictions. They are based on the number of cases per 100,000 inhabitants, people in hospitals, or the virus’s R-value (reproduction rate). But most governments have been forced to change those rules and thresholds many times. It has given reason for citizens and opposition parties to criticize their actions. The reality is that it is challenging and not very intelligent to manage only by a set of numbers if many factors remain unknown. This scenario makes sense to change rules and metrics as we learn more about the situation.
Many companies focus on optimizing their operations based on the numbers they follow and measure. It is precisely why a disruption in an industry lands an incumbent company in trouble when they focus on numbers in the domain they know and recognize. Still, disruption often changes factors that they don’t monitor or are unknown to them.
The most famous examples involved mainframe computer companies when personal computers came out; for Nokia, when Apple introduced the iPhone and print publishers when online content came. Those companies focused on optimizing their operations, products and metrics in the existing business and environment. For example, Nokia was optimizing the production costs of phones, model ranges for different customer segments and their software features. The iPhone looked too expensive based on their metrics, not suitable for many customer segments and had too few features for users.
Known unknowns and unknown unknowns are relevant categories for businesses to analyze in more detail. Known unknowns are factors they know about but cannot get details or data on. For example, competitor’s future products, economic growth and the future availability of components.
Unknown unknowns include factors that we don’t know at all but that are likely to impact us. The pandemic, for example, came as a surprise, and we had no idea what impact it would have on our lives and businesses. There are many factors we don’t know about or can even imagine, but they might have a lot of impact on us.
We can also think of one more category, unknown knowns. It would mean things we know, but we don’t recognize how they impact us. For example, using available data but never thinking it will be relevant, like a company being aware of climate change data but not recognizing it as a factor in their business.
However, if we only focus on the ‘known knowns’, we can still be surprised when something changes and still not understand the real reasons for it. Many businesses and people focus only on the ‘known knowns’ and try to understand and explain everything based on that, but then they miss those three other areas discussed above, and external factors surprise them, or they reach wrong conclusions if they don’t understand that things outside their focus have an impact.
Can we do something to handle unknown areas better? Maybe the most important thing is not to think you know and understand everything, and that you must keep your eyes open for other things too. We can assume at least three categories of actions that you can do better with data and metrics:
- Don’t focus only on data that explains your current business but try to have a vision and insight to understand how the future could look, even if it changes your business fundamentally.
- Don’t think your data and models are perfect. They might be useful to measure and optimize your current activities, but there are other factors too, and if external factors change, your existing model might stop working.
- Prepare for unexpected events with scenarios. You can’t know all things that might happen, but you can make scenarios to act fast when things don’t go as you expect.
The COVID-19 pandemic and its impact on governments, businesses and individuals has taught us how companies can and need to be better prepared for unexpected events. Some of them can be big significant global events, some smaller ones like why some sales go down and why people are no longer interested in a particular product.
Maybe the most important things to remember are: 1) you cannot know it all; 2) your models are not perfect, and 3) when something unexpected happens, don’t think you can explain or handle it using old models only. As the famously coined Darwin quote goes, “the species that survives is the one that is able best to adapt and adjust to the changing environment in which it finds itself.”