Excellent Guidelines for Identifying Metrics

Book Review: Kenneth G. McGee - Heads Up: How to Anticipate Business Surprises and Seize Opportunities First

This book was almost great. A great spin on just about every business process monitoring (BPM) approach going - that business surprises can be avoided through predicting the present following his reporting methodology. Some well chosen examples illustrating each of his well-developed steps build to a major case study at GM that unfortunately seems to lie well outside the book's core approach. McGee concludes with inventive speculation about the future role of BPM that taints the well-reasoned logic of his thesis.

The book begins with examples from recent history and business to support the idea that all business surprises actually have early warnings, but these aren't recognized in time to respond. He forms three categories - surprise events, suspected events, and surmounted events. The difference between the three types of events is whether early warnings are ignored, received too late to act, or acted upon. The book focuses on what is required to identify causal events in time to act. I couldn't help but think of this as shortening an OODA Loop to avert a crisis.

Chapter 2, Identifying and Justifying the Right Real-Time Information, provides excellent guidelines for identifying metrics for a departmental operational dashboard. McGee starts with Goals, prioritizes them, then filters the goals that can benefit from real-time information. Once the candidate goals have been identified, early warning metrics are brainstormed. Using corporate priorities, an ROI analysis identifies which metrics will be tracked. The analysis begins and ends with goals - not with what metrics might be readily available.

Following some case studies, McGee moves on to the Real-Time Enterprise. In this case, the objective is to shorten the reaction time to trigger events to surmount surprises regularly. Metrics are chosen with maximum corporate impact in mind. Critical business processes are retooled to allow continuous data collection. The problem I had with this was the example McGee used, reengineering at GM.

GM's recent dramatic improvements in quality and efficiency are used as an example of this Real-Time process, but unlike earlier cases, there is no direct connection to the steps in the text. Some actions are contradictory - the CEO monitoring daily sales figures - while developing strategy. The improvements seem impressive, although recent delays with the Solstice introduction may indicate that some processes are a bit too lean.

The last part of the book is dedicated to the exciting future of the Chief Monitoring Officer. This sounds like an interesting position, a C-level executive with few staff and a wide mandate, reporting directly to the board; but politics and Sarbanes-Oxley will make this a stressful position. I grasp the importance of real-time monitoring and I really like his discussion of selecting metrics, but I didn't make the leap to CMO with the author.

Settling the Settled Data Issue

Bill Inmon concludes a recent article Settled Data is Best for the Data Warehouse with this statement:

The most accepted school of thought should be, that when transactions occur, that those transactions should not be rushed to the data warehouse but should be allowed to "settle" in the operational environment.

I think this conclusion says more about Bill Inmon's view of a data warehouse than it provides practical advice. Data volatility is an issue that needs to be tackled head on and not ignored until the data is settled.

Mr. Inmon suggests that pulling data into a data warehouse in real time leads to messy complications that would be avoided if we wait 24 hours to a month or more to transform each transaction. For some strategic applications this might be reasonable, but for more and more applications, the data is less valuable as time goes by.

Mr. Inmon's example is an incorrect phone bill, where a call to India was identified by the customer weeks after the bill was issued. By waiting a month or more, these issues are supposed to settle. Even if this was a legitimate approach to handling data volatility, how would you decide when data was settled? Perhaps the customer had already paid the bill, then found the error. Perhaps the customer didn't find the error for three months. If the correction came after the settlement period, wouldn't we still have a mess?

Suppose we can wait for transaction information to settle for analyzingng long-term payment patterns. It would be reckless to use the same settlement rules to manage call volumes. To have variable settlement times for different applications of the same call transaction adds another level of complexity.

We need to stand back and assess the data volatility in the context of our business requirements. If our business needs require near real-time analysis, then we must either accept added volatility or added complexity. If our business needs require an absolutely static view of the data, we need to accept that these are historic snapshots and may contain errors that have since been corrected elsewhere. In either case, the business requirements must come first.