Being a piece of data is not as easy as it sounds.  How would you like to be trapped in a vestibule only to be accompanied by the likes of endless tables and foreign keys?  The lifecycle of a piece of data may sound as simple as CREATE, RETRIEVE, UPDATE, and DELETE, but I assure you that the frustrations that abound make life as a particle of information a bit more difficult than you might imagine.  Put another way, I could be doing much more efficient work for my organization, but due to the way I am treated, I am unable to even come close to reaching my potential.  But rudely, I have not even introduced myself.

My name is A. Little Datum, and I am a morsel of digital info that lives an arduous life as a fundamental unit of information that my company relies on to carry out its processes and uses to make important business decisions.  We are a global computer manufacturing company with thousands of suppliers and millions of customers around the world.  Although our profit and loss statements seem to please our directors and officers, I spend the majority of my time in silos struggling to break free in order to interact with the data in my company’s other information systems, all the while driving maintenance costs up and reducing data quality.  Let me give you a quick example of a day in my life, but let’s start at the beginning when we were a small business with under 20 employees.

Back in the day, I would find myself existing as a piece of information that explained the supply cost our company would have to pay to purchase HARD DRIVE A to use in our finished computers.  At a certain time, the hard drive manufacturer, our supplier, increased the price of HARD DRIVE A by $100, and therefore it cost us $100 more to buy each hard drive.  However, our purchasing department failed to alert the billing department of the price increase, so we were still charging the same amount for each of our finished computers.  As a result, our gross margin was cut by $100 per computer that we sold.  In this scenario, I only existed in the minds of the purchasing department, and by their failing to communicate to the other organizational departments, I was confined to the silo defined by the purchasing department.  In small companies, the silos can exist as simply communication failures between employees of the company, but the same problem exists (and compounds) for large companies with robust Enterprise Resource Planning (ERP) systems.  Often, large companies have multiple instances of their ERP systems (possibly a server in each country or region, or for each line of business) rather than a single consolidated database and therefore often experience communication failures among each other due to poor integration of a number of systems that use the same type of data.  These larger systems replace but carry out the same functionality as the minds of the employees in the small business, so reliable communication between them is essential to the elimination of silos.

My company exists today as one of the world’s largest computer manufacturers, and I am currently residing in our US inventory system as the product number (#435363) of one of the hard drives we purchase from one of our suppliers to use in the ITTY BITTY MODEL LAPTOP computer that we make.  My sole function as a product number in our inventory system is to track the #435363 hard drives as they move through our manufacturing process (i.e. quantities on hand, quantities on order, bills of material, work in process, and quantity shipped).  I have cousins in the other systems around the world who are used in our manufacturing operations, but I don’t know them very well – they have different numbers, different names, and different attributes.  As an order comes in for the ITTY BITTY MODEL LAPTOP, I get selected and installed in the computer.  Today, a very large order came in, and I needed the help of my cousins to fill it.  OOPS, I just ran into the silo walls put in place because we have more than 10 different installations of our ERP software and there is no communication among them.  I can’t find my cousins, and I don’t even know where to look – I don’t even know who is really related to me.  This must have happened hundreds of times, and since our inventory instances could not communicate with each other (even though we use the same ERP software all over the world), we ended up with a plethora of orders that we were not able to fill.  Needless to say, our customers were not happy, and we lost a large portion of our business. 

I dream of the day when I am not trapped in this silo.  I want to be part of the other systems so that my company can grow, and so our customers are happy.  I want to get to know my cousins, and I want to know their history.  Most of all, I long for a family reunion in a single, consolidated system.

Almost everyone has had some experience during which two or more people have been working on the same task at the same time.  In such cases, it is often hard to determine who has the “master copy” of the project, and changes made by one individual can easily get lost or overwritten by changes made by someone else working on the same thing.  If, for example, you and I are both given the same spreadsheet and told to perform some analysis and report on the results, it would be very difficult to combine both of our changes and different calculations after the fact to create a single report, and we would likely come up with different conclusions.  Now imagine that we are not working with a spreadsheet, but with millions of pieces of data across an enterprise, and there are not only you and I, but hundreds of other individuals and systems manipulating the data, all at the same time.  Because situations like this happen every day within large organizations, it is of utmost importance to establish a comprehensive trusted source of data in order to maintain systems that are complete, consistent, and correct.

An organization will have established a trusted source of “master” data across the enterprise when its information systems provide a single version of every item of shared data that is consistent with the rest of the data in the system – and when everyone who needs a data item knows where to go to find that authoritative version.  While establishing such a trusted source is extremely important, there are obstacles and challenges that threaten the trusted source in numerous ways:

• A user may be confronted with conflicting information from different sources and not offered any help in interpreting or resolving the differences.  Worse, the user may happen on one version of the data, never suspect that different (and perhaps truer) versions exist, and make decisions based on inaccurate information.
• The information obtained by a user is often incomplete, because the missing information is stored in a system that is “too many clicks away” from the portal through which the user is accessing the data.
• There is a lot of redundant data entry.
• Sources sending information into the enterprise database often have conflicts.
• Attempts to achieve a single trusted source by multi-directional synchronization of independent databases (containing, for example, order and return information) present the danger that timing problems may cause valid data to be overwritten by invalid data.

Even if a database contains a single definitive copy of all the data, those data are of no use unless the user application’s interface and other user aids enable people to find the information they need, when and where they need it.  The master database needs accurate, legible, documented, understandable, and up-to-date models to support the development of user interfaces and aids that make this data available to its users.

Finally, once a user has obtained the one true version of some piece of data, one more thing has to happen to turn the data into valid and valuable information:  The user must accurately understand what it means.  Standardizing procedures and processes across the enterprise can help an organization to ensure that users understand the data:

• Standard codings, terminology, and classifications for crucial shared entities (accounts, projects, etc.) need to be developed, adopted and put into common use.
• Time-tagging data items can prove useful in informing users how up-to-date they are.
• Educating new users of an application (and educating current users of a changing application) on the meaning and timeliness of the information they will be obtaining through that application is an essential component of maintaining the integrity of the database and the enterprise.

Finding the single source of truth

With the proliferation of silos and channels comes the problem of knowing whose version of the “truth” is correct: where did it originate, and how trustworthy is it? The first part of any information management effort should be to ascertain the owner and original source of each type of data in an information flow. Only this “single source of truth” should be accessed for that data by all the other organizations in the system. It is also important to pay attention to the management of metadata and master data to improve transparency into the lineage and quality of data.  Organizations need to consolidate systems that provide data for similar business processes, identifying and resolving duplicate data and incomplete data, so that data is entered and updated consistently in one place…the single source of truth.