More can cost less, and still be more. 30 Jul ’09
Chris Anderson’s recent article in Wired Magazine (“Tech is too cheap to meter”) makes the provocative case that we are wasting a lot of time and money trying to manage technology resources for efficiency. Disk space, processing cycles and bandwidth are so cheap now that managing them as limited resources is just a waste of time.
Anderson gives the excellent example of the phone system where your mailbox “fills up” after a hundred messages and you are forced to sit down and take a half an hour and delete things. This of course annoys the user (who, on a public phone system, is a CUSTOMER) to no end. In a business, it takes up valuable employee time.
And what exactly is being saved here? How much disk does a garbled thirty second voicemail message take up, anyway? Well, we can figure this out. A low-fi VOIP call needs a bandwidth of 30 kbits/sec, or about 4 kbytes/sec. For a 30 second message, that’s 120 kbytes, or about one-tenth of a megabyte. If I have a hundred messages in my voicemail box, it’s taking up about one megabyte. (Of course it should be much less, but we’ll assume this company has never heard of compression).
What does my clogged inbox cost the company? I recently bought a terabyte drive for about $200. Of course the price went down the next day, but let’s be generous and use the $200 figure. That calculates out to about a thousandth of a cent per megabyte.
If you spend half an hour cleaning out your voicemail, the cost in lost labor is going to be somewhere between four dollars and four thousand dollars, depending on where you are in life. And what did you save the company? One thousandth of one penny.
What does this have to do with Data Warehousing, you ask? A few things. The truth is that Data Warehousing is very much a child of cheap technology, which means the cheaper it gets, the better our systems should be able to deliver good information.
First off, we are no longer allergic to redundant sets of data. Most DW professionals have long ago seen the benefits of denormalizing dimensional data and aren’t worried that every single attribute isn’t tucked off into its own uniquely-valued table somewhere. But more important than that, we are less conservative about creating downstream data marts, cubes, summaries, and extracts. If we can save some financial analyst time and headache by giving them a custom data mart then we are happy to do it, even if the marketing department asked for a similar-but-not-quite extract the week before.
More important still is the change to our development model. Software development projects often go through lengthy and expensive requirements phases, with extensive user interviews and fat specification documents written up. Back in the 1990s, DW projects did the same thing. In fact, many DW project STILL do the same thing, despite the evidence of the benefits of Agile methods. But what exactly are we saving through a lengthy requirements and design process? Disk space? You may as well clean out your voicemail box. Programmer time? Not if you’ve got the right tools. User reports and dashboards are created by drag-and-drop now, as is the back-end data warehouse, thanks to WhereScape RED.
“Technology too cheap to meter” is an opportunity for us. Why go through a lengthy requirements phase? Why not have a workshop instead where you roll out a dozen reports or data marts to your user group and see which one tickles their fancy? It will take less time than interviews and writing specifications and probably have better results.
Yes, deploying and managing many downstream data objects can be a royal headache, but WhereScape RED is the perfect tool for this new environment. As you can drag and drop to create new downstream objects, RED tracks each aggregate, summary, data mart and data cube that you create and makes sure that they will be loaded and updated appropriately and included in the documentation. RED allows you to be generous with information while still maintaining a simple and manageable environment.
