WhereScape RED Builds data warehouses, fast.

We have just started working with IBM, and I have to say what a pleasurable experience it has been.

A big thanks to Boris, Laura, Ceyhan and their team for all their assistance.  We are now in the IBM Global Services Directory, the IBM developerWorks wiki and have been validated with the “Ready for IBM DB2 data server software” mark.  We still can’t believe how quickly you guys got all this done.

The official press release will follow, but WhereScape RED for DB2 is now available…roll on IOD.

Morgo is a two day conference for hi tech entrepreneurs from New Zealand held in the picturesque Bay of Islands.  The official theme this year was “Partnering for Success” but some of the strongest presentations were on giving back rather than partnering.

Selwyn Pellet, on the board of AIM listed Endace and now with Imarda talked about an entrepreneurship apprenticeship he is looking at setting up.  Born from the idea that you need entrepreneurs to grow economies he is looking at program that apprentices promising gen-ys (which he acknowledges may be a contradiction) with dark side CEOs – he is looking for sponsorship so the youth can fully participate with the CEO – joining them in all meetings, travel etc.  Sounds to me like a great idea and I would be in.

Scott Farquar from Atlassian (one half of the Australian Entrepreneurs of the year a couple of years ago) talked about www.kiva.com as well as Atlassian’s 1% initiative – giving 1% of revenue, equity and time into the Atlassian Foundation.  They also offer non-for-profit licences, and he offered the details from his web site if anyone wants to copy the idea.

Other highlights of the conference were Ben Anderson from CT Partners on the talent war and Andy Lark from Dell using an Olympics medal metaphor to discuss New Zealand’s relative performance to other countries. 

Glenn Martin’s talk about his recently released jet pack was highly engaging.  After spending 27 years working on a dream he was justifiably proud of his invention – and there is no denying flying with a jet pack is pretty cool.  Unfortunately the pack didn’t arrive in time so we were not able to get a demo.  Didier Elzinga, CEO from Rising Sun Pictures, gave a great talk about working with Hollywood from Adelaide, and how for the people he does business with if you outside their zip code you might as well be in Adelaide.

The Honorable David Cunliffe launched the Government’s Digital 2.0 Strategy at the conference.  His comparison of politicians to entrepreneurs was a bit of a stretch – I don’t know of too many politcos who have mortgaged their homes and borrowed money off their friends to go into Parliament.  I was a little surprised the Digital 2.0 targets were not a bit more ambitious. They are throwing some money at making sure the entire country can get broadband which is great, but surely we can plan for a bit faster speed especially in the urban areas?

Overall a great couple of days. As usual it is the informal discussions that are as useful as the actual content, it is great to catch up with a bunch of people all trying to make a difference, and a conference without people trying to sell you stuff is always refreshing.  Thanks to the Morgo team for organizing it and inviting me.

RED operates with dimension types that are recognized by modern data warehouse theory. When a developer starts working on dimension tables he gets to chose from four options for the default generation of the dimension table and its update procedure via the following dialog box:  

A Normal Dimension is where a dimension record is updated and changed whenever any of the non-business key information changes.   This is commonly called a type 1 dimension and is used when historical values of the change are not kept. 

Overwriting the old values.  A Normal dimension type is the most common dimension type. RED auto-generates a stored procedure to handle update and insert paths. Consider this simple example. The “Product” table contains a product named, “Thingamabob” with Product ID being the primary key. In the year 2007, the price of Thingamabob was $150 and over the time, the product’s price changes from $150 to $350. 

In the year 2007, if the price of the product changes to $250, then the old values of the columns “Year” and “Product Price” have to be updated and replaced with the new values. In this Normal Type, there is no way to find out the old value of the product “Thingamabob” in year 2006 since the table now contains only the new price and year information. 

The logic that WhereScape RED applies to the stored procedure handling the Normal dimension type works as follows:

- The stored procedure gets all unknown records from the dim table. If an unknown dim record does not exist, then the procedure allows an explicit value to be inserted into IDENTITY field

- The procedure loops through all the input records, attempting to update the dimension records based on the business key value. If the update attempt fails, then the procedure performs an insert. 

A Slowly Changing dimension is where new dimension records are created when certain identified columns in the dimension change.   This is commonly called a type 2 dimension and tracks point in time historical changes. 

Creating another added record. In this type, the old values will not be updated but a new row containing the new values will be inserted in the product table. Consequently, at any point in time, the difference between the old values and new values can be retrieved and easily compared. This can be very useful for reporting purposes. 

The problem with the above mentioned data structure is the “Product ID” column cannot store duplicate values for “Thingamabob” since “Product ID” is the business /primary key. Also, the current data structure doesn’t clearly specify the effective date and expiration date of the product when the change to its price happened. So, it would be better to change the current data structure to overcome the above primary key violation. 

In the changed Product table’s Data structure, “Product ID” and “Effective Datetime” are now a composite business / primary key. Therefore, there will be no violation of the primary key constraint. Addition of the new columns, “Effective Datetime” and “Expiration Datetime” provides the information about the Thingamabob’s Effective date and Expiration date adding more clarity and noticeably improving the scope of this table. The slowly changing dimension (SCD), type 2 approach, may need added space in the database, since for every changed record; an extra row has to be stored. Since dimensions are usually not that big in the real world, extra space is negligible. 

RED performs the following additional tasks in the stored procedures to handle Slowly Changing Dimensions (SCD), type 2 dimensions.

- Three new special variables are added: insert indicator, version indicator and version number.

- Four extra columns are added to support slowly changing dimensions: current_flag; version; start_date; end_date.

- RED automatically generates an alter table SQL script, based on type. It also adds a unique, business key index to support the slowly changing columns.

- In the update stored procedure loop, if the record is not found, the procedure inserts a new record and sets the insert flag on. Otherwise, the procedure checks to see if we have an existing dimension row and have just had a field change for data that is not part of the slowly changing data that we are interested in and performs an update. If there is an existing dimension row and it verifies that any of slowly-changing fields have changed. The procedure then updates the version indicator and version number, and inserts a new row. 

A Previous Values dimension commonly called a type 3 dimension, allows the storing of the last value of selected fields in secondary columns.

Creating new fields. In this dimension type, the latest update to the changed values can be seen. The example shown below illustrates how to add new columns and keep track of the changes. From the example, we can see the current price and the previous price of the product, “Thingamabob”. 

The problem with the Previous Values approach is over time if the product price has multiple changes; the complete history may not be stored, only the latest change will be stored. For instance, in year 2008, if the Thingamabob’s price changes to $350, then we would not be able to see the complete history of 2007 prices, since the old values would have been updated with 2008 product information. 

- RED guides you through the process of adding secondary columns. 

A Date Ranged dimension supports source systems that provide date ranged information. It rolls over time based on specified columns and date range and the history is kept. Otherwise, it looks just like the Slowly Changing dimension type. The difference in that the date range dimension is valid for a given range or set of dates on specified columns. 

The following example might be appropriate here.  Thingamabobs are on sale for the date range 1/1/08 – 1/31/08 at $150 and on sale for the date range 2/1/08 - 2/29/08 for $200.  So the dimensional records would be for 1/1/08…1/31/08, 2/1/08…2/29/08 and 3/1/08 to present each range would be represented by a date range dimension key and the product on sale for that dimension key could be reported on.   

- RED will guide you through the process of identifying which column(s) will be date ranged columns, i.e. “from” datetime column followed by the “until” datetime column. E.g. such “from” column could be “Effective Datetime” and “until” column - Expiration Datetime  

Note: RED is using slightly different naming convention.

The above table will look like this:

(*)          = (Business Key)

(**)       = (Primary Artificial Key)

(+)          = (Slowly Changing Business Key) 

A Get Dimension Key Stored Procedure 

The ‘Get Key’ stored procedure is auto-generated by RED to return an artificial key when a business key is supplied. The normal syntax is to call the function (usually from update_stage_% stored procedure) passing the business key and be returned a status indicating the result of the lookup.  

For example:

EXEC   @v_return_status = get_dim_product_key 

      @v_stage_product     

     ,’N’ — Auto add flag     

     ,’N’ — Write error trail entry if not found flag     

     ,@p_sequence

     ,@p_job_name

     ,@p_task_name

     ,@p_job_id

     ,@p_task_id     

     ,@v_stage_product_key OUTPUT     

     ,@v_getkey_status OUTPUT

On a successful lookup, the artificial key for the dimension record is also returned. If the lookup fails because the business key is not found in the dimension then an appropriate status is returned and the artificial key is set to 0. 

The main logic as follows:

- Lookup the key: If dss_end_date is NULL, then we are just looking up the current record. If not null then we are trying to match to a specific slowly changing record, so look through all of the records until finding the appropriate one.

- The procedure then loops through all the old keys looking for a match on the date. If key is found then the procedure sets the key and the flag.

- If no match found then the procedure gets the current version.

- If a dimension key is not found, then the procedure will add it if we have the auto_add flag set.

- If requested (auto add log message), the procedure will log a message about the auto add and log a message about the lookup failure. 

I’m soaking up sunshine on the terrace outside the annual TDWI World Conference in San Diego.   Pelicans are diving into the harbor just a few yards away.  Must remember to add tube of expensive French sunblock to expense report.  This is a great town for a conference.  Sales Veep Mark Budzinski and I walked to the Gaslamp district for dinner last night and it was hopping.  Restaurants and bars full of people and a big private party with a live band that took up a whole street.  Note to Michael: I’ve got the details if we want to do something similar for next year’s conference.  Maybe we can book Flight of the Conchords.

The exhibition hall has been mobbed and Mark, Raphael and I are talking non-stop to people who want to know about RED.  The new booth is really drawing them with the bright graphics and hard-to-ignore message: save TIME and MONEY.  With budgets tight, folks are really interested in what RED can do for them.

There’s also a growing buzz here about using Agile (or RAD) development methodologies for data warehouse development.  Agile methods are old hat, of course,  to Wherescape and our community of RED fanatics.  Six months and a zillion dollars to gather user requirements that will be outdated before the first draft is even written?  No thanks, mister multi-national tasseled-loafer consulting company.   How about we just GET IT DONE using RED?  Develop right alongside the users so they can show us exactly what they want and so we can respond quickly as the business evolves.  

This idea is catching on throughout the DW/BI community and several folks came up to talk about RED as enabling technology for Agile projects.  Stay tuned for more developments around Agile methodologies, including a White Paper and ultra-secret Project X.

It’s not all sunshine and carrot cake here in San Diego, though.   We get a little punchy after a couple of hours of talking about data warehousing.  Here’s a conversation that took place late yesterday afternoon between me and Raphael Klebanov, Wherescape’s main technical brain in Denver.  Raphael is originally from Russia, so you have to apply the correct accent to his lines:

Raphael:  I am sorry, Vince, I have maybe stupid question.

Vince:  There are no stupid questions, Raphael.  How can I help you?

Raphael: It is “TDWI”.  I have figured out that “DWI” stands for “Data Warehouse Institute”.  But what does “T” stand for?

Vince: You were right before.  That is a stupid question.  It stands for “The”.

Raphael: The what?

Vince: The Data Warehouse Institute. TDWI.

Raphael: I have that already.  This is Data Warehouse Institute.  But what does “T” stand for?

Vince: ”The”!

Raphael: The what?  This is what I want to know.  If it stands for something I would like to know what it is.

Vince: It stands for THE.  Just that word.

Raphael: What word?

Vince: The word THE!  THE DATA WAREHOUSE INSTITUTE!  TDWI!  That’s what it stands for!

Raphael: Yes, I know it stands for that.  But the “T” I don’t understand.

Vince: [turning to address the waiter who was clearing our coffee cups] Sir, could I tell you about how Wherescape RED can save you time and money on your next data warehouse project?

That’s it for now, live from TDWI 2008 in San Diego.  A big shout-out to our homeboy Sid from Sid Adelman and Associates;  Larissa Moss from Method Focus; Grace from Tagzfly Solutions in sunny Kingston, Jamaica; the BI crew from P.F. Changs (nice work snapping up all those raffle prizes);  and our booth neighbors Pervasive software, who were out the door and on the trolley to Tijuana as soon as the exhibit hall doors closed.  Don’t bother calling Pervasive tech support tomorrow. Those boys are going to be sportin’  major headaches, and maybe even a few new tattoos. 

Join CIBER, WhereScape, and Strategy Companion for a complimentary one-hour webinar to learn how you can launch a business intelligence / data warehouse initiative without an overwhelming investment in time, tools, and technologists.

The webinar will take place on Wednesday 10th September Noon Eastern / 9.00am Pacific Time.  

Registration is free and can be completed here.

When your software enables rapid building, eases change and simplifies management, one of the really nice things is that the cost for our customers of building and maintaining  data warehouses comes tumbling down.  While this is good for everyone in this economy, it is especially valuable in the mid and departmental markets where budgets are traditionally tight. 

Doing more with less is the theme for our next seminar, “Business Intelligence on a Shoestring” to be held in Portland, Oregon on Tuesday 9th September.  If you are in the area check us out.  One of the things we will be doing is building, live and in real time, a working data warehouse from scratch during the seminar.  For more information or to register see the seminar invite.

It is coming up to conference season again, and WhereScape will be participating in a number of events around the world in the next couple of months.  You can catch us at:

- 9th Annual Business Intelligence Conference July 24-25th in Sydney, Australia.  This will feature a case study (Data Warehousing Lifecycle Management - The new approach) by Andrew Smailes, Business Intelligence Program Manager - Information Management Branch, Department of Families, Housing, Community Services and Indigenous Affairs (FaHCSIA) and Steve Hitchman, Managing Director MIP (WhereScape’s Australian distributor). 

- TDWI Silicon Valley Chapter Meeting July 31 in San Francisco, California.  Held at The Thirsty Bear, 661 Howard Street this sounds all good - free entry, no presentations and held over happy hour.  WhereScape is a sponsor of this event.

- TDWI World Conference August 17-22 in San Diego, California.  WhereScape will be exhibiting at this conference

- MicroStategy Business Intelligence Symposium September 10 in San Francisco, California.  WhereScape is sponsoring this one day event at the Grand Hyatt in Stockton Street.

- Microsoft Business Intelligence Conference October 6-8 in Seattle, Washington.  We had a great time at the first event last year and will be back exhibiting again this year. 

- 2nd Annual Business Intelligence Summit October 8-9 in Auckland, New Zealand.  WhereScape’s Doug Barrett will be running a one day seminar entitled “Show me the money - Laying the foundation for a successful BI initiative.”

Our thanks to Rosalie and Yohan at ASB Bank for the great case study on their Human Resources data mart.  It was the sort of project our guys really enjoy working on, with committed and involved users.  When you are doing a prototype based build engagement is the critical factor.  It can be used to decrease cycle times, as Yohan said “We could iterate the warehouse daily, and immediately work around any roadblocks or new requirements that came up.”  Engagement also results in a better solution.  We find that when issues arise the developers and users can jointly attack them, together working through any compromises, changes in design or changes in approach.  The data warehouse development in this case was 30% ahead of schedule, a result that is also not unusual for prototype-and-iterate projects undertaken with committed users.

I recently came across a useful Excel macro that calculates NZ holidays - I have adapted it to generate SQL to update our date dimension.  Feel free to contact us on support if you would like a copy.

Every action in the information database and warehouse in particular has a consequence. Database developers, business analysts and DBAs  feel agonizing pain when an organization makes changes to their data warehouse applications without having complete insight into the true impact and cost that a modification will have.

So, what is an impact analysis?  It is an identification of the consequences of change on an object to its related objects. Impact analysis, in case of a data warehouse, consists of:

• understanding of the processes performed within a data warehouse as a whole and its components;
• comprehension of interdependencies between processes and objects;
• assessing the effect the change brings to the data warehouse structure;
• developing recovery procedures in case of  a failed modification.

WhereScape RED provides a variety of needed mechanisms to identify a data warehouse’s weak spot when it comes to a change.

In the development process we have put our best effort into creating a perfect addition to our data warehouse. Now it’s time to deploy our changes to a higher environment. But how can we do so without crashing things that we had developed previously? How do we insure that the new modification will not destroy the integrity of the data warehouse instead of improving it? With WhereScape RED it’s easy.

Here are a few recommendations on how to avoid oopses and ouches:

1. Structural Impact Analysis.  i.e.  what objects have been changed.
Click “Tools” >  Click “Search for string…” > Fill out dialog box for string (e.g. either enter the procedure name being modified or the column being changed) > examine “Object search results” screen for all objects that contain this string.

For the structural-impact analysis you can use the Browser screen for “DataWarehouse” connection.
Click “Invoke SQL Admin” > execute something like this (SQL Server):

select distinct(a.name) from sys.procedures a inner join sys.syscomments b on a.object_id=b.id
where charindex(‘stored procedure or column’,b.text)<>0;
– Check load where clause too
select distinct(a.lt_table_name)from ws_load_tab a
where charindex(‘stored procedure or column’,a.lt_where_clause)<>0;

2. Evaluation of the Impact. i.e. what possible impact the data warehouse’s alteration renders on various data warehouse objects. WhereScape uses an array of pre-defined reports. Click the “Reports” tab from the main Browser menu. The list of available reports expands from release to release. There are about two dozen reports that are directly related to Impact Analysis. Some of them are:
•  “Object modification dates (without indexes)” with date created and date modified, same for “Index modification dates”;
• “Modified procedures” i.e. procedures that have been modified since creation;
• “Table objects that have no associated procedure or script” and “Procedures that are not related to a table”;
• “All objects that have been Refreshed or Imported”;
• “Differences between current and selected repository…”;
• “Query data warehouse objects… where custom validations scripts can be run”;
• “Track back on specified column usage…” and many others.

3. Impact during the Promotion from one environment to the other. e.g.  from DEV to TEST,  when managing such deployments using the “Build Application tables…” built-in utility. While the developer plunges through the wizard-like dialog boxes, WhereScape RED continuously warns him/her of the possible implications of this activity. E.g. if the particular object is to be overwritten, warning message pops up to alert the developer on the possible impact on the existing object.

4. Organizational Documentation Update. The considerable problem for a conventional data warehousing project is that database changes within data warehouse are not reflected in documentation: nobody has time to update them. WhereScape RED automatically generates a new documentation set after each iteration of data warehousing development.

5. Versioning of Metadata Objects. Several kinds of version management are utilized in WhereScape RED. For example, choosing the “Automated Version Creation” option from the Tools menu will thwart a developer from losing valuable DDL scripts, procedural code and other metadata and database information.

6. Standardization and Best Practice Usage.    WhereScape RED uses a state-of-the-art methodology in building warehousing prototypes. Following the developer’s guidelines delineated by the WhereScape RED methodology, wizards and the automatic generation of database objects, in combination with an enterprise standard “look and feel” dramatically decreases the chances of glitches in the data warehouse processing.

The are several different ways to load data from files and source systems into the data warehouse load table using WhereScape RED.  Each load table type has different attributes, options and performance advantages in different situations.

RED automates loading data from files by either parsing files into columns as they’re loaded or by loading files into single column tables.  The following kinds of flat files:

• fixed width
• delimited
• XML

The default is to have RED manage the code for loading file.  This is called a file load.  You may decide to generated a script and manage this yourself.  This is called a script load.

When load fixed width and delimited files, RED uses the load utilities in the target data warehouse database environment.  For SQL Server this is BULK insert.  In an Oracle data warehouse, SQL*Loader is used.  Teradata has several different loaders and RED can utilize fastload, multiload and TPT.  DB2 uses the LOAD statement.

All options available for these loaders are able to be used by RED.

Two options are available for XML.  The default is to allow RED to read and load the XML file itself.  Another option available for SQL Server data warehouses.  This involves using the XML OLEDB provider for SQL Server via a Script load.

RED also automates extracting and loading data from other databases into the data warehouse.  Again, there are several approaches available:

• ODBC loads
• Native ODBC loads
• Database Link loads via a database link (Oracle warehouses only)
• Database Link loads via a linked server (SQL Server warehouses only)
• Externally Loaded load tables

For large volume loads from another database into the data warehouse, the fastest option is almost always the Native ODBC load.  The exceptions for this rule are:

• In an Oracle data warehouse loading data from another oracle database, a database link load may be faster if one of the following is the case:
  • the two databases are connected via a really fast network
  • the two databases on the same physical server

For smaller volumes, you should always use database link loads for SQL Server->SQL Server and Oracle->Oracle load tables.  Similarly, for small volumes, loading data from a different kind of database, it may be better to use ODBC loads rather than Native ODBC loads.

The final option RED provides is Externally loaded load tables.  These provide integration with existing ETL or EAI infrastructure.  RED knows about the load table in its metadata and can build downstream objects from the load table.  RED does not load the load table.  It waits for the other system to load it.  Job integration is achieved using a combination of RED and the other products APIs.

Refer to the WhereScape RED User Guide for full information on all of the load types discussed.

We are very pleased to have attracted Mark Budzinski to WhereScape as Vice Preseident of Sales.  Mark has had a long and distinguished sales career.  We first worked to together at Sequent Computers over 10 years ago where Mark was well known to the field sales organization in his role at Sequent University.  Since then he has undertaken a range of sales and executive sales roles, and will be a great addition to the team.  Mark is based in Portland, OR.

At the same time we have created a new position for Rick Smith, who is now in charge of Business Development.  Our partner business has been growing from strength to strength, and Rick has accepted the challenge to grow our partner channel even more.

We are also building our West Coast operation, and Vincent P Donovan, another Sequent alumni,  has joined us in a technical role in the Bay area.

Thanks to all the new and existing customers who have enabled us to grow.

All 400 load tables use Connection Site Generic as the connection defined in the metadata.

Thanks to Clearwater Seafoods for providing a case study on how WhereScape RED sped up their development.  We really appreciate when our customers are prepared to talk about how WhereScape RED has assisted them. 

WhereScape RED Enables Clearwater Seafoods To Adopt a Self-Service Reporting Model

RED Slashes Time in Building Data Warehouse Prototype by 30 Percent.

From its humble beginnings in 1976 when founders John Risley and Colin MacDonald, with one pickup truck, operated their company as a small lobster distributor in Halifax, Canada, Clearwater Seafoods has grown to become a premier supplier of premium shellfish and sea food globally.  Today, with a large fleet of freezer processor vessels and several processing plants throughout eastern Canada, Clearwater Seafoods has delivered 70 million pounds of premium seafood –including clams, scallops, shrimp, lobster, crab and ground fish –to 30 countries.

As part of its corporate structure, Clearwater operates separate companies, located in Canada, the U.S., U.K., Belgium, and Argentina, to sell and distribute their seafood around the world.  These distinctive operations, however, were adding to the complexity and accuracy of corporate-wide reporting.  Each business unit was reporting differently through a variety of methods that included spreadsheets and manual input of data. 

Since Clearwater sells their products worldwide, business analysis reports such as profit margins, inventory, and sales are critical to their business. As a global company, fuel and other costs as well as foreign exchange rates affect how the company evaluates its profit margins, according to Tony Scallon, business systems analyst at Clearwater Seafoods,   As a company selling internationally, Clearwater needs to know the true cost and profitability of what they are selling by region.  For instance, if Clearwater sold a particular seafood for $5 a pound  in the West Coast of the U.S., are they making money for that region once fuel costs, exchange rates, harvesting and production costs are factored in. With each company using different reporting tools and formats, this type of business analysis reporting would be time consuming and challenging. With its IT staff of 21 employees, Clearwater Seafoods had to dedicate two IT professionals full time to running business analysis reports for managers throughout the company. Depending on the complexity of the reports, it would take several hours to numerous days to complete reports.   

To improve its reporting capabilities, the decision was made to centralize all of its data into one data warehouse.  Clearwater uses Great Plains as its core ERP system that includes financials, sales and inventory management with Siebel CRM as the front end system used for order entry, and MS SQL Server as the RDBMS.

‘We selected WhereScape RED to manage our entire data warehouse,” said Scallon.   With RED, Clearwater Seafoods can now manage all their schema DDL (data definition language), into one meta data repository which is fully documented. 

“RED automated the time consuming process of writing SQL scripts and documentation and slashed the time to do this by more than 20 percent.  If you are manually writing SQL scripts, you also are susceptible to numerous errors. WhereScape RED not only automated this process but it provided the documentation where you could compare the  data to assure accuracy.” RED also has a fully integrated scheduler that  performs ongoing loads and logs all activities during processing.

From an IT perspective, WhereScape RED allows Clearwater to develop data transformations when needed and to automatically generate the documentation that shows all the data lineage based on the data that is loading into the warehouse.  These transformations allow the application of  business rules to be applied to  make a company’s data usable. For instance,  Clearwater may want to run a report on the total weight, in tons, of how much fish they distributed. However, the data they have been capturing was only based on the number of boxes packed. 

With WhereScape RED, a prototype of the sales data mart, with analysis cube and reporting services, was built within 30 days.  Scallon projected the prototype would take at least 90 days and was “very impressed” how quickly it took to prototype.  “WhereScape Red proved to be very scalable and an all in one tool that we needed in building our data warehouse,” he said.

With RED, data rates were proving faster than ever since Clearwater was able to build multi-dimensional cubes by leveraging Microsoft Analysis Cubes.  Once its data warehouse is in full production, scheduled for April 2008, Clearwater expects maintenance on the data warehouse to be easy and fast with RED since many of the mundane maintenance processes are all automated.

Another key benefit that may not be as tangible to measure is the peace of mind that RED provides, says Scallon.  “I no longer have to validate the data and worry about the accuracy of the scripts.  RED automates all that and gives me the time to focus on more strategic IT initiatives that will impact the company for the future.” 

The Business Problem:

• Business required more sophisticated business analysis reporting

• Separate business units within Clearwater Seafoods were all reporting differently

• Reports took several hours to several days to prepare

• Remove the need for IT department to produce reports and make it into a self-service model

The WhereScape Solution:

• A prototype was built with RED within 30 days compared to the projected 90 days estimated by Clearwater’s consultants 

• Reporting is now a self-service model with users now specifying and running their own reports

Return on Investments:

• Faster and Accurate reporting  across its separate business units operating throughout North America and Europe

• Able to run more sophisticated analytics and “what if” business scenarios

• More productive use of IT staff with two IT professionals, previously running reports, now reassigned to more strategic IT projects

It is never a good thing to see a company you know go under.  Compudigm, who developed visulaization software primarily for the gaming industry, went into receivership on March 28.  Their last accounts filed with the New Zealand companies office (they were based in Wellington New Zealand and Las Vegas Nevada) show a loss of 2.5m NZ (approximately 2m US) in 2005, and a similar loss in 2004.  They had recently sold the rights to the gaming software to Bally Technologies.  It looks like Bally is going to roll the Compudigm software into their own business intelligence solution so we will have to wait and see what happens to current Compudigm customers, which include some WhereScape RED customers.