WhereScape Builds data warehouses right, now

A client site of mine had a detail report that was running in BW over MaxDB that was taking 5-15 minutes to return a small selection of data.
The report was pretty much unusable which frustrated key users in the business as the subject area was of key interest to them.
As a proof of concept I combined my business knowledge and SQL skills with Doug’s Reporting Services knowledge and SQL skills and recreated the very slow report.
The source fact table has 200 million rows in it and is partitioned by month.
Without any additional performance improving indexes the report now runs in 2-4 seconds!

Colin White wrote an article last week for BeyeNetwork, Can Enterprise Data Warehousing and Master Data Management Projects Survive the Recession? , following up from a blog he wrote at the end of last year.

I totally agree with his comment that “the solutions that will have the most impact in 2009 will be those that offer quick and low-cost approaches that help organizations reduce costs and enable business users to become more productive and self-sufficient.”  It is in line with our experience that the BI initiatives being funded these days are project oriented, quick wins that deliver real value to the person funding it.

I am not so convinced that data quality or master data management projects are winners in a recession.    There are thousands of data warehouses and data marts built using WhereScape RED, and I am sure they could all benefit from cleaner source data.   In our experience, however, data quality projects are falling into the nice-to-have, no-money-for-it-now, maybe-next-year bucket.  I like Colin’s ideas on providing a data quality value or score, a number of our customers already do this, but in most cases it turns out to be of limited value – the users are all too aware of the problems with their own data and already provide their own filters.

While I am on a roll, talking about data quality and data warehousing in the same sentence always worries me.  If no one in the business cares enough to fix up data, why should the data warehouse team?  Especially when they are often not funded for it, it is the wrong place to fix any problems, and it delays the project…end of soapbox.

A couple of years ago Yann and Steve got the WhereScape team into running.  The first half marathon we did saw seven of us hit the course.  Last year there was a real drop off – one into the marathon and one entering the half.  This year, however, has seen a rise in new sports.  Steve, Jason and Doug took on the Kona Colville Classic, a 72km (45 mile) gentle pedal in the middle of summer along some mostly paved roads with the occasional puddle.

A couple of weeks before I did a gruelling 4.6km (2.9 mile) swim from the island of Rangitoto to St Heliers beach.

The island we started from can be seen behind Jeremy, while the next photo doesn’t quite capture how good it is to be back on dry land.

This is just a quick blog to remind you that WhereScape have pre-built data marts built for several modules of Oracle Financials. These quickstarts allow us to install, tweak (people have a habit of customising these applications) and process data marts built to analyse Oracle Financials information (eg GL or AP) very quickly, within a day or so per module. Once in a generic dimensional data mart the data can be reported and analysed using a wide range of user friendly tools (eg BO, Cognos, etc). That frees people from the built-in reporting and provides the ability to integrate this data with other analysis areas.

Just to add some spice to the story – this week we had a request to install these Oracle Financials quickstarts into a SQL Server data warehouse. This is a new twist – normally its Oracle Financials into an Oracle data warehouse. But by using the database conversion functionality in RED (described here http://blog.wherescape.com/technology/migrating-between-rdbms-platforms/) a consultant has delivered 2 main areas (AP and GL) in SQL Server with 2.5 days of effort. In addition they added Analysis Services OLAP cubes over the top of the star schemas (because RED is good at that too) so that the accountants can slice and dice the data directly from Excel. A very quick way to make friends with Accounting! Now they can also analyse how little the solution cost them!

It is interesting to see the number of articles being published about getting more from your BI investment.  Was it only last year when the main discussion areas were about scalability and performance?   Currently free from the Forrester website (you do have to register) is a paper called BI Belt Tighening in a Tough Economic Climate.  I can’t help but agree with their comment (in the context of where to look to save costs) “When your BI projects grow large, complex and recurring, consider…end-to-end BI lifecycle management…” especially when we are listed as one of the vendors in this category. 

In these current economic conditions getting to value quickly and removing cost and complexity from management of the entire data warehouse lifecyle are more critical than ever.