Posted by Reilly Atkinson on August 13, 1999 at 20:09:25:

In Reply to: Brief defining characteristics of a Data Warehouse posted by Martyn R Jones on August 13, 1999 at 17:14:05:

Martyn -- Data Warehouses, problems and opportunuties, and, I think, great opportunities for KM. I've spent a lot of time dealing with data provided by IT organizations. More often than not, after my analyst colleagues and I got through checking a cleaning the data, we were able to give the IT people practical advice on how to clean up their act. And our advice was often along the "rules of the data warhousing game". Not so long ago many IT organizations didn't have a clue about their data, other than storage formats, updating rules and the like. That is they often did not understand what the data meant, nor did they have much of a notion of how data might be used for sales forecasting, for market analysis and so on. Data Warehousing has hightened the IT communities appreciation of data as other than a passive element of a database. In many respects DW is a reflection of "best practices" associated with quantitative research.

My problems, for one, are the implied "one size fits all" treatment of data. The initial discipline and order imposed will provide new insights, for many organizations by virtue of uniform data practices -- finally everybody is on the same page. But at some point, as the sophistication of users grows, the uniformity will create problems.

As examples I'll start with the analysis of the efficacy of price promotions, or coupons or advertising. I've spent a lot of years doing this kind of stuff. Simple plots, suggestive as they may be, tell you next to nothing. You need to use quite advanced time series analysis techniques. And you may well have to experiment with different time scales -- daily, weekly, monthly. That is you need the raw data, not someone's notion of the best way to characterize sales by time. The analysis tells you what is best, and another analysis might give you a different cut.

Similar problems occur with age, or income, and so forth. Again, to support solid analysis, only the raw data will do -- sometimes you want continuous variables, sometimes various ranges.

These kinds of problems certainly can be solved, but only if the seasoned analyst plays a strong advisory role in system development. There's a big KM problem here: there's an enormous body of experience and knowledge on doing quantitative analysis and research, and it is minimally used in the DW universe. One impractical solution is to have IT folks take several courses in anlysis -- many fall of the log dealing with statistics, which, of course, is the starting point. Even more impractical is to require IT people to take lots of physics courses -- the best commercial DW setups I've seen are designed and run by Ph.D physicists -- who don't always follow the conventional DW conventions. (After all one of the first world-class DWs was constructed by Tycho Brahe in the late 16th century.)There's tons of non-propriatery knowledge out there that
can be enourmously valuable to an organization. How does KM address such a problem?

Regards
Reilly

---

Follow Ups:

• Data Warhousing in Context Martyn R Jones 20:59:47 8/15/99 (0)

---

Click Here to Post Follow Up in New Forums

    Knowledge Management Think Tank (New)

  

 Subject: 

Message: