Posted by Su (Susanne Williams) on September 14, 1998 at 13:53:03:

In Reply to: The Science and Art of Knowledge Management posted by Yogesh Malhotra,Ph.D. on August 12, 1998 at 12:58:23:

Two things:

Definitions of Art and Science
Rasch scaling as bridge building between the Science of "Hard Science" and the art of social "sciences".

First...Art and Science

First comment: do some background reading on the foundations of science...esp. A Korzibsky "General Semantics".

A piece of art can be defined as the representation of an individual experience as an attempt to communicate the same.

It is the creation of a "sign" to signify to a domain, which has the capacity to receive that sign, that which the artist wishes to communicate.

Science is recognised by a universal system of measurement, by the ability to replicate the experiment.

The so-called soft sciences have long had the problem of lacking a common standard of measure, sothat the findings of any particular study have, by the above definitions, been much more of an art than a science.

In KM I believe we need to exhibit more scientific rigour, as well as more free expression of art.

Results of experiments in the social sciences has been more of an artistic expression than a scientific one. Non-replicable experiments, no common measure for comparison.

This brings me to the Rasch Model. A Probabilistic method for objectivising experiment results.

Much has been written on Rasch and it would be for the reader to research the methodology more closely than I would attempt in this short piece.

It is essentially a technique used to create a baseline in an experiment and thereby produce results that are replicable and therefore comparable.

The application of the distribution DURING the construction of the experiment allows the bias to be gradually removed. This is done by working with the results to fit the data to the Poisson distribution, for instance, by removing questions which cause a bias.

***************************************8
from http://www.swan.ac.uk/cals/vlibrary/js96a.html:

Rasch Scaling
Rasch scaling is now widely used as the preferred method of analysis in mainstream testing. A full explanation of the theory underlying Rasch analysis is beyond the scope of this paper, but its merits have been described in detail within broad areas of test measurement (Lord, 1980; Wright and Stone, 1979; Hambleton, Swaminathan and Rogers, 1991) and within the field of EFL testing (Henning, 1987; Woods and Baker, 1985). The essential features of Rasch analysis are:

The difficulty of items and students' ability are measured on a common scale - the logit scale - which allows for a direct comparison to be made between the difficulty of an item and the probability of a student at any level of ability getting it correct.

Tests scores are independent of the restrictions of item difficulty and test population that limit classical test theory and analysis.

Items (or subjects) whose responses deviate from the population norm can be readily identified.

Items that have been pre-tested and calibrated can be used to anchor untested items so as to maintain a consistent scale.

The application of Rasch scaling to Yes/No tests is no different to that of traditional language tests, and assumes that the scores from the test reflect a student's underlying competence/proficiency in vocabulary. One prerequisite for the application and interpretation of Rasch analysis is that the skill being measured is unidimensional and that the tests used are valid measures that adequately assess this latent trait. If any tests cannot be shown to be unidimensional, then Rasch analysis, strictly speaking, should not be used.

A second prerequisite of Rasch analysis is that the target subjects should be representative of the broader target population and that the subjects should be a stable sample. Any marked deviation from the normed values of items will result in inaccurate measures and usually leads to misfits in items and subjects. Any misfitting items can be identified by large outfit statistics and need to be examined for inconsistencies. Students who respond inconsistently to items whose difficulty levels are inconsistent with their estimated ability, will be also flagged as misfitting and their scores have to be interpreted with caution.

*******************************************
Some opponents of the Rasch school have argued that weightings take into account the variability exposed by the Rasch method, but in many ways the application of these weightings themselves was an artistic expression of the creator of the experiment.

Ironically, the best of those experimenters have eventually come to create experiments where the distribution approaches Poisson, but they have had to do this by an iterative, expert knowledge based, set of subjective judgements.

Using the Rasch model it becomes much simpler, (and replicable) to create a data set that HAS a baseline, and is therefore validly comparable with another dataset.

So, in the pursuit of more rigour in our "scientific approach" and more freedom in the creation of our artistic expression, I believe the application of the Rasch model will assist the KM movement in gaining "scientific" credibility and validitity.

What other approached and avenues do we have?
How can we best benefit from the increased defenses we have against the "you're too unscientific" school?
How does increasing our scientific rigour promote the freer artistic expression?

Su

• On specific objectivity

---

Follow Ups:

---

Click Here to Post Follow Up in New Forums

    Knowledge Management Think Tank (New)

  

 Subject: 

Message: