Monday, February 05, 2007

The mess behind CAT Analysis

Ever wondered about what owning responsibility should mean. This year's CAT and its aftermath is a burning example of how high pitch ad campaigns, promotional programmes and the likes are empty and worth nothing. Moreover they misguide and confuse students to no ends. Read on......

Sample this. About an hour after CAT 2006 ended - most major names in the business of MBA Entrance training started flashing answerkeys and results on their websites. By the evening all major players had uploaded the complete answerkeys and solutions to each and every question of CAT (which this year were too few - 75 only) properly arranged in various sets. All of them also provided their analysis of what the test was like and what a particular score could mean for a student.

Thousands, if not Lakhs, of students logged in to these sites and tried to calculate their scores. On the basis of these scores the students worked out what could be their possible ranks - on the suggestions of these sites and the so called CAT Experts. It is important to note that the students take decisions based on these scores and suggestions about institutes that they would apply to for admissions. And this practice is followed for years now.

So what was new this year??
Not much except that this year IIM-Ahmedabad decided to be more transparent than earlier. Along with the result the institute put on their site
1. the criteria for sending GD/PI calls based on CAT percentiles and
2. they also uploaded their version of answerkey on the site.
For many people like me it should be eyeopening. Following table list the number of errors (wrong answers) in the answer keys of major MBA Entrance players across India.
Ps. All Comparisons with the first answer key published by respective classes on their official sites, collected the day after CAT 2006 was conducted. The author has a copy of all of them. The classes have obviously either removed the answerkey from their sites or have changed it later. Also the comparison is with the answer key published by IIM Ahmedabad on their official site which can be visited here

The only thing that is funny is the perfect AP that the nubers create - otherwise this is a big question mark on the competency of some of the best names in the industry who claim to shape the careers of the youth in India.

Implications
Do some arithmetic to digest the implications of these errors. This year there were just 75 questions asked in CAT. Now a simple calculation will tell us that the most visited sites in India for analysing solutions had an error rate that ranges from 13.33% to 18.66%.

Thus on an average the students checking their results based on this answerkeys would be wrong on their analysis of results by about 10%.

This is a whooping error because a less than 1% difference in marks can make a difference of several hundred ranks if not thousands. This means that if somebody makes a decision of which institutes to apply for based on these errorenous calculations he/she might be wrong by miles on his approach to his/her career. CAT happens just once in a year so it is expected that a large number of students have lost their year or at least they will compromise on the levels of college that they will finally get admitted to.

I am personally in touch with students who calculated a negative score in the verbal section based on these sites and are now sitting with a CAT score card with 90+ percentile in the section.

Why Does this happen?
Because the aim of the coaching classes is different from that of the students. For coaching classes this is a time to grab eyeballs/attention of stundents to their brands. In management its called ensuring strong brand recall.

Its speed which will sell? The site which puts up solutions to most questions fastest will get maximum attention from students - never mind the quality of solutions or the accuracy of it.

What helps further is short public memory - even those students who might have taken their application decisions based on these solutions do not care once they are done with the application process. Later if they do not achieve the expected percentile scres they would attribute it to fate, some unexplained glitch in the system and/or simply their misfortune.

Where does one go?
Take this as one of the following
1. A reason for students to become more aware customers as education is no more a job of teachers - it is the business of education and they are the target customers.
2. A reason for watchdog bodies to come up - though personally I am against it.
3. As an opportunity for some or the other coaching classes to become a TATA of education - a name one can trust - by being ethical, slow but sure, and dependable

Any takers