Ubuntu Certified Professionals

[Daniel Carrera]

Alan McKinnon wrote:
> Red Hat has 
> been asked many times to show the psychometric validity of their 
> exams, and to date they have not been able to do so.

I'm not familiar with RH's exam, but it may be that making the exam 
"psychometically measurable" would also make it less useful.

> Research shows that practical tests are influenced by the testee's 
> mood - having a bad day can ruin your exam results. But multiple 
> choice by measurement doesn't suffer from this

Yes, multiple choice test absolutely suffers from this. I can say that 
because I've been a testee many times. If I can't concentrate because my 
gf broke up with me or I have a cold that'll hurt me just as much on a 
multiple choice.

>>You can't compare two people writing different multiple choice
>>exams and prove you are comparing apples and apples.
> 
> But you can, using Angoff Standards Setting. First, the testee profile 
> is defined, then every question in the database is rated by subject 
> matter experts as to what % of testees are expected to get it right. 
> The exam objectives themselves are assigned weights according to 
> their relative importance. This gives a common base from which to 
> start.
[snip]

I'll have to give this some thought. Thanks for the explanation. It's 
much better than I thought.

> Looks like ingot has a streamlined operation, so well done for that. 
> Why do you say multiple choice exams produce bureaucracy?

That's how I understood what you wrote. You listed clerical staff, 
proctors and transportation and said that this made it necessary to 
charge $100. Sorry if I misunderstood.

> LPI gets by with something like 10 staff in Canada. There's a team of 
> about 6 to 10 working on exam development, and a network of volunteer 
> proctors who deliver the exams in their respective areas. You can't 
> get much leaner than that :-)

:-)
I admit that's pretty lean.

INGOTs has 5 staff for the UK. 1 clerk, 1 techie (that's me) and 3 
moderators (one of the moderators designs the criteria). This covers 
2600 pupils in the UK distributed over 50 academies. That's thanks to 
the large network of assessors (whom we train) who do most of the work.

> Historically, LPI exams delivered computer based style cost $100, and 
> more than half of that goes to Prometric or Thomson Vue. Paper based 
> exams cost $25 and that includes the cost of couriering them all over 
> the world. For security reasons, exams are printed and marked only in 
> Canada, hence the need to courier stuff.

Who are Prometric and Thomson Vue? Why do they take such a big chunk of 
the exam? I guess that the $100 is not bureocracy after all...

Cheers,
Daniel.
-- 
      /\/`) http://opendocumentfellowship.org
     /\/_/
    /\/_/   A life? Sounds great!
    \/_/    Do you know where I could download one?
    /