LPI exams - Ubuntu Certification

[Alan McKinnon]

On Wed, 2006-07-05 at 10:06 +0100, Daniel Carrera wrote: 
> Hello,
> 
> I've been going through sample LPI exams, and I don't like some of the
> questions. In brief, I think they're broken, and I wonder how useful LPI
> really is for measuring Linux proficiency.

You can relax, every "sample exam" out there (bar one) is seriously
broken and has very little relevance to the real exam. Here's how sample
exams get created: arb person A decides to write a "test exam" and
dreams up a bunch of dumb questions. None of them are psychometrically
valid, none are subjected to QA and they are that person's idea of what
the various Objectives are about.

The exception is the samples at praxis.de which are actually quite good

> For example: One sample question asked "You are installing an ISA card;
> what do you need to do?". Two options were (a) Nothing, Linux will
> configure it automatically and (b) Use isapnp and pnpdump.

ISA has been dropped from the LPI Objectives, and replaced with 
PCI/SATA/USB in more detail than it was before. There is a tendency 
for old stuff to hang around in the objectives, sometimes this upsets people.
ISA is a good example, boards with it on were in use for many many years
so it was actually relevant till quite recently. Because of the nature of the 
LPI exams and what it takes to make a new exam, only very mature topics that we 
are certain won;t suddenly be dropped actually make it in. devfs is a good 
example - when Linus accepted it into the kernel, it looked like it was 
there to stay. But we now know better. Imagine if a whole exam Objectives was
made around devfs in haste and now it had to be taken out.

> First, there's the matter of relevance. ISA disappeared years ago. Any
> PII or newer won't have ISA slots. Second, the question is flawed
> because the answer depends on the kernel you're using. With the old 2.2
> kernel you had to use pnpdump to get information which could then be fed
> into isapnp. Starting with the 2.4 kernel, this functionality was added
> into the kernel and they no longer exist. Debian Woody uses the 2.2
> kernel. If we're going to ask a question about hardware that was
> obsolete 6 years ago, it's not beyond reason to consider a distribution
> version that was current 18 months ago.

You've just made a classic flaw :-) Such hardware is indeed obsolete if you want
to buy a machine today. But they hang around for years after the vendors stop 
selling them, and admins need to know how the hardware works.

The LPI exam certifies an individuals knowledge of Linux, not their knowledge 
of what is the latest and greatest software today, or today's hardware, or 2.6.17-git9.
Even the 2.4 kernel is very relevant today and yes, you will receive questions on it -
it's on the objectives after all.

> The exam also asks questions about vi. I understand the logic in asking
> about the one text editor that any system, including a rescue system, is
> guaranteed  have. But they seem to ask the wrong questions about it...
> For example, it asks how to move the cursor up 4 spaces. I may not
> remember which of hjkl moves the cursor where, but in real life I can
> find out by pressing them. Plus, currently vim understands the arrow
> keys. And what if I write "4k" and the computer expects "kkkk"? Also,
> when it asks how to "save and exit", what if I am used to using :wq and
> the exam expects me two say :x ?  Does it really matter if I know
> that :x can save and exit as long as I am able to save and exit it?

Tricky question, it has an obtuse answer :-)

Technically, LPI tests the cognitive domain, i.e. your understanding of the 
fundamental underlying concepts. It doesn't test if you can read a man page, 
or observe that you just hit the wrong keystroke and to know the difference.
Psychometrically, questions like the one you cited are relevant, but in 
moderation. If you want to score 100% on this exam, you are expected to know 
everything, including obscure details. So two, maybe three such questions are 
a good thing. 10 of them would be bad.

Submitted questions are put through a QA process to pick up stupid mistakes like 
:wq and :x are the same thing. A question like that will never make it onto the 
real exam as it is obviously flawed.

> In some questions the exam seems to be measuring the wrong things.

Seems more like the sample exams are deeply flawed :-)

If you find some error in the actual exam you write, there is a feedback 
section at the end where you can enter your comments and improve it for the next 
candidate.

Do keep in mind that there is no known method to measure what we techies would like
an exam to measure. We want to know if someone knows Linux well, but you can't measure
that - it's intangible and exists only in your mind. So instead all exams measure some 
measurable side-effect and rely on that to correlate to the real thing. 

alan