Ubuntu Certified Professionals

[Alan McKinnon]

On Friday 07 April 2006 21:38, email.listen at googlemail.com wrote:
> Am Fri, 7. April 2006 20:25 schrieb Alan McKinnon:

> > I'm not sure what you mean here, and I know nothing about how
> > things work in Germany. You also use the word 'most' a lot
> > without saying who these 'mosts' are, could you be more specific
> > please?
>
> I'm using 'most' because it might be that there are companies out
> there which do not believe in braindump tests I'm not aware of.

OK, so you use 'most' instead of 'all' which is demonstrably wrong as 
it's absolute. But why do you have the viewpoint that multiple choice 
is universally disapproved? Is there something fundamentally wrong 
with the multiple choice methodology?

I can't see that your assertion about multiple choice not being 
supported by the market is logically sound. It sounds like your own 
personal opinion, to which you are entitled, but if it's your own 
belief then please state so. Meanwhile I'm still happy to talk to you 
about the exam and answer questions, if only to persuade you that 
it's a very different beast to the MCSE

> > > For me this is close to those coders who insist in beeing
> > > called hackers, ignoring the meaning of hackers in public
> > > today. A total fade-out of reality.
> >
> > OK, so how else would you build a rating system for system
> > administrators?
>
> We are talking about two pairs of shoes here.
>
> A rating system for professional administtrators in my eyes is a
> RHCE certification. Which ist often done after having a
> professional course, not by private self study.

See my reply to Daniel about why I consider the RHCE approach can't 
work for a general Linux certificate.

The RHCE only tests knowledge of what RH built into their OS. It does 
not test fundamental understanding of the underlying principles, 
because that's not the intent of the exam.

I've said it before in this thread, and I'll say it again in a 
different way: The RHCE is a good exam, but it's not perfect and it 
has a very specific testing domain - can someone get specific useful 
stuff done on an RH box. It does not test the same thing as LPI 
exams, the same way as Gnome and KDE are not really comparable but 
they belong to the same general class of thing.

> LPI is not a rating system for system administrators it is only a
> braindump test which says nothing at all about the skills of those
> who did this test.

And LPI is very clear about that on their website and state that a 
cert is only one factor in judging someone's suitability for a job.

There is only one way to *really* know if someone can do a job and 
that is to have observed them on the job over an extended period. No 
test or cert is ever going to be able to do that, the best that can 
be done is to provide a statistically meaningful indication of the 
probability of success.

I defy anyone to successfully braindump a series of LPI exams. The 
exam rotates too rapidly for braindump sites to be of much use, but 
they are a problem that won't go away. The RHCE has the same weakness 
btw. What's to stop you from telling me which tasks you were asked to 
do on your exam so that I can go and practice the same steps and hope 
I get the same task on my exam? Nothing stops you except your own 
integrity, so RH has lots of tasks in the database to get around it. 
Just like LPI.

If by braindump you mean the ability to parrot off answers rote style, 
then there is two answers for that:

- Some things are just learned rote, like the contents 
of /etc/services. You and I know that POP3 runs on port 110 because 
we have done it so many times by now it's second nature. So asking a 
candidate what port does POP3 run on tells you something useful about 
his knowledge.
- it's real easy to word a question so that monkey see, monkey do 
answers just don't help. Describe a scenario with technology A, and 
list 4 possible solutions. If you don't know how A works or have 
never done it yourself, you have to guess so a 25% score is the 
baseline and can be included in statistical corrections. 

Either way you have tested usefully and know something useful. The 
interpretation of those results is up to you.


> > > > > Finally, Ubuntu's done an excellent job of building a
> > > > > community of non- technical users.  I'm not certain what
> > > > > sort of "certification" is appropriate, but something other
> > > > > than membership in the Community Council might be useful. 
> > > > > I don't know if there's a good objective way to measure who
> > > > > qualifies. Whatever sort of recognition is available, it
> > > > > should be something that carries some weight outside of IRC
> > > > > meetings -- an endorsement of the individual that says to
> > > > > an outsider, "We the great and powerful
> > > > > [Oz|CC|Canonical|sabdfl] do hereby stand behind so-and-so
> > > > > as someone who has the best interests of the community at
> > > > > heart, and has demonstrated that via x, y, and z.  Although
> > > > > perhaps not the most technically skilled person we've ever
> > > > > encountered, we encourage you to trust this individual in
> > > > > helping you build your community. (Pay no attention to that
> > > > > man behind the curtain.)"
> > > >
> > > > Good idea, as people like to be recognized for their work. I
> > > > can think of a few folk I'd like to nominate :-)
> > >
> > > What is disappointing to me is the fact that a man like the so
> > > called self believed dictator who stands for a very well
> > > reputated educational foundations /projects (shuttleworth
> > > foundation SA, schooltool project) and by this should be able
> > > to have a wider foresight of what in the long run may result in
> > > such a (<sarcasm>stupid</sarcasm>) decission.
> > > Seeing his background he should known better what this will be
> > > for educational institutions and educational infrastructure.
> > >
> > > What makes it very disappointing for me is to see that this
> > > dictator in his position as chaiman of canonical seems to act
> > > against the honestly principles and prosperous projects I see
> > > fron shuttleworth foundation SA.
> >
> > Why is this Ubuntu cert possibly a stupid idea? I can see that
> > something here has upset you, but I can't figure out what it is.
> > You approve of TSF - are you aware that TSF is the South African
> > LPI affiliate?
>
> Where did I approve TSF, please?

You said "the honestly principles and prosperous projects I see fron 
shuttleworth foundation SA". I'm not attacking you here, I just want 
to know if you are aware that TSF fully supports LPI, and their 
reasons for doing so.
>
>
> As I said above, it is only a braindump test which says nothing
> about the real skills of an examined person at all.

You have asserted this several times, now please back it up. Please 
show evidence for why multiple choice is a flawed testing methodology 
and where it breaks down. Then also show why the RHCE for example is 
superior and back up your assertion with measurables.

> If I may be sarcastic, it is only following the myriad flies
> principle, 'All those myriad of flies eat sh..., myriads of
> individuals can't be wrong. So sh.. must be good, we must have it
> too.'

No, this logic is fundamentally flawed and it's called "Fallacious 
proof by association". I'll show you why it's wrong:

Microsoft uses a mouse driven GUI for their Windows OS.
The Windows architecture is demonstrably flawed in terms of security.
*nix can run Gnome which is a mouse driven GUI.
Linux is a *nix clone.
Therefore Linux is fundamentally flawed in terms of security.

See the flaw in the logic? The Ubuntu exam may or may not be 
scientifically sound, that can only be answered after the beta test 
phase is complete, but to form an opinion about it you will have to 
examine the Ubuntu exam, not other exams in the same class from other 
vendors.

> > Thomas, correct me if I'm wrong here, but did you research how
> > LPI exams and study material (and hence the Ubuntu exams) work?
>
> You can bet on, I did.
>
> I had a long lasting discussion with some LPI guys, e.g. Evan
> Leibovitz, at one of the last CeBIT fairs. In the end they they had
> to agree that LPI exam as test is not able to grant a prooven
> skill. Only for beeing able to learn a lot of questions and to
> answer some of them in a test.

But no-one ever claimed that LPI demonstrates a proven skill! No test 
can do that - all you can do is provide a statistically meaningful 
probability that applies to the general case and not necessarily to 
the specific case. Including the RHCE, which does not test for 
underlying cognitive understanding. 

For example, if a practical exam tests for setting up Apache 1.1, you 
can derive nothing meaningful about the candidate's ability to handle 
Apache 2.0, or even if he can configure Apache on any other arb 
distro that modifies the Apache setup. You don't have enough data to 
know either way and you'll have to observe the person on the job to 
get the data.

> I've been involved in the conceptional and organisational phase of
> establishing a curiculum for professional IT education at
> Universities of applied science.
>
> Plus some other minor activities , e.G. AK-ZERT (AKZERT; Research
> Group CERTification)
>
> > It's completely unlike every other IT cert out there. There is no
> > prescribed curriculum, no lesson plan, no prescribed materials
> > you must use - basically not much in the way of rules as to how
> > anyone should perform the teaching step.
> >
> > LPI candidates are free to get their knowledge anyway they want -
> > old timers got it from man pages and mailing lists, newbies might
> > want to attend a formal course. Both are OK. If you want to
> > organize a LUG-style after hours class to teach stuff, you can go
> > right ahead and do it - in fact you are encouraged to do so. You
> > can deliver a course in a classroom, over the web or any other
> > way you think would work for you and your students. You can teach
> > it in any order you like and take as long as you want, you can
> > offer any kind of preparation, assessment and practice tests you
> > think are appropriate, and no-one can tell you otherwise.
>
> And thats just what I'm talking about.
> It don't matter if you are skilled or just learned a lot of
> questions for a test (what I like to call a braindump test)

But you are missing the point, and assuming that one can learn a lot 
of answers to pass the test. I've watched 500 people go through these 
tests and so far only two managed to pass by learning stuff by rote. 
True, this is just my own observation and is largely based on my 
interpretation of how they prepared, but even more telling is the 
results of the experienced syadmins. It's real easy to spot those who 
know their stuff, and their pass rate is over 95% of those I've seen. 
The unlucky 5% were usually very specialized in their knowledge (like 
only knowing sendmail and almost never worked with any other 
service).

If LPI was a simple braindump test, then the pass rate would be 
significantly higher than what it is, and currently it's about 50%

> > The one thing you can't change is the list of objectives (areas
> > to be tested). They are developed with community input and there
> > was a wiki page about this, but once published they are fixed
> > till the next update.
> >
> > There is a Channel Partner Program in the works, that is for the
> > Corporate Training industry who demand a worked-out curriculum to
> > deliver and don't want to go to the effort of designing one
> > themselves. You don't have to agree with that and don't have to
> > sign up for it.
>
> You missed me I'm not interested to sign up for anything or to have
> a degree or whatever.
>
> I'm just mentioned that supporting another LPI examination will be
> desastrous and contraproductive in the end.

No, you are saying that it's your opinion that the testing methodology 
is flawed and/or specific other exams are weak. You have yet to show 
any evidence that this exam is the same. In fact you have not 
provided any evidence for your entire position, only your opinion.

I really do want to engage you on this and see if there's a way we can 
work together towards the common goal of increasing Ubuntu competence 
out there. But I can't do it if all you say is "LPI/Ubuntu is cr at p 
because all similar tests are cr at p".

You see, it's not the exam that's the most important thing here, it's 
the training. Real knowledge needs to move into the students head 
where he/she can use it, and the exam is the last final step to 
measure how successful that was. Passing the exam is not the goal, 
getting the knowledge to pass any fair exam is the goal.

> But for this you may ask other people from germany who must see
> most high rated and well known education institutes passing away
> over the last ten years and this because of the acceptance of this
> stupid braindump tests, MCSE, ..., and LPI.

It's well known that Microsoft handled their exam program badly - they 
did nothing to prevent braindump sites from appearing. The way their 
program is built makes it possible for the 'study from braindump 
site' technique to work. I've had many MCSE qualified people come 
through my Linux courses and *every*single*one* has told me that 
their class instructor gave them the exam questions in the final two 
weeks of the course! I mean that literally, as in the instructor 
printed pages from braindump sites, xeroxed them and handed them out 
to the students. This is over 30 students from different colleges. 
And all the MCSE instructors I ever spoke to eventually admitted that 
they were given pass quotas by the company they worked for, the quota 
usually being an 80 to 90% pass rate. No wonder that exam is flawed, 
no wonder they go out of business and no wonder the exam itself is 
now completely meaningless.

That doesn't mean we have to fall into the same trap.

> What I was talking about is a community driven strategy/system
> which scopes the education from a novice (a private users) point of
> view up to a professional training and examination. As high as an
> university degree if offered.
>
> What you mentioned as 'Channel Partner Program' again is a
> shortsighted way because ther is a gap between this professional
> training offers and no training offers/structures for private or
> novice users.

The Channel Partner Program includes an ICDL component for delivering 
professional training to end users. But that's not my field.

The community is perfectly free to devise ways to train ordinary users 
in unique innovative ways. A good start would be to get existing 
training groups to offer Linux training and use Ubuntu as the 
installed OS. I don't know what a university degree could possibly be 
with regard to Ubuntu, that doesn't fit with how universities work. 
What would be on offer? A BSc Comp Sci (Ubuntu)? But there isn't a 
BSc Comp Sci (Windows) or BSc Comp Sci (Solaris). What would the 
subjects be? Coding? That's adequately covered with existing 
assembler, C, Java and Python courses. Sysadmin? I don't know of an 
(SA) university that does sysadmin training - that's the domain of 
colleges and technikons.

A real example of this program in operation: the first pilot is 
happening right here in Cape Town SA right now, it's a Go OpenSource 
initiative intended to train 100 people on Ubuntu sysadmin basics. 
TSF and Go Open have a good relationship with 6 local universities 
through various education programs such as providing FOSS labs for 
students and the general public. Phase 1 is to train the trainers who 
will train the public and these first trainers are existing 
university lecturers and support and training staff - they do 
training and testing for a living. These people completed step 2 of 
their training yesterday and today they wrote the 101 exam (they were 
trained on that earlier this year and have been getting practical 
experience since). 

So 35 people knowledgeable about testing wrote LPI 101. The 
overwhelming feedback after the exam was that it was "tough, but fair 
and a good measure of overall competence"

> What you will loose by doing so is the chance to have a tremendous
> motivational argument to involve the community in many ways.
> It is again a gradet system which has big gaps (better hurdles)
> between the diferent grades. It will again be a manifestation of
> barriers in education.

The community can still be involved - this exam is only for sysadmins 
who will provide support to companies using Ubuntu. Every other use 
case for Ubuntu training is up for grabs (disclaimer - that's *my* 
point of view only, I don't work for LPI, Canonical, TSF, Go Open, or 
any Linux training college so I can't make policy statements on their 
behalf [my email domain is now my ISP, I haven't worked there for a 
year] )

>
> regards,
> Thomas

-- 
Alan McKinnon
alan at linuxholdings dot co dot za
+27 82, double three seven, one nine three five