[LTP] LTP - Include upstart whitebox / blackbox testing API's?

[Subrata Modak]

On Thu, 2008-06-26 at 05:26 -0700, Garrett Cooper wrote:
> Hello LTP gurus (and upstart gurus),
>       As I mentioned before on the upstart-devel list, one of the
> goals of the groups that I'm working with is to bring upstart -- the
> init replacement -- to Cisco's Linux based platform for process
> monitoring and management. As part of that we (my teammates and I)
> were thinking of including whitebox and blackbox tests with LTP (Linux
> test project) to try and unify testing of critical Linux components,
> and also provide deterministic output also with greater visibility in
> the testing community.
>       LTP has a number of whitebox and blackbox tests in place [3],
> most of the whitebox tests being C API's and the blackbox tests being
> shell invocations of Unix commands, as well as a well-defined set of
> test reporting API's and functions already in place.

Ah!. That reminds me of the testcases for commands in LTP:

http://ltp.cvs.sourceforge.net/ltp/ltp/testcases/commands/

I have been merging lots of patches and we were totally engaged with our
white box test cases, that we completely forgot about those black box
test cases, which are of immense help for:

1) Increasing code coverage for the kernel,
2) Testing the actual/mostly-used interfaces to the Linux OS.

Thanks Garrett for reminding this valuable testcases piece. And the
important point here to make is:

Writing white box test cases requires fair knowledge of Kernel
Internals, whereas the Blackbox test cases just requires user knowledge
of the OS. With guidance from the Man Pages information, a huge
community of administrators and normal users can write these black box
tests. And they are a huge group of people to count. I need to look into
this seriously from now.

>       So, my question is two-fold:
>       1. Would the upstart project be willing to work with LTP (via my
> team as a proxy in the beginning) to enter some unit test code and
> other test cases into LTP's test framework / overall testsuite, and
> improve acceptance in the Linux testing community?

I would be providing you the support with testing on the architectures i
have at my disposal and speedy patch merge to LTP. We definitely need to
do something to increase the code coverage.

>       2. Would either group be willing to work with my team to help
> maintain these testcases and develop new ones?

Of course, i will.

> Thanks,
> -Garrett
> 
> PS. Sorry for the cross-posting ; I try not to do this, but
> considering that both groups can benefit from the discussion I wanted
> to involve both.

Nothing to worry about. When it comes to making Linux better, we need
collaboration on various fronts. The livest example being the work done
by Masatake Yamato from Red Hat in porting Crackerjack´s
(https://sourceforge.net/projects/crackerjack) regression tests to LTP
format. Thanks Garrett for taking this initiative. We need to
collaborate much more with others as well.

Regards--
Subrata

> 
> 1. LTP -- Linux test project: http://ltp.sourceforge.net/
> 2. Upstart -- init(1) replacement: http://upstart.ubuntu.com/
> 3. LTP cvsweb -- http://ltp.cvs.sourceforge.net/ltp/ltp/ (see docs for
> relevant documentation items, lib/ltp for test lib API's, and
> testcases/commands for existing Linux command blackbox tests).
> 
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> Ltp-list mailing list
> Ltp-list at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/ltp-list