[LTP] LTP - Include upstart whitebox / blackbox testing API's?

[Subrata Modak]

Garrett,

Is there any headway with upstart developers regarding this initiative.
I dug out this mail from my mailbox to find this. Let me know if we can
resume this discussion once again.

Regards--
Subrata

On Fri, 2008-06-27 at 19:06 +0530, Subrata Modak wrote:
> On Thu, 2008-06-26 at 05:26 -0700, Garrett Cooper wrote:
> > Hello LTP gurus (and upstart gurus),
> >       As I mentioned before on the upstart-devel list, one of the
> > goals of the groups that I'm working with is to bring upstart -- the
> > init replacement -- to Cisco's Linux based platform for process
> > monitoring and management. As part of that we (my teammates and I)
> > were thinking of including whitebox and blackbox tests with LTP (Linux
> > test project) to try and unify testing of critical Linux components,
> > and also provide deterministic output also with greater visibility in
> > the testing community.
> >       LTP has a number of whitebox and blackbox tests in place [3],
> > most of the whitebox tests being C API's and the blackbox tests being
> > shell invocations of Unix commands, as well as a well-defined set of
> > test reporting API's and functions already in place.
> 
> Ah!. That reminds me of the testcases for commands in LTP:
> 
> http://ltp.cvs.sourceforge.net/ltp/ltp/testcases/commands/
> 
> I have been merging lots of patches and we were totally engaged with our
> white box test cases, that we completely forgot about those black box
> test cases, which are of immense help for:
> 
> 1) Increasing code coverage for the kernel,
> 2) Testing the actual/mostly-used interfaces to the Linux OS.
> 
> Thanks Garrett for reminding this valuable testcases piece. And the
> important point here to make is:
> 
> Writing white box test cases requires fair knowledge of Kernel
> Internals, whereas the Blackbox test cases just requires user knowledge
> of the OS. With guidance from the Man Pages information, a huge
> community of administrators and normal users can write these black box
> tests. And they are a huge group of people to count. I need to look into
> this seriously from now.
> 
> >       So, my question is two-fold:
> >       1. Would the upstart project be willing to work with LTP (via my
> > team as a proxy in the beginning) to enter some unit test code and
> > other test cases into LTP's test framework / overall testsuite, and
> > improve acceptance in the Linux testing community?
> 
> I would be providing you the support with testing on the architectures i
> have at my disposal and speedy patch merge to LTP. We definitely need to
> do something to increase the code coverage.
> 
> >       2. Would either group be willing to work with my team to help
> > maintain these testcases and develop new ones?
> 
> Of course, i will.
> 
> > Thanks,
> > -Garrett
> > 
> > PS. Sorry for the cross-posting ; I try not to do this, but
> > considering that both groups can benefit from the discussion I wanted
> > to involve both.
> 
> Nothing to worry about. When it comes to making Linux better, we need
> collaboration on various fronts. The livest example being the work done
> by Masatake Yamato from Red Hat in porting Crackerjack´s
> (https://sourceforge.net/projects/crackerjack) regression tests to LTP
> format. Thanks Garrett for taking this initiative. We need to
> collaborate much more with others as well.
> 
> Regards--
> Subrata
> 
> > 
> > 1. LTP -- Linux test project: http://ltp.sourceforge.net/
> > 2. Upstart -- init(1) replacement: http://upstart.ubuntu.com/
> > 3. LTP cvsweb -- http://ltp.cvs.sourceforge.net/ltp/ltp/ (see docs for
> > relevant documentation items, lib/ltp for test lib API's, and
> > testcases/commands for existing Linux command blackbox tests).
> > 
> > -------------------------------------------------------------------------
> > Check out the new SourceForge.net Marketplace.
> > It's the best place to buy or sell services for
> > just about anything Open Source.
> > http://sourceforge.net/services/buy/index.php
> > _______________________________________________
> > Ltp-list mailing list
> > Ltp-list at lists.sourceforge.net
> > https://lists.sourceforge.net/lists/listinfo/ltp-list
> 
> 
> -------------------------------------------------------------------------
> Check out the new SourceForge.net Marketplace.
> It's the best place to buy or sell services for
> just about anything Open Source.
> http://sourceforge.net/services/buy/index.php
> _______________________________________________
> Ltp-list mailing list
> Ltp-list at lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/ltp-list