Testcase Standards
- Testcases should be written in C and/or one of a very
limited set of scripting languages:
Unusual languages (e.g. Lisp, Prolog, Standard ML) and
"advanced scripting languages" (e.g. Perl, Python, Ruby)
are forbidden, as are utilities atypical of standard
UNIX userspace (seq, jot, etc.) and exotic shells (e.g.
zsh, tcsh, bash extensions).
- Unusual libraries are forbidden except where they provide
essential testcase support e.g. kernel aio.
- Linux Kernel coding style standards should be obeyed.
- Any system or library call which may return an error must
be checked for error returns.
- Any library-like functionality must check for errors in
its callees and properly return them to the caller.
- Hardcoded parameters are forbidden; all options should
be passed on the command-line, not adjusted by
editing and/or recompilation or similar.
- The testcase should run for a stringently controlled
duration, set by command-line options.
- When coding similar tasks carried out by varying
methods, driver-like arrangements should be made so
that additional methods may be added by supplying
additional operation tables.
- Any error affecting the validity of the test must
result in an abnormal exit code from the testcase.
- Short reads and writes must be handled regardless of
the anticipated medium to which the IO is to be sent.
- Signals should be handled just as errors are in any
case where a signal is used in lieu of an error
return from a system or library call esp. SIGPIPE.
- All memory allocations must be checked for failure.
- SIGBUS must be handled for mmap() IO on any file that
may be truncated while the test is in progress
regardless of whether the testcase itself truncates
the file.
- Output should be consistent and machine-parseable.
- Output should include either a meaningful performance
metric or pass/failure status.