Testcase Standards

• Testcases should be written in C and/or one of a very limited set of scripting languages:
  • Bourne shell
  • awk
  • sed
  Unusual languages (e.g. Lisp, Prolog, Standard ML) and "advanced scripting languages" (e.g. Perl, Python, Ruby) are forbidden, as are utilities atypical of standard UNIX userspace (seq, jot, etc.) and exotic shells (e.g. zsh, tcsh, bash extensions).
• Unusual libraries are forbidden except where they provide essential testcase support e.g. kernel aio.
• Linux Kernel coding style standards should be obeyed.
• Any system or library call which may return an error must be checked for error returns.
• Any library-like functionality must check for errors in its callees and properly return them to the caller.
• Hardcoded parameters are forbidden; all options should be passed on the command-line, not adjusted by editing and/or recompilation or similar.
• The testcase should run for a stringently controlled duration, set by command-line options.
• When coding similar tasks carried out by varying methods, driver-like arrangements should be made so that additional methods may be added by supplying additional operation tables.
• Any error affecting the validity of the test must result in an abnormal exit code from the testcase.
• Short reads and writes must be handled regardless of the anticipated medium to which the IO is to be sent.
• Signals should be handled just as errors are in any case where a signal is used in lieu of an error return from a system or library call esp. SIGPIPE.
• All memory allocations must be checked for failure.
• SIGBUS must be handled for mmap() IO on any file that may be truncated while the test is in progress regardless of whether the testcase itself truncates the file.
• Output should be consistent and machine-parseable.
• Output should include either a meaningful performance metric or pass/failure status.