Testing without executing, with code reviews, walkthroughs, inspections, linting.
- Errors hide other errors
- Code doesn’t need to be complete to test statically (compilers need not apply)
- Can consider code quality
Executing with test cases.
Tests derived from control/data flow of system.
- Statement adequacy: All statements are tested.
- Statement coverage: Executed statements / Total statements
- Path coverage: achieving coverage by exploring all state transitions of a system. Deriving a program’s control flow helps to derive tests for a system.
Tests derived from formal component specification.
- Identify functions you expect system to be able to perform
- Create input data, and define expected outputs
- Run test cases
- Compare output data with expected data
- Check the application works as per customer needs
- Ensuring all objects and methods are working.
- Must be done for all objects, so automate as much as possible!
- Aim for effectiveness: when used as expected, action performs as intended
Testing interactions between unit components. Types of interface errors to look for:
- Interface misuse: a component is passing wrong parameters, or receiving an unexpected return value
- Interface misunderstanding: a component doesn’t understand the behaviour of another one
- Timing errors
Looking for emergant behaviour in a complete system.
- Use case testing is useful, test for expected and unexpected emergant behaviour
- Force interactions to occur between components in the system
Exhaustive testing is impossible, so we make test cases which:
- Execute each statement ≥1 time
- Tests all functions via all menus/interfaces
- USes correct and incorrect test data/user input
Originally introduced with XP, gained popularity across the board.
- Identify an increment, write an automated test for the new feature
- Run it - it will fail
- Implement the functionality and rerun the test, and repeat until the test passes
- Alpha: Small set of users working with the development team very early on
- Beta: Much larger group of users testing an almost-complete system
- Acceptance: Customer deciding whether the system is ready to be deployed