Automating

• configuring the system

• generating data for the system

• operating some aspect of the system

• observing aspects of the state of the system before/after configuration

• observing some aspects of the system's behaviour

• observing some aspects of the system's output

• observing records that the system produces as it behaves (i.e., typically, log files)

• observing aspects of the state of the system after it has performed some function

• applying decision rules to the operations and observations above

• aggregating the outcome of any of the operations or observations above

• parsing, sorting, or filtering data

• performing analysis of that data

• comparing that output to some specified and desired (or perhaps undesired) output

• alerting someone to the the outcomes of operations and observations

• representing the aggregated outcomes in different forms (for example, rendering data from log files as tables, charts, or graphs, in order to help testers see patterns of behavior or output)

Scientific Method says:

1. Observe & Question

2. Form a Hypothesis

3. Test with experiment

4. Analyze data

5. Interepret the data

6. Report conclusions