Using Data to set Action Standards for Decision Making: An Introduction to Risk Assessment

Every product test is an experiment. Their data collected are used as the basis for business decisions and planning next development steps.

Every statistical test run on the data advises on the direction of those decisions and each test is accompanied by risk of advising poorly.

Statistical test results run the risk of making one of two errors:

1. making the tester believe there is a difference between the products being tested when there are none.
2. making the tester believe there is no difference between the products being tested when there is.

The goal here is provide a context for risk:

• To help the tester quantify the risk (in a monetary sense, if possible)
• Adjust the statistical testing process accordingly and reduce, or at least prepare for, the risk.