Here is the problem, almost repeating the description by Teddy.
Suppose you have a car, and the car stops working. You go to a garage. Here is what the mechanic tells you. The engine timing belt broke while you were traveling at high speed. The mechanic sayes that he can't tell if there was resulting engine damage until completing most of a several hundred dollar replacement of the belt. So, you have to decide whether to gamble about $150 (out of a $230 total timing belt repair) to find out whether your car (with 98.5K miles on it) has sustained engine damage - in which case it wouldn't pay to replace the belt and the $150 would be lost! The car is crucial, and you estimate that replacing it with all the involved costs would take some $4,500. You also have extra info of a "subjective" kind to help with the decision: When the timing belt broke, you recognized what had happended and almost immediately shifted the car into neutral. So, the engine speed dropped to 0 rpm naturally, under its own weight, without the weight of the car forcing it to turn out of synch.To solve this problem, consider the options:
This is a case in each we can be reasonably confident that decisions are robust, even if we are not very confident about the value of p. Say we think 0.1 < p < 0.4, which characterizes a set of probability distributions. Even then it is best to pay for the measurement.
Things change if you decide to buy a very cheap car. If the cost of the new car is $1000, then the graph of expected loss is as follows.
In this case, the interval 0.1 < p < 0.4 does not lead to a robust decision. It is best to test as long as p > 0.163043; at this point, you can either buy a new car or pay for the test. With the preferences and opinions you indicated, there is nothing that forces you to go either way.
Fabio Cozman, The Robotics Institute, Carnegie Mellon University