When You get a test result from Your engineers, do You feel confident about reporting it?
Any test involves sampling and a sample is a just glimpse of the real world. Therefore the conclusion made from the sample may not be rock solid.
Imagine a picture of some connected steel bars. One may jump to the conclusion that this is a picture of the Eiffel Tower while the truth is that it is a close up of a high power mast.
A larger picture could have helped along the right way. The same goes for sampling.
If You have a box with 500 blue and 500 red balls mixed up there is a good chance that if You take five balls it will be three blue and two red. You conclude that the box contains 60% blue and 40% red balls.
Then another person takes five and gets two blue and three red and concludes that the box contains 40% blue and 60% red balls.
Had both of you taken 20 balls it is very likely that you would agree more – that you would perhaps say 48-52% instead of 40-60%.
This is what confidence is all about when it comes to statistical analysis.
If you take a small sample it is likely that someone else might get a different result, You are not so confident in Your own conclusion.
If You take a large sample You are confident that someone else will reach the same conclusion as You did.
In statistical analysis there are ways to calculate how confident You are from a given sample size and that is one way to feel comfortable about reporting analysis conclusions.
In the example above one might say: “I am 90% confident that there is at least 40% and no more than 60% blue balls in the box.”
That way it is clear that the box contains approximately 500 of each colour but there is small risk that there is only 400 blue balls.
If that is critical to quality, then You need to look at more balls and get perhaps 95% sure that the number is between 48 and 52%
There will be more to follow on this topic, so stay tuned