What the story with reliability and confidence?
Recall the table I showed a couple of weeks ago? Risk evaluation and requirements for a process reliability (See it here).
When I wrote that 90% reliable is NOT 10% unreliable; what did I mean, how should You understand this, and what are the consequences?
When I say the process is 90% reliable it means that I have evidence for these 90%. My evidence is my confidence. So when I am 95% confident it simply tells that I have decided that:
“objective evidence” = 95% certain.
If you imagine the process as 100 suspects, I can tell you that I have objective evidence for 90 of these 100 persons. I am 95% certain about their innocence (remember we are to “free” the process through objective evidence when we are validating a process).
The last 10 persons, what about them?
Well I do not have quite as strong a confidence in them, but it does not mean that I have NO confidence.
In fact, I am 92% confident that there are 93 innocent persons – so if the judge is happy with 92% evidence, then I can set free three more people.
This is where You have to make a risk based decision. When is You evidence strong enough? And how big a process risk are You willing to accept?
Think carefully about the reliability (or capability) requirements You define. If the possible harm from a failure is small, then the accepted reliability may be lower.
The best way to show it is the matrix below.
If you require a high level of confidence before accepting you process, you will often reject the validation even though the process is perfect.
If You require a high reliability you will also reject a good process sometimes simply because it is more difficult to prove “innocence” to such a high degree.
My suggestion is to consider a lowering the confidence a bit for several reasons.
First of all, You are already familiar with the process, so even if there is no statistical confidence, there is at least the confidence in Your engineers. Secondly process validation is done in several steps and when production starts there will be regular monitoring of the quality.
Each time You perform a test successfully Your confidence will increase. It is very unlikely that You make a mistake over and over again without seeing it.
In total, the risk of a failing process causing hazards to occur is quite small.
The reliability should reflect the hazard a defect could cause.
So give it some thought. What is worse? Rejecting a perfectly good process or approving one that it not truly valid? It depends on Your product, Your customers and Your risk analysis
In these dire times it can be translated to:
The Reliability is the expected effect of lockdown,
The Confidence is how sure the authorities are about the effect.