Q: Could you explain the difference between supplier audits and a supplier evaluations?
As I recall from attending your training classes, you’re a proponent of evaluations. Could you explain further?
A: It’s not so much that I’m a proponent of supplier evaluations but that I want people to recognize the inherent limitations of audits.
An audit is typically done as an on-site visit — one that quite often doesn’t give you enough time to get into depth on issues that may be important to you.
When you’re conducting supplier audits, the supplier might dictate how long you’re going to be there. If they think you’re going to be on-site too long, they’ll limit the number of days. They’ll even limit the people you can talk to.
In some cases you have very little control over these limitations. Even though you may be buying a product that’s very critical to your operations, you might be a small player in that supplier’s world, with very little clout.
As a result, I think all of us who conduct audits recognize that most of them are nothing more than snapshots in time.
In contrast, an evaluation — which is required by 21 CFR Part 820.50 and may or may not include a site visit — gathers and analyzes all kinds of data from all kinds of sources, without some of the time and physical constraints associated with audits.
“In an evaluation, you want to gather and analyze as much data, externally as well as internally, about the supplier.”
In performing supplier evaluations, you want to gather and analyze as much data, externally as well as internally, about the supplier. Beyond gathering information about the supplier’s operations and procedures, you want to devote more time and attention to evaluating the supplier’s performance, the health of their quality management system, and the effectiveness of their products and services.
As much as possible, you want input from people who have first-hand knowledge about the supplier. Talk to those who use their product. Talk to those who have stopped using their product. Spend time reviewing the vendor’s website and compare them with their competitors.
Look at reports of problems, especially FDA Warning Letters and 483s if the vendor is regulated by the FDA. If not, talk to user groups and members of professional societies and trade associations.
In FDA’s written regulations, the Agency does not require supplier audits at this time. However, there is an expectation that if you’re dealing with an important (defined by your risk analysis) supplier, you should audit them, and if it’s a critical supplier, you’re absolutely expected to audit them. These audits should be performed as part of your overall supplier evaluation.
That brings me back to understanding the limitations of audits. Consider the FDA audits you’ve experienced. Did FDA find everything during their audit? I’ve never experienced that, and I worked for FDA for 22 years.
FDA’s audits are just like yours. They’re snapshots, just moments in time.
So be careful how much reliance you place on audits. Focus your energies and efforts on doing a thorough supplier evaluation, which certainly can include an audit if warranted.
Over time, your evaluation data and knowledge will continue to build and improve, allowing you to better understand your supplier’s strengths and weaknesses and spot negative trends before they impact the quality of your own products.
Answered by Martin Browning, President and Co-Founder of EduQuest, who served as an expert field investigator and the Special Assistant to the Associate Commissioner for Regulatory Affairs during his 22-year career at FDA. Martin also is the instructor of EduQuest’s Managing and Auditing Supplier Quality training class.