Blog
Updated 7 Oct 2022
NIST is a non-regulatory agency of the U.S. That means there are actually no NIST regulatory requirements for eKYC vendors to comply with. But that doesn’t mean NIST plays no role in the eKYC industry.
NIST is in the testing business, and one of the things they decided to test beginning back in 2006 is face recognition algorithms. Known as the FRVT 1:1 Verification, or Face Recognition Vendor Test, NIST uses “very large sets of facial imagery to measure the performance of face recognition algorithms developed in commercial and academic communities worldwide.” The data sets include passport photos, mugshots and photos “in the wild”.
The goal of FRVT is to measure “the performance of automated face recognition technologies applied to a wide range of civil, law enforcement and homeland security applications including verification of visa images, de-duplication of passports, recognition across photojournalism images, and identification of child exploitation victims.”
The metric produced by FRVT testing is called the False Non-Match Rate (FNMR). According to NIST, “FNMR is the proportion of mated comparisons below a threshold set to achieve the false match rate (FMR) specified. FMR is the proportion of impostor comparisons at or above that threshold.”
In simpler terms, FNMR is the probability that the system fails to detect a match. And of course, the lower the number the better. Accordingly, companies periodically submit their algorithms to NIST to get an FRVT evaluation and to see just how low their FNMR is.
Why is it important for face recognition vendors to have their algorithms periodically evaluated by a trusted third-party like NIST? Because if they get a really good score, they can brag about it.
When shopping around for an eKYC vendor, one of the criteria you should use is their NIST FNMR score. NIST actually ranks vendor algorithms by their FNMR score in several different categories. In each category, the lower the number, the higher the ranking.
How will you know if a particular vendor ranks high in one of these categories? You won’t have to look very hard, because they put front and center.
Here is one vendor, identifAI. From their home page, “Ranked Top 30th Percentile in NIST Leader Board. Nodeflux passed the US National Institute of Standard and Technology (NIST) Face-Recognition Vendor Test (updated 2019).”
Here is another vendor, HyperVerge. “HyperVerge made news by topping the NIST rankings for facial recognition against several benchmarks recently.” In this case, HyperVerge’s face recognition system had a false non-match rate as low as 0.66%, which matched the leader in the NIST rankings.
NIST even conducted FRVT testing with face masks. Beginning back in November, 2020, NIST began “quantifying face recognition accuracy for people wearing masks.” There were 152 algorithms evaluated. But as of May, 2022, NIST has stopped running that test, although vendors who want their algorithm evaluated can get it done on an ad-hoc basis.
NIST has no real regulatory authority in the eKYC industry. But the organization still makes an impact by evaluating facial recognition algorithms of eKYC vendors. And that standardized metric, FNMR, can be used to objectively evaluate eKYC vendors.
If you run a fintech SaaS and need help navigating eKYC, including evaluating facial recognition vendors, Ostride Labs would be happy to assist you. Reach out to us here to arrange a quick call to see how we can help.
Our newsletter (you’ll love it):
