Researchers reawaken iris-ageing debate
30 November 2015 11:12 GMT

Biometrics researchers from the University of Notre Dame have again raised questions over the accuracy of a National Institute of Standards and Technology (NIST) study into the potentially detrimental effects of ageing on iris-based biometric identification – claims which NIST has rebutted as “quantitatively irrelevant”.

In a report presented at the IEEE Computer Society Biometrics Workshop in June, Estefan Ortiz and Kevin W Bowyer focus on potential issues with a prior NIST investigation of iris recognition stability over time – the IREX VI report published in July 2013.

The IREX VI NIST report had concluded that there should be no concern about iris recognition accuracy degrading over time, contradicting previous findings by Bowyer and Ortiz’s Notre Dame research group. A prior study of 217 subjects over a three-year period had found that the recognition of the subjects' irises became increasingly difficult, consistent with an ageing effect.

In their new report, “Exploratory Analysis of an Operational Iris Recognition Dataset from a CBSA Border-Crossing Application,” Bowyer and Ortiz focus specifically on the IREX VI report’s analysis of an iris recognition dataset from the NEXUS border-crossing program run by the Canadian Border Services Agency (CBSA). NEXUS is a joint Canadian and American program used by frequent travelers to move quickly across the Canadian border.

Ortiz and Bowyer state that the NIST IREX VI report on iris recognition ageing “relies heavily” on the “OPS-XING” dataset, which they state drawn from logs recorded by the CBSA’s operational border-crossing program. The authors write that this operational scenario uses "1-to-first" matching, “not 1-to-N” as claimed in the IREX VI report, noting that the importance of this is that it means that the dataset contains “false matches that were labelled as authentic”.

The recorded match scores are normalised match scores, resulting in a "pulse" at the value of 0.0, writes the Ortiz and Bowyer report, meaning that the regression method used assumes that the data does not have this property.

“The right iris scores are on average worse than the left iris scores. This is because a right iris was mostly matched only if the left iris failed. But the protocol for matching right irises changed over the life of the system. This would complicate time sequence analysis of the data, but this is not handled in the IREX VI approach,” the authors told Planet Biometrics.

The writers add in their report: “It is certainly not appropriate to perform an explanatory regression analysis based on left and right iris scores together in a regression model that simply differentiates between the two with an indicator variable in the fixed effects, as is done in IREX VI. “ Additionally, although the “OPS-XING” dataset is actually an aggregate of datasets from 70 different kiosks in 11 different cities, Ortiz and Bowyer note that the IREX VI report does not mention this.

Writing in response to the concerns to Planet Biometrics, NIST’s Biometric Testing Project Leader, Patrick Grother, wrote that “The issues brought up by Professor Bowyer’s group are quite technical and we lean towards addressing them in the technical literature or in collaborative discussions”.

“Our key rebuttal of Notre Dame is that they make qualitatively correct points that are quantitatively irrelevant.  A prime example here is the Notre Dame assertion that the border crossing dataset suffers from the presence of ‘false matches labelled as authentic.’”

“Notre Dame is silent on their prevalence.  Instead they fault our study for using such a dataset.  However, in their forthcoming IET Biometrics paper, Notre Dame estimates that ‘hundreds of impostor scores’ are present, but declines to express that as a proportion of the total – it is below 0.01%. That proportion is tiny, to the point of risible.  Notre Dame did that computation before their CVPR paper, yet did not disclose its magnitude.”

In January 2014, Bowyer and Ortiz released an earlier report that attempted to explain why academic researchers have been achieving seemingly incompatible results on the effect of iris ageing.


Related articles

Iris template ageing debate rolls on…
No evidence of widespread iris ageing effect, says NIST