UFPR Fairness - Across Age

Dataset

  • Dataset includes 33,660 ocular samples from 1,122 subjects captured by 196 different mobile.

  • All images were resized to224×224. the complete dataset is classified into three age-groups: namely young (18 to 39), middle-aged (40 to 59), and older adults (60 to 79).

  • User recognition and gender classifier models are fine-tuned on randomly selected gender and age-group balanced subset from 780 participants. Subject-disjoint, gender and age-group-balanced subset selected from 342 subjects are used as the test set for authentication and 260 subjects (130 males and females) for gender classification.

  • Age classifier models are fine-tuned on balanced subset selected from 432 subjects and evaluated on subset from 132 subjects, across 6 age-groups, namely 18−29, 30−39, 40−49, 50−59, 60−69 and 70−79.


TABLE I: EER and FNMR at 0.01 and 0.1 FMR for user authentication using CNN models for Left (L), Right (R) ocular region,and their score-level fusion (L+R) for Young, Middle-Aged and Older adults evaluated on balanced version of UFPR ocular datasets.
CNN
Age-Group
EER(%)   FNMR(%) @ FMR
0.01 0.1
L R L+R L R L+R L R L+R
ResNet-50 Young 8.60 9.52 9.06 37.63 35.35 36.49 53.74 54.03 53.89
Middle-Aged 8.62 9.08 8.85 30.76 25.04 27.9 52.23 48.55 50.39
Older 11.01 11.00 11.01 15.47 19.34 17.405 30.67 30.67 30.67
MobileNet-V2 Young 7.75 7.32 7.54 33.21 26.11 29.66 51.61 48.28 49.95
Middle-Aged 9.08 8.68 8.88 29.18 28.14 28.66 51.17 51.31 51.24
Older 8.04 9.63 8.84 18.54 13.47 16.005 39.47 36.00 37.74
ShuffleNet-V2 Young 6.93 6.96 6.95 37.63 38.09 37.86 56.44 56.26 56.35
Middle-Aged 8.32 9.25 8.79 37.38 46.07 41.73 55.08 61.10 58.09
Older 9.72 8.18 8.95 30.53 30.80 30.67 44.93 53.47 49.20
EfficientNet-B0 Young 7.64 9.61 8.63 32.54 27.00 29.77 55.40 48.38 51.89
Middle-Aged 6.95 9.03 7.99 38.07 26.69 32.38 49.12 49.31 49.22
Older 9.34 12.08 10.71 20.80 23.33 22.065 39.73 41.74 40.74

TABLE II: Accuracy of CNN-based Gender Classification on Left Ocular Region among Young (18 to 39 years), Middle (40 to 59 years) and Older Adults (60 to 79 years).
CNN Young Middle-Aged Older
Male[%] Female[%] Male[%] Female[%] Male[%] Female[%]
ResNet-50 98.39 98.19 100 96.67 99.17 98.06
MobileNet-V2 99.97 99.9 100 99.7 100 100
ShuffleNet-V2-50 98.23 97.57 98.28 97.56 94.76 98.89
EfficientNet-B0 95.89 97.54 98.58 96.44 95.23 86.94

TABLE III: Accuracy of the CNN-based Gender Classification on Right Ocular Region among Young (18 to 39 years), Middle (40 to 59 years) and Older Adults (60 to 79 years).
CNN Young Middle-Aged Older
Male[%] Female[%] Male[%] Female[%] Male[%] Female[%]
ResNet-50 97.98 99.61 97.07 99.78 92.86 98.61
MobileNet-V2 96.84 92.35 95.86 98.66 94.76 97.5
ShuffleNet-V2-50 98.51 98.91 98.79 99.33 97.38 98.61
EfficientNet-B0 97.18 95.14 95.15 97.33 94.52 90

TABLE IV: Exact and 1-off accuracies of Age-group Classification for Young Adults
Left Ocular Right Ocular
CNN Exact [%] 1-off [%] Exact [%] 1-off [%]
ResNet-50 46.72 93 52.87 93.16
MobileNet-V2 54.14 91.78 59.76 94.9
ShuffleNet-V2-50 52.47 91.16 45.16 85.27
EfficientNet-B0 28.225 53.195 31.6 62.64

TABLE V: Exact and 1-off accuracies of Age-group Classification for Middle-Aged Adults.
Left Ocular Right Ocular
CNN Exact [%] 1-off [%] Exact [%] 1-off [%]
ResNet-50 31.61 81.73 35.03 78.43
MobileNet-V2 27.95 86.35 31.86 72.69
ShuffleNet-V2-50 28.46 75.2 24.61 47.98
EfficientNet-B0 20.93 57.86 19.56 55.73

TABLE VI: Exact and 1-off accuracies of Age-group Classification for Older Adults.
Left Ocular Right Ocular
CNN Exact [%] 1-off [%] Exact [%] 1-off [%]
ResNet-50 28 82.3 29.485 71.43
MobileNet-V2 38.07 79.82 32.93 76.65
ShuffleNet-V2-50 36.04 89.65 18.54 56.85
EfficientNet-B0 4.97 32.3 4.74 27

References

Bibliography called, but no references
Sreeraj Ramachandran
Sreeraj Ramachandran
Graduate Research Assistant

My research interests include Computer Vision, Biometrics, Bias AI, GANs and adversarial Attacks

comments powered by Disqus