Results - Tables and Charts

Training Datasets

  • VISOB - VIS, Gender, Subject, Race, Device, Lighting, Synthetic
  • ND-IRIS - NIR, Gender, Subject
  • UFPR - VIS, Gender, Subject, Age
  • Notredame - NIR, VIS, Face, Ocular, Gender, Subject, Race
  • Polyuiris - NIR, VIS, Subject
  • Casia-Lamp - NIR, Subject
  • Casia-Thousand - NIR, Subject
  • FFHQ-Aging - VIS, Face, Ocular, Gender
  • Casia-Syn - NIR, Subject, Synthetic

Evaluation Datasets

  • MOBIUS - VIS, Subject, Device, Light, Gaze, Quality, Gender, Age, Color
  • VISA-IRIS - VIS, Subject, Gender, Face, Ocular
  • IMP - NIR, VIS, Subject
  • I-SOCIAL-DB - VIS, Subject
  • Cross-Eyed - NIR, VIS, Subject
  • UNAB - NIR, Subject, Gender, Makeup, Contacts

Subject Verification

Across Model - VIS

Experiment ID Gender GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
efficientnet_b0 male 0.8789 0.0922 0.5488 0.1155 3.1597 0.9841 0.8735 0.7872 0.0634 0.9189 0.5142 0.2289 0.0797 0.0380 0.7743
efficientnet_b0 female 0.8882 0.0923 0.5531 0.1198 3.1324 0.9825 0.8679 0.7875 0.0663 0.9937 0.5311 0.2258 0.0853 0.0469 0.6158
efficientnet_b0 all 0.8826 0.0931 0.5169 0.1230 3.3535 0.9879 0.8892 0.8098 0.0556 0.9846 0.4508 0.1924 0.0615 0.0286 0.6649
xception male 0.8990 0.0772 0.5978 0.1080 3.2076 0.9858 0.8784 0.7930 0.0612 0.9404 0.4962 0.2207 0.0728 0.0353 0.5414
xception female 0.9060 0.0753 0.5843 0.1122 3.3665 0.9884 0.8900 0.8173 0.0554 0.9204 0.4277 0.1767 0.0603 0.0302 0.6335
xception all 0.9027 0.0763 0.5608 0.1156 3.4906 0.9914 0.9057 0.8366 0.0473 0.9231 0.3650 0.1535 0.0453 0.0195 0.5561
inception_v3 male 0.8782 0.0883 0.5715 0.1063 3.1380 0.9831 0.8645 0.7925 0.0686 0.8564 0.4704 0.2236 0.0886 0.0477 0.6369
inception_v3 female 0.8875 0.0859 0.5823 0.1120 3.0586 0.9803 0.8549 0.7717 0.0731 0.9805 0.4848 0.2506 0.1008 0.0542 0.6807
inception_v3 all 0.8824 0.0877 0.5527 0.1100 3.3143 0.9870 0.8821 0.8187 0.0594 0.9899 0.3959 0.1813 0.0683 0.0355 0.5895
efficientnet_b4 male 0.8875 0.0878 0.5639 0.1119 3.2175 0.9855 0.8798 0.7982 0.0605 0.9901 0.4483 0.2129 0.0722 0.0359 0.7102
efficientnet_b4 female 0.8888 0.0872 0.5453 0.1286 3.1250 0.9839 0.8685 0.7833 0.0660 0.9908 0.5407 0.2292 0.0834 0.0403 0.5421
efficientnet_b4 all 0.8875 0.0885 0.5199 0.1248 3.3971 0.9897 0.8988 0.8309 0.0507 0.9178 0.3987 0.1602 0.0514 0.0250 0.5573
mobilenetv2_100 male 0.8814 0.0884 0.5906 0.1056 2.9872 0.9790 0.8476 0.7610 0.0770 0.9975 0.5424 0.2699 0.1083 0.0577 0.8323
mobilenetv2_100 female 0.8821 0.0913 0.5739 0.1129 3.0011 0.9776 0.8507 0.7682 0.0756 0.9542 0.5026 0.2521 0.1028 0.0582 0.8417
mobilenetv2_100 all 0.8813 0.0906 0.5495 0.1146 3.2125 0.9843 0.8760 0.7881 0.0621 0.9973 0.4841 0.2235 0.0767 0.0372 0.7842
densenet121 male 0.8994 0.0773 0.5724 0.1168 3.3015 0.9879 0.8922 0.8055 0.0540 0.8526 0.4291 0.2003 0.0595 0.0278 0.5406
densenet121 female 0.9117 0.0725 0.5896 0.1194 3.2606 0.9872 0.8895 0.8262 0.0558 0.9971 0.3966 0.1706 0.0613 0.0330 0.8040
densenet121 all 0.9047 0.0762 0.5482 0.1274 3.3964 0.9902 0.9032 0.8292 0.0487 0.9963 0.4061 0.1619 0.0470 0.0228 0.6879
vit_base_patch16_224 male 0.8114 0.1053 0.5404 0.0887 2.7835 0.9722 0.8257 0.7620 0.0894 0.6571 0.4540 0.2733 0.1303 0.0809 0.8871
vit_base_patch16_224 female 0.8308 0.0977 0.5332 0.0971 3.0545 0.9828 0.8572 0.7912 0.0716 0.7064 0.4213 0.2291 0.1000 0.0508 0.9264
vit_base_patch16_224 all 0.8208 0.1027 0.5183 0.0922 3.0995 0.9831 0.8648 0.8059 0.0689 0.7126 0.3885 0.2094 0.0873 0.0494 0.9088
inception_resnet_v2 male 0.8951 0.0815 0.5851 0.1123 3.1598 0.9837 0.8667 0.7707 0.0670 0.9938 0.5751 0.2568 0.0900 0.0405 0.5841
inception_resnet_v2 female 0.9098 0.0752 0.5764 0.1203 3.3232 0.9876 0.8886 0.7877 0.0559 0.9915 0.5765 0.2262 0.0629 0.0271 0.5822
inception_resnet_v2 all 0.9013 0.0799 0.5530 0.1176 3.4639 0.9901 0.8997 0.8197 0.0503 0.9851 0.4776 0.1721 0.0506 0.0198 0.7681
resnet50 male 0.9039 0.0782 0.5945 0.1140 3.1642 0.9842 0.8763 0.7587 0.0623 0.9976 0.6627 0.2778 0.0796 0.0358 0.6409
resnet50 female 0.9121 0.0757 0.5908 0.1184 3.2342 0.9862 0.8785 0.7852 0.0616 0.9999 0.6015 0.2251 0.0774 0.0341 0.4078
resnet50 all 0.9073 0.0782 0.5675 0.1215 3.3251 0.9878 0.8926 0.7834 0.0539 0.9999 0.6340 0.2305 0.0585 0.0241 0.6420

Experiment ID GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
efficientnet_b0 0.9368 0.0904 0.5288 0.1756 2.9221 0.9776 0.8277 0.7413 0.0864 0.9921 0.8714 0.3251 0.1339 0.0758 0.3009
xception 0.9441 0.0843 0.5329 0.1532 3.3259 0.9852 0.8634 0.8136 0.0689 0.9941 0.7061 0.2077 0.0903 0.0455 0.2372
inception_v3 0.9317 0.0808 0.5263 0.1465 3.4270 0.9887 0.8825 0.8175 0.0593 0.5404 0.3579 0.2038 0.0701 0.0308 0.3405
efficientnet_b4 0.9352 0.0906 0.5437 0.1613 2.9938 0.9777 0.8598 0.7410 0.0737 0.9971 0.9262 0.3178 0.1376 0.0422 0.5591
mobilenetv2_100 0.9412 0.0905 0.5949 0.1517 2.7732 0.9643 0.8145 0.6793 0.0930 0.9980 0.9847 0.5632 0.1602 0.0858 0.6524
densenet121 0.9370 0.0784 0.5972 0.1387 3.0151 0.9789 0.8696 0.7744 0.0652 0.9961 0.8337 0.2676 0.0874 0.0444 0.6985
vit_base_patch16_224 0.8929 0.1004 0.5296 0.1201 3.2821 0.9813 0.8480 0.8365 0.0773 0.5550 0.4242 0.1810 0.1038 0.0632 0.7181
inception_resnet_v2 0.9518 0.0812 0.6074 0.1554 2.7772 0.9660 0.8216 0.6502 0.0901 1.0000 0.9831 0.5887 0.1828 0.0830 0.4802
resnet50 0.9622 0.0667 0.7156 0.1463 2.1683 0.9278 0.7657 0.5405 0.1181 1.0000 1.0000 0.9988 0.4356 0.1451 0.7610

Experiment ID Gender GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
mobilenetv2_100 male 0.9033 0.1187 0.5883 0.1483 2.3448 0.9417 0.7859 0.6601 0.1137 1.0000 0.9681 0.4047 0.1766 0.1194 0.8597
mobilenetv2_100 female 0.9023 0.1146 0.5879 0.1625 2.2356 0.9379 0.7399 0.6214 0.1390 1.0000 0.9677 0.5797 0.2241 0.1661 0.8284
mobilenetv2_100 all 0.8982 0.1219 0.5425 0.1596 2.5049 0.9514 0.7829 0.6077 0.1128 1.0000 0.9048 0.3714 0.1780 0.1209 0.7575
vit_base_patch16_224 male 0.8816 0.1032 0.6175 0.1129 2.4419 0.9411 0.8264 0.7247 0.1001 0.9944 0.5979 0.2927 0.1254 0.1001 0.9423
vit_base_patch16_224 female 0.8318 0.1117 0.5547 0.1258 2.3294 0.9386 0.7491 0.6736 0.1293 0.9565 0.7230 0.3896 0.2207 0.1533 0.8928
vit_base_patch16_224 all 0.8581 0.1113 0.5617 0.1245 2.5095 0.9466 0.7847 0.6733 0.1171 0.9125 0.6008 0.2980 0.1668 0.1265 0.9096
efficientnet_b4 male 0.8838 0.1558 0.5761 0.1547 1.9818 0.9162 0.7576 0.5984 0.1280 0.9993 0.9236 0.4870 0.2222 0.1469 0.9975
efficientnet_b4 female 0.8709 0.1354 0.5582 0.1629 2.0880 0.9259 0.7277 0.5818 0.1455 0.9978 0.9604 0.5914 0.2475 0.1750 0.9676
efficientnet_b4 all 0.8747 0.1516 0.5180 0.1656 2.2470 0.9335 0.7722 0.5628 0.1180 0.9983 0.8645 0.4404 0.1831 0.1325 0.9898
densenet121 male 0.9075 0.1190 0.5744 0.1599 2.3632 0.9413 0.7709 0.6004 0.1161 1.0000 0.9744 0.5579 0.2014 0.1306 0.8241
densenet121 female 0.9194 0.1211 0.6327 0.1648 1.9830 0.9280 0.7451 0.5886 0.1349 0.9955 0.9682 0.6182 0.2330 0.1611 0.9984
densenet121 all 0.9069 0.1249 0.5440 0.1668 2.4628 0.9491 0.7913 0.5645 0.1083 0.9991 0.9515 0.4515 0.1775 0.1123 0.9983
inception_v3 male 0.8648 0.1214 0.5509 0.1138 2.6681 0.9494 0.8086 0.7051 0.1076 0.9989 0.7133 0.3194 0.1421 0.1102 0.7604
inception_v3 female 0.8420 0.1306 0.5501 0.1225 2.3061 0.9294 0.7502 0.6908 0.1355 0.9744 0.6990 0.3679 0.2057 0.1566 0.9818
inception_v3 all 0.8512 0.1280 0.5184 0.1200 2.6821 0.9518 0.7987 0.6786 0.1181 0.9996 0.6441 0.2980 0.1535 0.1235 0.9735
resnet50 male 0.9246 0.1141 0.6757 0.1648 1.7559 0.9001 0.7124 0.4448 0.1459 1.0000 0.9978 0.8846 0.3813 0.1977 0.9434
resnet50 female 0.9081 0.1087 0.5838 0.1727 2.2472 0.9417 0.7389 0.5896 0.1327 0.9994 0.9699 0.5925 0.2570 0.1695 0.7624
resnet50 all 0.9136 0.1156 0.5837 0.1855 2.1344 0.9321 0.7433 0.4394 0.1301 0.9996 0.9953 0.7496 0.2526 0.1578 0.7662
inception_resnet_v2 male 0.9028 0.1216 0.5815 0.1495 2.3580 0.9397 0.7989 0.6567 0.1094 1.0000 0.8858 0.4425 0.1595 0.1142 0.9251
inception_resnet_v2 female 0.9074 0.1121 0.5982 0.1560 2.2764 0.9446 0.7949 0.6664 0.1094 0.9967 0.9365 0.4482 0.1644 0.1148 0.9436
inception_resnet_v2 all 0.9012 0.1203 0.5435 0.1581 2.5463 0.9522 0.8188 0.6307 0.0999 1.0000 0.8551 0.3448 0.1325 0.0999 0.8854
efficientnet_b0 male 0.8901 0.1503 0.5740 0.1547 2.0722 0.9208 0.7730 0.6057 0.1168 1.0000 0.9629 0.5167 0.2055 0.1306 0.9987
efficientnet_b0 female 0.8763 0.1536 0.5579 0.1761 1.9276 0.9182 0.7233 0.5995 0.1466 1.0000 0.9810 0.5658 0.2486 0.1784 1.0000
efficientnet_b0 all 0.8791 0.1571 0.5212 0.1678 2.2022 0.9306 0.7770 0.5664 0.1167 1.0000 0.9241 0.4425 0.1883 0.1265 0.9999
xception male 0.8998 0.1162 0.5673 0.1386 2.5998 0.9524 0.8038 0.6923 0.1043 1.0000 0.8561 0.3457 0.1499 0.1083 0.8151
xception female 0.8900 0.1277 0.5998 0.1503 2.0811 0.9247 0.7379 0.6330 0.1439 0.9961 0.8946 0.4705 0.2408 0.1672 0.9646
xception all 0.8913 0.1264 0.5314 0.1485 2.6096 0.9529 0.7954 0.6417 0.1072 0.9996 0.7766 0.3268 0.1587 0.1106 0.9227

Experiment ID Gender GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
mobilenetv2_100 male 0.8514 0.1122 0.5768 0.1359 2.2029 0.9371 0.7492 0.5003 0.1266 0.9773 0.8977 0.5909 0.2808 0.1591 0.9283
mobilenetv2_100 female 0.8803 0.0823 0.5791 0.1355 2.6871 0.9664 0.8388 0.6290 0.0862 1.0000 0.9949 0.6066 0.1701 0.0635 0.4994
mobilenetv2_100 all 0.8604 0.1065 0.5208 0.1462 2.6547 0.9636 0.8265 0.5480 0.0897 1.0000 0.8898 0.4459 0.1476 0.0797 0.8322
vit_base_patch16_224 male 0.8207 0.0797 0.5776 0.1132 2.4832 0.9594 0.8030 0.6154 0.0990 0.9399 0.7045 0.4318 0.2078 0.0990 0.9780
vit_base_patch16_224 female 0.8188 0.0897 0.5603 0.1120 2.5482 0.9600 0.7877 0.6466 0.1073 0.8020 0.6447 0.4365 0.2259 0.1269 0.4637
vit_base_patch16_224 all 0.8172 0.0912 0.5274 0.1194 2.7279 0.9676 0.8400 0.6202 0.0829 0.9065 0.6181 0.3563 0.1358 0.0620 0.9959
efficientnet_b4 male 0.8872 0.1045 0.5756 0.1372 2.5549 0.9597 0.8289 0.6104 0.0877 0.9935 0.8604 0.4545 0.1591 0.0747 0.9983
efficientnet_b4 female 0.8953 0.0910 0.5517 0.1411 2.8942 0.9770 0.8512 0.6937 0.0767 0.9086 0.6726 0.3959 0.1244 0.0635 0.2713
efficientnet_b4 all 0.8879 0.1041 0.5261 0.1481 2.8270 0.9714 0.8623 0.6117 0.0692 0.9882 0.7884 0.3553 0.1024 0.0551 0.9834
densenet121 male 0.8943 0.0997 0.6036 0.1416 2.3737 0.9555 0.8228 0.5678 0.0941 0.9984 0.8734 0.5390 0.1883 0.0877 0.9876
densenet121 female 0.9186 0.0811 0.5792 0.1478 2.8473 0.9737 0.8298 0.6671 0.0913 0.9949 0.8173 0.4365 0.1523 0.0812 0.2981
densenet121 all 0.9006 0.1025 0.5412 0.1519 2.7744 0.9706 0.8625 0.6009 0.0699 0.9990 0.8041 0.3898 0.1024 0.0492 0.9866
inception_v3 male 0.8463 0.0930 0.5691 0.1147 2.6551 0.9629 0.8280 0.6395 0.0880 0.9919 0.8003 0.3994 0.1477 0.0812 0.8243
inception_v3 female 0.8644 0.0893 0.5630 0.1115 2.9841 0.9753 0.8347 0.6902 0.0837 0.8934 0.7259 0.3807 0.1624 0.0685 0.3241
inception_v3 all 0.8513 0.0958 0.5374 0.1135 2.9882 0.9738 0.8558 0.6564 0.0738 0.9921 0.6663 0.3081 0.1043 0.0522 0.7719
resnet50 male 0.9018 0.0807 0.6396 0.1381 2.3180 0.9478 0.7868 0.4737 0.1136 0.9968 0.9708 0.7078 0.2987 0.1364 0.8464
resnet50 female 0.9297 0.0695 0.6588 0.1516 2.2974 0.9554 0.7771 0.5513 0.1200 0.9898 0.9315 0.6523 0.2690 0.1396 0.3325
resnet50 all 0.9094 0.0883 0.5755 0.1581 2.6074 0.9653 0.8465 0.4918 0.0838 0.9961 0.9380 0.5699 0.1860 0.0571 0.9779
inception_resnet_v2 male 0.9187 0.0846 0.6257 0.1326 2.6344 0.9728 0.8595 0.6894 0.0722 0.9854 0.7451 0.3328 0.0990 0.0519 0.9961
inception_resnet_v2 female 0.9360 0.0577 0.6383 0.1343 2.8802 0.9853 0.8949 0.7232 0.0583 0.7893 0.6827 0.3274 0.0685 0.0152 0.2547
inception_resnet_v2 all 0.9228 0.0831 0.5793 0.1461 2.8908 0.9795 0.8964 0.6780 0.0531 0.9833 0.7106 0.2539 0.0561 0.0276 0.9872
efficientnet_b0 male 0.8707 0.1054 0.5792 0.1451 2.2984 0.9469 0.7846 0.5330 0.1104 1.0000 0.9448 0.6071 0.2338 0.1250 0.9972
efficientnet_b0 female 0.8933 0.0907 0.5803 0.1417 2.6309 0.9628 0.8255 0.6276 0.0894 0.9975 0.9924 0.5761 0.1853 0.0812 0.4653
efficientnet_b0 all 0.8781 0.1021 0.5175 0.1523 2.7814 0.9691 0.8434 0.5635 0.0817 0.9990 0.9232 0.4193 0.1270 0.0620 0.9854
xception male 0.9040 0.0785 0.6062 0.1250 2.8541 0.9733 0.8664 0.6735 0.0672 0.9870 0.8019 0.3555 0.0990 0.0406 0.8377
xception female 0.9336 0.0658 0.6408 0.1336 2.7805 0.9784 0.8613 0.6809 0.0761 0.9746 0.8959 0.4594 0.1168 0.0406 0.2267
xception all 0.9129 0.0825 0.5647 0.1381 3.0623 0.9800 0.8947 0.6704 0.0551 0.9902 0.7579 0.2628 0.0581 0.0217 0.7985

Across Model - NIR

Experiment ID Gender GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
efficientnet_b0 male 0.8993 0.0842 0.5490 0.1456 2.9459 0.9809 0.8741 0.7988 0.0635 0.9998 0.8866 0.3483 0.0834 0.0364 0.8669
efficientnet_b0 female 0.9056 0.0751 0.5735 0.1492 2.8128 0.9779 0.8567 0.7945 0.0735 0.9961 0.8165 0.4178 0.1213 0.0444 0.8595
efficientnet_b0 all 0.8993 0.0821 0.5356 0.1420 3.1367 0.9841 0.8864 0.7767 0.0573 1.0000 0.8344 0.3169 0.0681 0.0268 0.8622
xception male 0.9266 0.0761 0.5851 0.1496 2.8776 0.9822 0.8780 0.7904 0.0621 0.9987 0.8112 0.3670 0.0824 0.0304 0.8504
xception female 0.9289 0.0727 0.5596 0.1669 2.8684 0.9804 0.8573 0.8119 0.0720 0.9999 0.7715 0.3386 0.1055 0.0468 0.5676
xception all 0.9266 0.0746 0.5448 0.1603 3.0535 0.9860 0.8895 0.7842 0.0560 0.9992 0.7492 0.2902 0.0652 0.0240 0.7631
inception_v3 male 0.8861 0.0809 0.5612 0.1257 3.0732 0.9830 0.8753 0.8209 0.0625 1.0000 0.6924 0.2480 0.0770 0.0377 0.9405
inception_v3 female 0.8905 0.0723 0.5651 0.1285 3.1210 0.9799 0.8783 0.8152 0.0630 1.0000 0.9555 0.4776 0.0901 0.0275 0.7817
inception_v3 all 0.8845 0.0789 0.5459 0.1280 3.1841 0.9846 0.8826 0.7874 0.0590 1.0000 0.7688 0.2756 0.0711 0.0304 0.9302
efficientnet_b4 male 0.8827 0.0947 0.5417 0.1357 2.9145 0.9785 0.8556 0.7947 0.0723 1.0000 0.6923 0.2955 0.1030 0.0507 0.9791
efficientnet_b4 female 0.9020 0.0797 0.5735 0.1364 2.9413 0.9804 0.8574 0.8079 0.0723 0.9813 0.7014 0.3392 0.1098 0.0457 0.7346
efficientnet_b4 all 0.8845 0.0932 0.5307 0.1388 2.9926 0.9809 0.8648 0.7775 0.0677 1.0000 0.6339 0.2770 0.0914 0.0441 0.9773
mobilenetv2_100 male 0.9152 0.0781 0.6038 0.1456 2.6654 0.9712 0.8379 0.7323 0.0822 1.0000 0.8846 0.5059 0.1495 0.0629 0.8131
mobilenetv2_100 female 0.9061 0.0717 0.5608 0.1530 2.8911 0.9785 0.8671 0.8029 0.0688 0.9999 0.8972 0.4724 0.1081 0.0361 0.7907
mobilenetv2_100 all 0.9107 0.0774 0.5601 0.1465 2.9922 0.9809 0.8710 0.7454 0.0654 1.0000 0.8380 0.3945 0.0911 0.0355 0.7550
densenet121 male 0.9114 0.0825 0.5876 0.1455 2.7378 0.9706 0.8353 0.7427 0.0827 1.0000 0.9077 0.4637 0.1425 0.0664 0.8814
densenet121 female 0.9320 0.0666 0.5886 0.1535 2.9030 0.9818 0.8760 0.8151 0.0643 0.9962 0.8506 0.4042 0.0929 0.0284 0.8407
densenet121 all 0.9136 0.0804 0.5711 0.1452 2.9188 0.9771 0.8571 0.7317 0.0719 1.0000 0.8705 0.4038 0.1104 0.0472 0.8796
vit_base_patch16_224 male 0.8640 0.0664 0.5637 0.1226 3.0461 0.9849 0.8791 0.8390 0.0605 0.8728 0.4539 0.2015 0.0715 0.0375 0.8241
vit_base_patch16_224 female 0.8643 0.0637 0.5424 0.1317 3.1130 0.9840 0.8711 0.8256 0.0653 0.9662 0.5880 0.2861 0.0894 0.0348 0.6545
vit_base_patch16_224 all 0.8612 0.0662 0.5481 0.1255 3.1208 0.9856 0.8811 0.8050 0.0595 0.9847 0.4966 0.2269 0.0709 0.0328 0.7415
inception_resnet_v2 male 0.9381 0.0658 0.6289 0.1506 2.6602 0.9786 0.8585 0.7772 0.0710 1.0000 0.8843 0.3887 0.1037 0.0460 0.7505
inception_resnet_v2 female 0.9348 0.0706 0.5826 0.1735 2.6585 0.9702 0.8302 0.7474 0.0891 0.9997 0.8771 0.5419 0.1916 0.0728 0.7969
inception_resnet_v2 all 0.9364 0.0679 0.5946 0.1579 2.8123 0.9807 0.8709 0.7467 0.0656 1.0000 0.8781 0.3887 0.0919 0.0342 0.7942
resnet50 male 0.9563 0.0681 0.6811 0.1476 2.3940 0.9654 0.8174 0.7150 0.0919 0.9992 0.9630 0.5721 0.1748 0.0828 0.8207
resnet50 female 0.9403 0.0659 0.6471 0.1498 2.5341 0.9471 0.8060 0.6879 0.1091 1.0000 0.9945 0.8846 0.3906 0.1330 0.9948
resnet50 all 0.9505 0.0690 0.6468 0.1450 2.6737 0.9696 0.8359 0.6752 0.0842 0.9999 0.9669 0.5935 0.1632 0.0644 0.9910

Experiment ID Gender GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
efficientnet_b0 male 0.9396 0.0670 0.5846 0.1345 3.3397 0.9913 0.9188 0.9001 0.0407 0.9994 0.5860 0.1439 0.0324 0.0144 0.6833
efficientnet_b0 female 0.9278 0.0784 0.5392 0.1340 3.5387 0.9897 0.9166 0.9028 0.0417 0.9872 0.6104 0.1438 0.0356 0.0207 0.6580
efficientnet_b0 all 0.9349 0.0719 0.5464 0.1453 3.3891 0.9904 0.9166 0.8962 0.0418 0.9997 0.6302 0.1658 0.0351 0.0168 0.6667
xception male 0.9336 0.0670 0.5901 0.1216 3.4989 0.9932 0.9236 0.9052 0.0383 0.9854 0.4809 0.1298 0.0284 0.0117 0.6040
xception female 0.9238 0.0651 0.5551 0.1151 3.9444 0.9964 0.9436 0.9380 0.0286 0.7788 0.1966 0.0602 0.0164 0.0078 0.6304
xception all 0.9292 0.0670 0.5647 0.1293 3.5405 0.9929 0.9271 0.9049 0.0367 0.9995 0.6875 0.1357 0.0254 0.0101 0.6551
inception_v3 male 0.9149 0.0757 0.5607 0.1125 3.6924 0.9915 0.9154 0.8911 0.0426 0.9964 0.5741 0.1698 0.0349 0.0137 0.6563
inception_v3 female 0.9068 0.0787 0.5414 0.1155 3.6968 0.9930 0.9199 0.9100 0.0404 0.7698 0.3123 0.1087 0.0334 0.0165 0.6720
inception_v3 all 0.9140 0.0775 0.5464 0.1168 3.7099 0.9927 0.9198 0.9007 0.0402 0.9156 0.4703 0.1360 0.0312 0.0139 0.6877
efficientnet_b4 male 0.9353 0.0706 0.5409 0.1420 3.5179 0.9926 0.9232 0.9067 0.0386 0.9955 0.5642 0.1236 0.0307 0.0144 0.5386
efficientnet_b4 female 0.9213 0.0840 0.5544 0.1442 3.1100 0.9830 0.8899 0.8715 0.0557 0.9919 0.6592 0.2298 0.0604 0.0360 0.7941
efficientnet_b4 all 0.9300 0.0761 0.5288 0.1445 3.4753 0.9909 0.9156 0.8986 0.0424 0.9972 0.5226 0.1428 0.0365 0.0171 0.7453
mobilenetv2_100 male 0.9348 0.0685 0.5751 0.1298 3.4656 0.9893 0.9114 0.8865 0.0444 0.9972 0.7619 0.2084 0.0391 0.0177 0.5296
mobilenetv2_100 female 0.9281 0.0684 0.5570 0.1300 3.5715 0.9927 0.9175 0.9039 0.0413 0.9927 0.4069 0.1331 0.0349 0.0144 0.5670
mobilenetv2_100 all 0.9304 0.0702 0.5608 0.1418 3.3038 0.9866 0.9091 0.8824 0.0455 1.0000 0.9860 0.3107 0.0412 0.0183 0.5766
densenet121 male 0.9493 0.0544 0.6073 0.1183 3.7150 0.9947 0.9305 0.9178 0.0349 0.9873 0.3724 0.0983 0.0238 0.0091 0.6036
densenet121 female 0.9321 0.0652 0.5426 0.1249 3.9092 0.9951 0.9380 0.9225 0.0311 0.8179 0.3640 0.0956 0.0184 0.0070 0.4765
densenet121 all 0.9433 0.0587 0.5630 0.1391 3.5626 0.9948 0.9308 0.9175 0.0346 0.9965 0.3238 0.0955 0.0232 0.0088 0.5444
vit_base_patch16_224 male 0.9107 0.0539 0.5669 0.1156 3.8116 0.9961 0.9423 0.9362 0.0293 0.6581 0.2140 0.0619 0.0191 0.0092 0.4798
vit_base_patch16_224 female 0.8945 0.0648 0.5423 0.1189 3.6781 0.9940 0.9308 0.9255 0.0360 0.6101 0.2370 0.0791 0.0297 0.0178 0.5852
vit_base_patch16_224 all 0.9062 0.0585 0.5376 0.1116 4.1382 0.9967 0.9487 0.9477 0.0267 0.5631 0.1583 0.0458 0.0176 0.0087 0.5590
inception_resnet_v2 male 0.9452 0.0675 0.6112 0.1489 2.8896 0.9822 0.8731 0.8448 0.0636 1.0000 0.8816 0.3081 0.0807 0.0390 0.6571
inception_resnet_v2 female 0.9328 0.0706 0.5576 0.1422 3.3412 0.9892 0.9023 0.8795 0.0490 0.9867 0.7116 0.1955 0.0479 0.0202 0.6399
inception_resnet_v2 all 0.9408 0.0694 0.5779 0.1571 2.9884 0.9842 0.8812 0.8463 0.0599 1.0000 0.8137 0.2923 0.0740 0.0304 0.6744
resnet50 male 0.9514 0.0561 0.6927 0.1045 3.0841 0.9823 0.8785 0.8382 0.0620 0.9981 0.8578 0.3722 0.0810 0.0307 0.6787
resnet50 female 0.9370 0.0596 0.6374 0.1156 3.2577 0.9887 0.8992 0.8766 0.0505 0.9792 0.6229 0.2068 0.0510 0.0223 0.6308
resnet50 all 0.9474 0.0579 0.6539 0.1275 2.9648 0.9848 0.8834 0.8488 0.0587 0.9985 0.7822 0.2983 0.0716 0.0289 0.6814

Experiment ID Gender GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
efficientnet_b0 male 0.9375 0.0710 0.5853 0.1311 3.3394 0.9888 0.8994 0.8790 0.0504 0.9945 0.7487 0.1906 0.0508 0.0235 0.4917
efficientnet_b0 female 0.9244 0.0748 0.5446 0.1348 3.4837 0.9924 0.9120 0.8906 0.0441 0.7683 0.4183 0.1681 0.0383 0.0136 0.3556
efficientnet_b0 all 0.9369 0.0693 0.5511 0.1452 3.3911 0.9915 0.9137 0.8905 0.0432 0.9888 0.6436 0.1761 0.0371 0.0135 0.4066
xception male 0.9577 0.0604 0.6738 0.1177 3.0343 0.9813 0.8812 0.8318 0.0616 0.9979 0.9648 0.3944 0.0864 0.0257 0.6970
xception female 0.9325 0.0664 0.5625 0.1299 3.5867 0.9912 0.9068 0.8790 0.0474 0.9909 0.6760 0.1967 0.0447 0.0093 0.2915
xception all 0.9514 0.0631 0.5766 0.1542 3.1826 0.9857 0.9020 0.8570 0.0509 0.9982 0.9540 0.3573 0.0525 0.0181 0.4111
inception_v3 male 0.9329 0.0639 0.6066 0.1128 3.5600 0.9922 0.9133 0.8884 0.0441 0.9773 0.4669 0.1663 0.0370 0.0123 0.4276
inception_v3 female 0.9170 0.0746 0.5604 0.1170 3.6345 0.9925 0.9144 0.9012 0.0430 0.9891 0.4411 0.1341 0.0366 0.0153 0.4412
inception_v3 all 0.9324 0.0652 0.5712 0.1332 3.4444 0.9933 0.9221 0.8992 0.0393 0.9817 0.4125 0.1408 0.0293 0.0110 0.4939
efficientnet_b4 male 0.9412 0.0619 0.5747 0.1365 3.4591 0.9950 0.9283 0.9094 0.0363 0.8967 0.3607 0.1146 0.0232 0.0059 0.2875
efficientnet_b4 female 0.9342 0.0638 0.5597 0.1480 3.2857 0.9911 0.9122 0.8895 0.0442 0.9798 0.6144 0.2002 0.0381 0.0143 0.3522
efficientnet_b4 all 0.9400 0.0626 0.5579 0.1414 3.4936 0.9944 0.9314 0.9103 0.0346 0.9740 0.5241 0.1226 0.0214 0.0067 0.4079
mobilenetv2_100 male 0.9383 0.0691 0.5978 0.1389 3.1041 0.9844 0.8827 0.8505 0.0593 0.9998 0.8915 0.3144 0.0720 0.0284 0.4805
mobilenetv2_100 female 0.9311 0.0692 0.5500 0.1659 2.9987 0.9883 0.8959 0.8641 0.0528 0.9858 0.6271 0.2650 0.0569 0.0185 0.4576
mobilenetv2_100 all 0.9391 0.0678 0.5612 0.1530 3.1932 0.9874 0.9026 0.8650 0.0496 0.9998 0.8541 0.3021 0.0490 0.0165 0.4945
densenet121 male 0.9546 0.0550 0.6085 0.1585 2.9180 0.9857 0.8932 0.8516 0.0551 0.9999 0.9367 0.3283 0.0640 0.0180 0.4042
densenet121 female 0.9436 0.0596 0.5832 0.1425 3.3000 0.9917 0.9218 0.8940 0.0403 0.9991 0.8240 0.2029 0.0286 0.0060 0.2703
densenet121 all 0.9523 0.0564 0.5715 0.1562 3.2439 0.9890 0.9163 0.8757 0.0438 0.9999 0.9156 0.2925 0.0342 0.0080 0.3706
vit_base_patch16_224 male 0.8946 0.0615 0.5662 0.1009 3.9293 0.9964 0.9448 0.9342 0.0278 0.7153 0.3842 0.0676 0.0146 0.0040 0.2626
vit_base_patch16_224 female 0.8727 0.0699 0.5572 0.1251 3.1140 0.9860 0.8729 0.8348 0.0653 0.7356 0.4686 0.2491 0.0931 0.0304 0.4172
vit_base_patch16_224 all 0.8889 0.0642 0.5367 0.1158 3.7621 0.9964 0.9420 0.9295 0.0291 0.7441 0.3658 0.0731 0.0145 0.0042 0.3635
inception_resnet_v2 male 0.9666 0.0495 0.6482 0.1350 3.1314 0.9910 0.9133 0.8872 0.0439 0.9998 0.6880 0.1787 0.0370 0.0125 0.4492
inception_resnet_v2 female 0.9605 0.0517 0.6908 0.1350 2.6389 0.9729 0.8589 0.8199 0.0707 0.9961 0.9596 0.6975 0.1067 0.0475 0.4423
inception_resnet_v2 all 0.9671 0.0481 0.6391 0.1351 3.2347 0.9916 0.9213 0.8996 0.0395 0.9999 0.7499 0.1748 0.0305 0.0116 0.4738
resnet50 male 0.9536 0.0627 0.6739 0.1209 2.9046 0.9792 0.8875 0.8533 0.0565 0.9995 0.8097 0.2915 0.0660 0.0303 0.8335
resnet50 female 0.9353 0.0662 0.6275 0.1367 2.8657 0.9794 0.8489 0.8118 0.0766 0.9976 0.7809 0.3581 0.1200 0.0519 0.4722
resnet50 all 0.9485 0.0630 0.6079 0.1378 3.1794 0.9866 0.8959 0.8675 0.0523 0.9895 0.6455 0.2323 0.0547 0.0252 0.6208

Experiment ID GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
efficientnet_b0 0.9238 0.1068 0.5277 0.1692 2.7998 0.9685 0.8269 0.7054 0.0872 1.0000 0.9719 0.4183 0.1474 0.0773 0.5724
xception 0.9247 0.1189 0.5631 0.1530 2.6396 0.9551 0.8207 0.7495 0.0956 0.9996 0.9792 0.3147 0.1345 0.0929 0.6104
inception_v3 0.9206 0.0984 0.5629 0.1223 3.2233 0.9814 0.8523 0.7833 0.0741 0.8950 0.5232 0.2558 0.1021 0.0563 0.4690
efficientnet_b4 0.9253 0.0953 0.5207 0.1581 3.1007 0.9792 0.8539 0.7612 0.0734 1.0000 0.9994 0.3053 0.1056 0.0540 0.5379
mobilenetv2_100 0.9400 0.0809 0.5792 0.1566 2.8942 0.9712 0.8486 0.6969 0.0767 1.0000 1.0000 0.6315 0.1286 0.0552 0.6422
densenet121 0.9532 0.0710 0.6160 0.1582 2.7512 0.9675 0.8375 0.6724 0.0814 1.0000 0.9941 0.7815 0.1400 0.0681 0.5239
vit_base_patch16_224 0.9031 0.0959 0.5237 0.1349 3.2421 0.9803 0.8372 0.8272 0.0852 0.5724 0.3259 0.1991 0.1152 0.0750 0.5226
inception_resnet_v2 0.9472 0.0848 0.6060 0.1466 2.8485 0.9720 0.8334 0.7429 0.0840 1.0000 1.0000 0.4199 0.1268 0.0707 0.5870
resnet50 0.9820 0.0652 0.8233 0.1088 1.7702 0.8986 0.6765 0.4372 0.1658 1.0000 0.9906 0.9480 0.5989 0.2984 0.7804

Experiment ID GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
efficientnet_b0 0.9334 0.0928 0.5199 0.1561 3.2198 0.9821 0.9030 0.8543 0.0525 0.9899 0.4977 0.1447 0.0534 0.0390 0.8075
xception 0.9581 0.0618 0.5784 0.1670 3.0155 0.9837 0.8886 0.7722 0.0571 0.9999 0.9823 0.3451 0.0707 0.0236 0.3871
inception_v3 0.9136 0.0845 0.5341 0.1203 3.6509 0.9910 0.9060 0.8312 0.0474 0.9769 0.4259 0.1824 0.0445 0.0172 0.6657
efficientnet_b4 0.9360 0.0777 0.5382 0.1486 3.3553 0.9888 0.8968 0.8404 0.0523 0.9837 0.4919 0.1671 0.0544 0.0338 0.4141
mobilenetv2_100 0.9360 0.0743 0.5412 0.1480 3.3722 0.9886 0.9044 0.8373 0.0481 0.9997 0.9412 0.1762 0.0470 0.0216 0.4964
densenet121 0.9581 0.0685 0.5693 0.1717 2.9750 0.9820 0.8900 0.7893 0.0556 1.0000 0.9764 0.3225 0.0604 0.0319 0.6326
vit_base_patch16_224 0.9139 0.0721 0.6071 0.1221 3.0600 0.9824 0.8536 0.8150 0.0735 0.6592 0.3752 0.2124 0.0993 0.0535 0.5855
inception_resnet_v2 0.9558 0.0625 0.5773 0.1480 3.3323 0.9917 0.9089 0.8587 0.0460 0.9978 0.4553 0.1382 0.0414 0.0163 0.4919
resnet50 0.9693 0.0512 0.5987 0.1606 3.1103 0.9899 0.9247 0.8190 0.0390 1.0000 0.9663 0.2481 0.0262 0.0090 0.2728

Experiment ID GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
efficientnet_b0 0.9148 0.0898 0.5102 0.1472 3.3185 0.9880 0.8889 0.7721 0.0558 0.9990 0.4908 0.2075 0.0626 0.0266 0.4694
xception 0.9375 0.0679 0.5209 0.1455 3.6693 0.9952 0.9328 0.8351 0.0344 0.9964 0.3650 0.1108 0.0201 0.0051 0.2302
inception_v3 0.8985 0.0884 0.5277 0.1218 3.4855 0.9893 0.8943 0.7975 0.0532 0.9832 0.3964 0.1754 0.0572 0.0259 0.4281
efficientnet_b4 0.9187 0.0899 0.5125 0.1571 3.1734 0.9845 0.8858 0.7235 0.0574 1.0000 0.7410 0.2658 0.0696 0.0299 0.6515
mobilenetv2_100 0.9132 0.0880 0.5313 0.1480 3.1375 0.9837 0.8719 0.7204 0.0642 1.0000 0.6052 0.2754 0.0867 0.0352 0.5601
densenet121 0.9372 0.0752 0.5262 0.1580 3.3215 0.9901 0.8994 0.7729 0.0507 0.9993 0.5829 0.1968 0.0513 0.0162 0.3135
vit_base_patch16_224 0.8500 0.1084 0.5345 0.1187 2.7756 0.9691 0.8009 0.7245 0.0998 0.7566 0.4720 0.3023 0.1616 0.0995 0.8664
inception_resnet_v2 0.9462 0.0581 0.5783 0.1346 3.5502 0.9944 0.9353 0.8162 0.0340 0.9972 0.4777 0.1294 0.0178 0.0050 0.4693
resnet50 0.9486 0.0760 0.6067 0.1610 2.7168 0.9715 0.8403 0.5957 0.0818 1.0000 0.9860 0.5494 0.1447 0.0615 0.4826

Across DataBalance - VIS

Experiment ID Gender Data Balance (% Females) GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
inception_resnet_v2 male 0.25 0.8988 0.0788 0.6012 0.1125 3.0639 0.9822 0.8643 0.7566 0.0680 0.9453 0.5751 0.2768 0.0938 0.0423 0.8464
inception_resnet_v2 female 0.25 0.9135 0.0750 0.5986 0.1163 3.2174 0.9861 0.8812 0.7758 0.0595 0.9983 0.5285 0.2398 0.0713 0.0317 0.5465
inception_resnet_v2 all 0.25 0.9057 0.0777 0.5698 0.1213 3.2980 0.9879 0.8919 0.7843 0.0548 0.9992 0.5373 0.2284 0.0600 0.0243 0.7508
inception_resnet_v2 male 0.75 0.9105 0.0758 0.6474 0.1196 2.6279 0.9679 0.8085 0.6364 0.0965 0.9998 0.8945 0.4762 0.1902 0.0931 0.6690
inception_resnet_v2 female 0.75 0.9075 0.0804 0.5760 0.1206 3.2346 0.9855 0.8762 0.7835 0.0624 0.9965 0.5619 0.2289 0.0760 0.0362 0.6984
inception_resnet_v2 all 0.75 0.9088 0.0785 0.5780 0.1299 3.0830 0.9833 0.8707 0.7378 0.0658 0.9999 0.7126 0.3130 0.0929 0.0347 0.6949
xception male 0.25 0.8967 0.0769 0.5643 0.1103 3.4962 0.9908 0.9067 0.8211 0.0470 0.9977 0.4512 0.1743 0.0435 0.0198 0.4963
xception female 0.25 0.9049 0.0755 0.5596 0.1151 3.5465 0.9917 0.9068 0.8441 0.0475 0.9419 0.3652 0.1396 0.0449 0.0185 0.5316
xception all 0.25 0.9002 0.0771 0.5359 0.1181 3.6540 0.9930 0.9168 0.8452 0.0417 0.9971 0.3976 0.1368 0.0348 0.0141 0.4475
densenet121 male 0.25 0.8785 0.0901 0.5621 0.1222 2.9475 0.9780 0.8442 0.7301 0.0788 0.9999 0.6003 0.3172 0.1218 0.0570 0.6785
densenet121 female 0.25 0.9058 0.0853 0.6191 0.1171 2.7978 0.9688 0.8123 0.6893 0.0946 0.9919 0.7220 0.3707 0.1661 0.0885 0.7503
densenet121 all 0.25 0.8911 0.0892 0.5530 0.1284 3.0581 0.9808 0.8550 0.7383 0.0726 0.9992 0.6120 0.2992 0.1064 0.0474 0.6493
xception male 1.0 0.8806 0.0847 0.6228 0.1105 2.6195 0.9652 0.8015 0.6700 0.0993 0.9994 0.7145 0.4080 0.1780 0.0990 0.9812
xception female 1.0 0.8775 0.0883 0.5329 0.1198 3.2735 0.9863 0.8801 0.7755 0.0601 0.9628 0.5804 0.2425 0.0736 0.0306 0.4711
xception all 1.0 0.8786 0.0876 0.5641 0.1200 2.9936 0.9798 0.8508 0.7470 0.0748 0.9464 0.5798 0.2862 0.1089 0.0540 0.9586
xception male 0.75 0.9257 0.0625 0.6843 0.1022 2.8487 0.9769 0.8451 0.7061 0.0777 1.0000 0.7800 0.3792 0.1213 0.0584 0.5581
xception female 0.75 0.9010 0.0810 0.5633 0.1141 3.4117 0.9894 0.8976 0.8374 0.0515 0.9837 0.3795 0.1526 0.0528 0.0243 0.8170
xception all 0.75 0.9146 0.0726 0.6049 0.1250 3.0298 0.9815 0.8641 0.7398 0.0681 0.9991 0.6899 0.3095 0.0933 0.0433 0.8812
densenet121 male 0.0 0.8767 0.0921 0.5304 0.1192 3.2517 0.9848 0.8718 0.7888 0.0647 0.9375 0.4914 0.2287 0.0810 0.0394 0.6005
densenet121 female 0.0 0.8894 0.0908 0.6057 0.1183 2.6908 0.9646 0.8040 0.7105 0.0996 0.9980 0.6076 0.3408 0.1603 0.0993 0.8323
densenet121 all 0.0 0.8819 0.0924 0.5504 0.1254 3.0102 0.9781 0.8414 0.7531 0.0799 0.9980 0.5494 0.2768 0.1147 0.0653 0.6747
inception_resnet_v2 male 1.0 0.8824 0.0897 0.6461 0.1074 2.3892 0.9494 0.7468 0.6121 0.1268 1.0000 0.8166 0.4900 0.2545 0.1573 0.9010
inception_resnet_v2 female 1.0 0.8982 0.0845 0.5840 0.1236 2.9682 0.9798 0.8553 0.7290 0.0727 0.9979 0.6897 0.3185 0.1093 0.0493 0.7088
inception_resnet_v2 all 1.0 0.8902 0.0879 0.6050 0.1165 2.7632 0.9714 0.8182 0.6932 0.0912 0.9979 0.6886 0.3682 0.1583 0.0828 0.8945
xception male 0.0 0.8683 0.0962 0.5691 0.1098 2.8983 0.9747 0.8406 0.7254 0.0800 0.9989 0.6556 0.3283 0.1205 0.0634 0.9303
xception female 0.0 0.8900 0.0847 0.6228 0.1080 2.7537 0.9690 0.8174 0.6905 0.0923 0.9958 0.7297 0.3759 0.1595 0.0836 0.7852
xception all 0.0 0.8779 0.0931 0.5730 0.1155 2.9071 0.9742 0.8371 0.7160 0.0817 0.9982 0.6377 0.3335 0.1309 0.0658 0.9275
inception_resnet_v2 male 0.0 0.8914 0.0833 0.6042 0.1093 2.9550 0.9783 0.8499 0.7217 0.0754 0.9930 0.7326 0.3441 0.1123 0.0526 0.6179
inception_resnet_v2 female 0.0 0.8984 0.0866 0.6259 0.1286 2.4865 0.9603 0.7903 0.6456 0.1050 1.0000 0.8795 0.4523 0.1945 0.1100 0.8264
inception_resnet_v2 all 0.0 0.8950 0.0849 0.6025 0.1200 2.8135 0.9735 0.8323 0.6933 0.0842 1.0000 0.7944 0.3778 0.1383 0.0693 0.8017
densenet121 male 0.75 0.9105 0.0734 0.6440 0.1026 2.9880 0.9788 0.8570 0.7365 0.0719 0.9974 0.6734 0.3150 0.1041 0.0483 0.8452
densenet121 female 0.75 0.8960 0.0847 0.5509 0.1264 3.2078 0.9844 0.8690 0.7926 0.0662 0.8997 0.4427 0.2191 0.0841 0.0442 0.7249
densenet121 all 0.75 0.9035 0.0798 0.5489 0.1377 3.1509 0.9860 0.8822 0.7870 0.0589 0.9998 0.5544 0.2249 0.0702 0.0338 0.7151
densenet121 male 1.0 0.8766 0.0919 0.6219 0.1171 2.4190 0.9508 0.7545 0.6119 0.1230 0.9998 0.8159 0.5020 0.2421 0.1506 0.8080
densenet121 female 1.0 0.8900 0.0870 0.5587 0.1308 2.9835 0.9789 0.8481 0.7299 0.0763 0.9998 0.6217 0.3108 0.1194 0.0550 0.4865
densenet121 all 1.0 0.8827 0.0908 0.5693 0.1278 2.8279 0.9728 0.8244 0.7040 0.0880 1.0000 0.7349 0.3546 0.1474 0.0769 0.6457

Experiment ID Gender Data Balance (% Females) GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
densenet121 male 0.25 0.9291 0.0851 0.5741 0.1256 3.3096 0.9830 0.8975 0.7964 0.0524 0.9999 0.5484 0.1582 0.0535 0.0330 0.9915
densenet121 female 0.25 0.9310 0.0897 0.6378 0.1320 2.5984 0.9618 0.8195 0.6746 0.0911 0.9998 0.9290 0.3967 0.1470 0.0841 0.9942
densenet121 all 0.25 0.9385 0.0849 0.5785 0.1304 3.2724 0.9819 0.8911 0.7691 0.0551 0.9997 0.5548 0.1608 0.0591 0.0353 0.9850
xception male 1.0 0.9187 0.0950 0.6108 0.1241 2.7867 0.9695 0.8358 0.6622 0.0825 1.0000 0.8480 0.3474 0.1264 0.0680 0.9991
xception female 1.0 0.9062 0.1063 0.5745 0.1283 2.8150 0.9650 0.8355 0.7070 0.0828 0.9995 0.7889 0.3342 0.1238 0.0704 0.9964
xception all 1.0 0.9227 0.1034 0.5785 0.1310 2.9171 0.9695 0.8532 0.6955 0.0745 0.9996 0.7148 0.2521 0.0976 0.0608 0.9963
densenet121 male 0.75 0.9348 0.0789 0.5893 0.1346 3.1322 0.9816 0.8803 0.7227 0.0601 1.0000 0.9445 0.2560 0.0721 0.0346 0.9833
densenet121 female 0.75 0.9261 0.1116 0.6116 0.1309 2.5851 0.9564 0.8486 0.7259 0.0775 0.9987 0.8702 0.3142 0.1040 0.0652 0.9994
densenet121 all 0.75 0.9363 0.1026 0.5761 0.1362 2.9863 0.9711 0.8892 0.7449 0.0575 1.0000 0.7865 0.1831 0.0623 0.0426 0.9973
densenet121 male 1.0 0.9190 0.0842 0.6071 0.1277 2.8847 0.9730 0.8424 0.6919 0.0790 1.0000 0.6950 0.3034 0.1199 0.0609 0.9942
densenet121 female 1.0 0.9003 0.1102 0.5718 0.1310 2.7144 0.9607 0.8319 0.7400 0.0866 0.9951 0.5888 0.2811 0.1222 0.0779 0.9934
densenet121 all 1.0 0.9197 0.1005 0.5797 0.1313 2.9078 0.9684 0.8596 0.7199 0.0721 1.0000 0.6059 0.2309 0.0907 0.0583 0.9965
mobilenetv2_100 male 0.0 0.9082 0.0954 0.5489 0.1344 3.0826 0.9796 0.8681 0.7269 0.0662 0.9998 0.7040 0.2532 0.0878 0.0414 0.9725
mobilenetv2_100 female 0.0 0.8975 0.0996 0.6050 0.1300 2.5267 0.9569 0.7892 0.6411 0.1067 0.9996 0.8747 0.4342 0.1907 0.1121 0.9772
mobilenetv2_100 all 0.0 0.9142 0.0961 0.5625 0.1326 3.0373 0.9767 0.8548 0.7039 0.0737 1.0000 0.6940 0.2467 0.0973 0.0568 0.9785
densenet121 male 0.0 0.9293 0.0813 0.5851 0.1315 3.1475 0.9813 0.8807 0.7518 0.0601 0.9994 0.6249 0.2180 0.0702 0.0373 0.9211
densenet121 female 0.0 0.9273 0.0831 0.6812 0.1145 2.4602 0.9532 0.7904 0.6416 0.1053 0.9998 0.9101 0.4330 0.1877 0.1101 0.9999
densenet121 all 0.0 0.9358 0.0833 0.6156 0.1304 2.9270 0.9743 0.8571 0.7013 0.0736 0.9998 0.7352 0.2449 0.0935 0.0616 0.9956
mobilenetv2_100 male 1.0 0.9097 0.0993 0.6126 0.1377 2.4750 0.9551 0.7893 0.5764 0.1056 0.9998 0.9424 0.4839 0.1897 0.1111 0.9953
mobilenetv2_100 female 1.0 0.8945 0.1164 0.5293 0.1432 2.7993 0.9657 0.8324 0.7020 0.0842 0.9991 0.7694 0.3379 0.1303 0.0731 0.9357
mobilenetv2_100 all 1.0 0.9115 0.1097 0.5557 0.1497 2.7112 0.9635 0.8258 0.6281 0.0891 1.0000 0.8610 0.3450 0.1320 0.0820 0.9785
mobilenetv2_100 male 0.25 0.9156 0.0913 0.5379 0.1346 3.2838 0.9840 0.8984 0.7709 0.0509 0.9998 0.6540 0.1864 0.0522 0.0260 0.9958
mobilenetv2_100 female 0.25 0.9106 0.0976 0.5840 0.1274 2.8793 0.9703 0.8524 0.7391 0.0744 0.9980 0.7556 0.2823 0.1005 0.0590 0.9488
mobilenetv2_100 all 0.25 0.9234 0.0944 0.5354 0.1341 3.3458 0.9826 0.8963 0.7780 0.0530 0.9999 0.5364 0.1482 0.0553 0.0358 0.9935
mobilenetv2_100 male 0.75 0.9196 0.0862 0.5815 0.1200 3.2361 0.9823 0.8849 0.7338 0.0577 0.9997 0.7274 0.2387 0.0668 0.0324 0.9691
mobilenetv2_100 female 0.75 0.9015 0.1069 0.5536 0.1258 2.9807 0.9710 0.8589 0.7608 0.0710 0.9999 0.6612 0.2482 0.0938 0.0540 0.9648
mobilenetv2_100 all 0.75 0.9205 0.0990 0.5492 0.1301 3.2118 0.9778 0.8824 0.7468 0.0596 0.9999 0.6443 0.1894 0.0678 0.0426 0.9582
xception male 0.0 0.9207 0.0882 0.5616 0.1303 3.2272 0.9808 0.8830 0.7243 0.0588 0.9999 0.8116 0.2524 0.0679 0.0338 0.9125
xception female 0.0 0.9075 0.0983 0.6062 0.1282 2.6379 0.9615 0.8089 0.6780 0.0966 0.9999 0.8033 0.3734 0.1551 0.0939 0.9816
xception all 0.0 0.9234 0.0933 0.5663 0.1278 3.1902 0.9787 0.8713 0.7275 0.0657 0.9996 0.6521 0.2156 0.0792 0.0475 0.9675
xception male 0.25 0.9310 0.0859 0.6058 0.1230 3.0651 0.9778 0.8762 0.7339 0.0627 1.0000 0.7649 0.2366 0.0749 0.0404 0.9998
xception female 0.25 0.9213 0.1013 0.6239 0.1267 2.5936 0.9564 0.8285 0.6657 0.0866 0.9999 0.9590 0.4329 0.1355 0.0774 0.9827
xception all 0.25 0.9325 0.0972 0.5925 0.1270 3.0059 0.9716 0.8725 0.7339 0.0662 1.0000 0.7622 0.1987 0.0782 0.0519 0.9988
xception male 0.75 0.9306 0.0853 0.5797 0.1320 3.1573 0.9804 0.8819 0.7195 0.0592 0.9999 0.8791 0.2611 0.0695 0.0342 0.9677
xception female 0.75 0.9165 0.0999 0.5470 0.1385 3.0597 0.9747 0.8765 0.7788 0.0628 0.9981 0.5862 0.2193 0.0738 0.0474 0.9027
xception all 0.75 0.9311 0.0973 0.5473 0.1403 3.1790 0.9773 0.8890 0.7491 0.0572 1.0000 0.6550 0.1798 0.0621 0.0417 0.9340

Across DataBalance - NIR

Experiment ID Gender Data Balance (% Females) GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
vit_base_patch16_224 male 0.75 0.8587 0.0696 0.5582 0.1274 2.9266 0.9812 0.8573 0.8005 0.0715 0.9079 0.5709 0.2750 0.1017 0.0489 0.8794
vit_base_patch16_224 female 0.75 0.8659 0.0658 0.5307 0.1395 3.0746 0.9866 0.8770 0.8443 0.0616 0.8819 0.4895 0.2292 0.0764 0.0329 0.6607
vit_base_patch16_224 all 0.75 0.8563 0.0686 0.5405 0.1321 2.9993 0.9830 0.8694 0.7712 0.0656 0.9693 0.6372 0.2924 0.0890 0.0384 0.8380
vit_base_patch16_224 male 1.0 0.8295 0.0769 0.5520 0.1189 2.7703 0.9728 0.8234 0.7585 0.0885 0.9834 0.6274 0.3445 0.1445 0.0775 0.9106
vit_base_patch16_224 female 1.0 0.8539 0.0700 0.5286 0.1469 2.8265 0.9695 0.8151 0.7405 0.0952 0.9828 0.7786 0.4668 0.1975 0.0885 0.7520
vit_base_patch16_224 all 1.0 0.8343 0.0764 0.5448 0.1267 2.7669 0.9715 0.8213 0.7038 0.0896 0.9885 0.7184 0.4024 0.1564 0.0788 0.8568
xception male 0.25 0.9167 0.0803 0.5934 0.1321 2.9576 0.9795 0.8628 0.7844 0.0692 0.9994 0.8064 0.3574 0.0992 0.0429 0.8626
xception female 0.25 0.9188 0.0807 0.6014 0.1323 2.8969 0.9777 0.8605 0.7955 0.0716 0.9901 0.7801 0.4265 0.1190 0.0418 0.7392
xception all 0.25 0.9146 0.0812 0.5651 0.1302 3.2221 0.9853 0.8908 0.7879 0.0550 1.0000 0.7489 0.2845 0.0622 0.0244 0.8135
vit_base_patch16_224 male 0.25 0.8644 0.0639 0.5473 0.1317 3.0634 0.9875 0.8894 0.8498 0.0557 0.9603 0.4509 0.1822 0.0607 0.0308 0.8047
vit_base_patch16_224 female 0.25 0.8727 0.0635 0.5385 0.1525 2.8614 0.9787 0.8484 0.7871 0.0780 0.9496 0.6559 0.3684 0.1338 0.0521 0.6009
vit_base_patch16_224 all 0.25 0.8651 0.0645 0.5334 0.1367 3.1039 0.9865 0.8844 0.8008 0.0580 0.9654 0.5477 0.2368 0.0682 0.0287 0.7704
inception_v3 male 1.0 0.8706 0.0902 0.6358 0.1151 2.2709 0.9400 0.7458 0.6166 0.1281 1.0000 0.9811 0.7124 0.3142 0.1693 0.9483
inception_v3 female 1.0 0.8935 0.0907 0.6015 0.1470 2.3918 0.9437 0.7332 0.6534 0.1352 0.9999 0.9277 0.6539 0.3242 0.1893 0.9179
inception_v3 all 1.0 0.8736 0.0918 0.6165 0.1237 2.3612 0.9445 0.7502 0.5705 0.1263 1.0000 0.9581 0.6706 0.3085 0.1666 0.9707
xception male 1.0 0.9009 0.0894 0.6249 0.1430 2.3148 0.9489 0.7656 0.6510 0.1175 1.0000 0.9269 0.6209 0.2629 0.1394 0.9219
xception female 1.0 0.9259 0.0772 0.5760 0.1804 2.5217 0.9594 0.8051 0.7126 0.1017 1.0000 0.9819 0.7091 0.2539 0.1044 0.5928
xception all 1.0 0.9034 0.0892 0.6094 0.1517 2.3618 0.9511 0.7734 0.6020 0.1141 1.0000 0.9618 0.6546 0.2631 0.1333 0.8889
vit_base_patch16_224 male 0.0 0.8401 0.0775 0.5341 0.1333 2.8061 0.9770 0.8393 0.7821 0.0804 0.9769 0.5863 0.3054 0.1216 0.0645 0.7912
vit_base_patch16_224 female 0.0 0.8572 0.0636 0.5522 0.1464 2.7017 0.9694 0.8252 0.7411 0.0913 0.9972 0.8108 0.5263 0.2021 0.0780 0.6756
vit_base_patch16_224 all 0.0 0.8417 0.0759 0.5378 0.1379 2.7300 0.9726 0.8266 0.7088 0.0870 0.9940 0.7171 0.3935 0.1504 0.0739 0.7933
xception male 0.75 0.9321 0.0689 0.5796 0.1501 3.0194 0.9825 0.8842 0.7866 0.0601 0.9991 0.8699 0.4117 0.0803 0.0242 0.6445
xception female 0.75 0.9195 0.0750 0.6043 0.1474 2.6944 0.9727 0.8397 0.7726 0.0818 0.9987 0.8983 0.5023 0.1517 0.0604 0.8654
xception all 0.75 0.9271 0.0717 0.5661 0.1491 3.0851 0.9830 0.8839 0.7505 0.0596 0.9999 0.8610 0.3974 0.0776 0.0242 0.6464
xception male 0.0 0.9362 0.0768 0.6250 0.1677 2.3858 0.9642 0.8175 0.6909 0.0937 0.9998 0.9518 0.6159 0.2053 0.0851 0.7230
xception female 0.0 0.9345 0.0731 0.6328 0.1356 2.7697 0.9608 0.8190 0.7204 0.0978 1.0000 0.9815 0.7417 0.2551 0.0938 0.9596
xception all 0.0 0.9352 0.0762 0.6232 0.1575 2.5218 0.9654 0.8219 0.6422 0.0918 0.9999 0.9559 0.6205 0.2027 0.0798 0.8857
inception_v3 male 0.75 0.8888 0.0789 0.5780 0.1226 3.0161 0.9818 0.8666 0.7957 0.0672 0.9983 0.6740 0.3024 0.0925 0.0401 0.8900
inception_v3 female 0.75 0.8891 0.0755 0.5582 0.1331 3.0586 0.9846 0.8767 0.8361 0.0618 0.9991 0.6899 0.2842 0.0786 0.0319 0.6730
inception_v3 all 0.75 0.8856 0.0784 0.5515 0.1253 3.1972 0.9856 0.8843 0.7873 0.0583 0.9967 0.6607 0.2714 0.0700 0.0280 0.7200
inception_v3 male 0.25 0.9015 0.0729 0.5681 0.1367 3.0445 0.9855 0.8862 0.8176 0.0574 0.9985 0.6708 0.2727 0.0677 0.0274 0.8340
inception_v3 female 0.25 0.9030 0.0769 0.5859 0.1338 2.9054 0.9714 0.8480 0.7666 0.0797 1.0000 0.9371 0.5917 0.1693 0.0522 0.7073
inception_v3 all 0.25 0.9003 0.0747 0.5537 0.1335 3.2026 0.9853 0.8914 0.7798 0.0551 1.0000 0.7893 0.3114 0.0626 0.0224 0.8213
inception_v3 male 0.0 0.8794 0.0880 0.5638 0.1310 2.8279 0.9748 0.8413 0.7566 0.0796 0.9998 0.8240 0.4084 0.1308 0.0603 0.9116
inception_v3 female 0.0 0.8833 0.0800 0.5787 0.1299 2.8242 0.9727 0.8305 0.7799 0.0851 0.9957 0.8186 0.4241 0.1445 0.0707 0.7438
inception_v3 all 0.0 0.8772 0.0864 0.5590 0.1272 2.9259 0.9772 0.8488 0.7355 0.0760 0.9998 0.7824 0.3696 0.1168 0.0544 0.9184

Experiment ID Gender Data Balance (% Females) GMean GSTD IMean ISTD Sensitivity index (d') AUC J-Index MCC EER ZeroFMR FMR1000 FMR100 FMR20 FMR10 ZeroFNMR
vit_base_patch16_224 male 0.75 0.9143 0.0559 0.5859 0.1238 3.4196 0.9927 0.9195 0.9163 0.0408 0.6632 0.2961 0.0933 0.0352 0.0200 0.7096
vit_base_patch16_224 female 0.75 0.8938 0.0690 0.5458 0.1138 3.6992 0.9918 0.9154 0.9110 0.0437 0.6250 0.2643 0.1029 0.0403 0.0249 0.6044
vit_base_patch16_224 all 0.75 0.9056 0.0630 0.5493 0.1212 3.6881 0.9924 0.9158 0.9083 0.0426 0.7039 0.3275 0.1091 0.0379 0.0200 0.6076
vit_base_patch16_224 male 1.0 0.8987 0.0641 0.5935 0.1240 3.0921 0.9851 0.8768 0.8615 0.0616 0.9730 0.6939 0.1985 0.0746 0.0358 0.9159
vit_base_patch16_224 female 1.0 0.8759 0.0751 0.5365 0.1167 3.4600 0.9905 0.8964 0.8879 0.0521 0.4589 0.2700 0.1402 0.0540 0.0252 0.5828
vit_base_patch16_224 all 1.0 0.8915 0.0696 0.5521 0.1269 3.3177 0.9865 0.8806 0.8592 0.0598 0.9757 0.5953 0.2080 0.0713 0.0315 0.8579
xception male 0.25 0.9365 0.0677 0.5669 0.1314 3.5369 0.9912 0.9188 0.8940 0.0408 1.0000 0.8822 0.1836 0.0319 0.0113 0.4341
xception female 0.25 0.9327 0.0688 0.5503 0.1338 3.5953 0.9962 0.9428 0.9319 0.0286 0.7589 0.3196 0.0707 0.0145 0.0063 0.4716
xception all 0.25 0.9358 0.0673 0.5511 0.1405 3.4916 0.9906 0.9232 0.8946 0.0389 1.0000 0.9366 0.2146 0.0276 0.0091 0.4374
vit_base_patch16_224 male 0.25 0.9031 0.0593 0.5446 0.1132 3.9683 0.9967 0.9440 0.9393 0.0283 0.5965 0.1814 0.0571 0.0165 0.0072 0.5236
vit_base_patch16_224 female 0.25 0.8930 0.0669 0.5536 0.1173 3.5540 0.9913 0.9117 0.9133 0.0461 0.6768 0.2510 0.0977 0.0438 0.0270 0.5205
vit_base_patch16_224 all 0.25 0.8977 0.0642 0.5229 0.1089 4.1926 0.9966 0.9424 0.9402 0.0294 0.5466 0.1448 0.0567 0.0189 0.0089 0.4272
densenet121 male 0.25 0.9398 0.0630 0.5811 0.1335 3.4374 0.9904 0.9097 0.8793 0.0464 0.9994 0.6817 0.2029 0.0416 0.0124 0.5566
densenet121 female 0.25 0.9314 0.0684 0.5729 0.1424 3.2093 0.9858 0.8869 0.8513 0.0580 0.9999 0.9281 0.3104 0.0702 0.0234 0.3936
densenet121 all 0.25 0.9371 0.0649 0.5693 0.1425 3.3215 0.9897 0.9078 0.8756 0.0471 1.0000 0.8274 0.2203 0.0434 0.0131 0.5313
xception male 1.0 0.9355 0.0745 0.6632 0.1209 2.7103 0.9673 0.8291 0.7711 0.0875 1.0000 0.9604 0.5931 0.1767 0.0713 0.8393
xception female 1.0 0.9228 0.0785 0.5944 0.1258 3.1320 0.9778 0.8611 0.8041 0.0733 0.9982 0.8982 0.4455 0.1292 0.0401 0.6115
xception all 1.0 0.9323 0.0764 0.6393 0.1332 2.6996 0.9709 0.8402 0.7864 0.0808 0.9997 0.9500 0.5459 0.1475 0.0604 0.7678
vit_base_patch16_224 male 0.0 0.8986 0.0654 0.5313 0.1303 3.5630 0.9926 0.9221 0.9087 0.0390 0.6750 0.2797 0.1115 0.0300 0.0145 0.8700
vit_base_patch16_224 female 0.0 0.8888 0.0707 0.5706 0.1277 3.0833 0.9869 0.8917 0.8900 0.0554 0.8043 0.3404 0.1384 0.0597 0.0361 0.9073
vit_base_patch16_224 all 0.0 0.8952 0.0685 0.5418 0.1265 3.4728 0.9908 0.9101 0.8991 0.0452 0.7087 0.2978 0.1254 0.0413 0.0214 0.8980
xception male 0.75 0.9506 0.0622 0.6576 0.1187 3.0924 0.9845 0.8784 0.8430 0.0617 0.9945 0.7408 0.3015 0.0803 0.0296 0.6218
xception female 0.75 0.9301 0.0758 0.5580 0.1145 3.8320 0.9942 0.9302 0.9049 0.0357 0.9106 0.4480 0.1435 0.0212 0.0056 0.3758
xception all 0.75 0.9451 0.0684 0.6114 0.1309 3.1939 0.9872 0.8899 0.8546 0.0561 0.9958 0.6981 0.2574 0.0648 0.0221 0.5562
densenet121 male 0.0 0.9220 0.0762 0.5761 0.1431 3.0182 0.9805 0.8701 0.8194 0.0675 0.9953 0.8360 0.4084 0.1034 0.0322 0.6273
densenet121 female 0.0 0.9237 0.0807 0.6407 0.1303 2.6112 0.9610 0.7980 0.7598 0.1012 0.9986 0.9617 0.5418 0.1907 0.1025 0.8058
densenet121 all 0.0 0.9249 0.0784 0.6041 0.1459 2.7397 0.9699 0.8234 0.7744 0.0889 0.9965 0.9042 0.4946 0.1639 0.0767 0.6831
xception male 0.0 0.9414 0.0682 0.6297 0.1257 3.0826 0.9824 0.8717 0.8363 0.0645 0.9944 0.6950 0.3324 0.0857 0.0380 0.6525
xception female 0.0 0.9333 0.0675 0.6427 0.1314 2.7815 0.9788 0.8546 0.8186 0.0733 0.9810 0.7647 0.3687 0.1116 0.0492 0.8384
xception all 0.0 0.9384 0.0671 0.6338 0.1285 2.9725 0.9803 0.8629 0.8215 0.0691 0.9990 0.7328 0.3682 0.1016 0.0429 0.8271
densenet121 male 0.75 0.9568 0.0588 0.6554 0.1291 3.0044 0.9836 0.8790 0.8460 0.0608 0.9965 0.8523 0.3279 0.0759 0.0306 0.5328
densenet121 female 0.75 0.9305 0.0703 0.5624 0.1471 3.1927 0.9899 0.9014 0.8809 0.0494 0.9264 0.4915 0.1816 0.0492 0.0226 0.4593
densenet121 all 0.75 0.9473 0.0653 0.5961 0.1550 2.9531 0.9827 0.8746 0.8371 0.0633 0.9993 0.9193 0.3354 0.0831 0.0343 0.5432
densenet121 male 1.0 0.9390 0.0663 0.6524 0.1429 2.5723 0.9669 0.8309 0.7763 0.0860 0.9991 0.9706 0.6567 0.1689 0.0697 0.7922
densenet121 female 1.0 0.9328 0.0786 0.5678 0.1480 3.0815 0.9808 0.8636 0.8402 0.0684 0.9990 0.8547 0.3072 0.0938 0.0451 0.6560
densenet121 all 1.0 0.9380 0.0708 0.6075 0.1594 2.6788 0.9737 0.8448 0.7994 0.0784 0.9994 0.9140 0.4692 0.1288 0.0571 0.7256

Across Race - VIS

Across Modality - VIS

Across Modality - NIR

Across Device - VIS

Across Lighting - NIR

Across Spectrum - Ocular

Across Spectrum - Face

Gender Classification

Across Model - VIS

Model Male Acc Female Acc Avg Acc MCC
mobilenetv2_100 0.8076 0.8342 0.8189 0.6362
vit_base_patch16_224 0.8164 0.7944 0.8070 0.6078
efficientnet_b4 0.8116 0.8465 0.8265 0.6521
densenet121 0.8996 0.7486 0.8355 0.6617
inception_v3 0.9000 0.7132 0.8206 0.6313
resnet50 0.8851 0.7993 0.8486 0.6890
inception_resnet_v2 0.5989 0.9321 0.7405 0.5428
efficientnet_b0 0.9025 0.7870 0.8534 0.6987
xception 0.8905 0.7934 0.8493 0.6901

Model Male Acc Female Acc Avg Acc MCC
efficientnet_b0 0.9443 0.9026 0.9251 0.8492
xception 0.8984 0.9564 0.9251 0.8521
inception_v3 0.9552 0.9179 0.9381 0.8754
efficientnet_b4 0.9421 0.9500 0.9457 0.8910
mobilenetv2_100 0.9290 0.9359 0.9322 0.8638
densenet121 0.9836 0.8910 0.9410 0.8836
vit_base_patch16_224 0.8918 0.8987 0.8950 0.7893
inception_resnet_v2 0.9038 0.9526 0.9263 0.8538
resnet50 0.9738 0.8705 0.9263 0.8541

Model Male Acc Female Acc Avg Acc MCC
mobilenetv2_100 0.9325 0.9405 0.9359 0.8697
vit_base_patch16_224 0.9563 0.7892 0.8856 0.7678
efficientnet_b4 0.9603 0.9081 0.9382 0.8733
densenet121 0.9286 0.9405 0.9336 0.8652
inception_v3 0.9643 0.8595 0.9199 0.8367
resnet50 0.6190 0.9405 0.7551 0.5690
inception_resnet_v2 0.9365 0.9459 0.9405 0.8790
efficientnet_b0 0.9206 0.9189 0.9199 0.8368
xception 0.9762 0.8216 0.9108 0.8205

Model Male Acc Female Acc Avg Acc MCC
mobilenetv2_100 0.9846 0.9770 0.9816 0.9616
vit_base_patch16_224 0.9846 0.9770 0.9816 0.9616
efficientnet_b4 0.9923 0.9885 0.9908 0.9808
densenet121 0.9846 0.9655 0.9770 0.9520
inception_v3 0.9692 1.0000 0.9816 0.9626
resnet50 1.0000 0.5517 0.8203 0.6515
inception_resnet_v2 0.9769 1.0000 0.9862 0.9718
efficientnet_b0 0.9923 0.9540 0.9770 0.9522
xception 1.0000 1.0000 1.0000 1.0000

Model Male Acc Female Acc Avg Acc MCC
efficientnet_b0 0.8594 0.8871 0.8744 0.7469
xception 0.9069 0.8300 0.8651 0.7342
inception_v3 0.8625 0.8579 0.8600 0.7189
efficientnet_b4 0.8875 0.8547 0.8697 0.7399
mobilenetv2_100 0.8722 0.8647 0.8681 0.7354
densenet121 0.9613 0.7439 0.8433 0.7114
vit_base_patch16_224 0.8057 0.8089 0.8074 0.6133
inception_resnet_v2 0.8903 0.8663 0.8773 0.7545
resnet50 0.9103 0.8097 0.8557 0.7179

Model Male Acc Female Acc Avg Acc MCC
efficientnet_b0 0.9319 0.9597 0.9470 0.8932
xception 0.9503 0.9476 0.9489 0.8971
inception_v3 0.9500 0.9526 0.9514 0.9022
efficientnet_b4 0.9391 0.9516 0.9459 0.8909
mobilenetv2_100 0.9185 0.9616 0.9419 0.8830
densenet121 0.9634 0.9447 0.9533 0.9064
vit_base_patch16_224 0.9238 0.8200 0.8674 0.7416
inception_resnet_v2 0.9591 0.9605 0.9599 0.9192
resnet50 0.9916 0.7673 0.8699 0.7659

Cross Dataset Evaluation

Across Model - NIR

Model Male Acc Female Acc Avg Acc MCC
efficientnet_b0 0.9614 0.8082 0.9038 0.7940
xception 0.9948 0.6811 0.8768 0.7470
inception_v3 0.9074 0.8648 0.8914 0.7695
efficientnet_b4 0.7438 0.9809 0.8330 0.7033
mobilenetv2_100 0.8911 0.9243 0.9036 0.8014
densenet121 0.9450 0.9336 0.9407 0.8744
vit_base_patch16_224 0.8079 0.8510 0.8241 0.6429
inception_resnet_v2 0.8388 0.9573 0.8834 0.7736
resnet50 0.9530 0.6291 0.8312 0.6382

Model Male Acc Female Acc Avg Acc MCC
efficientnet_b0 0.9570 0.7674 0.8838 0.7544
xception 0.9799 0.7365 0.8860 0.7636
inception_v3 0.9155 0.7905 0.8672 0.7173
efficientnet_b4 0.8780 0.7630 0.8336 0.6465
mobilenetv2_100 0.9397 0.7200 0.8549 0.6914
densenet121 0.9231 0.7740 0.8655 0.7134
vit_base_patch16_224 0.8988 0.7497 0.8413 0.6613
inception_resnet_v2 0.9501 0.7222 0.8621 0.7080
resnet50 0.9265 0.7442 0.8562 0.6932

Model Male Acc Female Acc Avg Acc MCC
efficientnet_b0 0.9979 0.8975 0.9591 0.9153
xception 1.0000 0.8732 0.9511 0.8993
inception_v3 0.9965 0.9283 0.9702 0.9378
efficientnet_b4 0.9515 0.9427 0.9481 0.8910
mobilenetv2_100 0.9917 0.9294 0.9677 0.9321
densenet121 0.9951 0.9361 0.9723 0.9420
vit_base_patch16_224 0.9938 0.9338 0.9706 0.9384
inception_resnet_v2 0.9764 0.9416 0.9630 0.9217
resnet50 0.7207 0.9449 0.8072 0.6498

Cross Dataset Evaluation

Model Male Acc Female Acc Avg Acc MCC
efficientnet_b0 0.9510 0.7466 0.8478 0.7116
xception 0.9965 0.5103 0.7509 0.5780
inception_v3 0.9825 0.6370 0.8080 0.6585
efficientnet_b4 0.8182 0.8425 0.8304 0.6609
mobilenetv2_100 0.9056 0.8014 0.8529 0.7103
densenet121 0.8042 0.8356 0.8201 0.6402
vit_base_patch16_224 0.7378 0.8630 0.8010 0.6060
inception_resnet_v2 0.9161 0.8253 0.8702 0.7440
resnet50 1.0000 0.4555 0.7249 0.5411

Model Male Acc Female Acc Avg Acc MCC
efficientnet_b0 0.4865 0.8795 0.6424 0.3776
xception 0.8090 0.3014 0.6076 0.1274
inception_v3 0.7495 0.5233 0.6598 0.2783
efficientnet_b4 0.4126 0.7863 0.5609 0.2064
mobilenetv2_100 0.6324 0.6548 0.6413 0.2812
densenet121 0.3279 0.7425 0.4924 0.0752
vit_base_patch16_224 0.3874 0.5534 0.4533 -0.0589
inception_resnet_v2 0.4270 0.6603 0.5196 0.0875
resnet50 0.4144 0.6822 0.5207 0.0976

Across DataBalance - VIS

Model Data Balance (% Females) Male Acc Female Acc Avg Acc MCC
efficientnet_b4 0.75 0.9443 0.8962 0.9221 0.8433
densenet121 0.25 0.9956 0.8244 0.9168 0.8413
inception_v3 0.75 0.9727 0.9167 0.9469 0.8937
inception_v3 0.25 0.9596 0.9167 0.9398 0.8790
efficientnet_b4 0.25 0.8918 0.9615 0.9239 0.8506
densenet121 0.75 0.9836 0.8500 0.9221 0.8481

Model Data Balance (% Females) Male Acc Female Acc Avg Acc MCC
efficientnet_b4 0.75 0.9443 0.8962 0.9221 0.8433
densenet121 0.25 0.9956 0.8244 0.9168 0.8413
inception_v3 0.75 0.9727 0.9167 0.9469 0.8937
inception_v3 0.25 0.9596 0.9167 0.9398 0.8790
efficientnet_b4 0.25 0.8918 0.9615 0.9239 0.8506
densenet121 0.75 0.9836 0.8500 0.9221 0.8481

Across DataBalance - NIR

Model Data Balance (% Females) Male Acc Female Acc Avg Acc MCC
mobilenetv2_100 0.25 0.8834 0.8989 0.8892 0.7702
mobilenetv2_100 0.75 0.8904 0.9330 0.9064 0.8082
efficientnet_b0 0.75 0.9078 0.9451 0.9218 0.8391
densenet121 0.25 0.9467 0.8354 0.9049 0.7957
efficientnet_b0 0.25 0.8928 0.9278 0.9060 0.8065
densenet121 0.75 0.9394 0.8885 0.9203 0.8298

Model Data Balance (% Females) Male Acc Female Acc Avg Acc MCC
efficientnet_b0 0.75 0.9570 0.7244 0.8672 0.7199
xception 0.25 0.9584 0.6714 0.8477 0.6795
xception 0.75 0.9841 0.6450 0.8532 0.6993
inception_v3 0.75 0.9335 0.7343 0.8566 0.6945
inception_v3 0.25 0.9667 0.7321 0.8762 0.7403
efficientnet_b0 0.25 0.9931 0.6582 0.8638 0.7244

Across Race - VIS

Model South Asian-Female South Asian-Male White-Male White-Female Latino-Female Black-Male Latino-Male East Asian-Male Black-Female
mobilenetv2_100 0.8284 0.8273 0.7967 0.8331 0.7532 0.6890 0.8515 0.8687 1.0000
vit_base_patch16_224 0.7583 0.8964 0.7148 0.8265 0.8831 0.7129 0.9208 0.9394 0.9024
efficientnet_b4 0.8329 0.8565 0.7717 0.8476 0.8701 0.6507 0.8911 0.8990 1.0000
densenet121 0.6981 0.9018 0.9316 0.7832 0.8701 0.6794 0.9109 0.9899 1.0000
inception_v3 0.6640 0.9348 0.8728 0.7819 0.4675 0.7847 0.9505 0.9192 0.9756
resnet50 0.7960 0.9263 0.8671 0.7832 0.8052 0.6986 0.8416 0.9798 0.9878
inception_resnet_v2 0.9021 0.6938 0.5453 0.9698 0.9221 0.2584 0.5842 0.6465 1.0000
efficientnet_b0 0.7583 0.9470 0.8622 0.8252 0.5974 0.8038 0.9406 0.9091 1.0000
xception 0.7853 0.9256 0.8613 0.8016 0.6104 0.7416 0.9901 0.9495 1.0000

Model South Asian-Female South Asian-Male White-Male White-Female Latino-Female Black-Male Latino-Male East Asian-Male Black-Female
mobilenetv2_100 0.8176 0.7552 0.7119 0.8647 0.9091 0.7560 0.9010 0.8182 0.9634
vit_base_patch16_224 0.8293 0.8104 0.8680 0.7083 0.7143 0.6651 0.9406 0.7172 0.9268
efficientnet_b4 0.7969 0.8526 0.8516 0.8200 0.7532 0.7321 0.7921 0.8586 1.0000
densenet121 0.8769 0.6907 0.6792 0.9264 0.9351 0.6507 0.5248 0.7475 1.0000
inception_v3 0.7520 0.8634 0.8632 0.7556 0.8052 0.7990 0.9406 0.8586 0.9878
resnet50 0.7376 0.9271 0.8960 0.8095 0.8182 0.7751 0.8911 0.8283 1.0000
inception_resnet_v2 0.5804 0.9578 0.9557 0.6965 0.5065 0.9091 0.8812 0.9293 0.9756
efficientnet_b0 0.9066 0.5932 0.7437 0.8830 0.9091 0.6316 0.7030 0.6162 1.0000
xception 0.9111 0.6930 0.7370 0.9120 0.9481 0.7464 0.7327 0.6465 1.0000

Across Age - VIS

Model Young-Male Young-Female Middle Aged-Male Middle Aged-Female Old-Male Old-Female
efficientnet_b0 0.9500 0.9105 0.9238 0.9030 0.9524 0.8556
xception 0.8767 0.9676 0.9286 0.9333 0.9619 0.9333
inception_v3 0.9667 0.9486 0.9238 0.8667 0.9524 0.8333
efficientnet_b4 0.9200 0.9581 0.9762 0.9394 1.0000 0.9222
mobilenetv2_100 0.9067 0.9467 0.9667 0.9212 0.9810 0.9000
densenet121 0.9783 0.9048 0.9905 0.8909 1.0000 0.8111
vit_base_patch16_224 0.8817 0.9238 0.8857 0.8606 0.9619 0.8222
inception_resnet_v2 0.8883 0.9714 0.9571 0.9152 0.8857 0.9111
resnet50 0.9683 0.8990 0.9810 0.8606 0.9905 0.7222

Cross Dataset

Across Modality - VIS

Across Modality - NIR

Across Device - VIS

Model Male Acc Female Acc Avg Acc MCC Device
xception 0.8382 0.7452 0.7987 0.5864 samsung
efficientnet_b4 0.7360 0.8077 0.7665 0.5376 iphone
resnet50 0.6524 0.8455 0.7345 0.4958 iphone
inception_v3 0.7549 0.7206 0.7403 0.4728 iphone
mobilenetv2_100 0.7753 0.7329 0.7573 0.5060 oppo
xception 0.8538 0.7619 0.8148 0.6194 oppo
vit_base_patch16_224 0.7585 0.7973 0.7750 0.5502 oppo
efficientnet_b4 0.8895 0.7924 0.8482 0.6880 oppo
vit_base_patch16_224 0.4920 0.8819 0.6577 0.3922 iphone
mobilenetv2_100 0.6098 0.8662 0.7188 0.4789 samsung
efficientnet_b0 0.8160 0.7280 0.7786 0.5457 iphone
efficientnet_b0 0.8491 0.7423 0.8037 0.5962 samsung
densenet121 0.6764 0.8436 0.7474 0.5161 samsung
resnet50 0.7498 0.8200 0.7796 0.5634 oppo
inception_resnet_v2 0.3556 0.9611 0.6130 0.3774 samsung
mobilenetv2_100 0.6982 0.8165 0.7485 0.5093 iphone
efficientnet_b4 0.7589 0.7821 0.7688 0.5358 samsung
inception_v3 0.7204 0.8032 0.7556 0.5177 samsung
vit_base_patch16_224 0.5444 0.9056 0.6979 0.4654 samsung
xception 0.7353 0.8028 0.7640 0.5320 iphone
inception_v3 0.7876 0.7432 0.7688 0.5290 oppo
densenet121 0.8793 0.7226 0.8127 0.6141 oppo
densenet121 0.8007 0.7378 0.7740 0.5380 iphone
inception_resnet_v2 0.8811 0.6572 0.7859 0.5587 iphone
efficientnet_b0 0.7276 0.8519 0.7805 0.5734 oppo
inception_resnet_v2 0.8811 0.7334 0.8183 0.6258 oppo
resnet50 0.5135 0.9351 0.6927 0.4742 samsung

Cross Dataset

Across Lighting - VIS

Model Male Acc Female Acc Avg Acc MCC Lighting
inception_v3 0.6240 0.7649 0.6839 0.3858 dim
vit_base_patch16_224 0.8036 0.7211 0.7686 0.5257 day
xception 0.7935 0.8052 0.7985 0.5937 office
inception_v3 0.8513 0.7378 0.8031 0.5947 office
inception_resnet_v2 0.4865 0.9021 0.6632 0.4114 dim
mobilenetv2_100 0.9338 0.5322 0.7631 0.5225 day
mobilenetv2_100 0.8029 0.7841 0.7949 0.5837 office
efficientnet_b4 0.8462 0.7432 0.8024 0.5937 day
inception_v3 0.6716 0.8028 0.7274 0.4699 day
resnet50 0.9491 0.5873 0.7953 0.5913 office
efficientnet_b4 0.7422 0.7698 0.7539 0.5068 dim
mobilenetv2_100 0.6993 0.8008 0.7424 0.4946 dim
resnet50 0.8858 0.6458 0.7838 0.5548 day
efficientnet_b4 0.7113 0.8593 0.7742 0.5653 office
densenet121 0.6775 0.8308 0.7426 0.5040 day
densenet121 0.7764 0.8116 0.7913 0.5821 office
vit_base_patch16_224 0.6636 0.8485 0.7422 0.5093 office
xception 0.8505 0.7442 0.8054 0.5996 day
vit_base_patch16_224 0.6145 0.8214 0.7025 0.4354 dim
xception 0.8044 0.7585 0.7849 0.5613 dim
efficientnet_b0 0.8935 0.6449 0.7878 0.5637 day
inception_resnet_v2 0.8335 0.7132 0.7824 0.5519 day
efficientnet_b0 0.8440 0.7619 0.8091 0.6082 office
resnet50 0.5058 0.9110 0.6780 0.4386 dim
inception_resnet_v2 0.9164 0.6645 0.8093 0.6105 office
densenet121 0.8851 0.5406 0.7387 0.4610 dim
efficientnet_b0 0.7615 0.7570 0.7596 0.5143 dim

Cross Dataset

Across Spectrum

Sreeraj Ramachandran
Sreeraj Ramachandran
Graduate Research Assistant

My research interests include Computer Vision, Biometrics, Bias AI, GANs and adversarial Attacks

comments powered by Disqus