DNV fine-tuned with SD1

Validation Scores of Models before fine-tuned with SD1 data

This table shows the validation score of the DNV model before fine-tuning with SD1. Utility A stands for DNV.

RUNID	Train dataset	Labelling Method	BackBone	validation F1 score	validation F2 score
61	Utility A (41,771 images)	B	resnet50	0.900	0.890
62	Utility A (41,771 images)	B	resnet101	0.863	0.847
60	Utility A (42,523 images)	A	resnet50	0.951	0.943
63	Utility A (42,523 images)	A	resnet101	0.951	0.943
64	Utiltiy A (43,536 images)	B	resnet50	0.977	0.967
cv_7	Utiltiy A (43,536 images) 90% train-valid 10% test	B	resnet50	0.988	0.982
cv_12	Utiltiy A (43,536 images) 95% train-valid 5% test	B	resnet50	0.993	0.990

95 train-val /5 test split

• 2296 SD1 images was split 95% training-validation and 5% testing
• The table shows how different models perform on SD1 data
• The first row represent a model that only trained and validated with SD1 data

Model	Dataset	Code Standard	Image labelling method for Utility A Images	Model Complexity	Transfer Learning/Fine Tune	Validation Score (F2 avg no ND)	Test Score (F2 avg no ND)	Test Dataset
1	SD1 ( 2179 images)	PACP	-	Resnet50	-	0.688	0.49	117 SD1 images
2	Utility A* (41,771 images)	PACP	B	Resnet50	SD1 ( 2179 images)	0.782	0.639	117 SD1 images
3	Utility A* (41,771 images)	PACP	B	Resnet101	SD1 ( 2179 images)	0.739	0.465	117 SD1 images
4	Utility A* (42,523 images)	PACP	A	Resnet50	SD1 ( 2179 images)	0.716	0.509	117 SD1 images
5	Utility A* (42,523 images)	PACP	A	Resnet101	SD1 ( 2179 images)	0.669	0.445	117 SD1 images
6	Utility A*(43,536 images)	PACP	B	Resnet50	SD1 ( 2179 images)	0.647	0.41	117 SD1 images
7	European (1.3 million images)	Fotomanual	-	Resnet50	SD1 ( 2179 images)	0.479	0.411	117 SD1 images

2266 images for train-validation and 30 images for testing

Model	Dataset	Code Standard	Image labelling method for Utility A Images	Model Complexity	Transfer Learning/Fine Tune	Validation Score (F2 avg no ND)	Test Score (F2 avg no ND)	Test Dataset
1	SD1 (2266 images)	PACP	-	Resnet50	-	0.748	0.792	30 SD1 images
2	Utility A* (41,771 images)	PACP	B	Resnet50	SD1 (2266 images)	0.719	0.853	30 SD1 images
3	Utility A* (41,771 images)	PACP	B	Resnet101	SD1 (2266 images)	0.704	0.826	30 SD1 images
4	Utility A* (42,523 images)	PACP	A	Resnet50	SD1 (2266 images)	0.714	0.791	30 SD1 images
5	Utility A* (42,523 images)	PACP	A	Resnet101	SD1 (2266 images)	0.741	0.849	30 SD1 images
6	Utility A*(43,536 images)	PACP	B	Resnet50	SD1 (2266 images)	0.727	0.859	30 SD1 images
7	European (1.3 million images)	Fotomanual	-	Resnet50	SD1 (2266 images)	0.193	0.296	30 SD1 images

The table below shows the validation score of the DNV model after fine-tuning with 2266 SD1 images and testing with 30 SD1 images

RUNID	Model's RUNID	BackBone	Validation F1 Score	Validation F2 Score	Test F1 Score	Test F2 Score
DNV_TL_2	61	resnet50	0.749	0.719	0.873	0.853
DNV_TL_3	62	resnet101	0.729	0.704	0.853	0.826
DNV_TL_4	60	resnet50	0.741	0.714	0.809	0.791
DNV_TL_5	63	resnet101	0.765	0.741	0.865	0.849
DNV_TL_6	64	resnet50	0.757	0.727	0.877	0.859
DNV_TL_cv_7	cv_7	resnet50	0.756	0.726	0.807	0.786
DNV_TL_cv_12	cv_12	resnet50	0.755	0.725	0.807	0.791