Runs Results from All Three Utilities

This markdown contains the F1 and F2 scores on validation and test data of each run. The model types are listed from 1 to 16 and TL1 and TL2 are models that I used for transfer learning. The information of each model can be found in the model configuration table.

Model Configuration Table

Model_Type	backbone	train/test split	pretrained?	epoch	freeze_epoch	image_aug?	weighted loss?	Note	Status
1	resnet50	80/20	TRUE	10	10	FALSE	FALSE	Baseline model	done
2	resnet50	80/20	TRUE	10	10	TRUE	FALSE		done
3	resnet50	80/20	TRUE	10	10	FALSE	TRUE
4	resnet50	80/20	TRUE	10	10	TRUE	TRUE
5	resnet50	80/20	FALSE	10	10	FALSE	FALSE
6	resnet50	80/20	FALSE	10	10	TRUE	FALSE
7	resnet50	80/20	FALSE	10	10	FALSE	TRUE
8	resnet50	80/20	FALSE	10	10	TRUE	TRUE
9	mobilenet_v3_Large	80/20	TRUE	10	10	FALSE	FALSE		done
10	mobilenet_v3_Large	80/20	TRUE	10	10	TRUE	FALSE
11	mobilenet_v3_Large	80/20	TRUE	10	10	FALSE	TRUE
12	mobilenet_v3_Large	80/20	TRUE	10	10	TRUE	TRUE
13	mobilenet_v3_Large	80/20	FALSE	10	10	FALSE	FALSE
14	mobilenet_v3_Large	80/20	FALSE	10	10	TRUE	FALSE
15	mobilenet_v3_Large	80/20	FALSE	10	10	FALSE	TRUE
16	mobilenet_v3_Large	80/20	FALSE	10	10	TRUE	TRUE
TL 1	cctv-multilabel-dnv-80-C-80_20-cv3-resnet50-prT-e10-fe10-augF-weightF-4gpus							for transfer learning	done
TL 2	cctv-multilabel-dnv-486-C-80_20-cv3-resnet50-prT-e10-fe10-augF-weightF-4gpus							for transfer learning	done

DNV

Compare results between different model types

1. Data extracted from 80 videos

Model Type	1	2	9
F1 normal validation score	0.969	0.968	0.964
F1 no ND validation score	0.963	0.966	0.957
F2 no ND validation score	0.948	0.953	0.940
F1 normal test score	0.983	0.980	0.973
F1 no ND test score	0.954	0.969	0.945
F2 no ND test score	0.938	0.959	0.924

2. Data extracted from 486 videos

Model Type	1	2	9
F1 normal validation score	0.980	0.975	0.969
F1 no ND validation score	0.940	0.943	0.932
F2 no ND validation score	0.930	0.933	0.917
F1 normal test score	0.975	0.968	0.957
F1 no ND test score	0.948	0.948	0.930
F2 no ND test score	0.933	0.930	0.910

Compare results between the baseline model and the model trained on all three utilities data

1. Data extracted from 80 videos

Model	Baseline model	Combined-692 Model
F1 normal test score	0.983	0.984
F1 no ND test score	0.954	0.962
F2 no ND test score	0.938	0.949

2. Data extracted from 486 videos

Model	Baseline model	Combined-692 Model
F1 normal test score	0.975	0.972
F1 no ND test score	0.948	0.935
F2 no ND test score	0.933	0.915

COV

Compare results between different model types

Model Type	1	2	9
F1 normal validation score	0.958	0.959	0.948
F1 no ND validation score	0.847	0.848	0.827
F2 no ND validation score	0.828	0.831	0.806
F1 normal test score	0.957	0.960	0.950
F1 no ND test score	0.931	0.934	0.925
F2 no ND test score	0.913	0.918	0.904

Compare the results between baseline model with transfer learning models

Model Type	1	TL 1	TL 2
F1 normal validation score	0.958	0.964	0.967
F1 no ND validation score	0.847	0.854	0.864
F2 no ND validation score	0.828	0.835	0.848
F1 normal test score	0.957	0.963	0.965
F1 no ND test score	0.931	0.936	0.939
F2 no ND test score	0.913	0.920	0.925

SD1

Compare results between different model types

Model Type	1	2	9
F1 normal validation score	0.978	0.978	0.971
F1 no ND validation score	0.899	0.906	0.910
F2 no ND validation score	0.882	0.896	0.895
F1 normal test score	0.977		0.970
F1 no ND test score	0.910		0.906
F2 no ND test score	0.897		0.894

Compare the results between baseline model with transfer learning models

Model Type	1	TL 1	TL 2
F1 normal validation score	0.978	0.982	0.984
F1 no ND validation score	0.899	0.944	0.945
F2 no ND validation score	0.882	0.935	0.937
F1 normal test score	0.977	0.981	0.983
F1 no ND test score	0.910	0.964	0.966
F2 no ND test score	0.897	0.953	0.956