CCTV Workflow Documentation

• The following is based on the current understanding on the work flow of CCTV apps.
• All the scripts mentioned here are available in gqc-utility-notebooks repository.
• It considers the latest workflow of using DB tables instead of CSV files.
• This document has the workflow exemplified using SD1D or May 26 2023 upload for understanding purposes.

Current Workflow

1. Data (videos and access DBs) is uploaded to Google drive on hydrotrekai@gqc.com, in this case the SD1 CCTV May 25 2023 upload.

2. The folder structure on which we work on MSI is below:

   ── git
        └── UTILITY_X
            └── Data
                └── Dataset_X
                    ├── Blurred_Frames
                    ├── Extracted_Frames
                    ├── Logs
                    ├── Received_Data
                    ├── Temp
                    ├── Video_DB
                    |     └── Video_DB.db
                    |           ├── video_groups
                    |           ├── distance_bounding_boxes
                    |           ├── video-groups-1_frames
                    |           ├── video-groups-2_frames
                    |           ├── video_defect_type
                    |           └── PACP_Condition_Code
                    ├── Video_Lists
                    └── cctv_settings.yml

3. The Videos uploaded to Google Drive are downloaded using rclone to the Recieved_Data folder.

4. We start by manually performing Distance Region extraction and Defect type annotations on videos and save the results to a DB called Video_DB.db under Video_DB folder.

5. Before we start populating tables using scripts, the DB has distance_bounding_boxes, video_defect_type, and PACP_Condition_Code. These are independent for intial steps mentioned in #8.

6. Creation and testing RegEx expressions for testing video types based on utility. This is being done manually and is dependent on the video_defect_type

7. Meanwhile, we run the metadata_DB_extract_inspection_condition_data on the access DB tables to get a CSV called all-conditions.csv. This is also independent for intial steps in #8 and is valid for SD1. -The schema of all_conditions.csv is below (reminder that this is specific to SD1 May 26, 2023)

      | all_conditions                   |
      |---------------------------------   |
      | Inspections_InspectionID           |
      | ConditionID                        |
      | Conditions_InspectionID            |
      | MediaID                            |
      | Media_Inspections_InspectionID     |
      | Video_Name                         |
      | Video_Location                     |
      | Distance                           |
      | Counter                            |
      | PACP_Code                          |
      | Continuous                         |
      | Inspections1_InspectionID          |
      | Conditions1_InspectionID           |
      | Media_Inspections1_InspectionID    |
      | Inspections2_InspectionID          |
      | Conditions2_InspectionID           |
      | Media_Inspections2_InspectionID    |

8. Start the CCTV usage notebooks run on the videos under Received_Data folder as follows:

   1. Setup YAML parameters with paths and provide the path to YAML file while running following scripts.

   2. Form video groups using video_list_creator script and save the output to video_groups table.

   3. Perform Frame extraction based on the video groups.

      • Using frame_extractor script, the resulting frames are saved to Extracted_Frames folder.

      • Corresponding data is saved to respective tables in format video-group-x_frames. The fields in the video_group_x_frames tables are below:

        TABLE "video-group-1_frames" (
           "video_id"              : TEXT,
           "frame_id"              : TEXT,
           "condition_code_list"   : TEXT,
           "OCR_distance"          : FLOAT,
           "distance"              : FLOAT,
           "distance_LF"           : TEXT,     # should be FLOAT
           "OCR_JSON"              : JSON,
           PRIMARY KEY("frame_id")
        );

   4. Run stitch_and_ocr script to generate stitched frames and send to the Azure OCR server. The resulting output is populated in the OCR columns in the video_group_x_frames tables.

   5. Then we use blur_frames script to create blurred frames ZIPs for each video group, saved in the Blurred_Frames folder.

   6. From the blurred frames, we extract distances using extract_distance_values script, that saves the output to Extracted_Frames as ZIPs. We populate the video_group_x_frames tables with the same.

   7. Using the video_defect_type table, we then extract condition data by running extract_condition_codes script, which populates the condition_code_list in video_group_x_frames tables.

9. After the above step, we have video_group_x_frames tables with required data to run the final step before training i.e; labelling the frames.

10. Use create_SD1_label_images script to generate labels for frames using 2 inputs - distance labels and Conditions data.

    • It is unclear what the inputs contain specifically.

11. Once we are done labelling, we can proceed to training.