Skip to main content

Week 03

Sept 12 2022​

CCTV​

  • Find the optimal image capturing rate so that we will not produce the images of the same frame. (Depend on the movement of the camera, frame rate).
  • Create sampling image of sewer-ml by grabbing 3000 or 5000 images that represent the same histogram of the whole sample.
  • Create sampling image via random sampling (eg. randomly grab 3000 images between image 1 to image 40000).
  • Find the optimal image size (resize) and batch size. This will affect GPUs memory.

Pipe Breaks​

  • Sort out email from Eric. (his response regarding water quality)
  1. Will Random Survival Forest benefit from using GPU?
  2. Implement jit to RSF to speed up the computation time? Deven said jit is already implemented in RSF.

Note​

  • Convert excel (containing all my tasks) to markdown

EVE presentation​

  • Include table 5-7 and table 5-1 from Eric Thesis.
  • Create GIS map showing top 10% of total pipes length that are likely to break.

Sept 13 2022​

  1. Discuss about EVE presentation

Collimator (BSM2)​

  1. Use Ideal sensor for now
  2. Hyddelay was used to break the algebraic loop in matlab. Need to find out whether this will be a problem with collimator. If no, get rid of hyddelay.

CCTV (cont)​

  1. Read this https://docs.fast.ai/torch_core.html#set_seed to understand on how to set seed to produce reproducible results. Deven will help with this tomorrow.
  2. Looking at train_loss and valid_loss to valid whether set seed works
  3. Need to set up new shellrunner and script runner
  4. Might need to git clone the sewer-ml github repository instead upload zip file to compute canada to be more efficient
  5. Hyper parameter training. Find out what threshold of F1 score the Sewer-ML used. Increase the threshold to see if F1 score performs better.

Sept 14 2022​

  • Create Python code to generate sample images

  • Will need to modify code to find the optimal learning rate

  • Figure out how to set seed

  • Sign up for Sewer ML leader board thing

  • Submit the result to leader board. Could not submit to the leaderboard at this moment because the link is broken.

  • Split polygon into multiple polygons using FME

  • Add some information related to the buildings (building id, centroid) in respective points. The file has projection coordinator 6527 New Jersey.If you have whitebox tools already installed in QGIS, you can try LidarRooftopAnalysis to find the maximum elevation of the building polygon. It is not what you guys asking for but it might be useful. Here is the link to the tutorial video: https://www.youtube.com/watch?v=V6VJxZyyTEg

  • EPANET and Collimator

  1. Use FFmpeg to create images from video https://ffmpeg.org/
  2. Link to on how to extract image for a video with ffmpeg at a given time https://stackoverflow.com/questions/27568254/how-to-extract-1-screenshot-for-a-video-with-ffmpeg-at-a-given-time
  3. Use cmd mkdocs serve to open the note on the browser
  4. add commit message: type fix or fixes #"issue number" to close the issue on GitHub

Sept 15 2022​

  • Ran fastai model with sample images (one from train00, one from train01, one from train00 and train01). Each run has around 3000 images
Last updated on by Viraj