Skip to main content

Week 02

Jan 9​

collimator​

  • Interpolation the output of collimator so that it matches the input's time interval

  • Replace the biorector 1 with the AI model

  • Pickle file: working with 15 variables (exclude Flow)

  • Output: need to map that 15 variables

  • AI collimator issue#1 is in progress.

    • Was able to run the model with pkl file successfully with sample input (one variable with 11 delay blocks)
    • Finished mapping the output from prediction to collimator format
    • Working on mapping the collimator input to prediction model input
    • Working on creating total Delay block so that it can be applied for other variables

CCTV​

  • Run 40, 41, 45, 46 - models trained with sharpened image performed better on test dataset than models trained with non sharpened images
  • Run 39, 43, 44, 47, 48 - models trained with non sharpened image performed better on test dataset than models trained with sharpened images
  • Run 42 test dataset did not have image with defect

Jan 10​

  • observation: Runs that contain label 'TBA' and 'TBB' have lower score than those runs that contain only 'TB' and 'TF'. Maybe we should not be specific whether TB is abandoned or active?

  • Is it legitimate to apply sharpening to predicted image?

  • Write idea what I want to do for CCTV?

  • Planning to finish of what I am doing right now. Present it to Sean and asking Sean for more data. Ask him if he knows other utility
  • Need to come up with features engineer part of the model
  • Get a status report of thesis next two weeks. Send her new thesis report.
  • Dr. Lence will be away til Feb 10.

Jan 11​

  • Model: soluble-particulate-t-TSTPlus-01-11-23.pkl

  • When having input as T, Particulate, Soluble:

    • T is negative
    • Soluble variables are larger than the expected values. The values are in hundreds
    • Ss is around 30. The values should be around 3
    • Xi and Xbh are negative
    • TSS is in tens
    • Xs is around 200 (should be around 60) and Xnd is around 50 (should be around 3)
    • Xba is negative

  • When having input as T, Soluble, Particulate:

    • T is in the right magnitude
    • Xi, Xbh, Xp, TSS are around 200. The values should be in thousands
    • Xs, Xba, and Xnd are in the right magnitude
    • All soluble variables are in the right magnitude.
    • So can go below zero (need to add a line to prevent that)

  • Model: soluble-particulate-t-InceptionTimePlus-01-11-23

  • When having input as T, Particulate, Soluble:

    • T is around 700
    • Soluble values are larger than the expected values. The values are in thousands and ten thousands. Snd values are around 60.
    • Xi is around 1300. The expected value is around 1500.
    • Xs, Xba, Xnd are larger than the expected values.
    • Xbh, Xp, and TSS are lower than the expected values.

  • When having input as T, Soluble, Particulate:

    • T is in the right magnitude.
    • All soluble variables are in the right magnitude. So can go below zero.
    • Xi, Xbh, Xp, TSS are around 300. Xi, Xbh, and TSS should be in thousands. Xp should be around 900.
    • Xs, Xba, and Xnd are in the right magnitude.

  • When I fed in the input with the order of T, Xi, Xs, Xbh, Xba, Xp, Xnd, TSS, Si, Ss, So, Sno, Snh, Snd, Salk:

    • the outputs were not in the correct magnitude.

  • However, when I fed the input with the order of T, Si, Ss, So, Sno, Snh, Snd, Salk, Xi, Xs, Xbh, Xba, Xp, Xnd, TSS:

    • temperature and soluble variables were in the correct magnitude.
    • Xs, Xba, and Xnd are in the correct magnitude.
    • Xi, Xbh, Xp and TSS, in which the expected values are in the thousands, were around 200 (TST model) and 300 (InceptionTime)

  • 15 inputs, single output

CCTV​

  • Review the videos SewerVideoUploads-Google Drive
  • Create items in Github that define how we will apply our AI models to these videos.
  • After Deven and I get chance to review those items, please predict the defects in these videos.

DNV streamlit​

  • The main issues I have right now are:

    • The image with space in its name cannot be read by the Sewer defect model (just an observation).
    • Is there an easy way to refer to different folder paths for unblur and blur directories? I want to have the ability to switch between DNV and SD1 images without changing their image folder name to unblur and blur. The same goes for the image path in CSV files.

  • I got an error regarding the keyname names vs fname because I selected "Yes" to "CSV with labels" when I actually uploaded the CSV file without labels.

  • How to run DNV streamlit

    • need csv that contains the path to image for prediction (shown on the top carousel)
    • need csv that contains the path to image inside unblur folder to view them at the bottom carousel
    • copy images inside blur folder (for image prediction; top carousel)
    • copy images inside unblur folder (for bottom carousel when you want to view nearby images. The order of image is depended on the order in un-blur.csv
    • Go to dnv-streamlit directory
    • In the terminal, activate environment that contains all necessary python library.
    • In the terminal, type "streamlit run app.py"

Jan 12​

Jan 13​

  • Leak modelling with EPANET

  • Mount_Washington_RMX_PumpCurv_09-19: without pump curve fix

  • Run the SewerML model on DNV images and vice versa

  • There was a simulation time-out when I ran the collimator model for 2 days end time. Therefore, I just ran it for 1 day end time for now. The csv files can be found here: https://drive.google.com/drive/folders/1jTTYi-af80DzHKItXVaa02UzEwkDbcMi.

  • As for EPANET, I have finished reviewing my material. I have recreated a leak at J-224 using Mount_Washington_RMX_PumpCurve_09-19.inp (the pump curve is not fixed). Please see the attached powerpoint for the results. Last time, we found that if we changed the time step from 5min to 1 hour when running PDA, the flow would not be zero in the system graph. I also compared the results between the two time steps (5min vs 1hour).

  • Simulate leak for ~400GPM (12 inch) assume 4h break? https://www.denverwater.org/tap/main-breaks-101-raising-our-infrastructure-gpa#:~:text=8%2Dinch%20main%20break%20%3D%2035%2C000,main%20break%20%3D%2045%2C000%2D378%2C000%20gallons

  • how to simulate leak in the middle of the situation

  • average water shut-off time. Contact Deven

  • format letter to ask for Sewer CCTV image from utility

  • pick public website and send format letter out city near us.

  • Sarah.Wells@vancouver.ca (COV) (in charge of CCTV)

  • Stadel, Cassidy (Cassidy.Stadel@vancouver.ca) (COV)

  • Reach out to Sean if he knows if utility that would like to do research with

  • any other city who is helpful where I can get CCTV image from

  • send email Deven can you send CCTV from Jack's video

  • how to double the image when applying transform in fastai?

Last updated on by Viraj