Week 04

Nov 21

Collimator

• The authors of BSM2 rewriting an ODE system to a differential algebraic equation (DAE) system to reduce stiffness of ADM1 model.
• However, we ran into the algebraic loop problem because the initial state of each variable was not defined correctly.
• The only way I can think of right now is we need to convert the system back from DAE to ODE. In this way, we can use integrator block to initialize the initial state value of each variable.
• I have converted pH_solver block and Sh2_solver block back to ODE system. However, the simulation was aborted due to time out.

Tomorrow tasks

• Write up the process to clean up access database
• implement cross-validation in fastai
• write up concern regarding train-validation-test data
  • get the dataset quantify
  • how many defect in test data out all test dataset
  • Generate a dataframe containing for 75-15-10 split runs :
    • video name
    • defect code
    • number of frame
    • location where are those frames? validation, train, or test dataset?
• write up how to set aside the testing dataset
  • get the distribution of pipe material. Set pipe materials that have more than two videos and set 2 videos from those list of videos for testing dataset. (Different pipe materials have different colour. We want to make sure that training includes all different type of pipe materials)
  • generate the distribution of defect in the test dataset and training/validation dataset.
• improve the document on the prediction run
  • write where the 10% test dataset are coming from
  • write the input dataset
• change learn.remove_cv(WandbCallback()) to learn.remove_cb(WandbCallback)

Nov 22

Collimator1

• It requires a long computation time when solving the 6 variables using ODE system.
• I was able to fix the problem with the pH solver. There was no Algebraic loop problem when I input the pH solver inside ADM1 (unlike the Sh2 solver). Furthermore, the run was faster when the variables inside the pH solver were solved using the DAE equation than the ODE equation. Therefore, we did not have a problem with time out anymore.
• After mapping all the output from ADM back to ASM using python script, I have finished the Anaerobic digester submodel block.
• create list of issue in collimator project

CCTV

• write up concern regarding train-validation-test data
  • get the dataset quantify
  • how many defect in test data out all test dataset
  • Generate a dataframe containing for 75-15-10 split runs :
    • video name
    • defect code
    • number of frame
    • location where are those frames? validation, train, or test dataset?
  • generate the distribution of defect in the test dataset and training/validation dataset.
• link talk about different between access database and annotation on the video
• link to my note regarding the tasks above

Nov 23

• Sending TSS and Q CSV to Pavan to create grafana dashboard.
• Sending email to Pavan asking how we can get output directly from submodel to cloud based TDengine database, and connect TDengine database to Grafana.
• Debugging BSM2 model on collimator.
• Refactoring the BSM2 model on collimator.
• Cross-validation
  1. problem/concern
     1. Multi labels: if there is label in validation that is not in training, how to deal with it? Do we drop it?
     2. How other people prepare the data for cross-validation for multi label (do some research)
     3. How do people do cross-validation based on different metric score: the f-score instead of accuracy. This can post a problem if we have different label in validation and training dataset.
     4. link to cross-validation
        1. Deven: It's a good resource. The only problem is the license under which the code is share. The code in this repo is shared under Attribution-NonCommercial-ShareAlike 4.0 International.
     5. Use Stratified method

Nov 25

1. Debug why Soluble output is negative. The problem is likely to be due to Activated Sludge block
   1. Start with So4 control in Activated Sludge
   2. Redesign the bioreactor block. Currently bioreactor4 outputs So = 300. The So should be around 2.
      1. The model was compiled successfully when replacing bioreactor1 with the new design.
      2. There is an algebraic loop when replacing bioreactor 3-5 with the new design due to we are feeding KLa back to bioreactor 3-5.
      3. The old design did not have produce this problem because I did not implement a block/code to check if KLa is negative before assigning So value.
      4. In Matlab, they have a concept of direct feedthrough to handle this problem
      5. The algebraic loop problem is solved when adding unit delay block in collimator.
      6. The outputs are no longer negative. However, the outputs are still incorrect
2. Refactoring the model
   1. Rename the block
   2. Rearrange the block
3. Email Brian regarding CODE GENERATION
   1. Brain's ans: it will generate c-code for that model. Only the discrete blocks are currently supported