Skip to main content

Common Practices at GQC

Below are some of the common practices we follow on Ubuntu and windows for handling and maintaining data in local and cloud.

Common Mistakes

  • While zipping data folders through CLI, make sure to provide relative paths as source instead of absolute paths. Other wise, the folders are on unzipping will have folder name as the absolute path provided and not just the file name.

  • Sample example for Ubuntu CLI is here:

        # Wrong
        # The folder name on unzipping will be /home/gqc/git/gqc/some-project/data
        zip -r input_data.zip /home/gqc/git/gqc/some-project/data

        # Correct
        cd /home/gqc/git/gqc/some-project
        zip -r input_data.zip -d ./data

  • Working directory can be anywhere as long as relative paths are provided.

Local​

  • Make sure data of large size is either on hard drives or SSD, instead of copying to C:Drive or local drives.

Cloud​

Github​

In cases where we need to track data files that are large in size or > 100 MB, we need to use git-lfs. Documentation of the same is available here: git-lfs