nbdev

nbdev setup on WSL

On Ubuntu 22.04

1. Install Ubuntu 22.04 on WSL in Windows if you have not already done so.
2. Open the WSL window
3. Setup your github SSH keys.
4. Install fastsetup package from fastAI.

   git clone https://github.com/fastai/fastsetup.git
   cd fastsetup
   source setup-conda.sh
   . ~/.bashrc
   conda install -yq mamba
5. Setup your python environment

   conda create -n nbdev python=3.9
6. (Optional) Add the alias ca for conda activate in your .bashrc

   alias ca="conda activate ''"
7. Install nbdev

   pip install nbdev
8. (Optional) Create .tmux.conf file in your home directory and copy the content of this dot file.
9. (Optional) Use tmux to work on your development.

Development with nbdev

1. If Jupyter notebook is not installed. Install it first.

   pip install notebook
2. Create a git repo. (Initializing it in github and cloning would be ideal as nbdev will automatically pickup information from there.)
3. Check the next steps from this tutorial.

nbdev + Colab

Pushing a new notebook to github and then pulling it to colab is a tedious process. Since Colab is a transient environment, you can't really use SSH keys to push/pull from github. Hence, we need to use Personal Access Tokens (PATs) and clone a repo on the Google Drive with https.

To do these git operations, we need to use terminal in Colab and the terminal is only accessible in Colab PRO accounts.

To upgrade to Colab Pro, you need to upgrade your current Colab subscription. GQC already has a Colab Pro subscription.

Once you have the Colab Pro Subscription, you can access the terminal from the bottom left corner.

This assumes we already have setup a git repo with nbdev template. Written on the basis of migrating already existing colab notebook to nbdev. Requires a Colab PRO account as we need terminal access in Colab.

In Colab

1. Open a new Colab notebook and run following.

   from google.colab import drive
   drive.mount('/content/drive')
   !pip install nbdev
2. Open the terminal in Colab and goto /content/drive/MyDrive/repos directory.
3. Clone the required repo via HTTPS (Pick the git clone command which has the https address). Provide your github user-name when asked for that and provide a token-key which you can create from github.com as Password. (See here if you don't already have a github token)
4. Create and checkout a new branch for yourself. Name it like colab-pavan,colab-deven, etc.
5. Using Google Drive, move any colab notebooks to the appropriate sub-folder in repo's nbs folder.
6. Rename the Colab notebooks if necessary.
7. Then git add the new notebooks, git commit them and finally git push them.
8. After the data is pushed to the new branch, we have to merge the new branch to the main branch. That can be done by either of,
   1. Creating a Pull Request through GitHub.
   2. Merging the changes manually by pulling both main and colab-<your_name> branch to WSL
9. Continue working on cleaning up on the notebooks through WSL by pulling in the latest changes to it.

In WSL

tip

nbdev has a slightly steep learning curve.
Before you start developing with nbdev, it is best that you either finish the written tutorial or video tutorial completely.

caution

1. nbdev is a python package not a command.
2. If you wish to see if nbdev is installed in your virtual environment, type pip show nbdev.
3. If you were to run nbdev --help in your terminal, you will get a command not found error.
4. Once, nbdev is installed; in the terminal, run nbdev_help to see all the available commands.
5. Sometimes, running nbdev commands on notebooks with outputs can throw errors and result in nbdev CI failing. This can be hard to trace back with the non-descriptive errors. In such cases, try clearing outputs before using nbdev_export or nbdev_clean.

1. Pull in the latest changes to the repo.
2. If not done already, create a new python virtual environment for the repo. For example:

   conda create -n gqc_utils python=3.9
3. Activate the python environment.

   ca gqc_utils
4. If running for the first time, install nbdev and upgrade quarto in the virtual environment.

   pip install nbdev
   nbdev_install_quarto
   nbdev_install_hooks
5. If not already done, install the dependancies of the current package (which are specified in the settings.ini) by

   pip install -e .
   or install submodule specific dependencies via

   pip install -e '.[vision]'
6. Useful nbdev commands (See a comphensive list here)
   1. nbdev_clean: Cleans notebook runtime metadata which causes merge conflicts if not cleaned.
   2. nbdev_preview: Locally builds and serves docs.
   3. nbdev_export : export the notebooks to py files
      1. Add the #|default_exp foo as the first cell on top of each notebook. (This will export the contents of the notebook to a py file called foo.py)
      2. Add the #|export flag on top of each cell that you wish to export.
      3. Finally, from the root of the repo, run the command nbdev_export to export the notebooks to py files.
      4. Optional: Learn about other nbdev flags → here
   4. nbdev_prepare : To export, test and clean all the notebooks.