Nbdev Tutorial

Resources for the nbdev + Quarto tutorial at JupyterCon, 2023.

Nbdev is an exciting literate and exploratory programming framework. With nbdev, you write your tests, documentation and software in one context: a Jupyter Notebook. Nbdev leverages Quarto to render documentation sites giving you additional power to customize your documentation.

In this tutorial, we will walk you through how to use nbdev from the ground up, and provide an overview of some of the underlying technologies such as Quarto and execnb.

Resources For This Tutorial

  • The slides for this talk.
  • The GitHub repo with all of the examples and the source code for this site.

Optional Pre-requisites

Optional because it’s best to watch and try this yourself at home.

  1. Install Quarto.
  2. Install JupyterLab with pip install jupyterlab.
  3. Install nbdev with pip install nbdev.
  4. Install VSCode by following the official docs.
  5. Install the nbdev VSCode extension 1.
  6. Install the Quarto JupyterLab extension.
  7. Optionally, install the github cli.

1. Intro (20 minutes)

Motivation for nbdev – literate & exploratory programming (slides).

2. Quarto Walkthrough (40 minutes)

  • Create a new project using quarto project create, see the _quarto.yml file.
  • basics.ipynb
    • No execution by default
  • cell-options.qmd
    • The qmd format is interesting and mirrors the experience of RStudio to a large extent
    • Running cells:
      • CMD + SHIFT + Enter
      • Jupyter cache (pip install Jupyter Cache)
    • Change value in cell (execution by default vs. notebooks not executing by default)
    • Quarto: Assist Panel, show how it works (only in .qmd now)
    • Customizing output
      • Code cell options:
        • echo: true|false|fenced
        • include: false
        • output: true|false|asis
      • Front matter
        • code-fold: true
        • code-tools: true
      • Other important cell options
      • General HTML options
    • Hotkeys:
      • CMD + SHIFT + O; outline navigator
      • CMD + T; see tags (project wide)
  • Other VSCode extension features
    • Options
    • Visual mode
      • Editor Toolbar: makes authoring tables so much nicer than plain markdown!
      • Remapping keys and toggling back and forth. superkey is a great tool for this.
  • Different formats
  • Some of my favorite features
    • annotation.qmd
      • You can make multi-line by commenting twice
    • Conditional formatting
    • Testing your docs by executing code:
      • Set execute: enabled: true in _quarto.yml or pass the—execute flag to quarto
      • Use the execution: freeze: true if you need to opt a notebook out of execution. reference.
      • The Jupyter cache can be helpful while developing: reference.
    • External listings example | docs.
    • Notes/TILs:
    • Notebook filters:
  • Publishing & Previewing
  • Configuration: _quarto.yml, front matter etc
    • Global options: _quarto.yml: docs
    • Page front matter: docs
    • Overriding global options per directory: docs
    • Repeatable options: listing_meta and merging that with metadata-files front matter
  • Where to get help:

3. nbdev Walkthrough (60 minutes)

This walkthrough is largely based on the official end-to-end walkthrough and notebook best practices guide.

  • Installation
  • Setup your nbdev project
    • Create an empty GitHub repo
    • Enable GitHub Pages
    • Initialise your repo with nbdev_new
    • Check out your workflows
      • CI – The CI (continuous integration) workflow streamlines your developer workflow, particularly with multiple collaborators
      • Deploy to GitHub Pages – Builds your docs with Quarto and deploys it to GitHub Pages
    • Check out your docs
  • Make your first edit
    • Install hooks for git-friendly notebooks
    • Build your library
    • Install your package
    • Preview your docs
    • Edit 00_core.ipynb – adding your own package’s code
      • #| default_exp directive (docs)
      • Add your own frontmatter (nbdev vs Quarto frontmatter formats) (docs)
      • Add your own function using the #| export directive (docs)
      • Add your own examples, tests, and docs using assert and fastcore.test (docs)
    • Prepare your changes with nbdev_prepare, a shortcut for:
      • nbdev_export: Export .py modules from .ipynb notebooks
      • nbdev_test: Test notebooks
      • nbdev_clean: Clean notebooks
      • nbdev_readme: Update README.md
    • Edit index.ipynb – your home page
    • Push to GitHub
    • Check the updated repo and docs
  • Advanced functionality
    • Doclinks: referencing related symbols (docs)
    • Docments: documenting callable parameters (docs)
    • Documenting error-cases as tests (docs)
    • @patch: Define classes across multiple cells (docs)
    • show_doc: Document external objects (docs)
    • Putting it all together: nbdev vs classical Python development (docs)

Awesome nbdev projects

By fast.ai

  • nbdev: The nbdev software development framework is itself an nbdev project
  • execnb: Fast and lightweight Jupyter notebook execution engine
  • fastai: Simplify training fast and accurate neural nets using modern best practices
  • fastcore: Python goodies to make your coding faster, easier, and more maintainable
  • ghapi: A delightful and complete interface to GitHub’s amazing API
  • ghtop: A real-time terminal user interface (TUI) for GitHub

By the community

  • AlphaPept: A modular Python-based framework for mass spectrometry
  • FastKafka: Build asynchronous web services that interact with Kafka topics
  • InstructGOOSE: Implementation of Reinforcement Learning from Human Feedback (RLHF) from the InstructGPT paper
  • nbdev-mkdocs: Extend nbdev to generate docs using Material for Mkdocs
  • nbdev-extensions: A collection of nbdev extensions
  • nbstata: A Jupyter kernel for Stata built on pystata
  • streamlit-jupyter: Preview and develop streamlit applications in Jupyter notebooks
  • sveltish: Svelte Stores in Python
  • UPIT: Unpaired image-to-image translation using fastai/PyTorch

Additional Reading