Webinar: Grading Jupyter Notebooks

Formatting a Jupyter Notebook for easy and effective autograding

The easiest and most common way to autograde Jupyter Notebooks is to convert them to code, i.e. appending all code cells to form a regular Python script. In CodeGrade, we can then assess this Python script like we would normally do.

There are a couple of ways to easily assess multiple assets of a Jupyter Notebook assignment in CodeGrade:

• We can assess the content of variables for correctness
• We can assess the correct working of functions
• We can run the whole sequence of code cells and check the output as a whole

In the webinar, we show you step by step how you can set up an AutoTest in CodeGrade to autograde these parts of a Jupyter Notebook assignment. What is important to note is that you need to know the names of the variables and functions to be able to assess them. A good practice is to provide your students with a template Jupyter Notebook file with predefined variables and functions that they can fill in themselves.

Autograding Jupyter Notebooks in CodeGrade

Autograding Jupyter Notebooks in CodeGrade becomes easy when formatting your assignment as explained above. The steps to do so are shown in the webinar, to summarize:

1. Convert the `.ipynb` file to a Python script in a Run Program step using `jupyter nbconvert --to script jupyter.ipynb`. (change the name `jupyter.ipynb` to the name of your Jupyter Notebook assignment, you can enforce this using Hand In Requirements)
2. Create an I/O Test in the same that runs `python -i jupyter` (change the name `jupyter` to the name of your assignment, this is the file you generated in the step above).
3. In this I/O test, you can now use regular Python as input and assess individual parts of the assignment. For instance, printing a variable or running a function and printing its result. You can write the expected output in the "Expected Output" field.

Ignoring superfluous print statements

One thing to be aware of is that we are checking the stdout of the scripts, which are run completely when importing. As a result, students can clutter the output with additional print statements outside of functions. There’s two ways to prevent this:

1. Importing the script with the stdout redirected. This can be done using this little code snippet, which you run via `python3 -i import_without_print.py`:

-!- CODE language-python -!-from contextlib import redirect_stdout
from os import devnull

with redirect_stdout(open(devnull, 'w')):
    import jupyter #<-- The name of your script

2. If you are providing Jupyter Notebook skeleton code to your students, you can make sure to add a `if __name__ == "__main__":` statement every time you print solutions. This way, the students can see their solutions while interacting with the notebook, but these solutions will not be printed when importing the code (using the script above or with a regular import).

‍

The above just lists a brief overview of what is discussed in the webinar. In the webinar, we went into detail on all things grading Jupyter Notebooks and went over all the steps on how to reproduce this for your assignment. We are here to make your course with CodeGrade a success, so please do not hesitate to reach out to us on Discord or via support@codegrade.com with any question, feature request or if you would like some technical or educational consultancy. We are always more than happy to help you out!