Automatically grading PHP code assignments

Input/Output autograding

Many programming courses, particularly introductory courses, teach students how to create programs and interact with them through the command line interface (CLI). For instance, a simple program might prompt a user to provide a number and, in return, print a statement telling the user that their number is odd or even. Input/Output (I/O) tests are a great way of checking that these simple programs work as expected. However, this would traditionally require teachers to download each of their students’ submissions and run them independently on their local environment, putting inputs to the program one at a time and making themselves vulnerable to human error.

Fortunately, CodeGrade’s I/O test step allows you to run a program as you would on your own CLI, enter an input (either standard input, stdin, or as input argument) to the program, and test that the predicted output matches the actual output. With CodeGrade, this doesn’t have to be an exact match, but can be made more flexible by the teacher by modifying case sensitivity, substring matching, whitespace sensitivity or by using a regex.

Unit Test autograding

Once courses begin to venture into more complex programming assignments, particularly those with functions and other encapsulated code, unit testing is a natural tool for autograding. Unit testing frameworks often enable multiple ways of testing that aren’t just limited to matching a predicted outcome with an actual outcome. For PHP, CodeGrade has support for PHPUnit out of the box. But frameworks like Codeception Selenium or Atoum can be installed too.

CodeGrade’s dedicated Unit Test Step parses the XML output of your unit tests and displays each test individually and whether or not they passed. The Unit test step makes customizing your unit tests really easy, enabling you to write custom descriptions for each test, weighting them differently, and even providing hints when the tests fail!‍

Code Structure autograding

A type of autograding that is exclusively offered by CodeGrade’s AutoTest is easy code structure checking. For this, we have implemented our own version of the powerful Semgrep tool. 

With this, you can now not only check if your students’ code works and whether the style is correct, but you can also check if they actually solved your coding challenge in the way that you want. You can automatically check if they use prohibited libraries, if they actually use a for-loop and not a while-loop and if they did not hardcode their answer but rather solved the problem. What’s more, you can also make sure that students define the correct variables, and functions, to make sure they get a helpful warning if they did not.