Autograding Code Quality and Common Bad Practices in Java Assignments

Setting up a basic Code Quality test

After choosing the Code Quality test in your AutoTest category, you can simply select a linter from the dropdown list. Choose Checkstyle or PMD for your Java assignment. After doing so, you can select your style guide. You can upload and use a custom one, or simply use one of the default style guides available. For Checkstyle, we recommend the very common Google style guide, based on the style guide universally used at Google with hard-and-fast rules, or Sun style guide, a more traditional style guide enforced by Oracle / Sun that is last revised in 1999, which can be downloaded here. For PMD, we recommend Apache Maven’s style guide, which can be downloaded here.

After selecting your style guide, you can optionally provide extra arguments for the selected tool and update the penalties for the different message severity levels. If you wish to just inform your students of style and not grade them, these penalties can all be set to 0%.

Designing a custom style guide for educational purposes

Autograding code quality can become even more effective when using a custom style guide. Custom style guides are relatively easy but time consuming to make, luckily, they lend themselves for iterative development and can easily evolve with your course. Just some of the benefits you can get with custom style guides for education are:

• Removing rules that are unnecessary, too advanced or too complicated for students
• Adding rules that may be unconventional for businesses but useful for education
• Creating a very small style guide with the most important and basic rules for introductory courses
• Creating a very specialized style guide for advanced courses (e.g. with common bad practices in multithreading)
  

In the following two sections, I will go over lists curated by CodeGrade with common rules to use in your style guides for education and teaching Java. These lists are by no means exhaustive, but should rather act as an inspiration and a great starting point for your own adventure in the literally hundreds of available rules and checks.

Checkstyle style guides for educational purposes

You can select the rules in your Checkstyle configuration file (style guide). A list of all available Checkstyle rules can be found on their website here. A Checkstyle configuration file is a simple XML document that specifies the rules to take into account, these rules are childs of the root module called “Checker”. 

-!- CODE language-xml -!-<module name="FileTabCharacter"/>

Checks that no tab characters (`\t`) are used.

-!- CODE language-xml -!-<module name="Indentation">
    <property name="caseIndent" value="0"/>
</module>

Checks if correct indentation is used, with a custom property value as recommended in the style guide of Oracle.

-!- CODE language-xml -!-<module name="ConstantName"/>

Checks the constants naming conventions, e.g.: `FIRST_CONSTANT1`.

-!- CODE language-xml -!-<module name="LocalVariableName"/>
<module name="MemberName"/>
<module name="ParameterName"/>
<module name="MethodName"/>
...

Checks the naming conventions of variables, parameters and methods, these should be lowerCamelCase.

-!- CODE language-xml -!-<module name="LineLength">
    <property name="max" value="80"/>
</module>

Checks the length of the lines, with a configurable maximum value, the style guide of Oracle recommends 80 characters per line.

-!- CODE language-xml -!-<module name="AvoidStarImport"/>

Checks that no star imports (`import java.io.*`) are done.

-!- CODE language-xml -!-<module name="WhitespaceAfter"/>

Checks that tokens are followed by whitespace, not `method(foo,bar)`.

-!- CODE language-xml -!-<module name="NeedBraces"/>

Enforce the use of braces around code blocks.

-!- CODE language-xml -!-<module name="MagicNumber"/>

Checks that no “magic numbers” are used (-1, 0, 1 and 2 are not).

PMD style guides for educational purposes

You can select the rules in your PMD configuration file (style guide). A list of all available PMD rules can be found on their website here. A PMD configuration file is a simple XML document that specifies the rules to take into account, these rule elements all have a `ref` attribute to specify the rule reference.

-!- CODE language-xml -!-<rule ref="category/java/bestpractices.xml/CheckResultSet"/>

Checks if return values of methods like `.next()` are checked.

-!- CODE language-xml -!-<rule ref="category/java/bestpractices.xml/UnusedImports"/>

Checks if none of the imports are unused.

-!- CODE language-xml -!-<rule ref="category/java/bestpractices.xml/UnusedLocalVariable"/>
...
<rule ref="category/java/bestpractices.xml/UnusedPrivateMethod"/>

Checks that there are no variables, private methods, etc. are unused.

-!- CODE language-xml -!-<rule ref="category/java/codestyle.xml/ForLoopShouldBeWhileLoop"/>

Checks if none of the for-loops should be simplified to while loops.

-!- CODE language-xml -!-<rule ref="category/java/design.xml/CollapsibleIfStatements"/>

Checks if two consecutive if-statements can be combined.

-!- CODE language-xml -!-<rule ref="category/java/errorprone.xml/EmptyCatchBlock"/>
...
<rule ref="category/java/errorprone.xml/EmptyWhileStmt"/>

Checks if there is no blocks of code or conditions that are empty, this is likely a bug or a bad coding practice.

-!- CODE language-xml -!-<rule ref="category/java/errorprone.xml/JumbledIncrementer"/>

Checks if multiple variables are referenced in for-loops  (e.g. `for (int k = 0; k < 20; i++)` ).

-!- CODE language-xml -!-<rule ref="category/java/errorprone.xml/UnconditionalIfStatement"/>

Checks if no if-statement is always `true` or always `false`.

-!- CODE language-xml -!-<rule ref="category/java/errorprone.xml/UseEqualsToCompareStrings"/>

Checks that `.equals()` function is used to compare strings, not `==` or `!=`.

Customised messages for meaningful feedback to students

The final step you can take towards the most student-friendly code quality autograding is customizing the messages and severity levels. All rules you can select have default messages, often very useful for experienced programmers, but sometimes complicated to understand for novice programmers. Furthermore, these messages are a great place to point students in the right direction, instead of just pointing out the mistakes. Changing “Avoid unused local variables” to “The variable ‘temp’ you initialized on this line is not used anywhere else in your code, it is better to remove such unused local variables” makes it easier to understand and more actionable for students.

Customizing messages and severity levels is possible for both Checkstyle and PMD and can be simply done by editing some attributes in the XML configuration file.

Customizing rules in Checkstyle

-!- CODE language-xml -!-<module name="RedundantModifier">
    <property name="severity" value="info" />
    <message key="redundantModifier"
        value="The ''{0}'' modifier you used is not needed here."
    />
</module>

In Checkstyle, a custom message can be set using the `<message>` property (a child of the rule module). You have to specify two attributes for this, a key, which can be found for each rule in the Checkstyle documentation here. And the custom message value, which may use placeholders. These placeholders can also be found in the documentation.

Setting a severity level can be done using a property too, named `severity`. The values for this are `info`, `warning` and `error`, corresponding to the severity levels found in CodeGrade AutoTest’s Code Quality step.

Customizing rules in PMD

-!- CODE language-xml -!-<rule ref="category/java/codestyle.xml/UselessParentheses"
    message="The parentheses here are not required and should be removed!"
>
    <priority>3</priority>
</rule>

In PMD, a custom message can be set with the `message` attribute to your rule. More information can be found in their documentation here. 

Setting a severity level is done using the `<priority>` property as a child of the rule. This is done with a number, corresponding to CodeGrade AutoTest’s Code Quality step as follows: 1 = fatal, 2 = error, 3 = warning, 4 = info.

Powerful Java autograding and plagiarism detection in CodeGrade

This guide has shown you all ins and outs of code quality autograding for Java assignments. But your CodeGrade assignment doesn’t have to stop here! There are many more things you can do to automate your grading workflow, provide your students with better feedback and manage your Java course effectively in CodeGrade.

In our recent webinar on Java autograding, which you can watch on demand here, we also discussed checking the functionality of student Java code, using JUnit unit tests to check individual methods, using JaCoCo to assess unit tests written by your students and how you can set up Hand In Requirements in such a way to enforce correct Java submissions to your Java assignments!

Next to autograding Java assignments, CodeGrade can also help you check for plagiarism in Java code and gives many tools for efficient manual grading. 

Would you like to learn more about setting up Java automatic grading for your code exercise or discuss certain Checkstyle or PMD rules mentioned in this article? Feel free to email me at support@codegrade.com and I’d be more than happy to help out!