How to automatically grade Java

Step 3: Configure Each Test

After setting up your environment (Java installation, Semgrep, etc.), it’s time to configure the tests themselves. We’ll go through each test type—Compile Tests, I/O Tests, Code Structure Tests, JUnit5 Tests, and Code Style Checks—and explain why they are valuable.

1. Compile Test

Compiling is the first hurdle any Java code must clear to run successfully. A Compile Test guarantees:

• No syntax errors or missing dependencies.
• A baseline check before proceeding to more advanced tests.
• Immediate feedback for students if their code cannot be compiled, highlighting the lines with errors.

How to Configure

1. Add a Connect Rubric block, selecting the "Compiles and runs without errors" rubric category.
2. Inside the Connect Rubric block, add a Java Compile block.
3. Provide the file name or glob pattern (e.g., LinkedList.java or *.java) to compile.
4. If the student code fails to compile, CodeGrade will annotate the errors in the submission and display them in the output.

In case of any errors, they will be reported:

If it successfully compiles, you will see this:

2. Input/Output Tests (I/O Tests)

I/O tests verify that the output of your program matches expected results given a particular input. This approach is straightforward and critical for:

• Ensuring correct functionality for different input scenarios.
• Quickly catching logical errors or mistakes in output formatting.
• Providing students with immediate, easy-to-understand feedback (i.e., “Your output should match this expected output”).

How to Configure

1. Add a Connect Rubric block for the "Produces correct results for given inputs (through I/O tests)" rubric category.
2. Place an IO Test block inside.
3. Specify how to run your compiled Java program (e.g., java LinkedList).
4. Within the IO Test block, add Substring match, Full match, or Regex match blocks for your test cases:some text
   • Enter the Input data and the Expected output.
   • Choose whether to compare case-sensitively, ignore whitespace, etc.

AutoTest output for the step

3. Code Structure Tests (Using Semgrep)

While I/O tests confirm output correctness, they don’t always guarantee the required structure or techniques. Code Structure Tests:

• Verify that students use specific syntax or design patterns (e.g., a Node class, certain methods).
• Help you ensure students follow best practices or meet specific assignment requirements (e.g., using recursion, implementing interfaces, etc.).
• Encourage students to write code that meets both functionality and structural requirements.

How to Configure

1. Add a Connect Rubric block, selecting "Is well-structured and organized".
2. Inside, add a Code Structure Test block.
3. Specify the student file(s) to scan (e.g., LinkedList.java).
4. Add Positive match or Negative match blocks:some text
   • Positive match checks if a specific pattern (like a class or method) is present.
   • Negative match ensures certain patterns (e.g., System.exit()) are not used.
5. Paste your Semgrep rule into the provided editor to define the pattern to search for. The Semgrep rule in the screenshot below checks for the presence of a Node class.

Autotest Output for the step

4. JUnit5 Tests

JUnit5 is crucial for in-depth, assertion-based tests, giving you:

• Fine-grained control over how each class or method behaves under various scenarios.
• The ability to test corner cases and intermediate states (not always possible with simple I/O tests).
• Industry-standard best practices for software testing, exposing students to real-world development workflows.

How to Configure

1. Add a Connect Rubric block, choosing "Passes unit tests at the component level (using JUnit5)".
2. Inside, include a JUnit5 block.
3. Ensure that the Test file name matches the class name in the editor.
4. Write your JUnit5 test cases with one test class per block. For example:some text
   • Constructor tests: Check if new nodes initialize properly.
   • Method tests: Verify correct behavior of insertion, deletion, or sorting methods.
   • Edge cases: Test empty lists, single-element lists, etc.
5. The example JUnit5 test below verifies that the Node constructor correctly initializes the data field to 10 and sets both next and prev references to null. Additionally, it ensures that the toString() method of the Node class returns the string representation "10".

Autotest Output for the step

5. Code Style Check (Using Checkstyle)

Code style is more than aesthetics; it’s about readability, maintainability, and professionalism. A Checkstyle test:

• Enforces industry-standard style guides (Sun, Google, or a custom configuration).
• Teaches students the importance of clean, readable code.
• Provides inline feedback, annotating lines in the code where style rules are broken.

How to Configure

1. Add a Connect Rubric block, selecting "Adheres to established coding standards for readability and maintainability".
2. Nest a Checkstyle block inside.
3. Select a style guide (Default, Sun, Google) as the config template and load it or customize your own rules.
4. Checkstyle will annotate any style violations in the students’ code. You can decide how many points are deducted for each severity level.

Autotest Output for the step

Step 4 (OPTIONAL): Setting up Assignment as Coding Quiz

If you want to create a more interactive learning experience for your students, CodeGrade offers a Quiz block feature in AutoTest. This allows you to ask multiple types of questions—from multiple-choice and select-all-that-apply to fully-fledged coding questions—all within the CodeGrade Online Editor.

• Targeted Assessment
  Coding quizzes allow you to assess students’ understanding of programming concepts by requiring them to write actual code snippets as part of their answers. This goes beyond multiple-choice questions to really test their ability to apply what they’ve learned.
• Instant Feedback
  By nesting automatic tests within each coding question, you ensure that students get immediate feedback on their solutions. This helps them learn through trial and error while saving you the time of manual grading.
• Single Submission Mode
  Since quizzes require using the Online Editor, you must make it the only allowed submission method. Enabling the Simple Submission Mode guides students directly to the Editor, ensuring they complete the quiz in one convenient location.

How to Configure

1. In your AutoTest configuration, add a Quiz block where you want to include your interactive quiz.
2. Inside the Quiz block, create a Coding Question block for each code snippet or mini-assignment you want students to write.
3. Configure the Coding Question

• Block Name: Give your coding question a descriptive title (e.g., “Implement a Node Class”).
• Question Field: Enter the instructions or prompt for the question. This field supports Markdown for formatting, adding links, or including short code snippets.
• File Name: Specify the name of the file that the student will produce (e.g., LinkedList.java).
• Starter Code: Optionally provide a template or partial solution to guide students (e.g., a skeleton class with TODO comments).

4. Add Automatic Testssome text
   1. Just like with other steps, nest your Compile Test, I/O Test, Code Structure Test, JUnit5, or Checkstyle blocks within a Connect Rubric block. 
   2. Place this Connect Rubric block inside your Coding Question block. This configuration ensures that the code students write for that specific quiz question is automatically tested and graded according to your rubric.

Here's the student view.

Step 5: Test and Publish Your AutoTest

Test with a Snapshot

1. Click BUILD SNAPSHOT in the tests sidebar to run a trial of your entire AutoTest configuration.
2. Provide a Test submission—for instance, a sample or reference solution.
3. Review the snapshot to ensure each test behaves as expected.

Publish to Students

Once satisfied, in the pop-up modal, click 'PUBLISH TO STUDENTS'.

Your students can now submit their solutions and immediately see the feedback from each test, allowing them to iterate quickly while you save valuable grading time!

Final Thoughts

Automated testing not only saves you time but also empowers students to continuously improve their code in a low-risk, supportive environment. By following these steps in CodeGrade, you’ll provide rich, automated feedback on their Java doubly linked list assignments—or any other project you want to use this setup for!