Canvas Integration: Showing only portion of test cases if incorrect solution.

Jason Isaacs asked 2 weeks ago

I am interested in using this system for an introductory course in python. I will be running the class through canvas.

Would it be possible to add a feature in the autograding that would allow the student to see a portion of the unit tests and keep a portion hidden for grading purposes?

I would like the students to be able to have a something to test against, but keep a portion of the test cases unseen for grading.

2 Answers

Ryan Staff answered 2 weeks ago

You’re thinking something like only show the results of the first X of however many test cases? Something like what CodingBat does with their “other tests.”

I really like that idea. It could solve the common problem where a student gets most of the answer right and just hard codes in a catch for the last couple of tests they’re missing.

Let me look into it and see if there’s a way to get that working. I’m pretty sure there is, but I don’t want to promise anything just in case.

Jason Isaacs answered 2 weeks ago

Hi Ryan,

Thank you for the prompt reply. That is correct. My experience is just as you described that some students will simply hard code for the last couple tests.

More detailed example of what I would like to see:

There are two public unit tests and two private unit tests. The student’s code passes the two public tests but fails one of the two hidden tests.

When the student clicks submit they would see:
Your code passes public test 1 and public test 2.
Your code passes hidden test 1 but fails hidden test 2.
Would you like to try again or complete the submission?
If they click complete the submission they would get a grade of say 75%, but they could have the chance to resubmit some number times (possibly infinite) in order to get full credit.

Your Answer