Try It: A Working Assessment-First Course

The article discusses a working assessment-first course called Doerkit, which uses a Large Language Model (LLM) to grade written answers against a rubric. The course includes six statistics lessons, quizzes, and cumulative review. The LLM never chats or does the student's work, but rather judges rubric criteria as booleans. The findings from the project include grader security living in the model-prompt pair, separable knobs for grader severity and warmth, and the cumulative-review feature having the biggest effect size in the source study. To run the course, engineers can clone the repository, install dependencies, and export an API key. The project includes two repositories, one for regression testing and one for the platform.