STAR Assessments for Mastery Learning

Update: Interested in enrolling for Spring 2024?

We will offer this course in Fall 2023 in person, Mondays 2:30-4:00pm in 320 Soda. You may take it for credit even if you have taken it previously.

In this special topics course, small teams (2-4) of graduate and undergraduate students will develop and rigorously evaluate rich, machine-gradable assessments that would address learning goals that might arise in typical EECS courses. The assessments will promote mastery learning (aka proficiency learning) Links to an external site. by following the acronym STAR:

• Specific to a learning goal in a particular domain or topic. Examples from EECS could include timing diagrams, graph labeling, connect the circuit elements, etc, but not generic-format multiple-choice/numeric-answer/fill-in-the-blanks questions. Developing and evaluating novel question formats and types are the main goals.
• Tagged to specific learning outcomes, skills to be demonstrated (competencies), etc. within the context of a full or partial concept inventory for the course, so that evaluation can be focused on whether the learning outcome is in fact reinforced by the assessment
• Autogradable with instant or near-instant automatic feedback
• Randomized, so each exercise has a large or very large number of variants and can therefore be used for practice (formative assessment for mastery learning)

In addition to developing assessments, student teams will evaluate them by using the methods of HCI and education research to run either informal or formal pilot studies. We will encourage students to make any resulting artifacts available as open educational resources, and if appropriate, submit results for publication.

See example content and projects from inaugural Spring 2023 offering.

Format & Prereqs, How to Enroll

• Instructors: Armando Fox, Dan Garcia, possibly others TBD & guest speakers
• In person, regular attendance and participation required to pass

• Permission of instructors required to enroll, based on these conditions:

1. Experience as a student, academic staff, or both, for one or more EECS courses that could serve as the basis for you to think up and develop novel assessment types for those course(s).
2. Evidence of reasonable proficiency in Python: B+ or better in CS61A, or we might ask you to do a brief diagnostic interview
3. Pre-proposal for a STAR exercise you could create based on your experience in a specific course. You’re not committing yourself to do this specific one, but it will serve as a starting point to make sure you have some good concrete ideas about what you might work on. The instructors will be available for informal consultation/discussion to help you prepare your proposal.
4. Helpful but not required: basic knowledge of HTML & Bootstrap CSS; we will provide basic learning materials and there will be a quiz to make sure you have enough background. Some knowledge of JavaScript and some of its major libraries (jQuery, d3). Comfort working with basic data analysis tools (spreadsheets, R/Pandas, etc.)

• Meetings will be part lecture/didactic, but (esp. after first 2-3 weeks) primarily demos, team presentations, discussion. 
• Required readings from the literature, which may be verified via periodic microquizzes.
• Grading: 3 units, letter graded; CS194 units count toward upper division EECS technical electives.
• Grading based on:
• Regular synchronous attendance, engagement, and participation in discussions. Contact instructors prior to a class meeting if it will be a problem for you to attend that meeting or meet these criteria generally.
• Project: In a small group, produce and evaluate a STAR assessment that works within the PrairieLearn assessment authoring system, tagged to concepts in a (partial) concept map for an appropriate course. 
• Team presentations: Paper summaries, project proposal, project progress, user studies, final presentation
• A few “checkpoint” assignments to keep projects on track: mini-presentations/progress reports, GitHub/deployment checkpoints, etc.
• Microquizzes to cross-check that people are at least skimming the readings.
• No midterm or final. A poster session will be held during the last day of class (November 27), but everyone gets an automatic extension until the end of RRR week to finish final coding, make pull requests, etc. and write a project report that is expected to be high enough quality to be submittable as a short or full paper to a reputable workshop, conference, or journal.

Projects will be proposed in week 2 or 3, instructors will provide feedback on project proposal to make sure it meets the general project criteria.

To enroll: please read the FAQ first to make sure the class is a good fit. At the bottom of that page is a link to fill out a short form to enroll.