Week 3: Getting more specific & finding related work

Great work on presentations today! I'm super excited about a great batch of projects. The next steps will be to narrow down your particular project scope and details and focus on what your project's novel innovation will be. To do this, you will understand the related work and get more concrete on one or more problem types.

I've linked each team's slide deck from the home page of the class; you can just add slides to this deck from now on as the project progresses, like a mini research notebook.

We encourage unlimited intra- and inter-team collaboration on this and every remaining assignment!

Getting More Specific

Time to get your hands dirty. Mock up one or both of the following:

1. An actual question (hardcoding the question is fine) that would be in the question.html file served by PL. The goal is to actually have the question displayed by PL, even if grading doesn't work yet (ie there is no server.py). I have invited you all to the PL Slack. Links to an external site.The #pl-help channel is primarily when you're having trouble creating a question; #pl-dev is for hacking PL itself, including creating new elements, which I believe many projects will probably do. It's fine if any direct-manipulation interaction doesn't work yet.
2. What the element markup would look like for a new question element (question type). For example, take a look in the PL docs Links to an external site. to see that when authoring (say) a "put these things in order" question, you would use the pl-order-blocks element, which defines some specific XML-like markup that will be turned into raw HTML by the element code. If you're creating a new element for (say) representing a red-black tree, or some graph in general, what would that markup need to express? Your element code would then turn that into a specific question. In essence, you're defining a mini-DSL (domain specific language) for your element markup. (Use the Ed discussion thread if you're confused about this and let's help each other out!)

Things to keep in mind from Feb 2 discussion:

• Try to focus your design as specifically as possible on the skill, thought process, etc. you are trying to get the student to demonstrate. How much can you use direct manipulation (in the HCI sense) to minimize the delta between the student's mental model of the problem and how they interact with your exercise? (For example, rather than labeling arcs in a graph with letters and having students select checkboxes for which arcs to remove, could they just click on an arc to remove it and then click somewhere else to put it back?)
• Similarly, the only errors it should be possible to make are conceptual ones. If the student has to write code, that's fine if the code itself is the point of the exercise; but if the point is to see if they can describe a process, could they use pseudocode instead? Or something like a Parsons or Faded Parsons pseudocode? In a Mongo query, could they 'construct' the query from building blocks in a way that makes trivial syntax errors impossible? etc.
• Can you thwart exhaustive guessing, wherein students just try every answer by brute force but don't learn anything? Try to address this in the design of the question itself, rather than (e.g.) by limiting the number of autograding attempts.

Deliverable: add a slide to your deck with screen shot(s) of one or both of the above as displayed in PL. In class, we hope to comment on it!

Related Work

Who else has done exercises targeting the problem types you're targeting? We're most interested in cases where some attempt has been made to evaluate them. We strongly encourage you to collaborate both within and across teams to discover related work, particularly scholarly articles.

Let's collaboratively author a list of venues: visit wikicfp.com and login as "acelab" with password "hearst360". Click on the "My List" and "My Archive" links to see upcoming and past conferences that ACELab researchers have found relevant to our work. Feel free to add additional entries to My List! Learning At Scale, SIGCSE, ITiCSE, CSEET, ICER, AIed, ACM CompEd, LAK, CHI (not an education conference but has an education track), UIST (ditto), ICSE (for software-specific stuff, it has a great education track) are all great choices to start from.

The goal right now is just to understand at a high level what the related work is, since you want to (ideally) do something novel.

Deliverable: add 1 or more slides to your deck with brief summary (1 bullet; screenshot/picture even better where available) of other exercises you've found that target similar skills/concepts, and what you see as your potential innovation.

Narrowing down

A result of doing all the above is that you should get to a more specific version of what, concretely, you want to try to build. Most of the Feb 2 presentations showed various types of problems; the above should help you prioritize what to work on (or at least what to do first).

Please use Ed discussion for questions, and see you next week!