[OH-Dev] OpenHatch missions pedagogy audit results

[Karen Rustad]

At long last: here's the results of the rest of the OpenHatch missions
pedagogy audit! (based on https://openhatch.org/wiki/Missions_pedagogy)

-- Karen

===SVN mission===

is_working: Pass.

Triangle: Pass. Objective and activity are mostly clear. (Pasting in a diff
isn't your typical patch submission process, though...) "Getting commit
access" (the assessment) is a little vague (how many patches do you have to
write to earn that?) but it's vague in real life too so that seems fine.

Plot: Pass. This is great! :D

Hints: FAIL. Need gradations of hints... the first step is too easy because
it just *gives* you the checkout command--it's just copy and paste. The
second is similarly non-gradatedly easy. The third builds upon the first
and second, so it's not quite so bad, but could be reconfigured to take
advantage of hints as well.

Quick reference: FAIL.

No blocking: Pass.

Other: OMG the user's username appears in the thank you message! This is
great!!

"About Subversion" appears as the second item in the left-hand list -- kind
of a random place, in the middle of two actual mission steps! It should
probably be moved to the bottom of the list.

===Git mission===

is_working: FAIL. When I paste in an incorrect patch to the second step,
the page doesn't give me any feedback -- just reloads the page & blanks out
the textarea!

Additionally, if you take the page copy *literally*, you will fail that
step (you're told to change it to "Hello world", sans exclamation point,
but you need the exclamation point to pass).

Triangle: Pass.

Plot: Pass. Not as great a plot as the SVN mission, though. The dinner
party thing is rather random. It would be great to extend the secret agent
scenario to this mission as well (with a different evil plot to foil, or
perhaps sabotage to correct)!

Hints: Pass, mostly. Doesn't use the new tabby hint system, but most steps
have appropriate hints.

Quick reference: FAIL. (See https://openhatch.org/bugs/issue456)

No blocking: Pass.

Other: Interesting. The objective of this mission is subtly different than
the svn mission. Where before your goal was commit access (and the patch
submission process was kind of hand-wavey), here patches are the focus.

The "About Git" material and the first step of the mission are on the same
page. You might want to separate those out, as they are separate in the SVN
mission.

On Sun, Dec 4, 2011 at 12:44 PM, Karen Rustad <karen at openhatch.org> wrote:

> Hey folks!
>
> One of our release goals this release was an audit to see to what
> extent our training missions meet the standards set in our pedagogy
> guide (http://openhatch.org/wiki/Missions_pedagogy).
>
> I've finished auditing the first two missions. Here are the results thus
> far!
>
> -- Karen
>
> ----
>
> ===In general===
>
> Check mark UI is horribly confusing. The checkbox appears whether or
> not you've completed the mission / step of the mission -- resetting
> the mission doesn't fix this. Faded / greyed-out check marks? Super
> weird / non-standard. Let's not do that!
>
> When I want to go from a mission back to the missions page, I
> instinctively look for a back link at the top left. Neither the header
> image nor anything else in that area of the page is a back link,
> however, which is confusing. The back link is instead at the top
> right, and hard to find.
>
> ===Tar mission audit===
>
> is_working: FAIL. The hinted command for the first part does not work
> on Macs. In addition, uploading (and, thus, the entire assessment
> functionality) is busted. See https://openhatch.org/bugs/issue632.
> [Editor's note: Jessica has a patch for this!]
>
> Curricular triangle: Pass.
>
> Plot: FAIL. No plot or even amusing examples.
>
> Quick reference: FAIL. No reference, though the current 'Hints' doc is
> somewhat similar.
>
> Levels of hints: FAIL. The highest level of help is the *first* one
> that the user sees -- the tooltip with the exact command needed to
> perform the operation. The "Hints" doc *would* be a good intermediary
> level of help, but it's positioned at the end of the mission -- you'd
> never look to it before looking at the tooltip!
>
> I would argue that a link to relevant outside documentation ought to
> be the first stage of hints for all missions. (Then our own guide,
> then verbatim commands/instructions.) This mission, in any case, does
> not have that.
>
> No absolute blocking: Pass; the two parts aren't dependent on each other.
>
> ===Diff/patch mission audit===
>
> is_working: FAIL. The second step fails due to whitespace errors (not
> anything meaningful) most of the time. We really ought to give people
> more meaningful feedback other than "No." -- otherwise people have no
> idea what to fix. Maybe a line number where the failure is? Or show
> the diff between what you submitted and what you were supposed to
> submit (possibly in some obfuscated way, possibly behind a Hint link)?
>
> Curricular triangle: Pass, mostly. See the is_working discussion: on
> the second step, what we're assessing doesn't quite match what we're
> trying to teach.
>
> Plot: FAIL. It could easily be changed to have a plot -- the third
> part talks about recipes in a very cute way -- but it doesn't
> currently.
>
> Quick reference: FAIL.
>
> Levels of hints: FAIL. On the first and second steps, all hints are
> available from the very beginning. There is a link to the patch man
> page, though, which is nice. Hints regarding grep use and how to send
> a patch to a file would help.
>
> No absolute blocking: Pass.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.openhatch.org/pipermail/devel/attachments/20120208/0f8e24bc/attachment.html>