[Events] Summer of Meta Mentorship in GSoC

[Asheesh Laroia]

Howdy all,

(Background: GSoC refers to Google Summer of Code, which you can read 
about at http://www.google-melange.com/ .)

I want to see GSoC mentors be better-equipped to handle their students 
during GSoC. So I want to try doing an optional meta-mentorship program 
within GSoC, where we help GSoC mentors be better mentors by connecting 
them with experienced mentors to answer questions, other current GSoC 
mentors to help them bounce ideas off, and thinking-aloud tools to help 
them reason through how to deal with their students.

I've written up a plan at https://etherpad.mozilla.org/somm ; I consider 
it "first draft" quality. Thanks to Michael Stone for his help getting to 
first draft quality! I'm also attaching its current state as a text file. 
Therefore, feel very free to to edit the Etherpad document.


We need to fix it in the following ways:

* State more clearly what results we expect (this also helps us perform 
power analysis, i.e. evaluate the experiment for statistical power)

* Decide if we want a pure control group (I think it is okay not to have 
one, since we want in this case everyone to get what we think is a 
beneficial treatment)

* Get someone who has some feeling for what is ethical in studies like 
this to make a personal (not necessarily official) statement about that

* Work out the resource dependencies listed in the document.


Shauna, I'd love your thoughts. Having read 
http://www.shaunagm.net/blog/2011/10/results-blind-reviewing-a-solution-for-publication-bias/ 
, it seems to me we should get as far as we can to writing an actual 
paper, with a results and methods section, before the study terminates, 
preferably before it begins.

After reading http://mstone.info/posts/willpower-20130421/ and 
http://www.shaunagm.net/blog/2012/02/degrees-of-freedom-arent-free/, it is 
also clear we should use the Simmons experiment quality checklist here.

Mako, you often have feelings about study design; I'd love to hear your 
thoughts here, too.

Really, anyone on this list -- I'd love feedback. On-list is preferable to 
keep it all in the same place, but if people want to email me off-list or 
get in touch via #openhatch IRC or private message or whatever, that is 
fine; I can work to summarize the updates on this list and credit you or 
not, at your option.

Also, if there some non-me person interested in helping run (or running!) 
this experiment under the OpenHatch banner -- for example, making sure we 
get the plan more well written, making sure the code to send out chapters 
of the book work properly, and so on, I would mega love if you would join 
in!

-- Asheesh.
-------------- next part --------------
Summer of Meta Mentorship

The goals

Increase the quality of Google Summer of Code in the following ways:

* Increase the average happiness of a GSoC mentor.

* Increase the average student success rate (real success, not just "marked as success in desperation at the end of the summer").

* Increase the rate at which bad students are failed early in the program.

A properly-functioning meta-mentorship group may have a higher failure rate than the non-meta-mentored group. That's okay, since it might mean that they failed students that deserved to fail.

Theory

* GSoC mentors do not fail students early enough because they think their situations are special. Therefore, hearing about other situations that others find themselves in may help them to realize that their situations are not special.

* GSoC mentors do not read the GSoC mentor book, but there is wisdom in there.

* GSoC mentors can provide general help to each other.

* The main mentors mailing list is not conducive to a sense of trusting your fellow mentors enough to ask them questions that make you feel vulnerable.

The strategy

A "participant" is a GSoC mentor.

A "meta-mentor" is someone who is helping organize this meta-mentorship program.

75% of participants will receive "the paperwork" combined with "the cohort" treatment. 12.5% will receive just "the paperwork" treatment; 12.5% will receive just "the cohort" treatment. Subject to rounding. I expect 100 participants or so, but I could imagine up to 200 participating.

"The cohort treatment"

Every participant in the cohort treatment receives:

* Membership in an email list containing 5 (including them) GSoC mentors.
** We try to select cohorts to be very geographically close so they can perhaps meet in person.

* A 15 minute group video or IRC chat to get to know their fellow cohort members, during which a meta mentor shows up to get to know them 5 min into the appointed time.
** Meta-mentors send out a doodle 

* A meta-mentor assigned to be part of their cohort to read their mails, and that if they write the word metamentor in the subject line to reply.
** There might be another convention we can use here; if so, great.

"The paperwork treatment"

Every participant in the paperwork treatment receives:

* Once every 5 weeks, we ask them to write to either their cohort group or their meta-mentor with one good thing about their time in GSoC so far, and one stressful thing about their time in GSoC so far.
** If they fail to write this email within a week of the request, they are considered to have failed the treatment and we remove them from the cohort group if they are in one.
** (what do we do to the 1/5 weeks scheduling thing when people fail out? Maybe we meta-mentor provide our own such report.)

* Once a week, we send them a "chapter" from the GSoC Mentors book.
** If they are doing the cohort group, we send it to their cohort mailing list.
** Two weeks before the final and midterm evaluations, we give each participant 7 days (with a reminder after 4 days) to write a brief 3-4 sentence statement of their student's progress with links to evidence to be checked by the meta-mentor.

Recommended practices that mentors can adopt

* Do a weekly check-in with your student -- privately or publicly, on IRC or IM or Hangouts or whatever -- or insist on a blog post weekly. If they ever fail to do the weekly check in, email them within 24 hours of them failing to do the check-in. If they fail to do their weekly check-in within 4 days of the day they were supposed to, fail them.

* Do we know other ones? If so, great. List them here.

Chapter schedule

We will release the chapters on a schedule starting with Monday 6/3. There are 17 chapters and 16 Mondays until (and including) the last Monday of the program, 9/16.

* Full schedule: TBD

Resource dependencies

* Collect meta-mentors
** (Write a brief bullet-point guide for meta-mentors so they know what to expect)

* Talk to Google and see what integration points with the Open Source Programs Office make sense


Meta-mentor office hours

* For one hour a week, some meta-mentor will be on some private IRC channel, which will be announced a week before the event. The hour chosen per week should be very close to random to enable geographic whatevers.
** (where do we indicate this? If you're in the paperwork treatment, in the paperwork email. If you're not, you'll get it since we'll also email the office hours time and person name to the general gsoc mentors mailing list. )
*** Maybe it's wise to make a paperwork-without-office-hours-email group?

Post-experiment evaluation

Metrics, often measurable of people within and not within the experiment.

* Stated level of happiness with GSoC by mentor
** maybe including a question of if you failed your student, did you feel like you waste lots of time on them? just a little? (:

* Stated level of happiness with GSoC by student.
* Percent of students who passed
** (inversely, percent of students who failed)
*** Of students who failed by the end, how many were failed at the mid-term? at the community bonding period?
*** Of students who failed, when did their mentor personally decide that they had failed?

* N ( = 3?) months later, are the students still active?
* N ( = 3?) months later, is the code the student worked on being used by the project?
* Did you read any chapters of the GSoC mentors book?