[Events] Lessons from the newcomer side of the Hackathon (fwd)

[Asheesh Laroia]

Dear OpenHatch Events,

Mike and I helped organize the Wikimania Hackathon, aiming to build a 
newcomer-oriented track into the Wikimania Hackathon.

This is the wrap-up email I sent to the Wikimedia tech mailing list. There 
was only reply, in case you're interested in following the thread: 
http://lists.wikimedia.org/pipermail/wikitech-l/2012-July/062066.html

If y'all have any thoughts, I'd love to hear.

-- Asheesh.

---------- Forwarded message ----------
Date: Tue, 31 Jul 2012 17:00:25
From: Asheesh Laroia <lists at asheesh.org>
To: wikitech-l at lists.wikimedia.org
Subject: Lessons from the newcomer side of the Hackathon

Hi all wikitech-ers,

I wanted to summarize a few lessons learned from the Wikimania Hackathon and 
make myself available to discuss any of these issues at greater length on the 
list.

Also, if you (as an attendee) have other thoughts you want to share about the 
Hackathon, feel quite free to email the list, just myself, or just Sumana if 
for any reason you want your identity or your feedback to be more private.

(Note: This is probably going to get long! If you just want the "lessons", look 
at just the header names.)

= Preface =

My major interest in the Hackathon was to encourage newcomers to step up their 
abilities, in terms of editing the encyclopedia, programming bots, or whatever 
exciting tech-related thing they want to be better at.

= Thanks =

Major thanks for the Hackathon go to:

* Sumana Harihareswara for inviting OpenHatch to lead the newcomer portion of 
the Hackathon;

* Greg Varnum, for being a hugely awesome local co-organizer;

* James Hare, for managing the local logistics and generally helping make 
thigns go smoothly behind the scenes;

* Katie Filbert, for helping tremendously with pre-event planning; and

* Mike Linksvayer, who was my other OpenHatch co-organizer, for taking care of 
lots of essential tasks like emailing local logistics folks and signing 
attendees in.

= Thoughts =

== The combination of newcomers and experienced people seemed to work ==

We got two different notes in the exit survey about this -- one from a person 
who said they were happy working in the big open main room and didn't feel 
distracted by noise, and from another person who said they went off because 
they found the main room too noisy.

I did really like being able to send people to nearby experienced folks to have 
a chat. This was especially helpful as I walked around and asked people what 
they wanted help on, or what they wanted to work on.

I did notice some experienced people, by the second day, had wandered off to 
quieter rooms than the big main room. I'm glad we had those rooms. (I'd love to 
hear from those people if they felt "pushed out", or if they instead felt happy 
that the quieter rooms were available.)

== We made a good impression by just running the event ==

One prospective attendee who, sadly, couldn't make it, indicated to me that 
just by reading the survey we sent to prospective attendees, and skimming the 
list of tasks, it was something that they'd be interested in attending, and 
that it was great that such an event was going on. In particular, they 
suggested it felt more like a "play-with-stuff-a-thon" rather than a 
"hack-a-thon", and described that as making them feel welcome.

If anything, we should have capitalized on this more. I heard from other 
prospective attendees that they didn't know the Hackathon was intended to be 
newcomer-friendly this year.

Personally, I think the term "Hackathon" gives an exclusionary vibe, and that a 
newcomer-oriented event should have a different name. For example, Boston Ruby 
recently started using "project night" at our (OpenHatch's) suggestion and it 
seems to have gone well for them: 
https://openhatch.org/blog/2012/the-steps-boston-ruby-is-taking-to-become-friendly-to-beginners/

== Many people did cool things for the first time ==

I heard from at least two people who made their first edit to Wikipedia during 
the Hackathon! I saw one person write a bot for what I believe was the first 
time, as well get a labs account and work on moving that bot there. In the exit 
survey, one attendee wrote, "I learned a lot about batch upload." Another wrote 
about making their first commits with Git and Gerrit.

I think this is the most exciting thing about a newcomer-friendly Hackathon -- 
it creates an environment where people can step up to the next level, and 
hopefully stay at that level through self-driven follow-up practice. This is 
how community growth happens.

That reminds me: Two people did indicate on the exit survey they'd be 
interested in follow-up mentorship. I'm going to see how we can best support 
them; I might just send them a periodic email to see what they're up to and see 
how I can personally help or direct them to help.

== Wi-fi was a serious obstacle ==

I personally had a lot of trouble with my laptop failing to stay on the wifi. I 
would estimate at least 10-25% of attendees had a serious problem with this.

It's especially tough because on one hand, the Hackathon is a very 
wifi-dependent event. On the other hand, the Hackathon is a pre-Wikimania 
event, which means that unless there's serious pre-pre-event testing of the 
wifi, Hackathon attendees are the ones that will run into any problems that 
occur.

(For those interested in minutiae, it seemed that roaming between access points 
triggered the problem. The problem as users experienced it was that randomly 
they would suddenly see 100% packet loss with no obvious way to fix it.)

A sample of things we saw from this:

* At least one person reports in the exit survey that it was impossible to get 
work done with the wifi the way it was.

* Mike and I couldn't sign people in using the wiki because our own wifi had 
failed, so we used a spreadsheet local to his computer. This meant that it was 
harder to use the wiki to locate like-minded people.

One organizer attempted to set up a separate wifi network, which was 
operational toward the end of the event. In the future, we need more testing of 
this, and more of a plan for what to do if it fails.

== Logistics concerns ==

The room that we had agreed we would use turned out not to have power strips 
lining the bottom of every table, so we switched rooms and had to update signs 
across the event.

Some exit survey respondents indicated they wanted the event to start later in 
the morning, and that they wanted the room to not close at 6pm.

I received more than a few emails from people who were, according to the 
wikimania2012 registration system, signed up, but who replied to me indicating 
that they couldn't in fact attend due to not receiving a visa as needed. It 
would have been nice if those people had not been in the registration system 
anymore marked as attendees.

I also sent out at least one email to the wrong address; the problem was that 
the user who did the registration isn't necessarily the person who's being 
registered. I think that in this case, one person in a company was responsible 
for registering another person. We could fix this by making personal email 
address a field that you enter at registration time (though I do realize data 
will basically never be entirely perfect).

(Similarly, the script I was given for extracting information from the 
registration system ended up suddenly failing, which slowed down the second 
pre-event email.)

== Exit survey suggests people overall liked the event ==

Of those people, all of them indicate they're either "likely" or "very likely" 
to recommend a hackathon organized by OpenHatch to a friend, so generally 
speaking people enjoyed themselves.

== Tutorials can use TAs ==

On the second day, I found volunteeres to be "TAs" for the tutorials. Their job 
was to wander around and help people with environment problems or people just 
having trouble following along because e.g. their web browser was different 
than the one being used by the presenter.

Another difficulty I found was that sometimes a tutorial speaker wasn't loud 
enough to be audible in the back of the room.

This is above the tasks I had labeled as needed for a "Talkmeister": 
http://wikimania2012.wikimedia.org/wiki/Hackathon/Volunteers#Talkmeister

Also, people who are having problem following along during a tutorial don't 
always speak up. I'm glad we added the TAs, although I think further work is 
required to find out how to non-intrusively convince people that asking 
questions is okay. The best way I've seen is to have a very small group, no 
bigger than 10, preferably about six people. I almost wonder if we'd be 
better-served to use pre-recorded video tutorials with a lot of TAs available, 
rather than live lecturers. Then we could easily have small group rooms, and 
could pause the video.

== Next time, I'll get firmer commitments from helpers ==

One thing I did was walk around the main room, asking if people needed 
anything. I did find some people willing to help out with this, but didn't have 
a good sense of when they could help. Next time, I'll do something like create 
an hourly sign-up sheet for this.

We did receive feedback at the end of the first day that at least some people 
were very happy with the helpers. One person indicated he wished there had been 
even more help. While I do think that everyone just about got the help they 
needed, it would have been nice to have had a more clear list of who's helping, 
partly to ensure we have more capacity, and partly for me to know when people 
are planning to help, and partly to encourage mentors to sign up for brief time 
slots and know they've helped the event.

== Favorites ==

People listed a wide variety of favorite activities, with all these being 
mentioned more than once in the exit survey:

* Tutorials

* Talking to people

* Coding/hacking

The laptop setup process showed up once, too, as did break-out rooms. Sadly for 
me, the list of "Tasks" I made wasn't a favorite for anyone who filled out the 
exit survey. (Surprisingly and pleasingly, the laptop setup was!)

== Diversity ==

I would estimate about 10-20% of our newcomer-oriented attendees were women.

One difficulty wasa that we experienced a lot of competition for attendees with 
AdaCamp DC, which took place on the same days and times. (I heard from more 
than a few attendees and prospective attendees that this was a conflict.)

(If you're not familiar with the event: "AdaCamp is a Ada Initiative event 
focused on increasing women’s participation in open technology and culture" 
<http://dc.adacamp.org/about-adacamp-dc/>)

Given that conflict, though, I think we did reasonably well.

In terms of other diversity, one attendee remarked to me that they were quite 
impressed at the diversity of ages in attendance. I was personally quite 
impressed by the diversity of experience levels in attendance.

I think we accommodate all that reasonably well.

== Exit survey: Qualtrics, etc. ==

The exit survey we ran received only 11 responses. (Our entrance survey 
received over 100, but I sent it to about 400 people. The exit survey was sent 
to about 45, so a 25% response rate is somewhat consistent. We had about 65 
people sign in on our spreadsheet, and about 45 of those people gave us email 
addresses.)

I used Qualtrics to run the exit survey, since it complies with Wikimedia's 
guidelines on data privacy/storage and relationships with vendors.

I ran into an issue with Qualtrics where the sample email that it sent me to 
preview what the form would look like wasn't an accurate simulation -- in 
particular, it didn't pre-fill the email address in the form in the same way as 
the real email did.

Anyway, it took too long to get this sent out, partly due to time spent 
learning Qualtrics that I didn't expect to be spending, partly due to my own 
conference travel post-Wikimania. Given the wifi, we could have just created a 
paper exit survey for people to fill out, or otherwise generally been faster at 
this if we had prep'd the exit survey questions before-hand. We can aim to do 
that for future events.

== Other thoughts ==

* One exit survey respondent wanted it to be easier to find people interested 
in using MediaWiki and related software for non-Wikimedia tasks, to chat with 
and work on tasks with.

* I treated laptop setup as a good thing for all attendees to go through first, 
because it was a prerequisite for some of the tasks, but by no means all of 
them. In hindsight, I would prefer to indicate on each "Task" what setup steps 
are required. The people that ended up just mostly editing Wikipedia didn't 
need to do them. (The other side of the spectrum is that it was good for many 
people to go through them, to enable them to do tasks that they might not have 
thought they could do!)

= Conclusion =

That's "it". If you read this far, thank you!

Discuss, if you like! I'm here to chat (although might take a day to see all 
responses, if any, and respond in bulk).

-- Asheesh.