Things I learned from organizing a datathon

A couple of weeks ago we had our first Global Datathon at my company. It was an initiative that I shamelessly copied from the Global Hackathon that our team of developers organizes annually. I thought it would be a good idea to let people in the data science and analytics teams to explore creative ways to use the data that we collect and analyze every day. There are extraordinarily smart people working with me and I felt that it would be fruitful to let them work on their own ideas in a short, intensive, and collaborative event.

There were some organizational challenges, because the data science and analytics teams are relatively new and their members are spread through offices in different continents, and that limited the things are activities that we could do –f.i., the final presentations had to happen over Google Hangouts at almost intempestive hours in San Francisco, which is definitely less entertaining that a final meeting with drinks.

In any case, I learned some valuable lessons from the experience:

  1. Encourage projects that target substantive ideas. It is tempting to use the Datathon as an opportunity to automatize/simplify a task that you perform every day, or to work on that module that you know Python is missing. However, I believe that our role in the organization is to be the people who find answers to analytical problems, and as such we are serving better our position by finding real applications to the data that we have at hand. Besides, answering relevant questions is something that will get the managers on board with the idea of a Datathon.

  2. Managers should not be left out. The event can only occur with their cooperation, but more importantly, their attendance at the final presentation can be encouraging for all the participants. We all like to be praised for our work by the people who evaluate our performance, and we all want our efforts to be conducive to a more fulfilling professional career. A final presentation that includes managers and supervisors, not only your peers, is also the most likely way for your idea to catch on if it is really valuable to the company. But one has to be careful to organize things in a way that the participation of managers is not intimidating. Finally, events like a Datathon in which people showcase their abilities and discuss projects are conducive to a work environment with better, more constructive vertical communication.

  3. Developers and sysadmins should not be left out either. You want to work closely with them and you want to learn from them. Let’s face it: most of your work involves coding and they are unquestionably better at that. Also, they are your main interface with the data —they built the infrastructure collect each and every datum you use and most of the time they also curate the databases on which your work depends. Because of that, I wanted the Datathon to include not only people whose primary task is not analyzing data, but also people who are responsible for the data being collected and used in the standard operations.

  4. Global is harder but more rewarding. This is probably specific to the way my company is structured. Geographic dispersion across several continents affects the sense of community among people who perform similar functions but that happen to work in different offices. With events that force people to work together in cross-continental teams, you build better relations with your peers, and that is valuable to foster horizontal communication. At a more practical level, it is also a good way to learn how to truly exploit online collaboration tools.