This semester I was tasked with creating a new recruitment video for the Computer Science department. I decided to do something a little offbeat, that might attract a different variety of folk. Here is the result!
Seven Reasons OU Computer Science is Better Than Hogwarts from Rachel Shadoan on Vimeo.
The internet is a many-splendored thing, and its advent and success has vastly expanded the reach of your average ethnographer. Information that we could once only access in face-to-face interactions can now be collected remotely, from across an ocean, across continents, across time and space. This is a great opportunity to learn more about people, largely unrestricted by geography and time.
However, it also presents a significant challenge. We have to develop new methods and tools to perform good remote ethnographies. In previous projects we have defaulted to Skype video calls–the next best thing, we thought, to in-person interviews. But when you live eight to ten time zones away from a participant, negotiating school, work, and leisure schedules to set up a Skype call can be nearly impossible. And, as much as we love our participants, at 3 am our time, we ethnographers much prefer to be sleeping.
The obvious solution to this was to develop an email interview. In our mind it had several advantages. It allowed the participant to respond to the questions in their own time, in a low-stress way. It removed the need for scheduling ninjitsu. And, it produced a ready-made transcript of the interaction, easing the field-note writing and analysis processes.
Our first iteration of the email interview was something like an open-ended survey. It explained the project and supplied the appropriate participant information and consent forms. Then, it listed the questions. It was in many ways a participant friendly version of the interview guide.
While the turnaround on the email interview surveys was really good from a time perspective, we felt that the answers we were getting were very short, to the point, and formal. This is in contrast to our in-person interviews, where answers to one question would often meander through several equally interesting subjects in the process of their completion.
So I thought a lot about how the in-person interviews were different from the email interviews, and I realized it was that with in-person interviews, the participant doesn’t know all of the questions you will be asking up front. Usually we tell them what kind of questions we will be asking, or what kind of information we are looking for, but the specific questions are unknown. As a result, the participant will often include a lot more information in the answer to each question. There was something about seeing all of the questions all at once that was cutting off this meandering; something about having all of the questions in front of the participant at once made the answers short and to the point.
So our solution was to send email interview questions one at a time.
This was a tremendous success. When we sent the questions one at a time, the answers were long, rich, and varied. Compare the two images below–the red circles surround answers to the same question.
We have tried out the second iteration of email interviewing on several participants, and have been blown away with their responses.
There are probably restrictions that come with this method. It is probably not appropriate for people who do not normally communicate via text-based mediums. (Our participants are very comfortable with the written communication of the internet, so in our case this has not been an issue.) It might also be less appealing to very busy executives–our coursemate Cora is doing a project with such folks, and she feels that her participants would become irritated with the process after three questions.
All things considered, however, we feel that the potential benefits of this method are numerous. Beyond the increased time that the participant has to respond to a question, there is also the increased time that the ethnographer has to consider the response, research, and contemplate a next question. While we thought that text would remove some of the richness present in non-verbal communication, we find that the opposite is true–for people who have grown up communicating via text, email, and chat, they are accustomed to explicizing the implicit, and often expand more on their feelings via text than they would likely do in person.
Finally, we feel it holds great promise for participants for whom an in-person interview would create a great deal of anxiety. This allows them time to answer the questions, and alleviates much of the pressure they might feel when faced with a note-scribbling, recording-toting ethnographer.
I want you to take a look at these visualizations. These are all charts depicting what skill a Plant Wars player is training, as well as how many stat points they receive as a result of their fertilizing of their plant. (There is a direct correlation between the amount of fertilizer used and the stat points gained. More fertilizer = more statpoints. So, the size of the bubbles indicates both the number of stat points gained, as well as the amount of fertilizer used.) All of the charts are on the same time scale, and so you can compare them to each other vertically.
Now, what I want you to notice in these charts are the large white spaces, which are times with the player is not fertilizing (and indeed, likely not playing the game at all, possibly even sleeping). Notice that around June-July 2009, the location of the white spaces begins to migrate. (This is most obvious in Roger Shant’s graph).
Now, when I first noticed this down-time (I call it that since I don’t know if they are sleeping are not) migration, I assumed that someone had had a shift change at work, or started a new semester. However, after seeing the same pattern on the same timescale across four different players, I began to think something was rotten in the state of Denmark. I assumed that something had gone horribly wrong with my data. Somewhere in the processing, something was terribly, terribly wrong. Maybe the log files themselves were corrupted. However, after hashing this out with Jon and ruling out all of the potential causes we could think of, we decided that all of these players might actually have changed their schedules at the same time. Indeed, because they are all really active players, it’s possible that they were changing their down-time schedules because of each other.
However, it’s impossible to know without asking them.
I can speculate all day long on why this pattern appeared. I can talk myself blue in the face spouting ideas and hypotheses on what exactly caused this shift in down-time.
But I won’t know. In fact, I can’t know, not until I ask the players whose data I’m looking at. Only once I have the story to connect with the data will the patterns in the numbers make sense. Only when I have the story and the data can we paint a complete picture.
For instance, notice that large bubble that appears on all four charts? That was another weird thing in my data that I assumed was a mistake. I asked Jon about it, and apparently there had been a server issue which had potentially eaten the stat points that players had gained during the day. So, on that day he gave players a huge stat point reward for fertilizing, to compensate for the stat points that the server issue had deleted.
And now, for more data visualizations than you can comfortably shake a stick at. Here I have charted all of the fertilizes for January-March 2009. These are a little hard to read, because there’s so much information here, but take note of the sort of density spike in the middle of the chart. Basically it looks like a stalagtite hanging from the top of the middle of the chart, and a stalagmite growing from the bottom of the middle of the chart to meet the stalagtite from the top. This is particularly visible in the chart showing only the Attack and Defense trains. Well, as it turns out, that increase in activity came from a spammer on Facebook. Apparently, around that time someone ripped off Plant Wars to create a Facebook game, and then began spamming Facebook like crazy. This resulted in a significant increase in traffic to the real Plant Wars, and the accompanying increase in trains.
For the full story on the Facebook Plant Wars rip-off, read Jon’s blog post about it.
We have some new visualizations for your viewing pleasure, this time of individual Plant Wars players!
In Plant Wars, players can increase their plant’s abilities by fertilizing, or “training”. Players can choose what ability they want to increase when they fertilize. Fertilizing Attack (referred to as Potency in the game) makes their plant’s attack stronger. Fertilizing defense (referred to as Girth in the game) makes their plant less suspectible to attack, while fertilizing speed (called Responsiveness in the game) makes their plant more agile and able to avoid attacks. When a player fertilizes, they occasionally have to successfully complete a CAPTCHA, a device to tell humans from computers. This weeds out cheaters. You can read more about the Plant Wars CAPTCHA here on Jon’s Plant Wars development blog.
Here, we have the training data of two Plant Wars players. One of the players is fairly new, having only been a member of the Plant Wars community for 100 or so days. The other player, a longer member, is now an admin.
Download the visualization PDF for a closer look!
Notice that recently, the only thing this new player has been training is Defense. This is possibly because of a Girth competition currently being held in the game. Also, notice the large blank spot from 12:00ish to 19:00ish. Nothing in happening there, so we think this is probably when the player is sleeping. (Otherwise, this player never to sleep and that blank spot is indicitative of work or school. However, given the way the blank spot travels later over time, we think it’s probably sleeping that the player is adjusting to the level of light as spring turned to summer.) Note that all times are Central time (ie, the same time as it is in Chicago, Dallas, or in this case, Tulsa, Oklahoma).
Download the visualization PDF for a closer look!
Our seasoned player takes a more methodical approach to training; he trains particular all day for several days in a row, and then moves to another skill. This appears to be a technique that he developed over time–in the early days of his tenure with Plant Wars, the kinds of skills he trained overlapped (take a look in the upper left hand corner to see that). Again, we think the blank spots are when this player sleeps, which seems to have changed over time–possibly because of a shift change.
A note on building the visualizations: these were once again processed from text log files using a Java program that I wrote, and then visualized using Excel. I’ve been experiementing with Prefuse Flare, but Flash doesn’t like large data sets one bit, so we’re kicking it old-school for the time being.
Part of the documentation we assemble before every project is an interview and observation guide. It’s supposed to serve as a checklist for supplies, as well as a guideline for how to conduct the interview. It includes the information we are trying to get at, as well as the questions we’re using to do so. Here is our interview guide for our current project. (If you would like a copy of our interview guide, you can download it here as a Word document, and here as a PDF.)
One of the greatest challenges for ethnographers is what to do with all of the data we collect. In addition to the physical notebooks full of scratch notes, we also have lots of digital data–from audio recordings of interviews to photographs taken during observations to the field notes we write up after an interaction with participants. Organizing and securely storing the data is a difficult proposition; how can we arrange things so that we can find them, but so that they will still be safe? These are the solutions we’ve implemented for this project.
Moving Data Into the Cloud
Nothing makes me more nervous than only having one copy of a document. I am a big fan of frequent backups, or, even better, letting someone else worry about the persistence of my documents. Enter Google Documents. If you’re not familiar with Google Documents, it’s definitely worth a look. For simple (read: not flashy) office processing tasks, Google Documents provides a solid set of free tools, and allows you to store your documents on Google’s servers. This has numerous advantages.
Safety: Protect your files from hard drive failure
Hard drives are vulnerable, and laptop harddrives are particularly so. Drop them one too many times, get them a little too close to a refrigerator magnet, spill some soda on them, and in a blink, your precious data is gone. I much prefer to store my data on someone else’s servers. Google’s servers are the lifeblood of their business–if those servers fail catastrophically, Google loses catastrophic amounts of money. So they hire loads of people to look after the health and wellbeing of Google’s servers to ensure that no such thing happens. Because Google and my interests as far as server reliability are in line, I feel very comfortable storing my documents there.
Security: No need to transfer via insecure physical media
Storing documents there also means that you can access them from any computer with an internet connection, removing the need for potentially insecure physical storage devices. Flash drives (aka pen or thumb or jump drives) are very handy, it’s true, but not only are they easy to misplace–a data security nightmare–but they are also notorious for transmitting viruses from one computer to another.
Sharing: Easily share documents with other researchers across the world
In an increasingly global society, researchers often end up collaborating remotely. Instead of emailing back and forth different versions of a document, Google Documents allows you to share a single document and edit it simultaneously and collaboratively.
For this project, we are storing all of our field notes on password-protected Google Documents. However, Google Documents is not the only cloud service we are using; we are also using the WordPress servers to facilitate our data management. Our audio, video, and image files are stored privately on our WordPress project blog, which offers the same advantages I described above. For this project we are not using Flickr, but it is also a fantastic cloud service that will help you protect your pictures for posterity.
File Naming for Fun and Profit
When you have many, many files that have to be organized, it’s important to be methodical in their naming. The convention we are using for this project is as follows:
Description/ParticipantPsuedonym_DDMonYYYY_Location_City_Researcher’sIntials
(Researcher’s intials are only required if only one researcher is involved in the creation of the file, as in the second example below in which I recorded audio).
Since that’s a little opaque, here’s a couple of examples:
Field Note File: LucasBowser_15Jun2010_RTW_Dundee
Audio File: LucasBowserAudio_15Jun2010_RTW_Dundee_RS
Note that we never use a participant’s real name in the naming of a file–it is always the participant’s psuedonym.
Speaking of psuedonyms, let’s talk about how we keep track of all of the folks we talk to.
Participant Psuedonyms and Tracking Database
Alicia and I are using a particularly fun mechanism for arriving at participant psuedonyms for this project. Each participant gets one name from a video game character. Their other name is something evocative for us. We spend a little bit of time discussing the names we choose so that we both understand the thought process behind the name selection, and then the name becomes a memory tool for us, encapsulating more information.
For instance, if I were to develop a psuedonym for Alicia, I might come up with something like, “Lara Chopin”.
Lara comes from Lara Croft, an image of whom is currently Alicia’s computer’s desktop wallpaper. I’ve also seen Alicia dress up as Lara Croft from time to time (for Halloween or other fancy dress occasions). Further, Lara is similar to Laurey, the name of the heroine from the musical Oklahoma, which is a nod to Alicia’s birthplace.
Chopin was an intensely creative Polish pianist and composer, which pays homage to Alicia’s creativity but also to her heritage.
So, as the researcher, I would read “Lara Chopin” on a file name, and immediately remember that this is the Polish-American participant from Oklahoma who is very creative, and kind of a badass. It’s a helpful memory device.
However, this all needs to be recorded somewhere. So we have created a participant tracking database (as a spreadsheet on Google Docs, naturally) that maps the participant’s name to their psuedonym, and indicates whether they are all squared away on ethical and field note fronts.
Field notes, for the ethnographically uninitiated, are detailed descriptions of field work (interviews with and observations of people).
Contrary to what the name suggests, field notes are not actually taken in the field while we are interviewing and observing people. Rather, while we are in the field we take scratch notes. In our scratch notes we write down things people say, make sketches of rooms, write down the time things happen, keep tallies of how many times that person has kissed a hamster in the last fifteen minutes, and a host of other information that might be useful to our research.
Once we leave the field and are back in our own homes or offices, we turn the scratch notes into field notes. This is a somewhat arduous and painful process, that involves turning things like, “6pham danc NW g 4:16-5 O” into a meaningful description like “At 4:16 pm, there were six people dancing in the northwest corner of the green, each holding small golden hamsters. They danced in a circle for five minutes.” It is best done as soon after the field work as possible, so that the appropriate details can be added in. Memory decays quickly, and then gets fuzzy and confused.
In traditional anthropology, field notes are not things that you share. They are, in many ways, considered the secret formulaes of anthropological research. Most researchers are loathe to share them. As a result, there is little standardization in how field notes are taken or taught, which creates a great deal of difficulty for an up-and-coming ethnographer. This year we have largely been left to our own devices in developing a methodology for writing field notes.
Old Methods
I have experimented with several different methods.
Ladies and gentlemen, the moment you’ve all been waiting for has finally arrived! The first data visualization of Plant Wars player interactions is finished!
The data used in this visualization tracked Plant Wars in-game banking transactions, of which there are several varieties. A player can deposit money into a bank, transfer it to another player, or transfer it to a “colony”. (A colony is an organized group of Plant Wars players, kind of like a club.) Players can also transfer donor days or seeds to other players or colonies. Finally, a member of a colony with appropriate administrative rights can transfer things to a user. This particular visualization doesn’t show the transactions by type, but the next version of it will do so.
The data started out looking like this:
[2009/06/17 09:51:55] 2842 deposited $1,782[2009/06/17 09:57:49] 1641 deposited $14,894[2009/06/17 10:00:10] 3361 deposited $2,807[2009/06/17 10:01:16] 3330 deposited $2,500[2009/06/17 10:03:04] 69 deposited $1[2009/06/17 10:05:40] 1641 deposited $16[2009/06/17 10:07:04] 2387 deposited $168[2009/06/17 10:07:22] 2387 deposited $201[2009/06/17 10:16:16] 418 deposited $3,734[2009/06/17 10:21:21] 644 -> 2720 : $1,000
I processed the data into comma-separated-value files (csv) using a simple Java program, and then built this visualization using Microsoft Excel.
Now the data looks like this (click on the image to go to Flickr, where you can get a GINORMOUS version of it. Or click here, to get a PDF):
The bottom and top of the chart are midnight, and the middle is noon. As you can see, banking transactions happen most often immediately after noon and immediately after midnight, thinning out significantly between 5 am and 9 am. While there is a slowdown in banking transactions in the afternoon and evening as well (after the noon rush), it is not as pronounced as the early-morning slowdown. We suspect that the transaction “rushes” happen because of the way time and money are handled in Plant Wars.
Future versions of this chart will show the transactions by type and by user.
In case I hadn’t bombarded you with enough content today, or in case you happened to be someone in my audience who could not care less about design ethnography (I suspect there are some of you out there), I offer you something new and completely different to investigate: my review of Machinarium.
I love Machinarium; I think it’s a great game, and I think you should play it. If you’re reading this blog, it’s likely that you can play it–it’s a downloadable Flash game, so you can probably even play it if you (*gasp*, the HORROR) only have access to a Mac. It’s point and click, and nothing jumps out to kill you, so it’s good for even high-strung folks. Arguably, it’s perfect for high-strung folks, because as I mention in the review, the game is so laid back that the calm seeps into your soul. It’s cheaper than a trip to a spa trip or a vacation on a remote tropical island! (Mom, Shanna, if you’re reading, you would love it and should download it immediately. Why procrastinate on Facebook when you can procrastinate with darling robots?)
Coming soon: a list of the things that I didn’t get to put in my article about Machinarium.
