Presented to an NSF-sponsored Workshop: “Future Technology to Preserve College Student Health and Foster Wellbeing (StudentHealth)”
Organized by Bonnie Spring and Andrew Campbell
Chicago, July 30-31 2015
Citizen science initiatives allow volunteer participants to collect observations, interpret data, and contribute to formal scientific projects. A large and growing number of opportunities are available for volunteer enthusiasts to contribute to science through field studies, organized events, classroom-based activities, and open science investigations.
Recently, along with my colleagues Dara Wald (who worked with Erik Johnston and me in the Center for Policy Informatics at Arizona State University, and is now at Iowa State University) and Rod Dobell of the University of Victoria, we completed a study of what I’ll refer to as virtual citizen science (or VCS).
Short History of VCS
VCS is one particular approach to citizen science, where volunteer participation is facilitated through websites and Internet-enabled mobile applications. This approach has grown since the mid-2000s, developing amid larger growth in opportunities for using the Internet to access information, enjoy entertainment, engage in game play, and pursue educational activities. Virtual citizen science has also developed in response to increasing scientific data flows that have outstripped the capacity of professional scientists to analyze them, and takes advantage of the desire of some Internet participants to contribute rather than just consume content.
You’re probably familiar with the most prominent example of this – the Zooniverse, which started as a platform to have Internet-enabled volunteers look at images of galaxies and then classify them by their shape.
Jointly headquartered at Oxford University and right near here at the Adler Planetarium, since its launch in 2009 the Zooniverse has grown to a current count of 29 active projects and 9 retired ones, with over 1 million registered participants.
The original version of this VCS model was called ClickWorkers, created by NASA to solve a particular problem: the NASA Mars Viking Orbiter had completely photographed the surface of Mars, but these photographs needed to be annotated in order to be useful for future exploration. But rather than do what they normally did – which was to create a postdoc position with the mind-numbing task of identifying the craters and classifying the features of the landscape – NASA thought that maybe this rocket science wasn’t rocket science after all and that circling craters on a meter-by-meter image was something that most people could do. So they put the images online, available to anyone with an Internet connection, and provided a simple annotation tool with instructions to draw circles around craters.
For a task that was estimated would take two postdocs two years to complete, the response was astonishing – within six months, 85000 volunteers had reviewed all images and had made 1.9 million annotations. To guard against inaccuracies and even mischief, each image was annotated independently 20 times and their annotations were averaged. An analysis of the quality of markings showed “that the automatically-computed consensus of a large number of clickworkers is virtually indistinguishable from the inputs of a geologist with years of experience in identifying Mars craters.” (“Clickworkers results: crater marking activity,” July 3, 2001; previously available at http://clickworkers.arc.nasa.gov (accessed April 25, 2005), quoted in Benkler, Yochai, and Helen Nissenbaum. “Commons‐based Peer Production and Virtue.” Journal of Political Philosophy 14.4 (2006): 394-419.; p. 397). The original Clickworkers site no longer exists. The article that describes the experiment (Kanefsky,B., N.G. Barlow, and V.C. Gulick. 2001. “Can Distributed Volunteers Accomplish Massive Data Analysis Tasks?” Lunar and Planetary Science XXXII (2001) 1272-1273) can be downloaded at http://www.digitalfishers.com/Clickworkers.pdf).
The Clickworkers project was a particularly clear example of how what was seen as a professional task requiring the efforts of highly trained individuals on full-time salaries could be reorganized so as to be performed by tens of thousands of volunteers in increments so small and simple that the tasks could be completed more quickly and on a much lower budget. With this successful demonstration of a crowd-based science activity, the VCS model was born.
This led to the creation of Galaxy Zoo and later the wide-ranging Zooniverse which includes VCS opportunities spanning astronomy, marine and land-based ecology, cell biology, climate sciences, and the humanities) and its many imitators (included a project I helped create at the University of Victoria called Digital Fishers).
What Makes a VCS Site Successful?
So as I said, we recently reviewed a sample of VCS projects to determine indicators of success. The central findings of our study is that it’s not enough for web and app design of VCS projects to focus on standard usability criteria. Rather, that VCS projects must focus on engagement (that is – why should an Internet-based volunteer choose their project over another, and – more to the point – why should they choose citizen science over one of the many other activities the Internet affords?) and retention (that it, how to keep them doing what they started? Because having many volunteers do one thing is great; having them do two is really great, but having them do several is really powerful). So we propose that successful VCS projects will do these three things well – engagement, usability, and retention. Unfortunately, what we found it that VCS sites do one thing well – usability – but they don’t consistently do engagement and retention well. And the reason they discount engagement and retention is because VCS projects rely on the inherent motivation of volunteers to contribute to citizen science projects and don’t worry themselves about whether the volunteer is gaining from the experience, or will find a reason to stay connected.
VCS is Dead
So that’s the current state of VCS, where large numbers of volunteers perform micro-scientific tasks for free so that science projects can plow through massive amounts of data. So far so good. Except for one big problem: this model is dead. And even if it’s not dead yet and doesn’t want to go on the cart, it will be dead soon enough. And the model is dead because the premise of the VCS model is now obsolete. The premise was elegant in its time: that when the increasing flood of data that computers could not interpret – largely images and sounds – coupled with shrinking science budgets, met the magic wand of Web 2.0 (where data could be provided to untold numbers of volunteers and their input easily collected), huge amounts of free labour were suddenly available in science labs around the world.
But there is one key element in that model that is obsolete, and it’s that the problem – too much data that computers can’t evaluate – is rapidly disappearing as machine learning is catching up to and surpassing the pattern recognition and interpretation skills of people. We tested this recently with the Digital Fishers video data, pitting volunteer citizen scientists against experts and machine algorithms. The findings demonstrate that the machine pattern recognition software is catching up and surpassing the abilities of experts and citizen scientists, signaling that the premise underlying the VCS model will soon be obsolete.
So the argument that science projects will be able to put their data up and have volunteers annotate it will become increasingly untenable as those volunteers come to understand that their efforts aren’t really necessary, further hampering the ability of projects to engage and retain volunteers.
(Now, the core motivation underlying citizen science will still be there, having the time, energy and commitment needed to do large numbers of small task-like activities. Because the VCS model isn’t just a demand side story – there’s also a supply side which speaks to the desire of the participants to do something meaningful – what Clay Shirky has called “cognitive surplus” and Yochai Benkler has called “commons-based peer production”. People are tired of passively watching TV and want to do something meaningful with their spare time. This gets into the motives behind citizen science, and how we respond to the displacement of the dominant model by continuing to engage that element of VCS motivation that will continue to exist even if the old model no longer works.)
So the task now is to take what worked in earlier incarnations and create new models. And this is where I think the interesting things in VCS are happening. Let me briefly go through three examples that illustrate what I think are some principles we should be using when considering future VCS initiatives, and I’ll conclude with some thoughts on how I think these developments are moving us towards a different VCS – what I’ll call (in order to salvage the acronym) virtual citizen-ship.
1. Technology enhanced community support networks through passive sensing, observation and networking: there are many examples of how we can use the ubiquity of mobile devices to contribute environmental observations that require very little effort on the part of the participant. The one I particularly like is the Tile, a bluetooth device that helps you find your keys when they’re lost.
I bought these for my sons after they lost their very expensive door fobs (I like giving gifts that have at their core an element of frustration: “stop losing your damn keys! and happy birthday!”).
Now, being able to use your phone to find your misplaced keys is a slightly interesting technology (though I would just as soon have this inside a golf ball because that would be really useful). But that’s not what I find interesting about the Tile. What I’m really excited about is the invisible community of searchers that Tile owners constitute. Because in installing the Tile app and having it running in the background on my phone, I’m also searching for and finding other Tiles that aren’t mine, and other people are doing the same for me. I tested this in Chicago by marking my Tile as lost, turning off my phone’s search capacity, and walking around the city with my Tile in my pocket. Within hours, someone found my Tile simply by walking near me. I got a notification it had been found
and I was able to send them an anonymous thank you.
Nerd alert! Two people completely unknown to each other are connected in a helper/helped relationship through a simple technology. This is what I’d call a low effort, invisible, unconscious, enhanced community support network built through passive sensing, observation and networking. This approach builds on earlier distributed computing projects like SETI@home, Folding@home and Fightaids@home where the unused computer processing cycles of millions of volunteers with computers connected to the Internet were harnessed to process vast data sets. But the Tile model is different because while the primary motivation is to get the benefit of being a Tile owner (i.e., being able to find my keys), what is really cool is the way that I have become part of a network that helped me while giving the finder an extremely low effort way of helping an unknown community member.
2. Technology enhanced community support networks through aggregating volunteers’ self-interested contributions. This model is akin to citizen science initiatives that ask volunteers to collect and upload data from their local environment as a way to expand the data collection capacity of science projects. The leading example here is the work of the Cornell Lab of Ornithology and the Audubon Christmas Bird Count, except that it relies on the self-interestedness (if that’s a word) of the contributor, rather than some intrinsic motivation like contributing to science. In this case, I think the prototypical example is Patients Like Me.
PatientsLikeMe is a patient-powered research network that aims to improve lives and develop a research platform to advance medicine. On PatientsLikeMe’s network, people connect with others who have the same disease or condition and track and share their own experiences. In the process, they generate data about the real-world nature of disease that can help researchers, pharmaceutical companies, regulators, providers and nonprofits develop more effective products, services and care. Participants engage with the site because they get answers to questions like “Is what I’m experiencing normal?” or “Is there anyone out there like me?”
Patients Like Me relies on a fundamental quid-pro-quo in VCS, that the reason that people contribute is because they get benefit from their contribution. And the more that community members contribute, the more benefit individual contributors get. (One thing that needs to be guarded against in such a system is the problem of free-riders who do not contribute but rather take the benefit of the collective contributions without contributing any new knowledge.)
Another good example is Waze, a GPS-based driving and navigation app for smartphones that provides turn-by-turn information. As users use Waze to help with their commutes and trips, they benefit from the travel times and route details like accidents and construction that are submitted by other users. As with Patients Like Me, users turn to Waze for the information it contains. But once they’re there, their use contributes more value back to other users.
3. Technology enhanced citizenship through a platform for DIY governance. The future of citizenship, in my opinion, will continue to see a shift from government (as institutional response to collective problems) to governance whereby people come together to address public problems either through government or some other alternative arrangement. DIY governance is a shorthand that suggests that there are problems that can be addressed through people coming together to solve them, and technology can be used to facilitate this coming together.
This is a bit of a stretch from citizen science, except that it builds on previous examples where volunteers contributed insights about what was happening in the field or through scanning digital data, and brought these issues to the attention of projects leaders. One of the best early examples of this is the Guardian’s use of crowdsourcing to interpret Members’ of Parliament expense claims from the UK (see a fuller description here: https://jlphd.wordpress.com/2009/07/05/more-public-policy-crowdsourcing-uk-mps-expenses/). Later examples are the UK Fixmystreet and the US SeeClickFix, which are web-based services designed to help citizens report non-emergency issues in their neighbourhood.
The submissions can be submitted via a web interface, by iPhone, Blackberry and Android reporting apps and a Facebook application. Local government officials receive alerts about submitted issues or can track issues that citizens submitted via the so-called “Watch Area” they are responsible for. The platform allows for direct feedback mechanisms: Local government officials assign a work order number and can change the status of the repair (from open, in progress, to fixed). Citizens are automatically informed about changes in the status of their reported issues allowing for a full feedback cycle. The service is integrated into the social networking services Twitter and Facebook and provides map-based reporting widgets for government and newspaper websites. The problem here is in getting past what people think to what they know or perceive, to get beyond web 2.0 platforms that serve as a medium for advancing an agenda to become platforms for sharing in our governance. And getting people to be active participants in their governance and not just complaining about what doesn’t work.
And one further step is providing a platform for people to take action in support of their governance once they’ve identified a governance failure. My favourite example here is Adopt-A-Fire-Hydrant, a Code for America project created in Boston. Adopt-a-hydrant allows citizens to claim responsibility for shoveling out fire hydrants after heavy snowfall. After a heavy snowstorms, buried hydrants cannot be immediately cleared by city work crews, and if uncleared can cause dangerous delays for fire fighters. This map-based web app allows individuals, small businesses and community organizations to volunteer in shoveling out specific hydrants. This approach to DIY governance gives community members a platform for identifying civic problems and making a public commitment to take action that can have personal as well as community benefit.
Principles for a New Approach to VCS
So the take home message, which I don’t intend as pessimistic but rather realistic, is that new models of citizen science need to focus on providing value to the participant first, reduce opportunities for agenda-setting, to do so in as easy a way as possible, and then using that “lazy self-interestedness” as the basis for encouraging people to take action in support of their own governance.
VCS: Transitioning from Virtual Citizen Science to Virtual Citizen-ship
The original model of virtual citizen science was built on a number of factors – including that there was a lot of micro-task work to be done in support of science but a lack of resources available to do it – but was ultimately an exercise in volunteers doing things to support science endeavours.
The models I’ve suggested might reflect what the future of citizen science could look like start by turning that relationship around, asking first what are the interests of the potential contributor? Coupled with a focus on simplifying the input of the contributor, VCS projects will have to focus on making a case to the potential participant that it is in their interests to contribute to the project, and that they will receive something in return.
But these shifts in VCS also serve to strengthen the community of participants, and linking them directly to their governance system. And this shift from a participant-project relationship to a participant-community relationship is why I’m suggesting that the future of VCS isn’t virtual citizen science but rather virtual citizen-ship.
If we think about the characteristics of the good citizen – critical thinking, civic participation, personal responsibility, helping others, fairness, rule-adherence, promoting inclusiveness, resolving disagreements peacefully, and participating actively to make a positive contribution to the community – these would seem to be ideal characteristics that we might wish to promote through technology. While VCS (virtual citizen science), with its narrow focus on providing free labour to science initiatives may be mostly dead, VCS (virtual citizen-ship) may be the model that can fill the void for people longing to do something meaningful with their cognitive surplus.