Wikipedia was founded on radically open collaboration. Pick any article you know something about, and the “edit this page” link at the top allows you to make an instant change.
By editing a Wikipedia article, you get instant access to the “guts” of the page. Whether you’re just changing some text, adding a reference, or inserting an image: Wikipedia is open to new contributions at any time.
Instead of moderating edits when they are made, the wiki model has always been to systematically review changes as they come in:
- by storing every version of every article ever created;
- by allowing anyone to restore prior versions;
- by providing numerous tools for experienced editors to review and patrol changes.
This gives writers the instant gratification to see their changes published, while - hopefully - leading to high quality articles over time as more and more people review and improve a page.
In addition to the constant mutual peer review, there are countless Wikipedia processes used to identify articles of the highest quality, articles with various problems, or articles that should be deleted. (The Wikipedia Signpost, a community newsletter, has just published an interesting history of the featured article candidacy process.)
New processes and technologies for quality assurance are developed and tested all the time. But few are as long-awaited and potentially game changing as FlaggedRevs.
The FlaggedRevs Extension
The German Wikipedia is currently trialing a new extension (what’s an extension?) to our software, called “FlaggedRevs“. The extension, which has been under development for more than a year, is a very powerful set of tools for reviewing, labeling and selecting changes made in a wiki. We believe that FlaggedRevs represents a milestone in the development of wiki technology. To our knowledge, there is no other tool available today that provides comparable functionality.
So what, exactly, does it do?
In a nutshell, FlaggedRevs (short for “flagged revisions”) can be used to give a defined group of authors the ability to attach quality labels (flags) to individual versions (revisions) of articles. It can also be used to determine which version of an article should be shown to a reader visiting the wiki: the most recent one, or the highest quality version available?
These two features are not necessarily linked. In the most basic use scenario imaginable, FlaggedRevs can simply be used to patrol a wiki for malicious changes (”vandalism“). When a change has been found not to be malicious, a trusted user can label it as such. This has two key advantages compared to the current patrolling model:
- It reduces duplicate effort in basic change patrolling, allowing users to focus on un-reviewed changes and thereby directing their attention more effectively.
- It ensures higher coverage of changes. In particular, when malicious changes are followed by good faith edits, malicious changes are sometimes overlooked. In the FlaggedRevs model, reviewers can systematically examine every change.
In addition, both human and non-human readers can select “known good” versions of Wikipedia articles which do not include malicious changes. Whether you’re a teacher printing Wikipedia articles for the classroom, a student using them for research, or a publisher creating a DVD copy, you can pick the articles which have been checked for basic vandalism by trusted editors, instead of simply choosing the most recent version.
As a user of the German Wikipedia, you will notice that some articles have the following icon in the top right corner:
This icon indicates that the version you are looking at hasn’t been checked for vandalism yet. (If an older version that has been checked is available, this is indicated below the icon.)
The End of Immediacy?
While this configuration is simple enough, it should be noted that until about a couple of weeks ago, the German Wikipedia was using a different setup in which any change by a user without the permission to review changes for vandalism (which includes all unregistered users and relatively new ones) had to be reviewed before becoming the default version shown to readers. In other words, if you were not in the group with permission to review edits, your own changes did not become the “live version” until someone else looked at them.
This was a controversial change, as some users felt it significantly reduced the incentive for new contributors to start editing Wikipedia. So far, there has been limited analysis of the data collected during this experiment, which lasted from May until July 2008, and we hope to analyze the effects in greater detail over the coming weeks. (Some real-time statistics are available, thanks to André Karwath.)
Should changes to Wikipedia by new and unregistered users be reviewed before becoming the default shown to readers? There might be a middle ground solution: On most articles, changes would continue to be applied immediately, under the assumption that the benefit of radically open collaboration is greater than the risk. But, on a subset of pages, changes by unregistered and new users would have to be reviewed before becoming visible. This subset could consist of articles which are frequently the target of vandalism, such as the biography of the US President, but it could also include those pages which have reached a very high standard of quality as determined by the Wikipedia community. In other words, when the drawbacks of radical openness outweigh the risks, editing would be throttled.
This would, in fact, represent an opening up of Wikipedia rather than a closing down, as many of the affected pages are currently “semi-protected”, meaning that they cannot be edited at all by new and unregistered users due to the perceived risks of malicious edits. Being able to make changes that do not immediately become visible is surely preferable to not being able to make changes at all.
What’s next?
The Wikimedia Foundation has authorized all Wikimedia project communities to conduct experiments with FlaggedRevs through a process of self-organization. The process by which a Wikimedia community (e.g. the French Wikipedia, the Russian Wikibooks, etc.) can request the FlaggedRevs extension to be enabled is open and transparent. As the process unfolds, we will try to support the communities by collecting data about the use of the extension. Depending on our findings, we may eventually make a simple configuration of FlaggedRevs the default for all wikis.
There are other potential future uses of FlaggedRevs:
- Use for identification of article versions which meet standards of accuracy and quality as determined by experts. Potentially, FlaggedRevs could interface with external expert communities (such as universities or expert-driven encyclopedia projects like the Encyclopedia of Life) to identify versions of Wikipedia articles which meet scholarly standards of quality.
- Use for identification of article versions which meet internally defined standards of quality beyond the simple check for vandalism. The original German Wikipedia proposal for FlaggedRevs includes a more in-depth community quality review stage, which is still being discussed. A simple way to tie into community review mechanisms would be to use FlaggedRevs to “tag” versions which have passed through processes like “Featured Article Candidates“.
- Use to collect basic reader feedback on articles. Asking our readers whether information in Wikipedia articles is useful to them, and whether it meets their quality standards, could be a good way to track reader satisfaction over time. The lead developer of the FlaggedRevs extension, Aaron Schulz, is currently implementing such reader feedback tools.
The development of this technology represents the commitment of the international Wikimedia community to achieving the highest possible standards of quality in all our projects. In particular, the German Wikipedia community and the German chapter have been leaders and pioneers in this process. Philipp Birken from the German chapter gave a compelling presentation at the recent Wikimania on this very topic.
We welcome your feedback in making this technology more useful. An English demo version is set up in the Wikimedia Labs.
Erik Möller, Deputy Director
