Wiki metrics, rubrics and collaboration tools

Draft

<pageby nominor="false" comments="false"/>

Introduction

The purpose of this article is survey various tools that allow to measure what participants (and students in particular) do in a wiki. Survey literature will include topics that are only indirectly related but of technical interest, e.g. the literature on trust metrics. In addition we will look at tools that will enhance wiki participation and collaboration. Finally, we will try to outline a few paths for further development.

Built-in wiki collaboration and quality tools

Some wikis include a reviewing process that allows authors to write drafts that only are published once it has been reviewed. A typical example is found in Wikimedia's Wikibooks. This is implemented with the Flagged Revisions extension.

Quality policies and guidelines

Quality in a wiki can be determined by user evaluation (including self evaluation). In this Wiki, we use a simple templates like "stub" and "incomplete" to convey self-assessed quality statements to the reader.

“Wikipedia has developed several user-driven approaches for evaluating the articles. High quality articles can be marked as “Good Articles” or “Featured Articles” whereas poor quality articles can be marked as “Articles for Deletion”” (Wöhner and Peters, 2009). Features and processes are documented in Wikipedia's Good article reassessment and also Good article criteria

Links

Specialized wiki conferences

• http://www.wikisym.org/

Bibliography

Trust and quality metrics

• Mark Kramer, Andy Gregorowicz, and Bala Iyer. 2008. Wiki trust metrics based on phrasal analysis. In Proceedings of the 4th International Symposium on Wikis (WikiSym '08). ACM, New York, NY, USA, , Article 24 , 10 pages. DOI=10.1145/1822258.1822291, PDF from wikisym.org

“Wiki users receive very little guidance on the trustworthiness of the information they find. It is difficult for them to determine how long the text in a page has existed, or who originally authored the text. It is also difficult to assess the reliability of authors contributing to a wiki page. In this paper, we create a set of trust indicators and metrics derived from phrasal analysis of the article revision history. These metrics include author attribution, author reputation, expertise ratings, article evolution, and text trustworthiness. We also propose a new technique for collecting and maintaining explicit article ratings across multiple revisions.”

• Aniket Kittur, Bongwon Suh, and Ed H. Chi. 2008. Can you ever trust a wiki?: impacting perceived trustworthiness in wikipedia. In Proceedings of the 2008 ACM conference on Computer supported cooperative work (CSCW '08). ACM, New York, NY, USA, 477-480. DOI=10.1145/1460563.1460639 http://doi.acm.org/10.1145/1460563.1460639, PDF from psu.edu

“Wikipedia has become one of the most important information resources on the Web by promoting peer collaboration and enabling virtually anyone to edit anything. However, this mutability also leads many to distrust it as a reliable source of information. Although there have been many attempts at developing metrics to help users judge the trustworthiness of content, it is unknown how much impact such measures can have on a system that is perceived as inherently unstable. Here we examine whether a visualization that exposes hidden article information can impact readers' perceptions of trustworthiness in a wiki environment. Our results suggest that surfacing information relevant to the stability of the article and the patterns of editor behavior can have a significant impact on users' trust across a variety of page types.”

• Andrew Lih, Paper for the 5th International Symposium on Online Journalism (April 16-17, 2004) University of Texas at Austin, PDF CiteSeerX

“This study examines the growth of Wikipedia and analyzes the crucial technologies and community policies that have enabled the project to prosper. It also analyzes Wikipedia’s articles that have been cited in the news media, and establishes a set of metrics based on established encyclopedia taxonomies and analyzes the trends in Wikipedia being used as a source.”

• Thomas Wohner and Ralf Peters, Assessing the Quality of Wikipedia Articles with Lifecycle Based Metrics, Proceedings of the 5th International Symposium on Wikis and Open Collaboration (WikiSym '09).

“[...] quality assessment has been becoming a high active research field. In this paper we offer new metrics for an efficient quality measurement. The metrics are based on the lifecycles of low and high quality articles, which refer to the changes of the persistent and transient contributions throughout the entire life span.”

• Stvilia, B., Twidale, M.B., Smith, L.C. and Gasser, L. 2005. Assessing information quality of a community-based encyclopedia. In Proceedings of the International Conference on Information Quality, 442–454, (November, 2005), Cambridge, USA

Collaboration metrics

• Sofia J. Athenikos and Xia Lin (2009), Visualizing Intellectual Connections among Philosophers Using the Hyperlink & Semantic Data from Wikipedia?, Proceedings of the 5th International Symposium on Wikis and Open Collaboration (WikiSym '09).

PDF

• Víegas, F., Wattenberg, M. and Dave, K. 2004. Studying cooperation and conflict between authors with history flow visualizations. In Proceedings of the SIGCHI Conf. on Human Factors in Computing Systems, 575–582, (April, 2004), Vienna, Austria.

Embedded Wiki tools

• Sabel, M. 2007. Structuring wiki revision history. In Proceedings of the 2007 International Symposium on Wikis. 125-130. (October, 2007), Montreal, Canada.