Learning management system: Difference between revisions

The educational technology and digital learning wiki
Jump to navigation Jump to search
 
(102 intermediate revisions by 11 users not shown)
Line 1: Line 1:
== Definition ==
== Definition ==


A Learning Management System (sometimes also called "Course Management System", "Pedagogical Platform",. "E-Learning Platform") is a software system that delivers [[courseware]] plus [[e-tutoring]] over the Internet.
A Learning Management System (sometimes also called "[[Course Management System]]", "Pedagogical Platform", "E-Learning Platform") is a software system that delivers [[courseware]] plus [[e-tutoring]] over the Internet.


LMS should not be your starting point. Typically, when decision makers talk about E-Learning they want to know what system to install. First of all, so called LMS are not the only answer, you also can implement E-Learning with other tools (e.g. [[groupware]], [[content management systems]] or even a [[wiki]] or other hypertext system, often in combination with a [[forum]]. Second, in any case you should start by thinking about an appropriate [[instructional design]] that uses appropriate [[pedagogic strategy|teaching strategies]] for various [[learning type]]s.
LMS should not be your starting point. Typically, when decision makers talk about E-Learning they want to know what system to install. First of all, so called LMS are not the only answer, you also can implement E-Learning with other tools (e.g. [[groupware]], [[content management system]]s or even a [[wiki]] or other hypertext system, often in combination with a [[forum]]. Second, in any case you should start by thinking about an appropriate [[instructional design]] that uses appropriate [[pedagogic strategy|teaching strategies]] for various [[learning type]]s.


See also: [[LCMS]] (Learning content management systems)
{{quotation|“LMS” is a catchall term in SCORM. It refers to a suite of functionalities designed to deliver, track, report on and manage learning content, learner progress and learner interactions. “LMS” can apply to very simple course management systems, or highly
complex enterprise-wide, distributed environments.}} ([http://www.adlnet.gov/ SCORM 2004 3rd Edition Overview Version 1.0])


== components of an LMS ==
See also:
* [[LAMS]] A good [[learning design]] system that we can recommend (for people with serious activity-based e-learning in mind)
* [[Rapid elearning]] (for mostly simple facts & skills training needs)
* [[LCMS]] (Learning content management systems)
* [[Student management system]]
* [[Classroom management software]]
 
== Components of an LMS ==


Feature lists are getting quite long and it is not easy to decide which features are the most important unless you have a good background in various areas like [[instructional design]], [[ergonomics]] and systems management.
Feature lists are getting quite long and it is not easy to decide which features are the most important unless you have a good background in various areas like [[instructional design]], [[ergonomics]] and systems management.
Line 13: Line 21:
Typical features are:
Typical features are:
* Course Management, e.g. lists of courses, registration, credit information and syllabus, pre-requisites
* Course Management, e.g. lists of courses, registration, credit information and syllabus, pre-requisites
* Teaching Materials, i.e. [[courseware]]
* Display of teaching Materials, i.e. [[courseware]] (either through content authoring and/or importing standard [[IMS]] or [[SCORM]] packages developed with an external tool )
* Self-assessment quizzes
* Self-assessment quizzes
* Lessons tools: Authoring for contents (structured XML or HTML) and quizzing/testing (e.g. Java Script generators) or alternatively ability to import standard [[IMS]] or [[SCORM]] packages developed with an external tool (e.g. Dreamweaver).
* Content authoring (structured XML or HTML) of learning materials and other contents (e.g. instructions)
* Asynchronous Communication: email, forums
* Asynchronous Communication: email, forums
* Synchronous Communication: chat, whiteboard, teleconferencing,  
* Synchronous Communication: chat, whiteboard, teleconferencing,  
* Student tools: Home page, self tests, bookmarks, progress tracking, ....
* Student tools: Home page, self tests, bookmarks, progress tracking, ....
* Student Management Tools: progress tracking, on-line grading ([[assessment]]), ....
* Student Management Tools: progress tracking, on-line grading ([[assessment]]), ....
* User and group management
* Learner feedback: [[course evaluation]] surveys, test evaluation surveys etc.
Other features include:
* ecommerce functionality
* certification
* integration with other systems, e.g. single-sign login, other learning platforms
* support for informal and social learning
Usually LMS are closed circuit platforms (logins, restricted access to classes), so the idea of sharing contents and reusing products generated during classes does not exist in the world of "LMSs" (main-stream [[e-learning]]).
== List of software ==


Usually LMS are closed circuit platforms (logins, restricted access to classes), so the idea of sharing contents and informal Noah's generated during classes does not exist in the world of "LMS" (main-stream [[e-learning]]).
These lists '''needs updating''', therefore, before looking our lists please check the links section at the end of this page. Others do a much better job than we do. In addition, it is very difficult to find truly independant reviews by real experts - [[User:Daniel K. Schneider|Daniel K. Schneider]] ([[User talk:Daniel K. Schneider|talk]]) 16:15, 9 October 2013. As of 2019, there are probably a few hundred systems.


==List of Free / Open Source Softwares==
=== Free / Open Sourcource Portalware ===


* [http://www.angellearning.com/ Angel LMS]
Software to install on your own server, some are probably also availble through commercial hosting or cloud services.
* [http://atutor.ca/ ATutor] (used at TECFA for student projects)
* [http://bazaar.athabascau.ca/ Bazaar]
* [http://www.softwarepublico.gov.br/ver-comunidade?community_id=9677539 Amadeus] (the Brazilian Federal Government public software LMS)
* [http://atutor.ca/ ATutor] (used at TECFA our student's to play with)
* [http://bodington.org/ Bodington]
* [http://bodington.org/ Bodington]
* [http://www.docebolms.org/index.php?special=changelang&newLang=english DoceboLMS]
* [[canvas (LMS)]]
* [http://www.dokeos.com/ Dokeos] (used at TECFA for student projects and one course)
* [http://www.chamilo.org/ chamilo] (made by people who left the Dokeos development).
* [http://exe.cfdl.auckland.ac.nz e-Learning XHTML Editor]
* [http://lcms.chamilo.org/ Chamilo LCMS Connect]. Same team, but very different from Chamilo, [http://trylcms.chamilo.org/ Try it]
* [http://fle3.uiah.fi Fle3]
* [http://www.claroline.net/ Claroline] (Older sister of Dokos and Chamilo).
* Claroline Connect (Beta, ready in summer 2014?) - [http://dev.claroline.net/ try it]
* [http://www.docebolms.org/ DoceboLMS]
* [https://www.joomlalms.com/ JoomlaLMS]
* [http://www.dokeos.com/ Dokeos] (popular in french and spanish speaking countries)
* [http://exe.cfdl.auckland.ac.nz e-Learning XHTML Editor] (contents can then be imported to most LMSs).
* [http://www.efrontlearning.net/ eFrontLearning] (fairly user friendly, not tested, there are commercial variants)
* [http://fle3.uiah.fi Fle3] (research system, maybe dead)
* [http://www.anemalab.org/ GaneshaLMS]
* [http://www.anemalab.org/ GaneshaLMS]
* [http://www.ilias.de/ios/index.html ILIAS]
* [http://www.ilias.de/ ILIAS]
*[http://www.interactlms.org interact]
* [http://www.interactlms.org interact] (Dead link)
*[http://www.kngforge.uwc.ac.za/ KEWL.Nextgen]
* [http://www.kngforge.uwc.ac.za/ KEWL.Nextgen] (Dead link)
* [http://dotlrn.org/ LRN]
* [http://dotlrn.org/ LRN]
* [http://moodle.org/ Moodle] (used at TECFA for several courses)
* [http://moodle.org/ Moodle] (used at TECFA for several courses)
* [http://www.olat.org OLAT]
* [http://www.olat.org OLAT]
* OpenUSS: [http://www.openuss.org/ platform itself] and [http://openuss.sourceforge.net/openuss/index.html sourceforge site]
* [http://www.openuss.org/ OpenUSS] and [http://openuss.sourceforge.net/openuss/index.html sourceforge site]
* [http://www.sakaiproject.org/ Sakai]
* [http://www.sakaiproject.org/ Sakai], “a community source software development effort to design, build and deploy a new Collaboration and Learning Environment (CLE) for higher education.”
* [http://segue.sf.net/ Segue]
* [http://segue.sf.net/ Segue] (dead project)
* [http://studip.de/ Stud.IP] (Studienbegleitender Internetsupport von Präsenzlehre). More like a portal, made for German higher education.
* [http://teleduc.nied.unicamp.br TelEduc]
* [http://teleduc.nied.unicamp.br TelEduc]
* [http://wplms.org/faqs/ WordPress LMS] (since summer 2012).
You can try out some of these system at [http://www.opensourcecms.com/ http://www.opensourcecms.com/] (in addition to '''many''' other portalware). You will have full administrator rights (all systems will be refreshed every hour).
=== Online services ===
Most of these hosted LMS are commercial, but some basic services may be free. In addition to providing LMSs or individual classes to organizations and teachers, there is also a trend for selling contents as service, e.g. like [http://openclassroom.com openclassroom] or [[MOOC]] services like Coursera.
* [http://www.myccnet.com/ CCNet]
* [http://www.haikulearning.com/ Haiku LMS]
* [https://www.canvaslms.com/ Canvas], a popular, more recent cloud-based system. As of 2013, many sites seem to migrate from Blackboard to this. It does have a [https://canvas.instructure.com/doc/api/index.html Auth2 / https / JSON] based API. Single teachers can sign up for free. There is also a community edition can be installed on your machines.
* [http://www.feathercap.net/ FeatherCap] (trial version available)
* [http://www.proprofs.com/training/lms-software/ ProProfs] (free version available)
* [http://www.koollearning.com/ KoolLearning], new in 2013. Offers good integration of resources.
* [http://www.adobe.com/ch_fr/products/captivateprime.html Adobe captivate prime]
* [https://www.schoology.com/ Schoology], A LMS-social platform hybrid. Single teacher free account.
* [https://www.easygenerator.com/ Easygenerator]. As of 2016, a new tincan/scorm compliant system with an easy to use interface.
=== Commercial ===
These are either available for self-hosting, as service or both. Some (e.g. blackbaord and canvas) target higher education, others industry and organizations. Most systems have similar features, in particular,  content authoring/delievery, user management, tracking/user surveys, quizzing, communication.
Below are a few examples, there are many more.
* [[Blackboard Vista]] (former WebCT)
* [http://interactyx.com/features/ TopYX], a service-based social LMS
* [http://iqpakk.com/ iQpakk]
* [http://www.brightspace.com/ Brightspace (formerly Desire2Learn]
* [http://www.halogensoftware.com/products/halogen-elms/ Halogen eLearning Manager]
* [http://www.rapidintake.com/ Rapid Intake] (several tools, both for corporate and school environments).
* [http://www.skilitix.com/ Skilitix], a service-based LMS using the new Tin-CAN API (2013). At its core is a roleplay application.
* [https://www.docebo.com/ Docebo]
* [https://www.expertus.com/expertus-one/ ExpertusOne]. Popular system in industry. Rather an LCMS, it can import learning objects such as SCORM 1.2. It includes gamification and a chat. Modules can be delivered as widgets (embedded contents in other websites).
* [https://www.ispringsolutions.com/ iSpring]
* [https://www.looop.co/ Looop]
* [https://www.talentlms.com/ Talent]
See also [[rapid elearning]], more popular in industry (low-level) training


== Links ==
== Links ==
Line 49: Line 113:
There are many sites that will give you advice on how to choose a standard, main-stream E-Learning System:
There are many sites that will give you advice on how to choose a standard, main-stream E-Learning System:


* [[http://www.edutools.info/ Course Management Systems]] (formerly Landonline). This site was built to assist higher education in using a more rational decision making process to review the many options for a course management system.
* [http://elearningindustry.com/ elearningindustry.com] maintains several lists (search the site), e.g.
** [http://elearningindustry.com/open-source-learning-management-systems The Ultimate list of Open Source Learning Management Systems]
** [http://elearningindustry.com/the-ultimate-list-of-cloud-based-learning-management-systems The Ultimate List of Cloud-Based Learning Management Systems]
** [https://elearningindustry.com/directory/software-categories/learning-management-systems/best/user-experience  The Best Learning Management Systems based on User Experience]
 
* [http://www.edutools.info/ Course Management Systems] (formerly Landonline).  
** Probably the '''best site''' out there, if you are interested in criteria-based selection of an LMS
** This site was built to assist higher education in using a more rational decision making process to review the many options for a course management system.


* [[http://www.edutech.ch/lms/  Elearning Platforms (Learning Management Systems)]] by EduTech, the technological support for the Swiss Virtual Campus project. Much shorter list, but enough for most of us ....
* [http://www.lmstalk.com/resources.php LMS Talk] List of products, forum, etc.
 
* [http://www.edutech.ch/lms/  Elearning Platforms (Learning Management Systems)] by EduTech, the technological support for the Swiss Virtual Campus project. Much shorter list, but enough for most of us ....
 
* [http://www03.lernmanagment.de/blogs/blog1.php LernmanagementSysteme.DE] German blog about LMS, includes a large comparative table of LMS's (added oct. 2011).


*[http://www.edutools.info/course/compare/byproduct/index.jsp LMS comparison]
*[http://www.edutools.info/course/compare/byproduct/index.jsp LMS comparison]


* [[http://www.unesco.org/webworld/portal_freesoft/Software/Courseware_Tools/ UNESCO free software portal]]
* [http://www.unesco.org/webworld/portal_freesoft/ UNESCO free software portal]
 
* [http://www.virtualcampuses.eu/index.php/Main_Page The Re.ViCa wiki] has been set up to provide an inventory and to show the results of a systematic review of Virtual Campus initiatives of the past decade within higher education throughout the world.
 
* [http://en.wikipedia.org/wiki/List_of_learning_management_systems List of learning management systems] (Wikipedia)
** See also [http://en.wikipedia.org/wiki/Category:Learning_management_systems Category:Learning management systems]
 
* [http://www.proprofs.com/c/category/lms/ What is an LMS] Free educational resource site with comprehensive articles explaining the different aspects of Learning Management Systems: features checklists, cost comparisons, support & technology, enterprise LMS checklists, LMS vendor comparisons, LMS & LCMS comparisons, proprietary/open source & SaaS LMS comparisons and many more.
 
* [http://chronicle.com/article/Amazon-s-Quiet-Dominance-of Amazon’s Quiet Dominance of Higher-Ed Learning Platforms], Chronicle of Higher Education, July 2016. Describe an increasing trend adopting cloud computing.
 
== Bibliography ==
 
; Usability / Evaluation of LMS studies
 
* Inversini, A., Botturi, L. & Triacca, L. (2006). Evaluating LMS Usability for Enhanced eLearning Experience. In E. Pearson & P. Bohman (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2006 (pp. 595-601). Chesapeake, VA: AACE. Retrieved March 4, 2014 from http://www.editlib.org/p/23070.
 
*  Chua B. B., & Dyson, L. E. (2004). Applying the ISO 9126 Model to the Evaluation of an e-Learning System. Proc. of ASCILITE 2004, December 5-8, Perth, Australia, pp. 184-190.
 
* Al-sarrayrih, Haytham S.; Lars Knipping, Erhard Zorn, Evaluation of a MOODLE Based Learning Management System Applied at Berlin Institute of Technology Based on ISO-9126, ICL2010. [http://www.icl-conference.org/dl/proceedings/2010/contributions/Contribution292.pdf PDF]
 
* Kakasevski, G.; Mihajlov, M.; Arsenovski, S.; Chungurski, S., "Evaluating usability in learning management system moodle," Information Technology Interfaces, 2008. ITI 2008. 30th International Conference on , vol., no., pp.613,618, 23-26 June 2008. doi: 10.1109/ITI.2008.4588480. [http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4588480&isnumber=4588363 http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=4588480&isnumber=4588363]
 
* Chandra, V., D. Fisher, and V. S. Chang. 2011. “Investigating higher education and secondary school web-based learning environments using the WEBLEI.” In Technologies for enhancing pedagogy, engagement and empowerment in education: creating learning-friendly environments, ed. Thao Le and Quynh Le, 93-104. USA: Information Science Reference, IGI Global.
 
* Chang, V., & Fisher, D. (2003). The validation and application of a new learning environment instrument for online learning in higher education. Technology-rich learning environments: A future perspective, 1-18.
 
* Chang, V. (1999). Evaluating the effectiveness of online learning using a new web based learning instrument. Proceedings Western Australian Institute for Educational Research Forum 1999. http://www.waier.org.au/forums/1999/chang.html
 
* Kurilovas, E. (2009). Methods of Multiple Criteria Evaluation of the Quality of Learning Management Systems for Personalised Learners Needs. European Conference on Technology-Enhanced Learning (EC-TEL'2009), 1-10, Retrieved from: http://wwwis.win.tue.nl/lms-ale-09/Kurilovas_paper.pdf
 
* Srdevic, B., Pipan, M., Srdevic, Z., Arh, T. (2012). AHP Supported Evaluation of LMS Quality. International Workshop on the Interplay between User Experience (UX) Evaluation and System Development (I-UxSED 2012). NordiCHI 2012.
 
* Vinesh Chandra, Darrell L. Fisher, Students’ perceptions of a blended web-based learning environment, Learning Environments Research, April 2009, Volume 12, Issue 1, pp 31-44.
 
* Jay Melton, The LMS moodle: A Usability Evaluation,  http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.124.7533&rep=rep1&type=pdf (ref to find/add)


Other references:
[[Category:Instructional design models]]


* Wikipedia's article on [[http://en.wikipedia.org/wiki/Learning_Management_System Virtual learning environment]] which also features a nice list of links to Systems plus other useful information.


*[[Category:Instructional design models]]
[[Category:E-learning tools]]
*[[Category:Technologies]]
[[Category: Portalware]]
*[[Category:Educational technologies]]
[[Fr:Système de gestion de l'apprentissage]]
[[Category:Edutech families]]
[[Category: Learning management systems]]
[[Category:Computer-mediated communication]]

Latest revision as of 16:48, 3 May 2019

Definition

A Learning Management System (sometimes also called "Course Management System", "Pedagogical Platform", "E-Learning Platform") is a software system that delivers courseware plus e-tutoring over the Internet.

LMS should not be your starting point. Typically, when decision makers talk about E-Learning they want to know what system to install. First of all, so called LMS are not the only answer, you also can implement E-Learning with other tools (e.g. groupware, content management systems or even a wiki or other hypertext system, often in combination with a forum. Second, in any case you should start by thinking about an appropriate instructional design that uses appropriate teaching strategies for various learning types.

““LMS” is a catchall term in SCORM. It refers to a suite of functionalities designed to deliver, track, report on and manage learning content, learner progress and learner interactions. “LMS” can apply to very simple course management systems, or highly complex enterprise-wide, distributed environments.” (SCORM 2004 3rd Edition Overview Version 1.0)

See also:

Components of an LMS

Feature lists are getting quite long and it is not easy to decide which features are the most important unless you have a good background in various areas like instructional design, ergonomics and systems management.

Typical features are:

  • Course Management, e.g. lists of courses, registration, credit information and syllabus, pre-requisites
  • Display of teaching Materials, i.e. courseware (either through content authoring and/or importing standard IMS or SCORM packages developed with an external tool )
  • Self-assessment quizzes
  • Content authoring (structured XML or HTML) of learning materials and other contents (e.g. instructions)
  • Asynchronous Communication: email, forums
  • Synchronous Communication: chat, whiteboard, teleconferencing,
  • Student tools: Home page, self tests, bookmarks, progress tracking, ....
  • Student Management Tools: progress tracking, on-line grading (assessment), ....
  • User and group management
  • Learner feedback: course evaluation surveys, test evaluation surveys etc.

Other features include:

  • ecommerce functionality
  • certification
  • integration with other systems, e.g. single-sign login, other learning platforms
  • support for informal and social learning

Usually LMS are closed circuit platforms (logins, restricted access to classes), so the idea of sharing contents and reusing products generated during classes does not exist in the world of "LMSs" (main-stream e-learning).

List of software

These lists needs updating, therefore, before looking our lists please check the links section at the end of this page. Others do a much better job than we do. In addition, it is very difficult to find truly independant reviews by real experts - Daniel K. Schneider (talk) 16:15, 9 October 2013. As of 2019, there are probably a few hundred systems.

Free / Open Sourcource Portalware

Software to install on your own server, some are probably also availble through commercial hosting or cloud services.

You can try out some of these system at http://www.opensourcecms.com/ (in addition to many other portalware). You will have full administrator rights (all systems will be refreshed every hour).

Online services

Most of these hosted LMS are commercial, but some basic services may be free. In addition to providing LMSs or individual classes to organizations and teachers, there is also a trend for selling contents as service, e.g. like openclassroom or MOOC services like Coursera.

  • CCNet
  • Haiku LMS
  • Canvas, a popular, more recent cloud-based system. As of 2013, many sites seem to migrate from Blackboard to this. It does have a Auth2 / https / JSON based API. Single teachers can sign up for free. There is also a community edition can be installed on your machines.
  • FeatherCap (trial version available)
  • ProProfs (free version available)
  • KoolLearning, new in 2013. Offers good integration of resources.
  • Adobe captivate prime
  • Schoology, A LMS-social platform hybrid. Single teacher free account.
  • Easygenerator. As of 2016, a new tincan/scorm compliant system with an easy to use interface.

Commercial

These are either available for self-hosting, as service or both. Some (e.g. blackbaord and canvas) target higher education, others industry and organizations. Most systems have similar features, in particular, content authoring/delievery, user management, tracking/user surveys, quizzing, communication. Below are a few examples, there are many more.

See also rapid elearning, more popular in industry (low-level) training

Links

There are many sites that will give you advice on how to choose a standard, main-stream E-Learning System:

  • Course Management Systems (formerly Landonline).
    • Probably the best site out there, if you are interested in criteria-based selection of an LMS
    • This site was built to assist higher education in using a more rational decision making process to review the many options for a course management system.
  • The Re.ViCa wiki has been set up to provide an inventory and to show the results of a systematic review of Virtual Campus initiatives of the past decade within higher education throughout the world.
  • What is an LMS Free educational resource site with comprehensive articles explaining the different aspects of Learning Management Systems: features checklists, cost comparisons, support & technology, enterprise LMS checklists, LMS vendor comparisons, LMS & LCMS comparisons, proprietary/open source & SaaS LMS comparisons and many more.

Bibliography

Usability / Evaluation of LMS studies
  • Inversini, A., Botturi, L. & Triacca, L. (2006). Evaluating LMS Usability for Enhanced eLearning Experience. In E. Pearson & P. Bohman (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications 2006 (pp. 595-601). Chesapeake, VA: AACE. Retrieved March 4, 2014 from http://www.editlib.org/p/23070.
  • Chua B. B., & Dyson, L. E. (2004). Applying the ISO 9126 Model to the Evaluation of an e-Learning System. Proc. of ASCILITE 2004, December 5-8, Perth, Australia, pp. 184-190.
  • Al-sarrayrih, Haytham S.; Lars Knipping, Erhard Zorn, Evaluation of a MOODLE Based Learning Management System Applied at Berlin Institute of Technology Based on ISO-9126, ICL2010. PDF
  • Chandra, V., D. Fisher, and V. S. Chang. 2011. “Investigating higher education and secondary school web-based learning environments using the WEBLEI.” In Technologies for enhancing pedagogy, engagement and empowerment in education: creating learning-friendly environments, ed. Thao Le and Quynh Le, 93-104. USA: Information Science Reference, IGI Global.
  • Chang, V., & Fisher, D. (2003). The validation and application of a new learning environment instrument for online learning in higher education. Technology-rich learning environments: A future perspective, 1-18.
  • Chang, V. (1999). Evaluating the effectiveness of online learning using a new web based learning instrument. Proceedings Western Australian Institute for Educational Research Forum 1999. http://www.waier.org.au/forums/1999/chang.html
  • Kurilovas, E. (2009). Methods of Multiple Criteria Evaluation of the Quality of Learning Management Systems for Personalised Learners Needs. European Conference on Technology-Enhanced Learning (EC-TEL'2009), 1-10, Retrieved from: http://wwwis.win.tue.nl/lms-ale-09/Kurilovas_paper.pdf
  • Srdevic, B., Pipan, M., Srdevic, Z., Arh, T. (2012). AHP Supported Evaluation of LMS Quality. International Workshop on the Interplay between User Experience (UX) Evaluation and System Development (I-UxSED 2012). NordiCHI 2012.
  • Vinesh Chandra, Darrell L. Fisher, Students’ perceptions of a blended web-based learning environment, Learning Environments Research, April 2009, Volume 12, Issue 1, pp 31-44.