Engagement theory

Draft

Definition

• Engagement is a concept that is not restricted to technology-based learning activities. Back in 1988, Meece et al. (1988) set a model for cognitive engagement in the classroom. Engagement from an educational point of view is seen as the learner participation, and interaction with the learning material, learning activities, and the learning community.
• O’Brien, H.L. & Toms, E.G.( 2008) set a conceptual framework defining user-engagement with technology. The framework explores the experience of users interacting with technology-based systems not limited to educational applications. The work resulted in a definition of engagement and a conceptual model that could be used in various application areas, including technology-based learning or citizen science projects, etc. According to O’Brien, H.L. & Toms, E.G. (2008), "Engagement is a quality of user experiences with technology that is characterized by challenge, aesthetic and sensory appeal, feedback, novelty, interactivity, perceived control and time, awareness, motivation, interest, and affect". The resulting conceptual model of engagement distinguishes different phases through an engagement process: Upon a point of engagement, the user initiates and sustains engagement in a task, he disengages, and potentially reengages several times with the system.

• the Engagement Theory is a framework for technology-based teaching and learning (Kearsley & Schneiderman, 1999). Its fundamental underlying idea is that students must be meaningfully engaged in learning activities through interaction with others and worthwhile tasks. While in principle, such engagement could occur without the use of technology, Kearsley and Schneiderman believe that technology can facilitate engagement in ways which are difficult to achieve otherwise. The general conceptual framework proposed by O'Brien and Toms joins the Engagement Theory on the importance of the self-directed, meaningful involvement with materials or applications based on cognitive challenge and motivation (O'Brien & Toms 2008).

The Engaged Learning model

Engagement theory is based upon the idea of creating successful collaborative teams that work on ambitious projects that are meaningful to someone outside the classroom. These three components, summarized by Relate-Create-Donate, imply that learning activities:

1. occur in a group context (i.e., collaborative teams)
2. are project-based
3. have an outside (authentic) focus

(Kearsley & Schneiderman, 1999).

• Relate emphasizes team work (communication, management, planning, social skills)
• Create emphasizes creativity and purpose. Students have to define (or at least identify in terms of a problem domain) and execute a project in context
• Donate stresses usefulness of the outcome (ideally each project has an outside "customer" that the project is being conducted for).

The Conceptual Framework of User Engagement

This framework deals with the description of what should be recognized as an engaged user experience with technology. This concept goes beyond the usability of a human- computer interaction. Through exploratory research, O'Brien &Toms conclude that engagement is a category of user experience characterized by attributes of challenge, aesthetic and sensory appeal, feedback, novelty, interactivity, perceived control and time, awareness, motivation, interest, and affect.

The framework describe an engagement process of four potential stages. These stages are the point of engagement (engagement is initiated), period of engagement, disengagement, and reengagement. This reengagement phase adds an iterative aspect to engagement where a user can stop the interaction and start again later. In addition the framework revealed that each stage is characterized with distinguishable attributes:

Point of Engagement Attributes	Period of Engagement Attributes	Disengagement Attributes
aesthetic	aesthetic & sensory Appeal	usability
novelty	attention	challenge
interest	awareness	positive affect
motivation	control	negative affect
specific or experiential goal	interactivity	perceived time
	novelty	interruptions
	challenge
	feedback
	interest
	Positive affect

The framework also indicates that there are an intrinsic relationship between usability and engagement that needs to be further explored: "Some of the engagement attributes are associated with usability variables of effectiveness, efficiency, and satisfaction. This demonstrates that usability is intricately woven into the experience of engagement; while an application may be usable, it may not be engaging, but engaging applications do appear to have an inherent baseline of usability."

methods to measure engaging user experiences

Based on the conceptual framework of user engagement, Ponciano and Brasileiro (2014) designed four engagement metrics to measure participant interaction and involvement with citizen science projects. These engagement metrics could be applied in other contexts where we are interested in user engagement in application with deadlines, such as moocs, online challenges and competitions, etc. Participant engagement over time takes into account their points of engagement, periods of sustained engagement, disengagements and reengagements.

The metrics rely on the time the participant could potentially remain linked to the project, the effective time during which the participant remains linked to the project, the active days through this duration, the time devoted on every active day, and the number of days elapsed between every two consecutive active days.

The four measures used by Ponciano and Brasileiro to study engagement profiles in human computation for astronomy projects are:

• Activity ratio: the ratio of the number of active days to the total period of activity
• Daily devoted work: the average activity duration through an active day
• Relative activity duration: ratio of the total period of activity to the period of possible activity, i.e. the period from the first activity till the end of the project (in case the project has a fixed deadline, e.g a challenge or a hackathon)
• Variation in activity: the standard deviation of the idle time between active days 

Other measures can also be used such as peak daily devoted work, total devoted work during the entire period of activity, and also the skewness of the daily work through the period of activity which might the pattern of engagement through time: whether the participant starts heavily and then looses interest, or on the opposite, he starts slowly and get more engaged as he participates further.

References

• Kearsley, G. & Schneiderman, B. (1999). Engagement theory: A framework for technology-based learning and teaching. Originally at http://home.sprynet.com/~gkearsley/engage.htm. Retrieved 14:42, 11 September 2006 (MEST) from google cache.

• Kearsley, G. (1997). The Virtual Professor: A Personal Case Study. [1]

• Meece, J. L, Blumenfeld, P. C, and Hoyle, R. H. (1988). Students’ goal orientations and cognitive engagement in classroom activities. Journal of educational psychology 80, 4 (1988), 514. DOI:http://dx.doi.org/10.1037/0022-0663.80.4.514
• O’Brien, H. L and Toms, E. G. (2008). What is user engagement? A conceptual framework for defining user engagement with technology. Journal of the American Society for Information Science and Technology 59, 6 (2008), 938–955. DOI:http://dx.doi.org/10.1002/asi.20801
• Ponciano, L. and Brasileiro, F. (2014) Finding Volunteers’ Engagement Profiles in Human Computation for Citizen Science Projects. Journal of Human Computation, 6 (2014), http://hcjournal.org/ojs/index.php?journal=jhc&page=article&op=view&path%5B%5D=12

• Shneiderman, B. (1994) Education by Engagement and Construction: Can Distance Education be Better than Face-to-Face? [2]

• Shneiderman, B. (1988), Relate-Create-Donate: An educational philosophy for the cyber-generation. Computers & Education, in press.

• Shneiderman, B., Alavi, M., Norman, K. & Borkowski, E. (Nov 1995). Windows of opportunity in electronic classrooms, Communications of the ACM, 38(11), 19-24.

• Miliszewska, Iwona and John Horwood. 2006. Engagement theory: a universal paradigm?. In Proceedings of the 37th SIGCSE technical symposium on Computer science education (SIGCSE '06). ACM, New York, NY, USA, 158-162. DOI=10.1145/1121341.1121392 http://doi.acm.org/10.1145/1121341.1121392