Improving the effectiveness of MOOCs to meet the 21st century challenges

Abstract

Massive Open Online Courses (MOOCs) are a type of online course designed using principles of education technology. It enables a massive number of participants to learn online in any course at any time. This affordance of scaling and open access to education is considered as the globalized solution for acquiring 21st century skills. However, unrealistic to the vision, pragmatically, MOOCs are facing challenges. Mainly the content-driven pedagogical structure with limited system design implications caused fewer interactions and isolations, thereby resulted in higher dropouts. Since MOOCs are introduced recently, the problems faced by participants or its effectiveness are less understood. Thus, a systematic understanding of arising problems and solutions to this newly emerged phenomenon is well needed. In this thesis, I explored MOOCs with a holistic view of understanding emerging problems with empirical pieces of evidence—whether MOOCs meet the 21st century skill requirements; what factors are affecting the effectiveness of a MOOC; how can we improve the effectiveness of MOOCs. By exploring the above questions, this thesis mainly contributes to 1) provide empirical evidence of the challenges that MOOCs are facing, 2) solicit a framework to identify the effectiveness of MOOCs, 3) design a novel peer review mechanism, and 4) develop the novel system PeerCollab to improve effectiveness of MOOCs. The research begun with exploratory research methods with active data collection using MOOC users. The analysis conducted using a combined approach of qualitative and quantitative methods to understand the challenges and explore the factors affecting the effectiveness of MOOCs. Initially, surveys were used to identify whether MOOC platforms are providing necessary 21st century skills such as collaborative skills, creativity skills, communications skills, and critical thinking skills. Next, a longitudinal qualitative study was used to gather MOOC experience using participants over 24 months period of time. Results of the qualitative study were incorporated to build an instrument to evaluate MOOCs' effectiveness. The instrument was empirically verified and validated using 121 MOOC participants. The initial survey to explore 21st century skills yielded results from 391 MOOC participants across six platforms. Descriptive statistics depicted that majority of participants reflect the gap in MOOCs to provide 21st century skills. Next, the qualitative analysis using Grounded Theory (GT) and quantitative analysis using Factor Analysis (FA) resulted in a detailed10-dimensional framework to evaluate MOOC effectiveness. Based on the high ranked dimensions in the framework such as Technology, Collaborativeness, Interactivity and Assessment, two systems were designed and developed to demonstrate the improved effectiveness in MOOCs. First, the “Identified Peer Review” (IPR) system demonstrated how peer identity, incentive algorithm, and effective communication in peer review enhance the MOOC's effectiveness. Next, the PeerCollab system demonstrated how social presence can integrate using theories of communities of practices (CoP) into MOOCs and thereby improve effectiveness. This system also demonstrated an articulation of CoP to MOOCs by a novel process named Rapid Communities on MOOCs (RCoM) design with four phases, viz. Cluster, Orient, Focus, and iii Network. Evaluations of the systems demonstrated the challenges and possibilities of integrating such systems into MOOCs and provided a direction to build effective interventions. These systems collectively empower interactions in isolated distributed individuals and form communities to work collectively bridging the gap to meet the 21st century skills. The work of this thesis actively contributes to the nuance of technologies that can be used in society specifically for large scale open and distributed learning contexts.