Rohloff, T., Sauer, D., Meinel, C.: Students’ Achievement of Personalized Learning Objectives in MOOCs.Proceedings of the Seventh ACM Conference on Learning @ Scale. pp. 147–156. ACM, Virtual Event, USA (2020).
Massive Open Online Courses (MOOCs) provide the opportunity to offer free and open education at scale. Thousands of students with different social and cultural backgrounds from all over the world can enroll for a course. This diverse audience comes with varying motivations and intentions from their personal or professional life. However, course instructors cannot offer individual support and guidance at this scale and therefore usually provide a one-size-fits-all approach. Students have to follow weekly-structured courses and their success is measured with the achievement of a certificate at the end. To better address the varying learning needs, technical support for goal-oriented and self-regulated learning is desired but very limited to date. Both learning strategies are proven to be key factors for students' achievement in large-scale online learning environments. Therefore, this paper presents a continuative study of personalized learning objectives in MOOCs to encourage goal-oriented and self-regulated learning. Based on the previously well-perceived acceptance and usefulness of the concept of personalized learning objectives, this study examines which learners select an objective and how successful they complete objectives. Concerning the learners' socio-demographic and geographical background, we could not identify any practical significant difference between students with selected learning objectives and the total course population. However, we have identified promising objective achievement rates, and we have observed a practical significant improvement of the certification rates comparing the total course population and students who selected an objective that included a graded certificate. This has also demonstrated a method for calculating more reasonable completion rates in MOOCs.
Rohloff, T., Schwerer, F., Schenk, N., Meinel, C.: openSAP: Learner Behavior and Activity in Self-Paced Enterprise MOOCs.Proceedings of the 13th International Conference on E-Learning in the Workplace. ICELW, New York, NY, USA (2020).
Massive Open Online Courses (MOOCs) have been a subject of research since 2012, especially in the context of professional development and workplace learning due to their flexible schedule and format, which is a prerequisite for on the job learning. But MOOCs often do not fulfill the promise of flexible learning as it is only possible to achieve a certificate during the course runtime. An unpredictable workload and thus a lack of time often results in not showing up to a course or dropping out during the course runtime. Therefore, some platform content remains accessible even after the course runtime in self-paced mode. These courses differ from live courses as participants still can access all of the content and the discussion forum in read-only mode, but are not able to take the graded assignments and exams which are a prerequisite to achieving a certificate at the end of a course. Even though it is only possible by paying a fee to earn a graded certificate for these self-paced courses, we identified a high share of additional enrollments after the course end that suggests there is still interest from participants. Nevertheless, learning behavior in self-paced courses has not been a major subject of research, yet. This work contributes to closing this research gap by exploring the learner behavior in self-paced courses. The results show tendencies of more time-efficiency and engagement of self-paced learners under certain conditions and pave the way for further research and practical applications.
Rohloff, T., Sauer, D., Meinel, C.: Student Perception of a Learner Dashboard in MOOCs to Encourage Self-Regulated Learning.Proceedings of 2019 IEEE International Conference on Teaching, Assessment, and Learning for Engineering. IEEE (2019).
In online learning environments like Massive Open Online Courses (MOOCs), where teachers cannot provide individual support and guidance for thousands of students, self-regulated learning (SRL) is a critical metacognitive skillset for students’ achievement. However, not every student intuitively self-regulates its learning and therefore technical solutions can help to apply SRL strategies. Learner dashboards with visualizations about the learner’s progress and behavior are able to create awareness, encourage self-reflection, and perhaps motivate students to plan and adjust their learning behavior to achieve their learning objectives. Hence, such Learning Analytics tools can support the SRL strategies self-evaluation and strategic planning. To examine this potential, a learner dashboard was integrated into the HPI MOOC platform. This work presents the design process, the concept, and an evaluation of the first dashboard iteration. The perceived usefulness and usability are investigated, and in addition, the question will be considered whether the dashboard encourages students to apply self-regulated learning. The positive results pave the way for future research and a next iteration of the learner dashboard.
Rohloff, T., Sauer, D., Meinel, C.: On the Acceptance and Usefulness of Personalized Learning Objectives in MOOCs.Proceedings of the Sixth ACM Conference on Learning @ Scale. p. 4:1--4:10. ACM, Chicago, IL, USA (2019).
With Massive Open Online Courses (MOOCs) the number of people having access to higher education increased rapidly. The intentions to enroll for a specific course vary significantly and depend on one's professional or personal learning needs and interests. All learners have in common that they pursue their individual learning objectives. However, predominant MOOC platforms follow a one-size-fits-all approach and primarily aim for completion with certification. Specifically, technical support for goal-oriented and self-regulated learning to date is very limited in this context although both learning strategies are proven to be key factors for students' achievement in large-scale online learning environments. In this first investigation, a concept for the application and technical integration of personalized learning objectives in a MOOC platform is realized and assessed. It is evaluated with a mixed-method approach. First, the learners' acceptance is examined with a multivariate A/B test in two courses. Second, a survey was conducted to gather further feedback about the perceived usefulness, next to the acceptance. The results show a positive perception by the learners, which paves the way for future research.
Rohloff, T., Oldag, S., Renz, J., Meinel, C.: Utilizing Web Analytics in the Context of Learning Analytics for Large-Scale Online Learning.2019 IEEE Global Engineering Education Conference (EDUCON). pp. 296-305 (2019).
Today, Web Analytics (WA) is commonly used to obtain key information about users and their behavior on websites. Besides, with the rise of online learning, Learning Analytics (LA) emerged as a separate research field for collecting and analyzing learners’ interactions on online learning platforms. Although the foundation of both methods is similar, WA has not been profoundly used for LA purposes. However, especially large-scale online learning environments may benefit from WA as it is more sophisticated and well-established in comparison to LA. Therefore, this paper aims to examine to what extent WA can be utilized in this context, without compromising the learners’ data privacy. For this purpose, Google Analytics was integrated into the Massive Open Online Course platform of the Hasso Plattner Institute as a proof of concept. It was tested with two deployments of the platform: openHPI and openSAP, where thousands of learners gain academic and industry knowledge about engineering education. Besides capturing behavioral data, the platforms’ existing LA dashboards were extended by WA metrics. The evaluation of the integration showed that WA covers a large part of the relevant metrics and is particularly suitable for obtaining an overview of the platform’s global activity, but reaches its limitations when it comes to learner-specific metrics.
Bothe, M., Renz, J., Rohloff, T., Meinel, C.: From MOOCs to Micro Learning Activities.2019 IEEE Global Engineering Education Conference (EDUCON). pp. 280-288 (2019).
Mobile devices are omnipresent in our daily lives. They are utilized for a variety of tasks and used multiple times for short periods throughout the day. MOOC providers optimized their platforms for these devices in order to support ubiquitous learning. While a combination of desktop and mobile learning yields improved course performances, standalone learning on mobile devices does not perform in the same manner. One indicator for this is the mismatch between the average usage pattern of mobile devices and the time to consume one content item in a MOOC. Micro learning builds on bite-sized learning material and focusses on short-term learning sessions. This work examines the potential of micro learning activities in the context of MOOCs. Therefore, a framework for video-based micro learning is presented, which features a personalized curriculum. Videos are suggested to the user in a non-linear order that is determined by content dependencies, users’ preferences and watched videos, as well as explicit and implicit user feedback. A mobile application was implemented to test the approach with restructured MOOC content resulting in 58 connected short videos about engineering education – e.g. web technologies and programming languages. The usage data indicates initial curiosity by the users. To improve retention rates, more user motivation will be required for future studies. A survey gathered additional qualitative feedback. While the content suggestions were seen as a vital feature for such an approach, the results showed good interest and acceptance rates to create a better learning experience for MOOCs on mobile devices.
Rohloff, T., Renz, J., Suarez, G.N., Meinel, C.: A Ubiquitous Learning Analytics Architecture for a Service-Oriented MOOC Platform. In: Calise, M., Delgado Kloos, C., Reich, J., Ruiperez-Valiente, J.A., and Wirsing, M. (eds.) Digital Education: At the MOOC Crossroads Where the Interests of Academia and Business Converge (EMOOCs 2019). p. 162--171. Springer International Publishing, Cham (2019).
As Massive Open Online Courses (MOOCs) generate a huge amount of learning activity data through its thousands of users, great potential is provided to use this data to understand and optimize the learning experience and outcome, which is the goal of Learning Analytics. But first, the data needs to be collected, processed, analyzed and reported in order to gain actionable insights. Technical concepts and implementations are rarely accessible and therefore this work presents an architecture how Learning Analytics can be implemented in a service-oriented MOOC platform. To achieve that, a service based on extensible schema-agnostic processing pipelines is introduced for the HPI MOOC platform. The approach was evaluated regarding its scalability, extensibility, and versatility with real-world use cases. Also, data privacy was taken into account. Based on five years of running the service in production on several platform deployments, six design recommendations are presented which can be utilized as best practices for platform vendors and researchers when implementing Learning Analytics in MOOCs.
Rohloff, T., Bothe, M., Meinel, C.: Visualizing Content Exploration Traces of MOOC Students.Companion Proceedings of the 9th International Conference on Learning Analytics & Knowledge (LAK19). p. 754--758 (2019).
This workshop paper introduces a novel approach to visualize content exploration traces of students who navigate through the learning material of Massive Open Online Courses (MOOCs). This can help teachers to identify trends and anomalies in their provided learning material in order to improve the learning experience. The difficulty lies in the complexity of data: MOOCs are structured into multiple sections consisting of different learning items and students can navigate freely between them. Therefore, it is challenging to find a meaningful and comprehensible visualization that provides a complete overview for teachers. We utilized a Sankey diagram which shows the students' transitions between course sections by grouping them into different buckets, based on the percentage of visited items in the corresponding section. Three preceding data processing steps are explained as well as the data visualization with an example course. This is followed by pedagogical considerations how MOOC teachers can utilize and interpret the visualization, to gain meaningful insights and execute informed actions. At last, an evaluation concept is outlined.
Rohloff, T., Bothe, M., Renz, J., Meinel, C.: Towards a Better Understanding of Mobile Learning in MOOCs.Proceedings of the 5th Learning with MOOCs Conference (LWMOOCs 2018) (2018).
The pervasive presence of mobile devices and growing trends like ubiquitous learning make new demands on Massive Open Online Courses (MOOCs). Users learn increasingly on the go and with multiple devices, instead of being tied to a fixed workstation. However, there is a lack of research how the usage of mobile devices influences the learning behavior and outcome in MOOCs. Thus, this paper presents a first quantitative study to examine this question. To enable a statistical analysis, a proof-of-concept implementation outline is presented, which enhances the Learning Analytics capabilities of the openHPI MOOC platform with contextual data to process various learning behavior metrics. Based on an analysis of four courses, it was found that users who additionally learnt with mobile applications showed a higher engagement with the learning material and completed the course more often. Nevertheless, the reasoning must be addressed with qualitative analyses in future, to better support their learning process and success on mobile and stationary devices.
Rohloff, T., Utunen, H., Renz, J., Zhao, Y., Gamhewage, G., Meinel, C.: OpenWHO: Integrating Online Knowledge Transfer into Health Emergency Response. In: Dimitrova, V., Praharaj, S., Fominykh, M., and Drachsler, H. (eds.) Practitioner Proceedings of the 13th European Conference On Technology Enhanced Learning (EC-TEL 2018). CEUR-WS.org (2018).
The platform OpenWHO was developed in 2017 in a cooperation between the World Health Organization (WHO) and the Hasso Plattner Institute (HPI). The Department of Infectious Hazard Management, under the WHO Health Emergencies Programme, worked together with the HPI to create a new interactive, web-based, knowledge-transfer platform offering online courses to improve the response to health emergencies. The platform was newly launched as there was an identified need of an open and scalable solution for fast distribution of life-saving content in disease outbreaks for frontline responders. The platform provides adjusted versions of the massive open online learning resources that are self-paced and at ease formats for the frontline and low-bandwidth use. The HPI already developed know-how in previous Massive Open Online Course (MOOC) projects like openHPI and openSAP. OpenWHO is based on the same technology as the aforementioned projects. This paper will provide insights into the practical deployment, the adaption of the MOOC concept, and lessons learnt within the first year of this platform.
Rohloff, T., Meinel, C.: Towards Personalized Learning Objectives in MOOCs. In: Pammer-Schindler, V., Pérez-Sanagustín, M., Drachsler, H., Elferink, R., and Scheffel, M. (eds.) Lifelong Technology-Enhanced Learning (EC-TEL 2018). p. 202--215. Springer International Publishing, Cham (2018).
Instead of measuring success in Massive Open Online Courses (MOOCs) based on certification and completion-rates, researchers started to define success with alternative metrics recently, for example by evaluating the intention-behavior gap and goal achievement. Especially self-regulated and goal-oriented learning have been identified as critical skills to be successful in online learning environments with low guidance like MOOCs, but technical support is rare. Therefore, this paper examines the current technical capabilities and limitations of goal-oriented learning in MOOCs. An observational study to explore how well learners in five MOOCs achieved their initial learning objectives was conducted, and the results are compared with similar studies. Afterwards, a concept with a focus on technical feasibility and automation outlines how personalized learning objectives can be supported and implemented on a MOOC platform.
Renz, J., Rohloff, T., Meinel, C.: Automatisierte Qualitätssicherung in MOOCs durch Learning Analytics. In: Ullrich, C. and Wessner, M. (eds.) Joint Proceedings of the Pre-Conference Workshops of DeLFI and GMW 2017. CEUR-WS.org (2017).
Dieser Beitrag beschreibt wie mithilfe von Learning Analytics Daten eine automatisierte Qualitätssicherung in MOOCs durchgeführt werden kann. Die Ergebnisse sind auch für andere skalierende E-Learning Systeme anwendbar. Hierfür wird zunächst beschrieben, wie in den untersuchten Systemen (die als verteilte Dienste in einer Microservice-Architektur implementiert sind) Learning Analytics Werkzeuge umgesetzt sind. Darauf aufbauend werden Konzept und Implementierung einer automatisierten Qualitätssicherung beschrieben. In einer ersten Evaluation wird die Nutzung der Funktion auf einer Instanz der am HPI entwickelten MOOC-Plattform untersucht. Anschließend wird ein Ausblick auf Erweiterungen und zukünftige Forschungsfragen gegeben.
Rohloff, T., Renz, J., Bothe, M., Meinel, C.: Supporting Multi-Device E-Learning Patterns with Second Screen Mobile Applications.Proceedings of the 16th World Conference on Mobile and Contextual Learning (mLearn 2017). pp. 25:1-25:8. ACM, Larnaca, Cyprus (2017).
Many providers of Massive Open Online Course (MOOC) platforms released mobile applications in the recent years to enable learning offline and on the go, for a more ubiquitous learning experience. However, mainly the MOOC content was optimized for small screens, but mobile devices provide the opportunity to enrich the MOOC experience even further by enabling new forms of learning. Based on a previous learning patterns evaluation and a user survey, this paper presents a second screen prototype for the MOOC platform of the Hasso Plattner Institute, whereby the mobile application can be used as a learning companion while using the web platform on a computer. Four different actions were implemented which can be done next to watching a video lecture. The evaluation showed that the prototype was helpful and made learning more efficient, as reported by users, and also ideas for further improvements were proposed.
