Introduction
Redesigns of training and courses are part of a continuous quality process for organizations with the goal of improving the experiences for learners and the efficacy of the organizational outcomes. Often, training and course redesigns start with assessing past experiences. The assessment will include analyzing both qualitative and quantitative data from the experiences, including surveys, facilitator notes, observations, and data collected from integrated technologies. Analyzing the data and metrics from past experiences can inform what components of the learning design could be improved.
The design of every learning experience is made up of the following key components:
- Resources, including information, content, books, and technology
- Learning Activities, including individual activities, peer engagement, and assessments
- Learning Support, including facilitation by instructors and services provided within the course or through organizational departments
The learning analytics from past experiences can support the instructional designer in refining the training or course components. There are two types of analytics within the course experience that can be reviewed to provide insight into the design effectiveness.
Checkpoint Analytics
Checkpoint analytics provides snapshot data on if the learner has met the prerequisites for learning or is progressing in learning by accessing the relevant resource. It measures access to the resources included in the learning design. These analytics are valuable for providing leading indicators of learner engagement.
However, in isolation, these analytics will not provide insight into learner processing of information and understanding of content. Examples of checkpoint analytics include logging into the course site, downloading a file or other resources, signing up for group work or support, and attending in-person sessions. The checkpoint analytics could provide insight into access and engagement opportunities and challenges as well as areas of support that may be needed, like alerts and other prompts.
Process Analytics
Process analytics provide insight into how learners process and apply information and knowledge as part of the learning experience. These analytics are tied to activities and tasks that the learner engages in and completes. For example, discussions, practice activities, and assessments are tasks that the learner would engage with and complete. The analytics from the interactions provide insight into how the learners are acquiring knowledge and demonstrating understanding through application.
Examples of process analytics include online and in-person discussions, narratives and peer engagement, versions of written or project drafts, reflection activities, and submitting tests. Process analytics could provide insight into how participants are learning, learning sequences that may increase retention and application of knowledge, and ways the design can be further adapted for new cohorts or contexts.
Conclusion
Additional resources to learn how learning analytics informs redesigns: