OUseful Info

Course Analytics, Part 1 - Visitor Behaviour

In Course Analytics - Prequel I set the scene for a couple of posts about student behaviour in an online course, as viewed through a Google Analytics lens.

To clarify things a little further before I start, I'm not really interested at this stage in reporting course grades or identifying individual students who maybe about to drop out, or need some sort of additional support, (although outlier data points may points towards the existence of such individuals). In short I'm not really interested in Academic Analytics.

For the moment, what I am interested in is how website analytics can be used applied to online course websites in order to gain a better understanding of online study habits and the bahaviour of students taking an online course.

The potential disconnect between this view of online-course analytics and academic analytics is something I may explore later - just not here, and not yet... (That said, finding ways of correlating/reconciling online-course and academic analytics (to create a fusion "elearning analytics"?!) to gain a more comprehensive view of individual performance on the one hand, and course performance in all its senses on the other, would I'm sure be an interesting exercise.)

One final disclaimer - the data shown here is partial data collected as a pilot exercise over the last 8 weeks or so from 10 week, online course, in the context of no other web stats reports.

The course is split into 10 one-week sessions, with assessment in weeks 5 and week 10. The online study materials amount to a nominal 2 to 4 hours of study per week (the course also includes reading from two supplied set books, and completing a series of practical activities using a desktop application, for which instructional material is provided via a set of downloadbale PDF documents and a desktop application launcher.

The intention of the exercise was to gain familiarity of Google Analytics in a familiar context, and identify meaningful course analytics that could inform a course analytics reporting strategy for future presentations of the course, and help identify appropriate conversion "goals" for a course website.

In this post I shall describe several reports relating to general visitor behaviour. In further posts I'll show reports on daily behaviour/weekly cycles, as well as activity surrounding assessment submission dates and user search behaviour from within the course materials.

Data that was not collected includes - printing behaviour (whether students printed out copies of the online course materials) and outlink click-thru behaviour (whether students followed links to external web resources.

Unique visitors are identified by Google Analytics alone; that is, although individual students logged in to the course using their own credentials, those credentials - or the individual students' login identities - played no part in the collection of the analytics data. As such, we would expect the number of unique visitors over the period to be in excess of the number of students registered on the course if students access the course materials from more than one browser configuration.

Visitor Loyalty

The Visitor Loyalty report describes how often each visitor accessed the course materials over the 8 week data collection period.

The absolute number of unique visitors (not shown) is way in excess of the number of students registered on the course, so this report must be taken with a pinch of salt (the actual number of logins per student over the course period should be obtainable from "offiical" learning system reporting tools).

The report would seem to indicate that students are accessing the site multiple times (at least 50% of them more than 10 times).

(I actually find the above report confusing - does it read: "Of N visits, the visitors who made those visits visited Number-of-Visits times"? If so, the Number-of-Visits has dimensions "visits per visitor", in which case isn't the Visits divided by the Number-of-Visits the Number-of-Visitors, which should be an integer value?!)

Length of Visit

The Length of Visit report describes the amount of time a visitor spent on the site in a particular session. I would hazard a guess that computers left switched on, mid-session, overnight would report a long length of visit.

The distribution suggests that the majority of students are logging in for more than 5 minutes at a time, with a significant proportion spending at least, of the order of, half an hour online.

The granularity of reporting is obviously geared to sites where long sessions are atypical. I need to check whether I can get better resolution data for longer durations/lengths of visit.

Depth of Visit

The Depth of Visit report shows how many pages a visitor accessed on each visit. As with the Length of Visit report, short visits are not so interesting (except insofar as we might wonder what a visitor is doing on a short a visit?!)

What is far more interesting is the long tail, and the significant percentage of visits in which students are engaged with the online course materials for a long period of time. As a guide, each week's study is approximately 20 HTML pages of material, accessed from the course index page or course calendar.

Once again, the low resolution reporting for "deep" visits means it works against us, although it is tempting to draw the conclusion that some students might study the week's material in a single session.

Summary

The above reports suggest that a significant proportion of students spend a significant amount of time studying the online materials. As a heuristic, designing materials in "chunks" that can be studied within a 30 minute online study session would seem to be reasonable.

Tags: analytics, course analytics

Posted by ajh59 at October 25, 2007 03:25 PM