August 17, 2007
Document Views (not) in Electronic Assignment Handling
In my recent post on the OpenMark Computer-Assisted Assessment Tool, I described an open source electronic assessment system that has been developed - and is actually used - here at the OU to support our course assessment strategies.
As well as Open Mark, the OU also hosts an electronic assignment handling system (the "eTMA system") that allows students to upload electronic copies of Office documents as submissions to an assignment and an eCMA system that provides computer based marking for multiple choice assessments (I haven't checked recently whether this is (still?!;-) based on students completing an OCR form that they have to post in to the OU, whence it is scanned in for electronic marking...)
One problem we have had until now is that as course team members we have been unable to see what our students see when using these assessment systems. For a start, we're staff, with staff authentication and corresponding staff access permissions. Added to which, we're not students registered on the courses we're actually running, so we don't have student permissions or access to the electronic assessment systems... After years of asking, it seems that this year the Technology Short Course Program course teams are going to get access to a dummy student account, which means that for the first time we'll be able to get a valid 'student experience' of our course materials - and hopefully the assessment systems too...
One issue that anecdotally keeps cropping up about the eTMA ad eCMA systems is that there is little feedback provided when you have uploaded an assignment and it is often difficult to tell whether everything has worked okay (not being a registered student on any of our courses means I haven't been able to personally check this...).
This was brought home to me recently whilst marking a batch of scripts when I opened one script to find it was for the wrong course. (There were - and always are - several half-completed scripts too, and I'm never really sure whether these have been deliberately submitted as a final offering, or were an early, 'holding position' upload that hasn't been replaced by a later resubmission.)
What would be useful, I think, would be a 'preview' facility to show students what they have uploaded - something like the document viewer you find in Scribd, for example:
In fact, this sort of facility would be useful all round, for markers as well as students.
Every time I mark scripts submitted via the eTMA system, I always get one that uses a 'different' document format. I don't remember what sort of submitted filetype it was this time round that I couldn't open on my desktop, but I do know I had to resort to viewing it in Google docs (which presumably breaks all sorts of OU rules?).
Added to the filetype problem (which really should be defended against - we specify the range of 'approved' document types in several places, so why doesn't the eTMA system check for these and warn the student that they have uploaded a file that their marker may not be able to open?) I occasionally wonder about virus issues - does the eTMA system automatically scan all uploaded files? And if so, what does it do if it finds an infection?
Using a system like the one adopted by Scribd, students would be able to check that they had uploaded what they intended to submit for an assignment, and markers could either view the file online, or download it using one of the (common!) supported formats. A commenting system could also be used to allow markers to comment back on the assignment to the learner.
The viewing technology could also be reused in our online course materials, as well as in student portfolios (speaking of which, Guy Carberry is soliciting comments on the latest design iteration for MyStuff...).
PS That's definitely it for a week or two, now... Let the holiday commence :-)
Posted by ajh59 at August 17, 2007 10:12 AM