A critique of the Edutechnica data for the Australian university sector

In 2013 Edutechnica published a blog entitled “Data-driven Campus LMS Strategy” and included Australian universities identified from the Australian Open Data site. At the time I responded in the comments section that I thought the results for Australia were flawed and a better data set to use was http://education.gov.au/selected-higher-education-statistics-2013-student-data but I see that the ‘Comments’ section is closed and this criticism was not published.

Now that Edutechnica have again analysed LMS usage in Australia (and other countries) using the “same set of institutions used in 2013 to keep this data uniform” it has prompted me to write this blog to correct the record about LMS usage in the Australian universities.

Put simply, it is generally accepted that there are 39 universities (38 public, 1 private) in Australia. Given the small number of universities and my knowledge of the sector I know that there are 20 (51%) universities which use Blackboard as their enterprise LMS, 16 (41%) use Moodle, and 3 (8%) use D2L. It is acknowledged that there are some departments within universities that use another LMS but according to Edutechnica’s methodology these were excluded from their analysis.

Australian university LMS usage
Figure 1: Enterprise LMS usage in Australian Universities 2014

In terms of EFTSL (Equivalent Full-Time Student Load) in 2013 the distribution of LMS usage in Australian universities is 475,676 (53.65%) Blackboard, 340,064 (38.35%) Moodle, and 70,890 (8%) D2L. [Source: Australian Government Department of Education].

The reference to the Sakai usage in 2014 and not in 2013 is interesting in that Charles Sturt University (CSU) was the only university in Australia using Sakai until they made a decision in 2013 to move to Blackboard. I can’t explain why this wasn’t shown in the 2013 data and in 2014 this would represent a legacy LMS as CSU is well advanced in their transition to Blackboard.

So what is the point of this critique? Firstly, it is very easy in today’s connected world for information to be promulgated without critical review – I know I fall into this trap daily with retweets! Secondly, the work of Edutechnica to automate the capture of LMS usage data is good in principle but it must still be seen as a “work in progress” as demonstrated by their results for the Australian university sector.

Addendum:
If we want to consider all HE providers (including universities) in Australia then a definitive list is available from TEQSA at http://www.teqsa.gov.au/national-register/search/provider. This lists 176 registered HE providers at time of access (25-Sep-2014).

The Australian Government Department of Education at http://docs.education.gov.au/system/files/doc/other/2013listofinstitutions.xls also provides a list of HE providers for 2013 but only lists 133 registered HE providers. The interesting point to note from this spreadsheet is that the 39 universities referenced above account for 94% of the total number of students attending Australian HE institutions as listed.

 

 

One thought on “A critique of the Edutechnica data for the Australian university sector

  1. George Kroner

    Hi Allan, Thank you for your feedback and your own independent analysis of Australian universities. My reply to you from last year still holds true. I copy it below verbatim from Phil’s original post last year. http://mfeldstein.com/new-data-available-higher-education-lms-market/ In short, I do not consider the list of the 39 universities to be a complete representation of higher education in Australia. In our analysis we include TAFEs, other private institutions, and other accredited institutions as detailed here and elsewhere: http://en.wikipedia.org/wiki/List_of_universities_in_Australia. We have a similar situation for Canada and the UK where there is no authoritative, comprehensive list of institutions. In the US, however, the US Department of Education provides a complete, authoritative list of all higher education institutions, and as such we do not have this type of variance. The Sakai institution that you mention, however, was indeed a single miss on our part last year, and it appears that they are still running Sakai today.

    —————————————–

    Thank you, Allan, for your feedback. Most of our initial effort with this particular project went towards cleaning the US dataset for all institutions with more than 2000 students. As Phil indicates, with our lessons learned from this exercise we are now better able to revise our international data.

    Perhaps I jumped the gun on Charles Sturt. Admittedly, I was involved with that project while a Blackboard employee and chalked it up as a Blackboard win. That said, for institutions who noted an impending decommissioning of an LMS, we noted this and eliminated the decommissioned environment from our rankings. For example, University of Melbourne also has a Sakai environment (https://sakai.unimelb.edu.au/portal), but it is clearly noted that it will soon be shut down. Other schools use Sakai as a research or collaboration environment and not as a primary LMS. There are also other reasons that we could have mis-identified an LMS. By default these were lumped into the “other” category for later analysis. It’s just that we haven’t gotten there yet for Australia.

    As for the discrepencacy about the number of institutions, we included TAFEs, for-profit, and private institutions in our analysis – not just public institutions. TEQSA, for example, shows 177 “education providers” in Australia: http://www.teqsa.gov.au/national-register/search/provider. To be consistent with the sources of institutional data in the other geographies, we included these institutions in our Australian analysis.

    Again, we do greatly appreciate your feedback and especially your regional perspective on how we can improve the quality of our data and analysis. Thank you, and please keep it coming.?

Leave a Reply

Your email address will not be published. Required fields are marked *