What We Measure
Our market data is organized in a dataset that captures system usage on a per higher education institution basis. For most schools, a campus is equivalent to an institution. But there are also cases where there are multiple campuses per university (e.g. University of Minnesota system with five campuses, or DeVry University with dozens of campuses) and the system decision is made centrally. In these situations, one decision will lead to multiple institutions listed in the data. In the U.S., the definition of an institution is guided by unique identifiers in the Department of Education’s IPEDS data, and each region or even county has its own way of defining an institution.
The dataset goes beyond “school X uses system Y”, as it also includes dates of implementation and decommission, usage as a primary or secondary system (there may be more than one system in use at a school), and hyperlinks to the public information documenting a system selection or usage. The definition of institutions includes information about its sector (public two-year, private non-profit four-year, etc) as well as student enrollments.
How We Measure
Looking deeper at selection, there are multiple layers of data gathering at different intervals. Some of the sources:
- Extensive search engine notification such as Google Alerts on product keywords in multiple languages;
- Social Media scrapers;
- Domain scrapers looking for system information at official school websites; and
- Targeted human-directed searches.
Each new data point is verified by someone using the associated hyperlinks tied to selection or usage data.
Our North American data is essentially saturated, in that we know the vast majority of degree-granting institutions based on U.S. Department of Education data or Canadian provincial governmental data. We have well above 90% of all schools in the dataset.
For the global regions outside of North America, we are building up the dataset and do not have saturated coverage yet. For example, in Europe, we estimate that we have 60 – 75% of institutions. We have less than that in Latin America and more than that in Oceania. Asia and African data is still a work in progress.
Where feasible, we include on-the-ground subjective coverage by visiting the global regions, testing theses, finding out unique context, and finding local sources who can provide QA to our data.
We plan to expand coverage to additional regions as we develop at least 30% coverage of institutions and have time to do additional research to back up our analysis.
Degrees of Uncertainty
Because higher education data is lumpy and based on extended implementation times, we offer the following caveats:
- Market share information provided in percentages and trends are more reliable than absolute counts outside of North America. When we do provide absolute numbers, we advise caution for readers or subscribers to not over-interpret the absolute numbers, at least without us providing additional details to keep the data in context.
- We typically separate North American data from Rest of World data (Europe, Latin America, Oceania) to avoid the problem of North America numbers dominating aggregates and obscuring important regional differences.
- When we have system usage information but do not have accurate implementation dates (per month or quarter), we assign these system records to June. Therefore the summer data for new systems will appear artificially high. We currently have implementation dates for approximately 70% of records.
- Put another way, annual data is more reliable (i.e. without additional data collection noise) than half-year or quarterly data. The more-granular data is provided to certain subscribers, but we take great care in attempting to describe sources of “lumpiness” in the data that should be understood for any analysis.
The Systems we Track
We have data on the following Product Categories :
- Advising and Degree Audit
- Alert Notification
- Applicant tracking system
- Assessment and Evaluation
- Business Intelligence – BI
- Career Readiness Solutions
- Catalog Management
- Classroom Response Systems – CRS
- Content Management Systems – CMS
- Continuing education
- Curriculum Management Systems
- Customer Relationship Management – CRM
- Document Management
- Enterprise resource planning – ERP
- Faculty & Course Evaluation
- Financial Aid
- Financial Systems
- Help Desk
- Housing – Residence
- Human Resources – HR
- Labor management
- Learning Analytics Platforms
- Learning Management Systems – LMS
- Library Information Systems
- Massive open online course – MOOC
- Mobile Computing
- Online Program Management
- Payment Solution
- Retention Management
- Scheduling – Room Management
- Student Information System – SIS
- Textbooks, Digital Course Materials & Courseware