Review websites have been part of our daily lives ever since we started buying products online. Reviews date from many decades before but were controlled by media and very few sources. Nowadays, thanks to the democratization of the Internet, people can submit what they think of a product or a service. When it comes to edtech solutions, these reviews help institutions sort products before launching an official selection process.
In this situation, institutions will browse specialized review websites, such as G2 or Capterra, to read the comments and evaluations made by users of the different solutions. On the vendor side, if companies want to appear on some of these sites, they subscribe. Then, reviews start to appear. Everything (theoretically) works on both sides.
However, it’s not always a rosy picture. The question you have the right to ask: are they real reviews? The answer is not as clear as it seems. Many reviewers only submit evaluations to get their “salary”: $10 per review, either in gift cards or in cash. As a company, you receive reviews from people who have never used your product, and may not even know about it.
How About Automated Reviews?
In recent years, we saw that artificial intelligence can produce automated content, as Rolling Stone described it in the Spotify article. Knowing that current reviews on review websites are often written by non-users, what is the difference between a non-user writing inaccurate reviews and using ChatGPT to produce similar content?
OpenAI recently launched ChatGPT for everyone’s enjoyment or fear (depending on which side you are on). We have asked the solution to produce some product reviews in the edtech world.
The first example is a 150-word product review for Blackboard LMS for the University of Ottawa:
The University of Ottawa has been using Blackboard as its learning management system (LMS) for several years, and overall, the experience has been positive. Blackboard is a comprehensive platform that provides a wide range of tools and features for delivering online courses and managing student learning.
One of the standout features of Blackboard for the University of Ottawa is its ease of use. The platform has a user-friendly interface that makes it easy for instructors and administrators to manage course content, assignments, quizzes, and more. The platform is also highly customizable, allowing the university to tailor the look and feel of the platform to match its brand.
Another key benefit of Blackboard for the University of Ottawa is its robust analytics tools. The platform provides detailed insights into student engagement and progress, allowing instructors and administrators to track the success of their courses and make informed decisions about how to improve the learning experience. These analytics tools have been instrumental in helping the university to identify areas where students are struggling and provide targeted support.
Overall, the University of Ottawa is satisfied with its decision to use Blackboard as its LMS. The platform provides a comprehensive solution for delivering online courses and managing student learning, and the university has been able to take advantage of its customization options and robust analytics tools to create a positive and engaging learning experience for its students.
The second review is a 200-word negative product review for Moodle LMS:
One challenge with Moodle is its interface, which some users might find confusing or cluttered. The platform has a lot of features and options, which can be overwhelming for those who are new to online learning or LMS platforms. Additionally, the interface may not be as intuitive or user-friendly as other LMS platforms, which can make it difficult to navigate or find specific features and options.
Another potential drawback of Moodle is its performance and speed. Depending on the size of the course and the number of students, the platform may become slow or unresponsive, making it difficult to manage content or track student progress. This can be frustrating for instructors and administrators, and can negatively impact the overall learning experience for students.
Finally, some users might also find that Moodle lacks certain features or integrations compared to other LMS platforms. For example, it may not offer as many tools for collaboration or multimedia integration, or it may not integrate with other popular tools or platforms used by educators and students.
In conclusion, while Moodle is a widely used and highly regarded LMS platform, it is not without its limitations. Depending on the specific needs and preferences of the users, Moodle may not be the best choice for every organization or institution. Before choosing Moodle, it’s important to consider these potential drawbacks and determine whether the platform is the right fit for your needs.
We were floored by the quality of these reviews. The database to produce such content certainly includes many existing reviews and comments on the different solutions. The fact that it can write a positive or a negative review clearly demonstrates how powerful this tool can be.
But one has to remain vigilant as, in this case, the University of Ottawa does not use Blackboard as its LMS. Hence the review could not only sway potential users, it also offers erroneous information.
How Can We Identify Fake Reviews?
With the artificial intelligence and paid reviewers, one thing we know is that we cannot stop entirely fake reviews. However, there are some ways to identify them. Using Amazon reviews as examples, Wired Magazine listed several things that may indicate a potential fake review:
- A very high percentage of five-star reviews
- Lack of detail in reviews and vague praise
- Generic review titles like “Nice product” or simply “Awesome”
- Mentions of competing products
- Wording similar to other reviews
- Poor grammar and spelling mistakes
- Multiple reviews on specific dates (especially if there are long gaps between them)
- “Customers also bought” section contains unrelated products
- Glowing reviews with one small negative that isn’t a deal breaker
- Explaining away cons
Even if these guidelines are rather useful for general public websites like Amazon or Best Buy, they can give us some indications on the quality of the reviews and possibly help us tell true from false.
Review Websites: A Competitive Market
The business model of review websites like G2 and Capterra is based on the constant influx of reviews to increase web traffic to their sites. To ensure that new evaluations are published each week, these sites are willing to pay, as mentioned earlier in the post. One year ago, I created a G2 account to better understand this business model. Since my account creation, I never wrote a review, but have received over 40 emails asking me to do reviews. Almost 75% of these emails were offering $10 for each submission. Here are some examples of email subject lines:
- 1 review = $10 donated to charity!
- 1 review for a $10 gift card!
- Each review = $10 to you!
The G2 business model is a freemium one and operates a review site for B2B software. The software vendors can create free profiles on the site, but companies that want to maximize their profiles need to invest in premium options. This will give them the option of sending G2 a list of current users and G2 will ask them to review specific products.
Whether they are real or fake evaluations, the fact remains that they are often based on emotions where technical issues or amazing features of each solution are overrepresented. When it is difficult to trust the content available on the Internet, we can always refer ourselves to one credible source of information: data. As always, we encourage you to have a look at our portal to find relevant market information. We are also available to help with custom data sets.
Note: we have mentioned G2 and Capterra (bought by Gartner in 2015) in this post, but other companies also based their business model on reviews: SourceForge, Experts Exchange, TrustRadius (bought by Gartner in 2014) and Software Advice (bought by Gartner in 2014).
