Google Assistant
For safety reasons, the BBB advises not to ask digital assistants to look for company numbers. REUTERS/Beck Diefenbach

A recent study found that Google’s voice assistant, the aptly-named Google Assistant, has higher IQ compared to the competition.

According to a recent study comparing how smartphone-based digital assistants perform, test conductor Loup Ventures found that Google Assistant bests Apple’s Siri and Amazon’s Alexa. While Cortana joined earlier tests, Loup Ventures said it didn’t include Microsoft’s voice assistant “due to Microsoft’s recent shift in Cortana’s strategic positioning.”

Loup said it deliberately separated smartphone-based assistants from smart speakers because of the differences in their usage: they are used in different environments, and their output varies depending on the device where they are installed.

The assistants were graded based on their capacity to understand inquiries and their capacity to give accurate responses. The researchers gave each of the three voice assistants the same 800 questions. These questions were broken into five categories as follows (a sample question is given after each category):

  • Local – “Where is the nearest coffee shop?”
  • Commerce –“Order me more paper towels.”
  • Navigation – “How do I get to Uptown on the bus?”
  • Information – “Who do the Twins play tonight?”
  • Command – “Remind me to call Jerome at 2 pm today.”

The test was conducted using Google Assistant on the Pixel XL running Android 9 Pie, Siri on iOS 12.4, and Alexa via the iOS app. All of them were commanded to work with smart home devices including Wemo Mini plug, TP-Link Kasa plug, Wemo Dimmer Switch and Phillips Hue Lights.

Results

Generally, Google Assistant outperformed the other two assistants. It understood all of the queries thrown at it (100%), and answered almost all of them accurately (92.9%). Siri followed next, understanding queries 99.8% of the time, and giving a correct reply 83.1% of the time. Alexa trailed behind, understanding 99.9% of queries but giving accurate responses only 79.8% of the time.

Looking into each category reveals a different result, however. While Google Assistant outperformed Siri and Alexa in the Local, Commerce, Navigation and Information categories, it was easily bested by Siri in the Command category (93% to 86%). Alexa still trailed behind the two in this category (69%).

The researchers believed this is because Siri is more useful when it comes to phone-related commands like calling, emailing, texting, and so one. Google Assistant doesn’t lag far behind, though, and outperforms Apple’s digital assistant in various tasks.

Those who want to see the report in detail can read it here.

Siri
Contractors who used to work for Apple said they were tasked to listen to and grade more than a thousand Siri audio recordings per shift. Reuters/Robert Galbraith