Loup Ventures has shared a new test on Apple’s new HomePod device whereby a number of features were put through their paces. Three separate HomePods were tested for categories including Siri, sound quality, and ease of use.
In total, 782 queries were asked and it was reported that while the devices understood 99.4% of queries asked of it, only 52.3% were answered correctly. This means that compared to previous tests of rival speakers, HomePod is in bottom place when it comes to performance of the AI assistant. In comparison, Google Home answered 81% correctly, Amazon’s Alexa answered 64% correctly, and Microsoft’s Cortana answered 57% correctly.
Loup Ventures’ Gene Munster broke this down further, stating that Siri is good for “local” and “commerce” questions, beating Alexa and Cortana when asked questions about where the nearest coffee shop is, for example. Despite such a low percentage, Munster explained that Siri’s overall performance still rose above expectations “given the limited scope of HomePod’s music focus.” Over time, HomePod and Siri should match, or even surpass, rival assistants by adding query domains such as calender, email, calling, and navigation.
Where the HomePod excelled was it’s superior listening skills. It allowed users to speak at a normal volume, even when the music from the speaker was particularly loud. Loup Ventures also favored the HomePod’s sound quality, stating that “it sounds incredible.” Unfortunately, this was countered by a lackluster performance from Siri – something that Munster believes will be changing very soon:
“Don’t be fooled by HomePod’s sound quality-focused first step into smart speakers; Apple has a grander vision than delivering a better sounding Echo. While not present in the first version of HomePod (i.e. you can’t even make a phone call with HomePod), we believe Apple’s goal is to make Siri a ubiquitous, ambient presence that connects and controls all your connected devices and services – and to make a leap forward in the transition to voice-first computing.”
