Nick Heer at Pixel Envy tested how well 2018 Siri performs commands given to the voice assistant in a 2010 demo video. The video takes Siri, which started as a stand-alone, third-party app, through a series of requests like ‘I’d like a romantic place for Italian food near my office.’ Just a couple of months after the video was published, Siri was acquired by Apple and the team behind it, including the video’s narrator, Tom Gruber, began integrating Siri into iOS.

That was eight years ago. Inspired by a tweet, Heer tested how well Siri performs when given the same commands today. As Heer acknowledges, the results will vary depending on your location, and the test is by no means comprehensive, but Siri’s performance is an eye-opener nonetheless.

What’s clear to me is that the Siri of eight years ago was, in some circumstances, more capable than the Siri of today. That could simply be because the demo video was created in Silicon Valley, and things tend to perform better there than almost anywhere else. But it’s been eight years since that was created, and over seven since Siri was integrated into the iPhone. One would think that it should be at least as capable as it was when Apple bought it.

Eight years is an eternity in the tech world. Siri has been fairly criticized recently for gaps in the domains it supports and their balkanization across different platforms, but Heer’s tests are a reminder that Siri still has plenty of room for improvement in how it handles existing domains too. Of course, Siri can do things in 2018 that it couldn’t in 2010, but it still struggles with requests that require an understanding of contexts like location or the user’s last command.

Voice controlled assistants have become a highly competitive space. Apple was one of the first to recognize their potential with its purchase of Siri, but the company has allowed competitors like Amazon and Google catch up and pass it in many respects. The issues with Siri aren’t new, but that’s the heart of the problem. Given the current competitive landscape, 2018 feels like a crucial year for Apple to sort out Siri’s long-standing limitations.