Learn everything about Analytics

Alibaba’s Neural Network Model Beat the Highest Human Score in Stanford’s Reading Test

Machines getting the better of humans is no longer a surprise. It started with IBM’s Deep Blue program beating Garry Kasparov in a game of chess more than 20 years ago and with the increasing breakthroughs in the world of machine and deep learning, machines continue to become powerful tools.

                                                                       Source: SFGate

Yesterday, Alibaba developed a model that beat out any human competition in Stanford’s reading comprehension competition. The dataset consists of more than 100,000 questions sourced from more than 500 Wikipedia articles. The purpose of the quiz is to see how long it takes the machine learning models to process all the information, train themselves and then provide precise or accurate answers.

Alibaba used a deep learning framework to build a neural network model. It’s based on the “Hierarchical Attention Network”, which according to the company, works by identifying first paragraphs, then sentences and finally words. The underlying technology has been used previously by Alibaba, in it’s AI-powered chatbot – Dian Xiaomi.

Alibaba achieved a score of 82.44, which beat out the human high of 82.304. Microsoft’s AI achieved a score on 82.650. The website lists that Microsoft submitted their model a couple of days before Alibaba but the team evaluating the models, Squad (Stanford Question Answering Dataset), officially released the results of Alibaba’s model first, and Microsoft’s a day later, thus giving Alibaba the unique distinction.

The competition leaderboard published by Squad

Companies like Google, Tencent, IBM and Samsung (among many others) have also participated in the competition but Alibaba became the first to beat the human best score.

Alibaba have mentioned that they will be sharing the model-building framework with the public in the coming weeks.

 

Our take on this

This just goes to show that machines are now able to answer complex objective questions with remarkable precision. Remember going to museums or historical monuments with a guide? That will be a thing of the past.

Customer service is expected to be fully automated in the next few years and Alibaba hope to lead the drive using their Natural Language Processing lab. The human input required for these tasks will be minimal.

You can also read this article on Analytics Vidhya's Android APP Get it on Google Play

7 Comments

  • Hi Pranav,
    It is in reality a nice and helpful piece of info.
    I am glad that you shared this useful info with us.
    Please keep us informed like this.
    Thanks for sharing.

  • sharad Pawar says:

    Thanks a lot Pranav

    Nice article

  • Mithlesh says:

    Thank for sharing the information, really interesting.

  • Mithlesh says:

    Interesting.

  • Felipe says:

    Hello! I got confused in the last paragraph: “Alibaba achieved a score of 82.44, which beat out the human high of 82.304. Microsoft’s AI achieved a score on 82.650.” With this information, isn’t Microsoft’s the highest score?

    “The website lists that Microsoft submitted their model a couple of days before Alibaba but the team evaluating the models, Squad (Stanford Question Answering Dataset), officially released the results of Alibaba’s model first, and Microsoft’s a day later, thus giving Alibaba the unique distinction.” The unique distinction is to be the first model to be evaluated?
    Thanks.

    • Pranav Dar says:

      Hi Felipe,

      Thanks for reading! Yes, Microsoft’s was the highest score but because the Stanford staff (Squad) evaluated Alibaba’s model first, they put them ahead of Microsoft in the leaderboard and gave them the distinction of being the first machine to beat the highest human score.

      You can see the rankings here – https://rajpurkar.github.io/SQuAD-explorer/

      Both Alibaba and Microsoft’s models are ranked 1, but Alibaba has been placed ahead of Microsoft.

  • Rohit Shukla says:

    It seems that we are getting closer to our science fiction movies, where machines are going to replace humans in many places like, better teacher, tour guides etc.
    On the other hand we can use them to solve mysteries of universe, health sector. Bright days ahead for new discoveries 🙂