An AI algorithm has trumped human intuition when it comes to big-data analysis, a recent experiment has shown.
The trial was conducted by researchers from the Computer Science and Artificial Intelligence Laboratory, at the Massachusetts Institute of Technology.
Previously, it had been widely believed that computers are ineffective at processing unfamiliar data and detecting the missing pieces of the puzzle, since that usually requires intuition that machines simply do not possess.
MIT researchers wanted to test this hypothesis however, and they created a Data Science Machine, based on several algorithms, in order to identify predictive patterns in large databases.
For example, the prototype had to spot fluctuations related to sales averages across certain periods of time, despite having information referring just to the beginning and end dates pertaining to a variety of sales promotions.
The software then had to vie against human teams in 3 data science competitions. One such test involved assessing the probability of students dropping out of a class, after analyzing the extent to which they had accessed course material. Other challenges were to anticipate the success of a crowd-funded project, or to predict the chances of a client becoming a repeat buyer.
Overall, the algorithm’s performance when it came to distinguishing buried patterns exceeded all expectations. In fact, the prototype succeeded in defeating 615 out of the 906 human teams.
Moreover, in two of the contests the predictions formulated by the algorithm were around 94% and 96% as reliable as those issued by its rivals. In the third competition, the AI software still had a 87% success rate, in comparison with its human counterparts.
These accomplishments are even more remarkable given the fact that the Data Science Machine only required a time interval of 2 to 12 hours in order to make its calculations and submit its predictions. In contrast, the human teams had to toil for days, weeks and even months so that they could come to their conclusions.
As researchers explained, the algorithm managed to perform so well by employing certain operations that can mimic human intuition. More precisely, some of the steps involved building a staggering array of metrics identified in the database, and detecting correlations between these numbers, using mathematical operations.
In addition, the Data Science Machine took into account data related to categories as well, such as brand names or periods of the year. This way, connections and trends were established between these categories and the previously identified metrics.
The surprising results of the trial will be presented this week, at the International Conference on Data Science and Advanced Analytics, organized by the Institute of Electrical and Electronics Engineers.
While lead researchers are disinclined to believe that the algorithms might eventually replicate human intuition entirely, they still think artificial intelligence could definitely assist in furthering big-data analysis. As they say, the Data Science Machine can be “a natural complement to human intelligence”.
Results are achieved must faster when employing such techniques, even if accuracy still needs some improvement. There are currently so many pools of data which haven’t even begun being analyzed, so using algorithms could be extremely beneficial for harnessing at least some of their potential.
Image Source: Pixabay