From Aristotle to Yann LeCun: The Evolution of AI

In this third article of our AI Science and Philosophy series, we will explore some key years and events in the history of AI.

Aristotle (384–322 B.C.) was the first to formulate a precise set of laws governing the rational part of the mind

Aristotle developed an informal system of syllogisms for proper reasoning, which in principle allowed one to generate conclusions mechanically, given initial premises. 

Much later, Ramon Lull (d. 1315) had the idea that useful reasoning could actually be carried out by a mechanical artifact. Thomas Hobbes (1588–1679) proposed that reasoning was like numerical computation, that “we add and subtract in our silent thoughts.” The automation of computation itself was already well under way. Around 1500, Leonardo da Vinci (1452–1519) designed but did not build a mechanical calculator; recent reconstructions have shown the design to be functional. The first known calculating machine was constructed around 1623 by the German scientist Wilhelm Schickard (1592–1635), although the Pascaline, built in 1642 by Blaise Pascal (1623–1662), Pascal wrote that “the arithmetical machine produces effects which appear nearer to thought than all the actions of animals.” Gottfried Wilhelm Leibniz (1646–1716) built a mechanical device intended to carry out operations on concepts rather than numbers, but its scope was rather limited. Leibniz did surpass Pascal by building a calculator that could add, subtract, multiply, and take roots, whereas the Pascaline could only add and subtract. Some speculated that machines might not just do calculations but actually be able to think and act on their own. In his 1651 book Leviathan, Thomas Hobbes suggested the idea of an “artificial animal,” arguing “For what is the heart but a spring; and the nerves, but so many strings; and the joints, but so many wheels.”

Philosophers staked out some of the fundamental ideas of AI, but the leap to a formal science required a level of mathematical formalization in three fundamental areas: logic, computation, and probability. The next step was to determine the limits of what could be done with logic and computation. The first nontrivial algorithm is thought to be Euclid’s algorithm for computing greatest common divisors.

Besides logic and computation, the third great contribution of mathematics to AI is the theory of probability. The Italian Gerolamo Cardano (1501–1576) first framed the idea of probability, describing it in terms of the possible outcomes of gambling events. In 1654, Blaise Pascal (1623–1662), in a letter to Pierre Fermat (1601–1665), showed how to predict the future of an unfinished gambling game and assign average payoffs to the gamblers. Probability quickly became an invaluable part of all the quantitative sciences, helping to deal with uncertain measurements and incomplete theories. James Bernoulli (1654–1705), Pierre Laplace (1749–1827), and others advanced the theory and introduced new statistical methods. Thomas Bayes (1702–1761), proposed a rule for updating probabilities in the light of new evidence. Bayes’ rule underlies most modern approaches to uncertain reasoning in AI systems.

History of Machine Learning

The origins of Machine Learning can be traced back to the 1950s and 1960s, when researchers began exploring the idea of building intelligent machines. During this period, researchers developed algorithms that could learn from data and make predictions. Some of the early algorithms developed during this period included the Nearest Neighbor algorithm and the Perceptron algorithm.

During 1970s and 1980s, researchers focused on developing decision tree algorithms and other methods for building predictive models. The introduction of backpropagation in the mid-1980s enabled the development of multilayer neural networks, which had the potential to learn more complex patterns than earlier algorithms.

• 1970s: Decision Trees are developed by researchers including Ross Quinlan and Leo Breiman. 1980s: Artificial Neural Networks (ANNs) are developed by researchers including David Rumelhart, Geoffrey Hinton, and Yann LeCun. 

The concept of Convolutional Neural Networks (CNNs) is also introduced by Yann LeCun. It is the year 1994, and this is one of the very first convolutional neural networks, and what propelled the field of Deep Learning.

• 1990s: Support Vector Machines (SVMs) are developed by researchers including Vladimir Vapnik and Corinna Cortes. 

• 2000s: Boosting algorithms are developed by researchers including Yoav Freund and Robert Schapire.

Limited progress in early 2000s due to the high computational costs

During that time researchers began to explore the use of Deep Learning for speech recognition and other tasks. However, progress was limited due to the high computational cost of training these algorithms. It wasn't until the mid-2000s, with the availability of high-performance graphics processing units (GPUs), that researchers were able to train Deep Learning algorithms more efficiently. One breakthrough during this period was the development of convolutional neural networks (CNNs), which are well-suited for image recognition tasks. 

In 2012, a team of researchers from the University of Toronto used a deep convolutional neural network to win the ImageNet Large Scale Visual Recognition Challenge, marking a major milestone in the field of Deep Learning. 

Another breakthrough during this period was the development of recurrent neural networks (RNNs), which are capable of processing sequences of data. This technology was used to develop language models that could generate coherent sentences and paragraphs of text, leading to the development of applications such as speech recognition and machine translation.

• 2006: Geoffrey Hinton and his colleagues publish a paper on deep learning, which sets the stage for its resurgence. 

• 2012: Alex Krizhevsky and his colleagues use deep learning to win the ImageNet competition, sparking a renewed interest in the field. 

• 2014: Google develops a deep learning algorithm that learns to play Atari games at a superhuman level. 

• 2015: Microsoft develops a deep learning algorithm that beats the human champion at the game of Go. 

• 2016: AlphaGo, a deep learning algorithm developed by Google, beats the world champion at the game of Go, marking a major milestone in the field.

Generative AI Breakthrough

After 2017, Generative AI made significant strides, with new algorithms and techniques being developed that enabled the creation of more realistic and sophisticated generated content. 

One of the key developments in this period was the use of Generative Adversarial Networks (GANs) to generate images, videos, and other types of content. GANs, introduced in 2014 by Ian Goodfellow, are a type of neural network that consists of two models: a generator model and a discriminator model. The generator model generates new content, while the discriminator model evaluates the quality of the generated content. The two models are trained together in a process called adversarial training, where the generator model tries to fool the discriminator model, and the discriminator model tries to distinguish between the generated content and real content. 

In 2017, Nvidia researchers introduced a GAN-based algorithm called Progressive Growing of GANs (PGGANs), which enabled the generation of high-resolution images. PGGANs worked by training the generator model on low-resolution images and progressively increasing the resolution of the generated images as the training progressed. This technique allowed for the generation of images up to 1024 x 1024 pixels, which was a significant improvement over previous GAN-based algorithms. Another key development in Generative AI after 2017 was the use of GANs for video generation.

In 2018, researchers from the University of California, Berkeley, introduced a GAN-based algorithm called Vid2Vid, which could generate realistic videos from input images. The algorithm worked by first generating a semantic segmentation map of the input image and then using the segmentation map to generate the corresponding video frame. 

In 2019, OpenAI introduced a new language model called GPT-2 (Generative Pretrained Transformer 2), which was capable of generating coherent and natural-sounding text. The model was trained on a massive dataset of internet text, and it was able to generate high-quality text that was difficult to distinguish from human-written text. 

In 2020, researchers from Nvidia introduced a new GAN-based algorithm called StyleGAN2, which improved upon previous GAN-based algorithms by generating more realistic and diverse images. The algorithm worked by separating the style and content of the generated images and allowing for fine-grained control over the image generation process. 

In November 2022, OpenAI introduced the world to its AI-enabled chatbot ChatGPT, leaving an indelible mark on the public and technology community, with the aim of benefiting humanity as a whole. It hit the world like a storm. 

Within five days of the platform’s initial release, the company drew more than one million daily users. 

Besides the publicity and energy behind the launch, what ChatGPT did was send shockwaves throughout the tech community and re-energized the belief in AI’s real-world applications.

As Generative AI continues to advance, we can expect to see even more impressive applications and use cases emerge in the coming years.

Recommended references and resources for AI Evolution

• https://aima.cs.berkeley.edu/

•  http://yann.lecun.com/

•  https://qbi.uq.edu.au/brain/intelligent-machines/history-artificial-intelligence 

• https://www.venturecrowd.com.au/s/learn/ai/generative-ai-impact-on-investing 

https://chat.openai.com/

helped me edit the article