A Brief History of AI
What is Intelligence
The history of artificial intelligence began with ancient stories about mythical beings that were intelligent and could perform tasks. These stories included Greek myths about a bronze guardian named Talos and Jewish folklore about a creature called a Golem. These early ideas about intelligent beings influenced people's thoughts on creating smart machines.
In modern times, the Turing machine was a major breakthrough that helped define AI. It showed that machines could manipulate symbols and perform logical tasks. This led to the Turing Test, a way to determine if a machine is truly intelligent by seeing if it can communicate like a human.
As we move toward a future with Artificial General Intelligence (AGI), machines may become even more intelligent than humans, capable of learning, understanding, and reasoning across a wide range of tasks. This progress will redefine the meaning of AI and may ultimately transform the way we live, work, and interact with technology.
The Birth of Artificial Intelligence (1960)
The history of artificial intelligence (AI) can be traced back to 1956, when the Dartmouth Conference laid the foundation for the field. It was organized by leading figures in computer science and related fields at the time. The conference aimed to explore the potential of computers to simulate human intelligence and problem-solving capabilities. Although the conference didn't result in immediate breakthroughs, it laid the groundwork for future developments in AI, setting the stage for the advancements we see today.
A year later, psychologist Frank Rosenblatt developed the perceptron, a digital neural network designed to mimic a few brain neurons. Rosenblatt's first task for the network was to classify images of men and women, expecting that the network would eventually learn to differentiate between the two. Despite the hype surrounding the perceptron, Rosenblatt's neural network failed to deliver on its promises due to its single-layer structure, which significantly limited its capabilities. Consequently, the computer science community abandoned the idea of AI by 1969, believing it to be a dead end.
During this early era of AI, another groundbreaking development took place: the creation of ELIZA, a chatbot developed by Joseph Weizenbaum in the mid-1960s. ELIZA was capable of engaging in basic text-based conversations. Although limited in its understanding, ELIZA demonstrated the potential of natural language processing and ignited further interest in AI research.
How Deep Neural Networks and Backpropagation Lead Into an AI Winter (1980)
However, it was Geoffrey Hinton's work that truly rekindled the field in the 1980s. Convinced that the human brain was a neural network, he saw potential in artificial neural networks, theorizing that they simply needed some adjustments. Hinton's work led to the development of multi-layered neural networks, which overcame the limitations of Frank Rosenblatt's single-layered approach. This breakthrough led to the evolution of Deep Neural Networks, a cornerstone of modern AI.
In 1985, Hinton co-authored a paper that introduced the Boltzmann machine, a fundamental building block of early deep neural networks. The Boltzmann machine allowed artificial neurons to learn basic features from data, paving the way for more sophisticated applications. In the 90s, Yan Lecun, a student of Hinton's, developed a program that recognized handwritten digits. Lecun's work, along with Hinton's theory of backpropagation, contributed significantly to the advancement of AI.
Despite the progress made, AI research faced significant challenges, such as slow computing power and a lack of data. The field's inability to meet the high expectations of its early days resulted in a period known as the "AI Winter". Research funding dried up, and many researchers shifted their focus away from AI. Hinton, however, remained steadfast in his belief in artificial neural networks, even as he faced ridicule from the computer science community for his unwavering commitment to a seemingly failed idea.
The Internet, Deep Blue and Self-Driving Cars (1993-2011)
Between 1993 and 2011, the field of AI experienced a transformation, thanks to the relentless pursuit of researchers like Geoffrey Hinton, the rapid advancements in computer processing power, and the dawn of the internet era.
One of the key factors contributing to the resurgence of AI was Moore's law, as observed by Intel's co-founder Gordon Moore. This law stated that the number of transistors per square inch doubles approximately every two years. The exponential growth in computer processing power made it possible for AI researchers to tackle more complex problems and develop sophisticated algorithms.
Another crucial factor was the advent of the internet. As the internet became increasingly popular, a wealth of data became available for researchers to utilize. This data helped solve the second major problem that AI faced: the lack of information to train and test AI models.
During this period, several milestones were achieved, one of which was IBM's Deep Blue. On May 11, 1997, Deep Blue became the first computer chess-playing system to defeat a reigning world chess champion, Garry Kasparov. This event demonstrated the potential of AI systems and their ability to process millions of moves per second, outperforming human capabilities.
Another notable achievement was by a team from Stanford University in the DARPA Grand Challenge. In 2005, their autonomous robot successfully navigated 131 miles of an unrehearsed desert trail. In the subsequent DARPA Urban Challenge, the team from Stanford University secured the second place by autonomously navigating 55 miles in an urban environment while obeying to traffic laws and avoiding simulated hazards created by human stunt drivers. Stefan Klumpp, one of the founding members of SIROC, was part of the Stanford team during the 2007 DARPA Urban Challenge. A fascinating video documentary is available on YouTube, providing an in-depth look at the Stanford team's journey during the DARPA Urban Challenge.
Deep Learning, Big Data and Cheap Computing Power (2011-2016)
This period of time marked the birth of the modern AI movement, which can be traced back to a single date: September 30th, 2012. On this day, Geoffrey Hinton and his team created the first artificial deep neural network, known as AlexNet, to be used on a widely recognized benchmark image recognition test called ImageNet. With a performance like no other, AlexNet achieved a success rate of over 75%, a staggering 41% better than the best previous attempt. Within seven years, the accuracy of identifying objects rose from 75% to 97%, surpassing human ability.
Another significant milestone during these years was the development of AlphaGo [watch this amazing documentary on YouTube], an AI system designed to play the ancient game of Go. In 2016, experts had predicted that it would take an AI around 12 years to beat a human at the game. However, these predictions were proven to be wrong when AlphaGo defeated the Go grandmaster that same year. The next version of the AI, AlphaGo Zero, not only learned to play the game from scratch but also managed to beat the previous version 100 games to 0 in just three days.
The advancements in AI during this period can largely be attributed to the emergence of Big Data, Deep Learning, and the increase in affordable computing power, especially in the form of GPUs. The availability of massive datasets provided the necessary foundation for training advanced machine learning models, while deep learning techniques facilitated the modeling of high-level abstractions using deep graphs with multiple processing layers. The rise of cheap and powerful GPUs enabled researchers and developers to process and analyze large amounts of data more efficiently, significantly reducing the time required to train complex models. These three factors combined allowed AI systems to learn from vast amounts of data, develop intricate models, and contribute to the groundbreaking achievements seen in image recognition and natural language processing.
Generative AI and Large Language Models (2017-2023)
The period between 2017 and 2023 witnessed a series of groundbreaking discoveries and innovations in the field of artificial intelligence. Among these advancements were the development of Transformer models, Diffusion models leading to the emergence of consumer-grade AI tools such as ChatGPT, Stable Diffusion and Midjourney which lead to an explosion of AI applications in 2023.
Transformer models, introduced by Google in 2017, revolutionized the field of natural language processing (NLP) with their self-attention mechanism, which enabled the models to process input sequences more efficiently. Unlike traditional recurrent neural networks, Transformer models can simultaneously process all tokens in a sequence, allowing for improved parallelization and faster training times. This innovation led to the development of large-scale language models, such as BERT (Google) and GPT (OpenAI), which significantly improved NLP tasks like machine translation, sentiment analysis, and question-answering.
Diffusion models, which gained attention around 2021, introduced a new approach to generative modeling. Growing in popularity, diffusion models operate by taking an image and progressively adding Gaussian noise, eventually transforming it into an unrecognizable static picture or pure noise. The fundamental principle behind diffusion models involves inverting this process by starting with a noisy image and successively removing noise to create a coherent image. These models have achieved success in image generation and have started to compete with, or even outperform, other generative models like generative adversarial networks (GANs) in specific tasks. As a result, consumer-grade applications such as DALL-E, Stable Diffusion, and Midjourney emerged in 2022, capable of producing photorealistic images indistinguishable from actual photographs or paintings.
The open-source movement played a pivotal role in the rapid advancement of AI technology during this time. Organizations like Hugging Face made it easier for researchers and developers to access and experiment with state-of-the-art AI models, fostering a collaborative environment and accelerating innovation. Hugging Face's Transformers library became the go-to resource for many AI practitioners, offering a comprehensive suite of pre-trained models and tools for NLP and other AI tasks.
As the AI landscape continued to evolve, several organizations developed their own large-scale language models, competing with OpenAI's GPT. Meta, formerly known as Facebook, introduced LLama, while Stanford University developed ALPACA. Google, another major player in the AI field, launched its own language model called LaMDA. These competing models further demonstrated the growing importance of AI in various industries and highlighted the potential for even more advanced AI systems in the near future.
The Future of Artificial General Intelligence (AGI) And Reaching Singularity
The pioneers of AI, such as Geoffrey Hinton and Yann LeCun, now hold prominent positions at Google and Facebook, respectively. Their groundbreaking work has led to AI's rapid growth in less than two decades, taking it from the fringes of science to the center stage of the world. The impact of these pioneers' work has created a significant milestone in human history, placing AI alongside the invention of fire, the wheel, electricity, computers, and the Internet.
The next generations of AI, including the pursuit of Artificial General Intelligence (AGI), have the potential to invent tools that humans could never conceive, sparking discussions about the future of intelligence. AGI represents a type of AI that can understand, learn, and apply knowledge across a wide range of tasks at a level equal to or beyond human capabilities, essentially making it capable of performing any intellectual task a human can do. Dr. Richard Sutton of the University of Alberta suggests that the singularity, a point where artificial intelligence surpasses human intelligence, is widely estimated to occur around 2040. By 2030, we should have the hardware capability to achieve this, leaving a decade for researchers like Sutton to develop the code that enables the singularity and brings AGI to fruition.
The thought of computers becoming smarter than humans within a decade or so can be quite unnerving. However, some experts believe that rather than developing fully autonomous AI, we might head towards a symbiotic relationship, where AI systems, including AGI, are designed to assist and develop alongside humans, fostering collaboration and mutual growth.
About SIROC
We hope you enjoyed this brief journey through the history of AI and gained insights into its fascinating evolution.
If you need assistance or advice with data analytics, data engineering, or integrating ML/DL models into your business, don't hesitate to reach out to us at SIROC. We specialize in developing custom AI solutions tailored to your needs, such as forecasting models, recommender and decision maker models, as well as anomaly detection models.
Not only do we have an extensive collection of pre-trained in-house models, but we also collaborate with industry-leading partners such as OpenAI and Aflorithmic to deliver cutting-edge solutions to optimize and automate your business. Find out more about our products and solutions on our website.