AI Science and Philosophy Series: Exploring the Capabilities, Ethics, Limitations, and Potential of Artificial Intelligence
Artificial intelligence (AI) is rapidly transforming our world, from the way we work and communicate to the way we make decisions and interact with each other. As AI systems become more advanced, they raise complex philosophical questions about the nature of intelligence, consciousness, and ethics.
In this series, we will explore these questions from a scientific and philosophical perspective, examining the latest research on deep artificial neural networks and their implications for our understanding of cognition, ethics, and safety.
Themes that we will examine:
The Capabilities and Limitations of AI: We will explore the current state of AI research and its applications in various domains, such as natural language processing, computer vision, and robotics. We will discuss the strengths and weaknesses of AI systems, including their ability to learn from data, their susceptibility to bias and error, and their limitations in understanding context and human emotions.
The Philosophy and Potential of Artificial Intelligence: We will introduce key philosophical questions surrounding AI, such as: What is intelligence? Can machines be intelligent? What is the relationship between human intelligence and artificial intelligence? We will discuss the history of AI research and the different approaches to creating intelligent machines, including symbolic AI and connectionist AI.
Ethics and Safety in AI: We will examine the ethical and safety implications of AI, including issues such as bias, privacy, accountability, and the potential for AI systems to cause harm. We will discuss the role of philosophy in developing ethical frameworks for AI and the importance of interdisciplinary collaboration between philosophers, scientists, and policymakers.
Let’s begin with some introductions and latest important developments on the field.
Generative AI / (GenAI) is a type of AI system capable of generating text, images, or other media in response to prompts. Generative AI systems use generative models such as large language models to produce data based on the training data set that was used to create them. Large language models have had a big year (Fall 2022-2023 developments), with the recent ChatGPT putting the cherry on top and stealing the show.
Large language models (LLMs) are a type of artificial intelligence that have been trained on vast amounts of text data using deep learning techniques. These models can generate text that is similar in style and structure to human-written text, and can perform a variety of language-related tasks, such as language translation, question-answering, and text summarization. These models are often referred to as "large" because they require massive amounts of data and computing power to train and operate, with the most advanced models containing billions of parameters. They have become increasingly important in natural language processing and have the potential to revolutionize a wide range of industries, including education, journalism, and customer service.
LLMs emerged around 2017-2018 and perform well at a wide variety of tasks. This has shifted the focus of natural language processing research away from the previous paradigm of training specialized supervised models for specific tasks. The current AI revolution for natural language only became possible with the invention of transformer models.
Before this, text generation was performed with other deep learning models, such as recursive neural networks (RNNs) and long short-term memory neural networks (LSTMs). These performed well for outputting single words or short phrases but could not generate realistic longer content. Google BERT's transformer approach was a major breakthrough since it is not a supervised learning technique. That is, it did not require an expensive annotated dataset to train it. BERT was used by Google for interpreting natural language searches, however, it cannot generate text from a prompt.
In 2018, OpenAI published a paper about natural language understanding using their GPT-1 language model. GPT stands for "Generative Pre-trained Transformer". GPT models are pre-trained on massive amounts of text data and can then generate new text in a variety of natural language tasks such as language translation, summarization, and question answering.
GPT Turbo is a variant of the GPT model developed by the EleutherAI team. It is a community-driven effort to create an open-source version of GPT-3, which is a proprietary language model developed by OpenAI. GPT Turbo aims to be a more affordable and accessible alternative to GPT-3 by providing similar capabilities with an open-source license.
GPT-3.5, also known as "Codex", is a new language model developed by OpenAI that goes beyond natural language processing and can also generate code in a variety of programming languages. Codex is built on top of the GPT-3 architecture and is trained on a large dataset of code, enabling it to generate functional code snippets in response to natural language prompts.
The main differences between these models are:
GPT and GPT Turbo are language models designed for natural language processing, while GPT-3.5 is designed for generating code.
GPT is developed by OpenAI, while GPT Turbo is an open-source community-driven project.
GPT-3 is a proprietary language model with a closed-source license, while GPT Turbo is open-source, and Codex is built on top of the GPT-3 architecture but adds the ability to generate code.
ChatGPT is an artificial intelligence chatbot developed by OpenAI and released in November 2022. It is built on top of OpenAI's GPT-3.5 large language models and has been fine-tuned using both supervised and reinforcement learning techniques. This version took the world by storm after surprising the world with its ability to generate pages of human-like text. ChatGPT became the fastest-growing web application ever, reaching 100 million users in just two months. OpenAI in March 2023 released GPT-4 which has been developed to improve model "alignment" - the ability to follow user intentions while also making it more truthful and generating less offensive or dangerous output. GPT-4 improves on GPT-3.5 models regarding the factual correctness of answers. The number of "hallucinations," where the model makes factual or reasoning errors, is lower, with GPT-4 scoring 40% higher than GPT-3.5 on OpenAI's internal factual performance benchmark. It also improves "steerability," which is the ability to change its behavior according to user requests. A further improvement is in the model's adherence to guardrails. If you ask it to do something illegal or unsavory, it is better at refusing the request. One major change is that GPT-4 can use image inputs (research preview only; not yet available to the public) and text. Users can specify any vision or language task by entering interspersed text and images.
In a recent research (March 2023) by the OpenAI Team, where they investigated the potential implications of Generative Pre-trained Transformer (GPT) models and related technologies on the U.S. labor market, by assessing occupations based on their correspondence with GPT capabilities, incorporating both human expertise and classifications from GPT-4, research team found that approximately 80% of the U.S. workforce could have at least 10% of their work tasks affected by the introduction of GPTs, while around 19% of workers may see at least 50% of their tasks impacted. The influence spans all wage levels, with higher-income jobs potentially facing greater exposure. Notably, the impact is not limited to industries with higher recent productivity growth. They concluded that Generative Pre-trained Transformers exhibit characteristics of general-purpose technologies (GPTs), suggesting that these models could have notable economic, social, and policy implications.
Another recent important development was the proposal of a 6-month moratorium on generative AI. Thought leaders Andrew Ng and Yann LeCun met and discussed on the topic which seem to be virtually unenforceable and can be seen as an extension of the “Killer AI Syndrome.” The discussion offers reasonable perspectives for how generative AI has turned the world on edge.
Given the momentum and speed of how the field is evolving, we hope that our AI Science and Philosophy series will provide a thought-provoking and informative introduction to the area, sparking further discussion, debate, and awareness.
Human slavery is wrong, insecure and demoralizing. On mechanical slavery, on the slavery of the machine, the future of the world depends. Oscar Wilde.
Recommended references and resources for introduction in AI Topics:
Attention Is All You Need paper https://arxiv.org/pdf/1706.03762.pdf
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding https://arxiv.org/abs/1810.04805v2
Formal Algorithms for Transformers https://arxiv.org/pdf/2207.09238.pdf
Ahead of AI
https://stanford-cs324.github.io/winter2022/lectures/modeling/
https://phildeeplearning.github.io/
Yann LeCun and Andrew Ng: Why the 6-month AI Pause is a Bad Idea
https://chat.openai.com/ helped me edit the article
https://openai.com/product/dall-e-2 helped me create some pictures inspired by Ghost in the Shell