Back to the Future: The Story of Generative AI (Episode 2)
Welcome back to our series on the history of generative Artificial Intelligence. In the first part, we explored the basics and saw how early statistical models like the Naïve Bayesian Classifier paved the way for today’s AI. Now, we take a significant leap forward and delve into the second epoch—a transitional period where neural networks and GPUs take center stage and revolutionize the world of AI.
Epoch 2 – Transition
From 2015 – Awe in the Pre-Stage
The AI winter is over, and neural networks as well as GPUs (Graphics Processing Units) have made their grand entrance. However, these new technological marvels are largely confined to tech experts. But that doesn’t mean impressive products and applications aren’t being developed—quite the opposite! StyleGANs (Generative Adversarial Networks) deliver unprecedented image quality, and transformer models like BERT (Bidirectional Encoder Representations from Transformers) capture text with remarkable detail.
Despite this, direct operation of these models remains out of reach for the general public due to their technical and specific nature. One must choose, expand, link, and train specific models and architectures. Nevertheless, applications such as chatbots, customer service automation, generative design, and AutoML solutions make their way to the market.
From 2019 – Lift-off
“Bigger is better” becomes the new credo. Open source is sidelined, and the world of Natural Language Processing (NLP) is turned on its head: Large Language Models (LLMs) are here! However, the first model, GPT-2, is not released in 2019 due to concerns over potential misuse:
“The Elon Musk-backed nonprofit company OpenAI declines to release research publicly for fear of misuse.” (Guardian, 14.02.2019)
The words “Musk,” “nonprofit,” and “fear of misuse” in one sentence – almost surreal in hindsight. By the end of the year, GPT-2 is eventually released. It finds significant use in research to explore the fundamental properties of LLMs and later serves to understand the implications of advancements by comparing it to larger models.
In 2020, GPT-3 follows—with ten times more data and a model a hundred times larger. In 2021, DALL-E is introduced, followed by DALL-E 2 in 2022. Texts can now be processed and created using natural (written) language, although not yet in the now-familiar dialogue form, but through Few-Shot Prompts. For images, however, this was not the case; in DALL-E and DALL-E 2, example images could not be provided in the prompt. In this paradigm, which is still common in non-chat variants of GPTs today, the model was not trained to conduct a conversation but merely to complete texts. This means it requires examples, such as question-answer pairs, to understand how to continue the text.
An example of a Few-Shot Prompt: After three provided examples, the actual user input follows up to the word “Label:”, expecting the model to grasp the task or meaning and continue the text by giving the correct answer.
The public, as well as developers, are impressively confronted with state-of-the-art technology, for example, through the first articles written with GPT-3.
In the next part of our series, we will look at the latest developments and the revolution of generative Artificial Intelligence. Read on to discover how we transition from Few-Shot Prompts to practical applications, making generative AI accessible to the general public!
Don’t miss Part 3 of our blog series.
