How Does ChatGPT Work?

Few technologies have caused as much of a stir over the last decade as ChatGPT. What started out as a demo to showcase what a chatbot could do has now become an all‑in‑one AI assistant. Today, ChatGPT can query the web, create and interpret images, write computer code, summarize documents, translate text between languages, and even take on difficult exams. It’s like having any kind of digital Swiss Army knife at one’s disposal.

ChatGPT just continues to grow in capabilities with every update and new model release — though continued evolution aside, it can be difficult to pin down what its central purpose is. Here’s how it works, broken down step by step, in plain language.

1. What Is ChatGPT?

ChatGPT is sort of a friendly front end for some of the most powerful AI bots in the world. You enter (or speak) a prompt, and in the background, ChatGPT selects the optimal combination of AI engines to service your queries. No need to jump between several apps or figure out new workflows — just toss out the tap and chat away.

Its main abilities include:

• Writing and editing text
  • Write draft emails, blog posts, reports, poems or anything else you fancy.
  • Rewrite or simplify existing text.
  • Learn how to use AI for content creation →
• Generating and analyzing images

• Turn text descriptions into pictures.
• Describe what appears in an uploaded photo or chart.
• Translating and coding

• Convert between human languages.
• Translate simple English instructions into code that can be programmed.
• Answering questions and brainstorming

• Search live on the web for research topics.
• Give summaries with cited sources.

JM: Under the hood, here’s what ChatGPT tends to use out of the box these days:

2. Core Concepts Under the Hood

To know why ChatGPT is so impressive, we need to go back to several fundamental building blocks: pre‑training, transformers, tokens, and fine‑tuning.

2.1 Generative Pre‑Training

• Pre‑training is teaching the A.I. by exposing it to enormous quantities of raw information — text, images, audio — without the information having been labeled by humans.
• Early A.I. models required neatly labeled data sets (say, “cat,” “dog”), which is costly to create. Instead, GPT models learn by scouring enormous collections of books, articles and web pages, finding patterns in language on their own.
• Once the AI is pre‑trained, it already “knows” grammar, facts and some level of reasoning without the need for training specifically.

2.2 Transformer Architecture

• Then the transformer arrived — a new type of neural network model that upended everything, in 2017.
• Because the transformers don’t read one word at a time, as older systems do, they can look at all words at once. This does so through a mechanism known as self‑attention, which compares every word (“token”) in a sentence to every other word to determine which ones are of the greatest interest.
• Parallel processing allows training to be faster and cheaper, which allows such massive models to be built in reasonable amounts of time.

2.3 Tokens

• Transformers don’t work with raw text or images directly. They chop up inputs into smaller pieces called tokens: for text, that can be several characters or a piece of a word; for images, little square patches of pixels.
• The numerical representation of each token is a vector. Related tokens get turned into similar vectors.
• GPT‑3 trained on 500billion text tokens. In more modern models, trillions more, plus image and audio tokens, deepen their understanding.

2.4 Fine‑Tuning and Safety (RLHF)

• Unsupervised raw pre‑training results in fluent yet still inconsistent text. In order to ensure that the AI is safer and more valuable also after these learning procedures, it becomes further subjected to Reinforcement Learning from Human Feedback (RLHF).
  • Demonstration data: Human teachers demonstrate the model an optimal response.
  • Comparator data: The trainers rank several model outputs by quality.
  • AI learns a reward model to prioritize the top-ranked responses.
• In this way it suppresses noxious or biased output and hence makes the AI human value compliant.

2.5 Chain‑of‑Thought (CoT) Reasoning

• When solving for multi‑step problems — math, logic puzzles, planning — the model is trained using chain‑of‑thought training.
• Instead of transpiring directly to an answer, it “thinks out loud” in stages, trying out partial solutions before arriving at a complete result.
• This additional inference allows it to be more accurate on complex queries, but it is more compute‑intensive and therefore reserved for specialized models like o1‑mini.

3. Multimodal Capabilities

ChatGPT isn’t limited to text. “Multimodal” is that it’s designed to work with various types of data together:

• Text + Image: Ask ‘What’s in This Photo?’ or “Draw me a mountain lake.”
• Text + Audio: Speak the questions and get spoken answers.
• Text + Live Search: Ask a question about something happening in the news, and the search model will find and quote the most recent information.

All of these modes are based on the same transformer and token formalism.

4. Integrations and the API

Outside of the chat window businesses, can access ChatGPT through an API:

• Integrate AI into customer-support tools, apps, or websites.
• Automate processes by integrating ChatGPT with services such as calendars and databases.
• Tailor to preference with fine‑tuned versions of the model for your industry or use case.

This versatility makes ChatGPT a module to be used in all kinds of applications, from generating legal documents to driving interactives at a museum. What is an API? →

5. What’s Next for ChatGPT?

In two years, ChatGPT has blossomed from a tech demo to a go‑to productivity tool. The things its good at — multimodality, third-party app integration, and high-level reasoning — only get stronger with every subsequent model. Looking ahead, we can expect:

• More subtle appreciation for context and nuance.
• Models that are faster and more efficient, and cost less to operate.
• Novel ways to combine text, images, audio and data—and new contenders like Gemini AI from Google.
• More finely-tuned versions adapted to specific professions (medicine, law, education).

There’s no obvious ceiling to what ChatGPT can accomplish. As its core technologies — transformers, tokens, and reinforcement learning — advance, so will its capacity to enable us to learn, create, and problem solve.

Share this post: on Twitter on Facebook