What AI Is and Isn't
- Terminology and concepts to look out for: retrieve · generate · hallucination
- Relevance to me: You know that a chatbot can give wrong answers. Today you learn why, using what you learned yesterday about the two files.You already know a chatbot can be wrong, the way you know the forecast can be wrong, usually on the day you trusted it. Today you find out why, using yesterday's two files.
- Board Question: Is asking a chatbot for the definition to a word the same as searching a dictionary (electronic or hardcopy)? Yes or no, and one reason.
Open the discussion form on the right and answer Section 1 from your first reading.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
I Do questions
-
notebook cue: retrieve
You search a library catalog for a book the library does not own. What comes back?
Check
Nothing comes back. A catalog can only retrieve records that were stored, and there is no record for a book the library does not own, so the search comes back empty. See How Search Engines and Catalogs Retrieve, paragraph 1.
-
notebook cue: generate
You ask a chatbot about a book that does not exist. What comes back, and why is that different from the catalog?
Check
Some text comes back, often a fluent description of the book. A model will generate text for almost any prompt by predicting one token at a time from its parameters, while the catalog has no stored record to return. See How a Language Model Generates and How Hallucinations Happen, paragraph 1.
-
notebook cue: hallucination
A chatbot gives a confident, wrong birthday for a real person. Using the two files, where did that date come from?
Check
The runtime generated it from the parameters, one token at a time. The date was never stored as a record, so the prediction produced a likely-looking date that was wrong, which is a hallucination. See How Hallucinations Happen, paragraph 1, where a model gave three different wrong birthdays.
Retrieving and Generating
This section explains two ways a computer can answer a question, and why one of them sometimes produces false details.
How Search Engines and Catalogs Retrieve
A search engine and a library catalog both answer a question by finding something stored earlier. These systems retrieve information, which means they find a stored item and return it unchanged. Google stores information about web pages in its index, which Google describes as a large database. When a person types a query, the search engine finds matching pages in the index and returns links to them. A library catalog works the same way. A search for a book the library owns will retrieve the record for that book. If the library does not own the book, there is no record to find, so the search comes back empty. A library catalog cannot generate a record for a book it does not hold.
A note on Google. In this reading, a Google search means the list of actual websites below the search box. It does not mean the AI Overview that Google sometimes shows automatically at the top.
How a Language Model Generates
As the previous lesson explained, a language model is made of two files: a large file of numbers called parameters and a smaller program called a runtime. Researchers have shown that large language models store factual knowledge in their parameters. A language model can generate text, which means it produces new text by predicting it one token at a time. A token can be a whole word, part of a word or a single character. First, the runtime uses the parameters to calculate how likely each possible next token is. Next, it selects one token and adds it to the text. Then it repeats the calculation until the answer is complete. For example, when a person asks a chatbot to explain how a battery works, the model will generate the explanation token by token. A chatbot answering from its parameters does not retrieve anything, because no stored copy of the answer exists for it to find. The writer Ted Chiang notes that a chatbot cannot reproduce an exact quote from a web page, because the model keeps only an approximation of the text.
How Hallucinations Happen
A model will generate text for almost any prompt, even when its parameters hold nothing reliable about the question. A hallucination is generated text that sounds correct but contains a false detail. In a 2025 paper, researchers asked a model for the birthday of one of the authors and requested an answer only if the date was known. Across three attempts, the model generated three different dates, and all three were wrong. The model is not broken when this happens. It is still predicting the next token in the usual way, and this time the prediction is wrong. The same paper explains that a birthday follows no pattern. A model often cannot predict such a fact correctly when it appeared rarely in the training text. A typo in a stored record is not an example of a hallucination, because a person made that mistake when typing the record. The system will retrieve the record exactly as it was stored.
Chiang compares a language model to a blurry JPEG, a photo saved with lossy compression. In his comparison, a hallucination is like a compression artifact, a detail filled in where the original was not kept. Lossy compression is one useful way to think about a model, although it is a comparison and leaves out parts of how a model works.
Tools That Retrieve and Generate
Some tools do both. They search first, and then they generate an answer from what they found. The AI summary at the top of a Google search page works this way. Google states that its AI Overviews come from a language model connected to its ranking systems, which find relevant pages in its index. In this tool, the search step retrieves pages, and the language model step generates the summary. The summary can still contain errors, and Google reported that some AI Overviews misread a question or treated a satirical web page as fact. Figure 1 shows both paths side by side, with a dotted path for a tool that does both.
A common mistake is copying an AI answer without checking it. The fix is to ask the same question every time, "What source can I check this answer against?", before using the answer.
Summary
A search engine or a library catalog retrieves stored items and returns them unchanged, or it returns nothing. A language model generates new text from its parameters one token at a time, and a hallucination is generated text with a false detail. Some tools search first and then generate, and any generated answer needs a source to check it against.
Sources: Google, In-depth guide to how Google Search works (2025); Google, Introduction to Large Language Models (2026); Library of Congress, Library of Congress Catalog (n.d.); Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020); Kalai et al., Why Language Models Hallucinate (2025); Chiang, ChatGPT Is a Blurry JPEG of the Web (2023); Reid, AI Overviews: About last week (2024).
Retrieving and Generating, or: Where Answers Come From
This entry explains the two ways a computer can answer a question, and why one of them occasionally invents things with a perfectly straight face.
Fetchers: Search Engines and Catalogs
A search engine and a library catalog both answer questions the same unglamorous way: by finding something somebody stored earlier. These systems retrieve information, which means they find a stored item and hand it back unchanged. No improvements, no opinions. Google keeps information about web pages in its index, which Google describes as a large database. When a person types a query, the search engine finds matching pages in the index and returns links to them. A library catalog works the same way. Search for a book the library owns, and you retrieve the record for that book. Search for a book the library does not own, and there is no record to find, so you get nothing. This is disappointing, but it is honest. A library catalog cannot generate a record for a book it does not hold, and it would not dream of trying.
A note on Google. For travelers: here, a Google search means the list of actual websites below the search box. It does not mean the AI Overview that Google sometimes places at the top without being asked.
Makers: How a Language Model Generates
As the previous entry explained, a language model is two files: a large file of numbers called parameters and a smaller program called a runtime. Researchers have shown that large language models store factual knowledge in their parameters, smeared across billions of numbers. A language model can generate text, which means it produces new text by predicting it one token at a time. A token can be a whole word, part of a word or a single character. First, the runtime uses the parameters to work out how likely each possible next token is. Next, it picks one and adds it to the text. Then it does the whole thing again, and again, until the answer is finished. Ask a chatbot how a battery works, and the model will generate the explanation token by token, like someone laying track just in front of a moving train. A chatbot answering from its parameters does not retrieve anything, because there is no stored copy of the answer anywhere for it to find. The writer Ted Chiang points out that a chatbot cannot reproduce an exact quote from a web page, because the model keeps only an approximation of the text.
Hallucinations, Delivered Calmly
A model will generate text for almost any prompt, including prompts its parameters know nothing reliable about. It does not get nervous. A hallucination is generated text that sounds correct but contains a false detail. In a 2025 paper, researchers asked a model for the birthday of one of the authors and asked for an answer only if the date was known. Across three attempts, the model generated three different dates. All three were wrong, which is a lot of birthdays for one person. The model is not broken when this happens. It is predicting the next token exactly as usual, and this time the prediction happens to be wrong. The same paper explains that a birthday follows no pattern. When a fact like that appeared rarely in the training text, a model often cannot predict it correctly. A typo in a stored record is not a hallucination, because a person made that mistake while typing. The system will retrieve the record exactly as it was stored, typo and all, with the loyalty of a dog bringing back a shoe.
Chiang compares a language model to a blurry JPEG, a photo saved with lossy compression. In his comparison, a hallucination is like a compression artifact: a detail filled in where the original was not kept. Lossy compression is a useful way to think about a model, although it is a comparison, and comparisons always leave something out.
Tools That Do Both
Some tools fetch and then make. They search first, and then they generate an answer from what they found. The AI summary at the top of a Google search page works this way. Google states that its AI Overviews come from a language model connected to its ranking systems, which find relevant pages in its index. The search step retrieves pages, and the language model step generates the summary. The summary can still be wrong, and Google itself reported that some AI Overviews misread a question or treated a satirical web page as fact. Figure 1 shows both paths side by side, with a dotted path for a tool that does both.
The most common mistake in this part of the galaxy is copying an AI answer without checking it. The fix is one question, asked every time before you use an answer: "What source can I check this answer against?"
Summary
A search engine or a library catalog retrieves stored items and returns them unchanged, or it returns nothing. A language model generates new text from its parameters one token at a time, and a hallucination is generated text with a false detail, delivered calmly. Some tools search first and then generate, and any generated answer needs a source to check it against.
Sources: Google, In-depth guide to how Google Search works (2025); Google, Introduction to Large Language Models (2026); Library of Congress, Library of Congress Catalog (n.d.); Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020); Kalai et al., Why Language Models Hallucinate (2025); Chiang, ChatGPT Is a Blurry JPEG of the Web (2023); Reid, AI Overviews: About last week (2024).
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
I Do questions
-
notebook cue: retrieve
You search a library catalog for a book the library does not own. What comes back?
Check
Nothing comes back. A catalog can only retrieve records that were stored, and there is no record for a book the library does not own, so the search comes back empty. See How Search Engines and Catalogs Retrieve, paragraph 1.
-
notebook cue: generate
You ask a chatbot about a book that does not exist. What comes back, and why is that different from the catalog?
Check
Some text comes back, often a fluent description of the book. A model will generate text for almost any prompt by predicting one token at a time from its parameters, while the catalog has no stored record to return. See How a Language Model Generates and How Hallucinations Happen, paragraph 1.
-
notebook cue: hallucination
A chatbot gives a confident, wrong birthday for a real person. Using the two files, where did that date come from?
Check
The runtime generated it from the parameters, one token at a time. The date was never stored as a record, so the prediction produced a likely-looking date that was wrong, which is a hallucination. See How Hallucinations Happen, paragraph 1, where a model gave three different wrong birthdays.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
I Do questions
-
notebook cue: retrieve
You search a library catalog for a book the library does not own. What comes back?
Check
Nothing comes back. A catalog can only retrieve records that were stored, and there is no record for a book the library does not own, so the search comes back empty. See How Search Engines and Catalogs Retrieve, paragraph 1.
-
notebook cue: generate
You ask a chatbot about a book that does not exist. What comes back, and why is that different from the catalog?
Check
Some text comes back, often a fluent description of the book. A model will generate text for almost any prompt by predicting one token at a time from its parameters, while the catalog has no stored record to return. See How a Language Model Generates and How Hallucinations Happen, paragraph 1.
-
notebook cue: hallucination
A chatbot gives a confident, wrong birthday for a real person. Using the two files, where did that date come from?
Check
The runtime generated it from the parameters, one token at a time. The date was never stored as a record, so the prediction produced a likely-looking date that was wrong, which is a hallucination. See How Hallucinations Happen, paragraph 1, where a model gave three different wrong birthdays.
Retrieving and Generating
This section explains two ways a computer can answer a question, and why one of them sometimes produces false details.
How Search Engines and Catalogs Retrieve
A search engine and a library catalog both answer a question by finding something stored earlier. These systems retrieve information, which means they find a stored item and return it unchanged. Google stores information about web pages in its index, which Google describes as a large database. When a person types a query, the search engine finds matching pages in the index and returns links to them. A library catalog works the same way. A search for a book the library owns will retrieve the record for that book. If the library does not own the book, there is no record to find, so the search comes back empty. A library catalog cannot generate a record for a book it does not hold.
A note on Google. In this reading, a Google search means the list of actual websites below the search box. It does not mean the AI Overview that Google sometimes shows automatically at the top.
How a Language Model Generates
As the previous lesson explained, a language model is made of two files: a large file of numbers called parameters and a smaller program called a runtime. Researchers have shown that large language models store factual knowledge in their parameters. A language model can generate text, which means it produces new text by predicting it one token at a time. A token can be a whole word, part of a word or a single character. First, the runtime uses the parameters to calculate how likely each possible next token is. Next, it selects one token and adds it to the text. Then it repeats the calculation until the answer is complete. For example, when a person asks a chatbot to explain how a battery works, the model will generate the explanation token by token. A chatbot answering from its parameters does not retrieve anything, because no stored copy of the answer exists for it to find. The writer Ted Chiang notes that a chatbot cannot reproduce an exact quote from a web page, because the model keeps only an approximation of the text.
How Hallucinations Happen
A model will generate text for almost any prompt, even when its parameters hold nothing reliable about the question. A hallucination is generated text that sounds correct but contains a false detail. In a 2025 paper, researchers asked a model for the birthday of one of the authors and requested an answer only if the date was known. Across three attempts, the model generated three different dates, and all three were wrong. The model is not broken when this happens. It is still predicting the next token in the usual way, and this time the prediction is wrong. The same paper explains that a birthday follows no pattern. A model often cannot predict such a fact correctly when it appeared rarely in the training text. A typo in a stored record is not an example of a hallucination, because a person made that mistake when typing the record. The system will retrieve the record exactly as it was stored.
Chiang compares a language model to a blurry JPEG, a photo saved with lossy compression. In his comparison, a hallucination is like a compression artifact, a detail filled in where the original was not kept. Lossy compression is one useful way to think about a model, although it is a comparison and leaves out parts of how a model works.
Tools That Retrieve and Generate
Some tools do both. They search first, and then they generate an answer from what they found. The AI summary at the top of a Google search page works this way. Google states that its AI Overviews come from a language model connected to its ranking systems, which find relevant pages in its index. In this tool, the search step retrieves pages, and the language model step generates the summary. The summary can still contain errors, and Google reported that some AI Overviews misread a question or treated a satirical web page as fact. Figure 1 shows both paths side by side, with a dotted path for a tool that does both.
A common mistake is copying an AI answer without checking it. The fix is to ask the same question every time, "What source can I check this answer against?", before using the answer.
Summary
A search engine or a library catalog retrieves stored items and returns them unchanged, or it returns nothing. A language model generates new text from its parameters one token at a time, and a hallucination is generated text with a false detail. Some tools search first and then generate, and any generated answer needs a source to check it against.
Sources: Google, In-depth guide to how Google Search works (2025); Google, Introduction to Large Language Models (2026); Library of Congress, Library of Congress Catalog (n.d.); Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020); Kalai et al., Why Language Models Hallucinate (2025); Chiang, ChatGPT Is a Blurry JPEG of the Web (2023); Reid, AI Overviews: About last week (2024).
Retrieving and Generating, or: Where Answers Come From
This entry explains the two ways a computer can answer a question, and why one of them occasionally invents things with a perfectly straight face.
Fetchers: Search Engines and Catalogs
A search engine and a library catalog both answer questions the same unglamorous way: by finding something somebody stored earlier. These systems retrieve information, which means they find a stored item and hand it back unchanged. No improvements, no opinions. Google keeps information about web pages in its index, which Google describes as a large database. When a person types a query, the search engine finds matching pages in the index and returns links to them. A library catalog works the same way. Search for a book the library owns, and you retrieve the record for that book. Search for a book the library does not own, and there is no record to find, so you get nothing. This is disappointing, but it is honest. A library catalog cannot generate a record for a book it does not hold, and it would not dream of trying.
A note on Google. For travelers: here, a Google search means the list of actual websites below the search box. It does not mean the AI Overview that Google sometimes places at the top without being asked.
Makers: How a Language Model Generates
As the previous entry explained, a language model is two files: a large file of numbers called parameters and a smaller program called a runtime. Researchers have shown that large language models store factual knowledge in their parameters, smeared across billions of numbers. A language model can generate text, which means it produces new text by predicting it one token at a time. A token can be a whole word, part of a word or a single character. First, the runtime uses the parameters to work out how likely each possible next token is. Next, it picks one and adds it to the text. Then it does the whole thing again, and again, until the answer is finished. Ask a chatbot how a battery works, and the model will generate the explanation token by token, like someone laying track just in front of a moving train. A chatbot answering from its parameters does not retrieve anything, because there is no stored copy of the answer anywhere for it to find. The writer Ted Chiang points out that a chatbot cannot reproduce an exact quote from a web page, because the model keeps only an approximation of the text.
Hallucinations, Delivered Calmly
A model will generate text for almost any prompt, including prompts its parameters know nothing reliable about. It does not get nervous. A hallucination is generated text that sounds correct but contains a false detail. In a 2025 paper, researchers asked a model for the birthday of one of the authors and asked for an answer only if the date was known. Across three attempts, the model generated three different dates. All three were wrong, which is a lot of birthdays for one person. The model is not broken when this happens. It is predicting the next token exactly as usual, and this time the prediction happens to be wrong. The same paper explains that a birthday follows no pattern. When a fact like that appeared rarely in the training text, a model often cannot predict it correctly. A typo in a stored record is not a hallucination, because a person made that mistake while typing. The system will retrieve the record exactly as it was stored, typo and all, with the loyalty of a dog bringing back a shoe.
Chiang compares a language model to a blurry JPEG, a photo saved with lossy compression. In his comparison, a hallucination is like a compression artifact: a detail filled in where the original was not kept. Lossy compression is a useful way to think about a model, although it is a comparison, and comparisons always leave something out.
Tools That Do Both
Some tools fetch and then make. They search first, and then they generate an answer from what they found. The AI summary at the top of a Google search page works this way. Google states that its AI Overviews come from a language model connected to its ranking systems, which find relevant pages in its index. The search step retrieves pages, and the language model step generates the summary. The summary can still be wrong, and Google itself reported that some AI Overviews misread a question or treated a satirical web page as fact. Figure 1 shows both paths side by side, with a dotted path for a tool that does both.
The most common mistake in this part of the galaxy is copying an AI answer without checking it. The fix is one question, asked every time before you use an answer: "What source can I check this answer against?"
Summary
A search engine or a library catalog retrieves stored items and returns them unchanged, or it returns nothing. A language model generates new text from its parameters one token at a time, and a hallucination is generated text with a false detail, delivered calmly. Some tools search first and then generate, and any generated answer needs a source to check it against.
Sources: Google, In-depth guide to how Google Search works (2025); Google, Introduction to Large Language Models (2026); Library of Congress, Library of Congress Catalog (n.d.); Lewis et al., Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020); Kalai et al., Why Language Models Hallucinate (2025); Chiang, ChatGPT Is a Blurry JPEG of the Web (2023); Reid, AI Overviews: About last week (2024).
Each table decides: same or not the same. The table agrees on one answer and on the one difference that matters most.
Chromebooks stay closed while your table talks. Part 1 of the discussion form is your answer from your first reading. After you talk, finish Part 2 and Part 3 right here under My View. Submit before the exit ticket.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
This form has three parts. Part 1 after your first reading, Part 2 during table talk, Part 3 at the end of table talk. Today's Board Question: Is asking a chatbot for the definition to a word the same as searching a dictionary (electronic or hardcopy)? Yes or no, and one reason.
CR 1. (Board Question · answer 1) From your first reading, answer today's Board Question: Is asking a chatbot for the definition to a word the same as searching a dictionary (electronic or hardcopy)? Yes or no, and one reason. Give your answer and one reason.
Write your answer in your lab notebook, labeled CR 1.
MC 1. (Where you stand) After talking with your table, which position is closest to yours now?
- Yes. Both give an answer when a person types a question.
- No. A search engine retrieves stored pages, and a chatbot generates new text.
- Sometimes. Some tools search first and then generate an answer.
MC 2. (Statement 1) When a chatbot has no information about a question, it gives no answer.
- Agree
- Disagree
MC 3. (Statement 2) A hallucination means the model is broken.
- Agree
- Disagree
MC 4. (Statement 3) A search result is easier to check against its source than a chatbot answer.
- Agree
- Disagree
CR 2. (My view) Write one thing you said or thought at your table.
Write your answer in your lab notebook, labeled CR 2.
CR 3. (A different view) Write the view at your table that was most different from yours. If everyone agreed, write the strongest argument against your table's answer.
Write your answer in your lab notebook, labeled CR 3.
CR 4. (Board Question · answer 2) Think about the Board Question again: Is asking a chatbot for the definition to a word the same as searching a dictionary (electronic or hardcopy)? Yes or no, and one reason. Did you reach the same conclusion you had after your first reading, or a different one? What evidence from the reading and schematic(s), your table talk may have caused you to change your mind. If you have the same conclusion as you did at the beginning of class, explain how the readings/schematic(s), your table talk reinforce your opinion.
Write your answer in your lab notebook, labeled CR 4.