The Model Is Two Files
- Terminology and concepts to look out for: parameters · runtime · prediction
- Relevance to me: Last week you learned about compression in photos and music. Today the same idea helps explain the language models you have used all semester.Last week you learned about compression in photos and music. Today the same idea explains the language models you have used all semester, which turn out to be less mysterious and more numbers than advertised.
- Board Question: If you cut a chatbot off from the internet, could it still answer you? Yes or no, and why.
Open the discussion form on the right and answer Section 1 from your first reading.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
I Do questions
-
notebook cue: parameters
If you opened the big file, would you find sentences or numbers?
Check
Numbers. The parameter file for Llama 2 70B holds 70 billion numbers in 140 GB and not one sentence. See The Parameter File, paragraph 1.
-
notebook cue: runtime
The parameter file sits on a drive and does nothing. What has to happen before it can answer anything?
Check
A runtime has to read the parameter file and do the arithmetic with its numbers. A parameter file with no runtime produces nothing. See The Runtime, paragraph 1.
-
notebook cue: prediction
The model does not write its whole answer at once. What does it produce on each step?
Check
One token. On each pass of the loop the model predicts a likely next token, adds it to the end of the text and goes again. See Prediction, One Token at a Time, and Figure 1.
A Language Model Is Two Files
This section explains what a language model is made of when it is stored on a computer, and what it does each time it runs.
The Parameter File
A language model is a computer program that predicts text. When a language model is stored on a computer, it can be described as just two files. One example is Llama 2 70B, a model that Meta released for anyone to download and run. Parameters are the numbers a language model uses to calculate what text comes next. The larger of the two files holds the parameters. Llama 2 70B has 70 billion parameters. Each parameter is stored as a number that takes up 2 bytes, so the parameter file for this model is 140 GB. A person who opened that file would see a long list of numbers and would not find a single sentence written out in it. A parameter file is not an example of a saved document, because a saved document holds its sentences and gives them back exactly. The parameter file holds no sentences, so its numbers are useful only when a program reads them and does arithmetic with them.
Where the Parameters Come From
The parameters are set by training, a process in which a computer adjusts the numbers while it processes a very large amount of text. For Llama 2 70B, the training data was about 10 TB of text taken from the internet. One useful way to think about the result is as lossy compression of the text that was used in training. Lossy compression makes a file smaller by removing detail that cannot be recovered, like the JPEG photos from last week. The 140 GB parameter file is far smaller than the 10 TB of text, so it cannot hold an exact copy of that text. This is a comparison and not an exact description of what training does. A JPEG still stores a picture, while the parameter file stores only numbers that help predict text.
The Runtime
The second, much smaller file is the runtime. A runtime is the program that reads the parameter file and does the arithmetic that turns those numbers into text. Andrej Karpathy's 2023 talk "Intro to Large Language Models" describes a runtime for Llama 2 70B that takes about 500 lines of code in the C programming language. Another example is llama.cpp, a runtime written in C and C++ that needs no other software libraries to work. A runtime with no parameter file has nothing to run, because the arithmetic needs the numbers that the parameter file stores. A parameter file with no runtime sits on a drive and produces nothing, because no program is reading its numbers. When both files are on the same computer, they form a complete package, and the model can run without any connection to the internet. Mr. Mugg's computer at home runs open models this way, as two files stored on a drive. MuggsOfPrompts and MuggsOfVibes, the tools used in this class, work differently. They send each prompt over the internet to a model that runs on Mistral's servers.
Prediction, One Token at a Time
A model reads and produces text as tokens, which are words or parts of words that a computer represents as numbers. Prediction is the process of producing a likely next token from the text so far. The runtime carries out prediction in a loop of four steps, as Figure 1 shows. First, it reads all the tokens so far, starting with the prompt. Next, it uses the parameters to score every possible next token by calculating how likely each one is. Then it picks one token. Finally, it adds that token to the end of the text and starts the loop again with a list that is one token longer. For example, given the words "cat sat on a," a model might predict "mat" as the next word. The model repeats the loop one token at a time until the answer is complete. Prediction is not the same as a lookup, because a lookup hands back something that was stored, while a prediction produces something new. For this reason, a model can generate a sentence that does not appear anywhere in its training text. That sentence may be correct, or it may contain details that are false.
Summary
A language model on a computer is two files: a large parameter file that stores numbers and a small runtime that does the arithmetic with them. Each time the model runs, it makes a prediction of the next token, adds that token to the text, and repeats. Training on a very large amount of text set the parameters, and lossy compression is one useful comparison for the result, though not an exact description.
Sources: Karpathy, "[1hr Talk] Intro to Large Language Models" (2023); Meta, Llama 2 Model Card (2023); Touvron et al., "Llama 2: Open Foundation and Fine-Tuned Chat Models" (2023); ggml-org, llama.cpp README (2026); Hugging Face, LLM Course, "Tokenizers" (n.d.); classroom data (2026).
A Language Model Is Two Files, Which Is Fewer Than You Would Think
This entry explains what a language model is actually made of when it sits on a computer, and what it does every time someone wakes it up.
The Parameter File
A language model is a computer program that predicts text. Stored on a computer, it can be described as just two files, which is a little underwhelming for something people argue about this much. One example is Llama 2 70B, a model that Meta released for anyone to download and run. Parameters are the numbers a language model uses to calculate what text comes next. The larger of the two files holds the parameters. Llama 2 70B has 70 billion parameters. Each parameter is stored as a number that takes up 2 bytes, so the parameter file for this model is 140 GB. A person who opened that file would see a very long list of numbers and would not find a single sentence written out anywhere, no matter how long they scrolled. A parameter file is not an example of a saved document, because a saved document holds its sentences and gives them back exactly. The parameter file holds no sentences at all, so its numbers are useful only when a program reads them and does arithmetic with them.
Where the Parameters Come From
The parameters are set by training, a process in which a computer adjusts the numbers while it works through a very large amount of text. For Llama 2 70B, the training data was about 10 TB of text taken from the internet, which is to say a great deal of everything. One useful way to think about the result is as lossy compression of the text used in training. Lossy compression makes a file smaller by removing detail that cannot be recovered, like the JPEG photos from last week. The 140 GB parameter file is far smaller than the 10 TB of text, so it cannot hold an exact copy of that text. Something had to go. This is a comparison and not an exact description of what training does. A JPEG still stores a picture, while the parameter file stores only numbers that help predict text.
The Runtime
The second, much smaller file is the runtime. A runtime is the program that reads the parameter file and does the arithmetic that turns those numbers into text. Andrej Karpathy's 2023 talk "Intro to Large Language Models" describes a runtime for Llama 2 70B that takes about 500 lines of code in the C programming language, which is roughly the length of a long essay nobody asked for. Another example is llama.cpp, a runtime written in C and C++ that needs no other software libraries to work. A runtime with no parameter file has nothing to run, because the arithmetic needs the numbers that the parameter file stores. A parameter file with no runtime just sits on a drive and produces nothing, like a cookbook in a house with no kitchen. When both files are on the same computer, they form a complete package, and the model can run without any connection to the internet. Mr. Mugg's computer at home runs open models this way, as two files stored on a drive. MuggsOfPrompts and MuggsOfVibes, the tools used in this class, work differently. They send each prompt over the internet to a model that runs on Mistral's servers, somewhere else entirely.
Prediction, One Token at a Time
A model reads and produces text as tokens, which are words or parts of words that a computer represents as numbers. Prediction is the process of producing a likely next token from the text so far. The runtime carries out prediction in a loop of four steps, as Figure 1 shows. First, it reads all the tokens so far, starting with the prompt. Next, it uses the parameters to score every possible next token by calculating how likely each one is. Then it picks one token. Finally, it adds that token to the end of the text and starts the loop again with a list that is one token longer. It does not get bored. For example, given the words "cat sat on a," a model might predict "mat" as the next word. The model repeats the loop one token at a time until the answer is complete. Prediction is not the same as a lookup, because a lookup hands back something that was stored, while a prediction produces something new. For this reason, a model can generate a sentence that does not appear anywhere in its training text. That sentence may be correct, or it may contain details that are false, and it will sound exactly the same either way.
Summary
A language model on a computer is two files: a large parameter file that stores numbers and a small runtime that does the arithmetic with them. Each time the model runs, it makes a prediction of the next token, adds that token to the text, and repeats, patiently, forever if necessary. Training on a very large amount of text set the parameters, and lossy compression is one useful comparison for the result, though not an exact description.
Sources: Karpathy, "[1hr Talk] Intro to Large Language Models" (2023); Meta, Llama 2 Model Card (2023); Touvron et al., "Llama 2: Open Foundation and Fine-Tuned Chat Models" (2023); ggml-org, llama.cpp README (2026); Hugging Face, LLM Course, "Tokenizers" (n.d.); classroom data (2026).
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
I Do questions
-
notebook cue: parameters
If you opened the big file, would you find sentences or numbers?
Check
Numbers. The parameter file for Llama 2 70B holds 70 billion numbers in 140 GB and not one sentence. See The Parameter File, paragraph 1.
-
notebook cue: runtime
The parameter file sits on a drive and does nothing. What has to happen before it can answer anything?
Check
A runtime has to read the parameter file and do the arithmetic with its numbers. A parameter file with no runtime produces nothing. See The Runtime, paragraph 1.
-
notebook cue: prediction
The model does not write its whole answer at once. What does it produce on each step?
Check
One token. On each pass of the loop the model predicts a likely next token, adds it to the end of the text and goes again. See Prediction, One Token at a Time, and Figure 1.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
I Do questions
-
notebook cue: parameters
If you opened the big file, would you find sentences or numbers?
Check
Numbers. The parameter file for Llama 2 70B holds 70 billion numbers in 140 GB and not one sentence. See The Parameter File, paragraph 1.
-
notebook cue: runtime
The parameter file sits on a drive and does nothing. What has to happen before it can answer anything?
Check
A runtime has to read the parameter file and do the arithmetic with its numbers. A parameter file with no runtime produces nothing. See The Runtime, paragraph 1.
-
notebook cue: prediction
The model does not write its whole answer at once. What does it produce on each step?
Check
One token. On each pass of the loop the model predicts a likely next token, adds it to the end of the text and goes again. See Prediction, One Token at a Time, and Figure 1.
A Language Model Is Two Files
This section explains what a language model is made of when it is stored on a computer, and what it does each time it runs.
The Parameter File
A language model is a computer program that predicts text. When a language model is stored on a computer, it can be described as just two files. One example is Llama 2 70B, a model that Meta released for anyone to download and run. Parameters are the numbers a language model uses to calculate what text comes next. The larger of the two files holds the parameters. Llama 2 70B has 70 billion parameters. Each parameter is stored as a number that takes up 2 bytes, so the parameter file for this model is 140 GB. A person who opened that file would see a long list of numbers and would not find a single sentence written out in it. A parameter file is not an example of a saved document, because a saved document holds its sentences and gives them back exactly. The parameter file holds no sentences, so its numbers are useful only when a program reads them and does arithmetic with them.
Where the Parameters Come From
The parameters are set by training, a process in which a computer adjusts the numbers while it processes a very large amount of text. For Llama 2 70B, the training data was about 10 TB of text taken from the internet. One useful way to think about the result is as lossy compression of the text that was used in training. Lossy compression makes a file smaller by removing detail that cannot be recovered, like the JPEG photos from last week. The 140 GB parameter file is far smaller than the 10 TB of text, so it cannot hold an exact copy of that text. This is a comparison and not an exact description of what training does. A JPEG still stores a picture, while the parameter file stores only numbers that help predict text.
The Runtime
The second, much smaller file is the runtime. A runtime is the program that reads the parameter file and does the arithmetic that turns those numbers into text. Andrej Karpathy's 2023 talk "Intro to Large Language Models" describes a runtime for Llama 2 70B that takes about 500 lines of code in the C programming language. Another example is llama.cpp, a runtime written in C and C++ that needs no other software libraries to work. A runtime with no parameter file has nothing to run, because the arithmetic needs the numbers that the parameter file stores. A parameter file with no runtime sits on a drive and produces nothing, because no program is reading its numbers. When both files are on the same computer, they form a complete package, and the model can run without any connection to the internet. Mr. Mugg's computer at home runs open models this way, as two files stored on a drive. MuggsOfPrompts and MuggsOfVibes, the tools used in this class, work differently. They send each prompt over the internet to a model that runs on Mistral's servers.
Prediction, One Token at a Time
A model reads and produces text as tokens, which are words or parts of words that a computer represents as numbers. Prediction is the process of producing a likely next token from the text so far. The runtime carries out prediction in a loop of four steps, as Figure 1 shows. First, it reads all the tokens so far, starting with the prompt. Next, it uses the parameters to score every possible next token by calculating how likely each one is. Then it picks one token. Finally, it adds that token to the end of the text and starts the loop again with a list that is one token longer. For example, given the words "cat sat on a," a model might predict "mat" as the next word. The model repeats the loop one token at a time until the answer is complete. Prediction is not the same as a lookup, because a lookup hands back something that was stored, while a prediction produces something new. For this reason, a model can generate a sentence that does not appear anywhere in its training text. That sentence may be correct, or it may contain details that are false.
Summary
A language model on a computer is two files: a large parameter file that stores numbers and a small runtime that does the arithmetic with them. Each time the model runs, it makes a prediction of the next token, adds that token to the text, and repeats. Training on a very large amount of text set the parameters, and lossy compression is one useful comparison for the result, though not an exact description.
Sources: Karpathy, "[1hr Talk] Intro to Large Language Models" (2023); Meta, Llama 2 Model Card (2023); Touvron et al., "Llama 2: Open Foundation and Fine-Tuned Chat Models" (2023); ggml-org, llama.cpp README (2026); Hugging Face, LLM Course, "Tokenizers" (n.d.); classroom data (2026).
A Language Model Is Two Files, Which Is Fewer Than You Would Think
This entry explains what a language model is actually made of when it sits on a computer, and what it does every time someone wakes it up.
The Parameter File
A language model is a computer program that predicts text. Stored on a computer, it can be described as just two files, which is a little underwhelming for something people argue about this much. One example is Llama 2 70B, a model that Meta released for anyone to download and run. Parameters are the numbers a language model uses to calculate what text comes next. The larger of the two files holds the parameters. Llama 2 70B has 70 billion parameters. Each parameter is stored as a number that takes up 2 bytes, so the parameter file for this model is 140 GB. A person who opened that file would see a very long list of numbers and would not find a single sentence written out anywhere, no matter how long they scrolled. A parameter file is not an example of a saved document, because a saved document holds its sentences and gives them back exactly. The parameter file holds no sentences at all, so its numbers are useful only when a program reads them and does arithmetic with them.
Where the Parameters Come From
The parameters are set by training, a process in which a computer adjusts the numbers while it works through a very large amount of text. For Llama 2 70B, the training data was about 10 TB of text taken from the internet, which is to say a great deal of everything. One useful way to think about the result is as lossy compression of the text used in training. Lossy compression makes a file smaller by removing detail that cannot be recovered, like the JPEG photos from last week. The 140 GB parameter file is far smaller than the 10 TB of text, so it cannot hold an exact copy of that text. Something had to go. This is a comparison and not an exact description of what training does. A JPEG still stores a picture, while the parameter file stores only numbers that help predict text.
The Runtime
The second, much smaller file is the runtime. A runtime is the program that reads the parameter file and does the arithmetic that turns those numbers into text. Andrej Karpathy's 2023 talk "Intro to Large Language Models" describes a runtime for Llama 2 70B that takes about 500 lines of code in the C programming language, which is roughly the length of a long essay nobody asked for. Another example is llama.cpp, a runtime written in C and C++ that needs no other software libraries to work. A runtime with no parameter file has nothing to run, because the arithmetic needs the numbers that the parameter file stores. A parameter file with no runtime just sits on a drive and produces nothing, like a cookbook in a house with no kitchen. When both files are on the same computer, they form a complete package, and the model can run without any connection to the internet. Mr. Mugg's computer at home runs open models this way, as two files stored on a drive. MuggsOfPrompts and MuggsOfVibes, the tools used in this class, work differently. They send each prompt over the internet to a model that runs on Mistral's servers, somewhere else entirely.
Prediction, One Token at a Time
A model reads and produces text as tokens, which are words or parts of words that a computer represents as numbers. Prediction is the process of producing a likely next token from the text so far. The runtime carries out prediction in a loop of four steps, as Figure 1 shows. First, it reads all the tokens so far, starting with the prompt. Next, it uses the parameters to score every possible next token by calculating how likely each one is. Then it picks one token. Finally, it adds that token to the end of the text and starts the loop again with a list that is one token longer. It does not get bored. For example, given the words "cat sat on a," a model might predict "mat" as the next word. The model repeats the loop one token at a time until the answer is complete. Prediction is not the same as a lookup, because a lookup hands back something that was stored, while a prediction produces something new. For this reason, a model can generate a sentence that does not appear anywhere in its training text. That sentence may be correct, or it may contain details that are false, and it will sound exactly the same either way.
Summary
A language model on a computer is two files: a large parameter file that stores numbers and a small runtime that does the arithmetic with them. Each time the model runs, it makes a prediction of the next token, adds that token to the text, and repeats, patiently, forever if necessary. Training on a very large amount of text set the parameters, and lossy compression is one useful comparison for the result, though not an exact description.
Sources: Karpathy, "[1hr Talk] Intro to Large Language Models" (2023); Meta, Llama 2 Model Card (2023); Touvron et al., "Llama 2: Open Foundation and Fine-Tuned Chat Models" (2023); ggml-org, llama.cpp README (2026); Hugging Face, LLM Course, "Tokenizers" (n.d.); classroom data (2026).
Each table decides: could it still answer, yes or no. The table agrees on one answer and says where the two files would have to be.
Chromebooks stay closed while your table talks. Part 1 of the discussion form is your answer from your first reading. After you talk, finish Part 2 and Part 3 right here under My View. Submit before the exit ticket.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
This form has three parts. Part 1 after your first reading, Part 2 during table talk, Part 3 at the end of table talk. Today's Board Question: If you cut a chatbot off from the internet, could it still answer you? Yes or no, and why.
CR 1. (Board Question · answer 1) From your first reading, answer today's Board Question: If you cut a chatbot off from the internet, could it still answer you? Yes or no, and why. Give your answer and one reason.
Write your answer in your lab notebook, labeled CR 1.
MC 1. (Where you stand) After talking with your table, which position is closest to yours now?
- Yes, if the two files are stored on the device being used.
- No. A chatbot needs the internet to look up its answers.
- No, if the two files are on a server somewhere else.
MC 2. (Statement 1) The parameter file contains the sentences the model was trained on.
- Agree
- Disagree
MC 3. (Statement 2) The runtime can answer questions without the parameter file.
- Agree
- Disagree
MC 4. (Statement 3) A model writes its answer one token at a time.
- Agree
- Disagree
CR 2. (My view) Write one thing you said or thought at your table.
Write your answer in your lab notebook, labeled CR 2.
CR 3. (A different view) Write the view at your table that was most different from yours. If everyone agreed, write the strongest argument against your table's answer.
Write your answer in your lab notebook, labeled CR 3.
CR 4. (Board Question · answer 2) Think about the Board Question again: If you cut a chatbot off from the internet, could it still answer you? Yes or no, and why. Did you reach the same conclusion you had after your first reading, or a different one? What evidence from the reading and schematic(s), your table talk may have caused you to change your mind. If you have the same conclusion as you did at the beginning of class, explain how the readings/schematic(s), your table talk reinforce your opinion.
Write your answer in your lab notebook, labeled CR 4.