What It Takes to Run One
- Terminology and concepts to look out for: CPU · GPU · memory
- Relevance to me: Yesterday you learned that the answer comes from a server. Today you learn which parts of a computer a model needs in order to run.Yesterday you learned that the answer comes from a server, somewhere far away and humming. Today you learn which parts of a computer a model needs in order to run, and why yours is politely sitting this one out.
- Board Question: Could the Chromebook in front of you run the model by itself? Yes or no, and what would it need.
Open the discussion form on the right and answer Section 1 from your first reading.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
I Do questions
-
notebook cue: memory
The parameter file is saved on the drive. Is it in memory yet? What has to happen first?
Check
No. A file saved on the drive sits in storage, and the computer has to copy it from storage into memory before the model can predict a single token. See Storage and Memory, paragraph 2.
-
notebook cue: GPU
Why does a model answer faster on a GPU than on a CPU?
Check
A model's prediction is made of very large groups of multiplications that can be done at the same time. A GPU carries out thousands of simple calculations at once, while a CPU works on only a few tens. See The CPU and the GPU, paragraphs 1 and 2.
-
notebook cue: CPU
A CPU can do the model's arithmetic. What goes wrong if it is the only part doing it?
Check
The answer still comes out, but slowly, because the CPU works on far fewer calculations at once than a GPU and produces tokens much more slowly. See The CPU and the GPU, paragraph 2.
What It Takes to Run a Model
This section explains which parts of a computer a language model needs to run.
Storage and Memory
A server is a computer, and it is built from the same main parts as the computers in a classroom. Those parts include a central processing unit, a graphics processing unit, memory and storage. Storage is the part that keeps files, and it holds them even after the computer is turned off. Memory is the part of a computer that holds the programs and data being used right now, and it is much faster than storage. Memory loses what it holds when the power goes off. Before a program can run, the computer copies the program and its data from storage into memory.
For example, a language model's parameter file sits in storage while nothing is happening, and the computer has to load it into memory before the model can predict a single token, and the parameter file stays loaded in memory while the model runs. A parameter file that is only saved on a drive is not in memory yet, because nothing has copied it there. A model needs enough memory to hold all of its parameters at once.
The CPU and the GPU
The CPU, or central processing unit, is the part of a computer whose main job is to run programs. It is built to finish one series of steps quickly, and it works on only a few tens of them at the same time.
A GPU, or graphics processing unit, is a part built to carry out thousands of simple calculations at the same time. For example, a model's prediction is made of very large groups of multiplications, and many of them can be done at the same time, which is the kind of work a GPU is built for. A CPU can do the same arithmetic, but it works on far fewer calculations at once, so it produces tokens much more slowly. A GPU is not the part that runs the rest of the computer, because the operating system and the other programs run on the CPU.
Model Size and Lab 1
In Lab 1, the three panes used three models from Mistral: Ministral 3B, Ministral 14B and Mistral Large 3. Ministral 3B has about 3.4 billion parameters for language, Ministral 14B has about 13.5 billion, and Mistral Large 3 has 675 billion in total. A bigger parameter file needs more memory to load. A model predicts one token at a time, and its parameters are read again for each new token. Mistral Large 3 uses about 41 billion of its parameters for each token, which is still more than either Ministral model has in total.
The seconds and tokens per second that students recorded in Lab 1 measured the whole trip through Mistral's service. Speed through a service also depends on the machines the provider uses and how it runs the model on them. For this reason, a slower pane fits with a bigger model, but the Lab 1 numbers alone do not show that size was the only cause.
Two Machines Side by Side
Figure 1 shows the inside of a server beside two spec cards. The first card describes a typical school Chromebook with 4 GB or 8 GB of memory and 32 GB or 64 GB of storage. Its GPU is built into the processor chip and shares the system memory with everything else on the computer. The second card describes an example machine that can run a model, with an NVIDIA GeForce RTX 4090 GPU that has 24 GB of its own memory and with 96 GB of system memory. This machine is an example and not the class server, because the class models run on Mistral's servers.
The model card for Ministral 14B says it fits in 24 GB of GPU memory, so it fits on the example machine's GPU and not on either Chromebook. Ministral 3B, the smallest class model, fits in 8 GB of GPU memory in the form Mistral publishes, which is all the memory the larger Chromebook has. The same model card says that a more compressed version needs less memory. A very small model could therefore load on a Chromebook, and the Chromebook would produce its tokens slowly. Mistral Large 3 needs far more, and its model card names a machine with eight NVIDIA H200 GPUs, each with 141 GB of memory.
Summary
A computer has to load a model from storage into memory before the model can run, and the memory must be large enough to hold all of its parameters. A GPU carries out the many simple calculations of prediction at the same time, while the CPU runs the rest of the computer and can run a model only slowly. A school Chromebook has enough memory for a very small model at most, and the class models need far more.
Sources: CompTIA, Tech+ FC0-U71 Exam Objectives (2023); Franchitti, Introduction to Computer Science, OpenStax (2024); NVIDIA, CUDA Programming Guide (2026); Verma and Vaidya, "Mastering LLM Techniques: Inference Optimization," NVIDIA (2023); NVIDIA, GeForce RTX 4090 and H200 specification pages; Lenovo, 100e Chromebook Gen 4 PSREF (2023); Mistral AI, model cards and "Introducing Mistral 3" (2025); MuggsOfCompSci, Lab 1 records and build spec (2026).
What It Takes to Run a Model, or: Where the Thinking Machine Keeps Its Stuff
This entry explains which parts of a computer a language model needs before it can do anything at all.
Storage and Memory
A server is a computer. It is not a glowing oracle in a mountain. It is built from the same main parts as the computers in a classroom: a central processing unit, a graphics processing unit, memory and storage. Storage is the part that keeps files, and it holds them even after the computer is turned off, like a closet. Memory is the part of a computer that holds the programs and data being used right now, and it is much faster than storage. Memory loses what it holds when the power goes off, which it accepts with great dignity. Before a program can run, the computer copies the program and its data from storage into memory.
For example, a language model's parameter file sits in storage while nothing is happening, and the computer has to load it into memory before the model can predict a single token, and the parameter file stays loaded in memory while the model runs. A parameter file that is only saved on a drive is not in memory yet, because nothing has copied it there. Owning a cookbook is not the same as cooking. A model needs enough memory to hold all of its parameters at once.
The CPU and the GPU
The CPU, or central processing unit, is the part of a computer whose main job is to run programs. It is built to finish one series of steps quickly, and it works on only a few tens of them at the same time. Think of a very fast, very focused chef.
A GPU, or graphics processing unit, is a part built to carry out thousands of simple calculations at the same time. Think of a stadium full of people who can each do one multiplication. For example, a model's prediction is made of very large groups of multiplications, and many of them can be done at the same time, which is exactly the kind of work a GPU is built for. A CPU can do the same arithmetic, but it works on far fewer calculations at once, so it produces tokens much more slowly. A GPU is not the part that runs the rest of the computer, because the operating system and the other programs run on the CPU. The stadium does not do the dishes.
Model Size and Lab 1
In Lab 1, the three panes used three models from Mistral: Ministral 3B, Ministral 14B and Mistral Large 3. Ministral 3B has about 3.4 billion parameters for language, Ministral 14B has about 13.5 billion, and Mistral Large 3 has 675 billion in total, a number best not thought about for too long. A bigger parameter file needs more memory to load. A model predicts one token at a time, and its parameters are read again for each new token. Mistral Large 3 uses about 41 billion of its parameters for each token, which is still more than either Ministral model has in total.
The seconds and tokens per second that students recorded in Lab 1 measured the whole trip through Mistral's service. Speed through a service also depends on the machines the provider uses and how it runs the model on them, none of which anyone tells you. For this reason, a slower pane fits with a bigger model, but the Lab 1 numbers alone do not show that size was the only cause.
Two Machines Side by Side
Figure 1 shows the inside of a server beside two spec cards. The first card describes a typical school Chromebook with 4 GB or 8 GB of memory and 32 GB or 64 GB of storage. Its GPU is built into the processor chip and shares the system memory with everything else on the computer, the way a younger sibling shares a room. The second card describes an example machine that can run a model, with an NVIDIA GeForce RTX 4090 GPU that has 24 GB of its own memory and with 96 GB of system memory. This machine is an example and not the class server, because the class models run on Mistral's servers, somewhere else entirely.
The model card for Ministral 14B says it fits in 24 GB of GPU memory, so it fits on the example machine's GPU and not on either Chromebook. Ministral 3B, the smallest class model, fits in 8 GB of GPU memory in the form Mistral publishes, which is all the memory the larger Chromebook has. The same model card says that a more compressed version needs less memory. A very small model could therefore load on a Chromebook, and the Chromebook would produce its tokens slowly, with visible effort. Mistral Large 3 needs far more, and its model card names a machine with eight NVIDIA H200 GPUs, each with 141 GB of memory.
Summary
A computer has to load a model from storage into memory before the model can run, and the memory must be large enough to hold all of its parameters. A GPU carries out the many simple calculations of prediction at the same time, while the CPU runs the rest of the computer and can run a model only slowly. A school Chromebook has enough memory for a very small model at most, and the class models need far more. Your Chromebook is fine. It is simply not a data center, and nobody expected it to be.
Sources: CompTIA, Tech+ FC0-U71 Exam Objectives (2023); Franchitti, Introduction to Computer Science, OpenStax (2024); NVIDIA, CUDA Programming Guide (2026); Verma and Vaidya, "Mastering LLM Techniques: Inference Optimization," NVIDIA (2023); NVIDIA, GeForce RTX 4090 and H200 specification pages; Lenovo, 100e Chromebook Gen 4 PSREF (2023); Mistral AI, model cards and "Introducing Mistral 3" (2025); MuggsOfCompSci, Lab 1 records and build spec (2026).
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
I Do questions
-
notebook cue: memory
The parameter file is saved on the drive. Is it in memory yet? What has to happen first?
Check
No. A file saved on the drive sits in storage, and the computer has to copy it from storage into memory before the model can predict a single token. See Storage and Memory, paragraph 2.
-
notebook cue: GPU
Why does a model answer faster on a GPU than on a CPU?
Check
A model's prediction is made of very large groups of multiplications that can be done at the same time. A GPU carries out thousands of simple calculations at once, while a CPU works on only a few tens. See The CPU and the GPU, paragraphs 1 and 2.
-
notebook cue: CPU
A CPU can do the model's arithmetic. What goes wrong if it is the only part doing it?
Check
The answer still comes out, but slowly, because the CPU works on far fewer calculations at once than a GPU and produces tokens much more slowly. See The CPU and the GPU, paragraph 2.
The big picture so far (newest on top)
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
I Do questions
-
notebook cue: memory
The parameter file is saved on the drive. Is it in memory yet? What has to happen first?
Check
No. A file saved on the drive sits in storage, and the computer has to copy it from storage into memory before the model can predict a single token. See Storage and Memory, paragraph 2.
-
notebook cue: GPU
Why does a model answer faster on a GPU than on a CPU?
Check
A model's prediction is made of very large groups of multiplications that can be done at the same time. A GPU carries out thousands of simple calculations at once, while a CPU works on only a few tens. See The CPU and the GPU, paragraphs 1 and 2.
-
notebook cue: CPU
A CPU can do the model's arithmetic. What goes wrong if it is the only part doing it?
Check
The answer still comes out, but slowly, because the CPU works on far fewer calculations at once than a GPU and produces tokens much more slowly. See The CPU and the GPU, paragraph 2.
What It Takes to Run a Model
This section explains which parts of a computer a language model needs to run.
Storage and Memory
A server is a computer, and it is built from the same main parts as the computers in a classroom. Those parts include a central processing unit, a graphics processing unit, memory and storage. Storage is the part that keeps files, and it holds them even after the computer is turned off. Memory is the part of a computer that holds the programs and data being used right now, and it is much faster than storage. Memory loses what it holds when the power goes off. Before a program can run, the computer copies the program and its data from storage into memory.
For example, a language model's parameter file sits in storage while nothing is happening, and the computer has to load it into memory before the model can predict a single token, and the parameter file stays loaded in memory while the model runs. A parameter file that is only saved on a drive is not in memory yet, because nothing has copied it there. A model needs enough memory to hold all of its parameters at once.
The CPU and the GPU
The CPU, or central processing unit, is the part of a computer whose main job is to run programs. It is built to finish one series of steps quickly, and it works on only a few tens of them at the same time.
A GPU, or graphics processing unit, is a part built to carry out thousands of simple calculations at the same time. For example, a model's prediction is made of very large groups of multiplications, and many of them can be done at the same time, which is the kind of work a GPU is built for. A CPU can do the same arithmetic, but it works on far fewer calculations at once, so it produces tokens much more slowly. A GPU is not the part that runs the rest of the computer, because the operating system and the other programs run on the CPU.
Model Size and Lab 1
In Lab 1, the three panes used three models from Mistral: Ministral 3B, Ministral 14B and Mistral Large 3. Ministral 3B has about 3.4 billion parameters for language, Ministral 14B has about 13.5 billion, and Mistral Large 3 has 675 billion in total. A bigger parameter file needs more memory to load. A model predicts one token at a time, and its parameters are read again for each new token. Mistral Large 3 uses about 41 billion of its parameters for each token, which is still more than either Ministral model has in total.
The seconds and tokens per second that students recorded in Lab 1 measured the whole trip through Mistral's service. Speed through a service also depends on the machines the provider uses and how it runs the model on them. For this reason, a slower pane fits with a bigger model, but the Lab 1 numbers alone do not show that size was the only cause.
Two Machines Side by Side
Figure 1 shows the inside of a server beside two spec cards. The first card describes a typical school Chromebook with 4 GB or 8 GB of memory and 32 GB or 64 GB of storage. Its GPU is built into the processor chip and shares the system memory with everything else on the computer. The second card describes an example machine that can run a model, with an NVIDIA GeForce RTX 4090 GPU that has 24 GB of its own memory and with 96 GB of system memory. This machine is an example and not the class server, because the class models run on Mistral's servers.
The model card for Ministral 14B says it fits in 24 GB of GPU memory, so it fits on the example machine's GPU and not on either Chromebook. Ministral 3B, the smallest class model, fits in 8 GB of GPU memory in the form Mistral publishes, which is all the memory the larger Chromebook has. The same model card says that a more compressed version needs less memory. A very small model could therefore load on a Chromebook, and the Chromebook would produce its tokens slowly. Mistral Large 3 needs far more, and its model card names a machine with eight NVIDIA H200 GPUs, each with 141 GB of memory.
Summary
A computer has to load a model from storage into memory before the model can run, and the memory must be large enough to hold all of its parameters. A GPU carries out the many simple calculations of prediction at the same time, while the CPU runs the rest of the computer and can run a model only slowly. A school Chromebook has enough memory for a very small model at most, and the class models need far more.
Sources: CompTIA, Tech+ FC0-U71 Exam Objectives (2023); Franchitti, Introduction to Computer Science, OpenStax (2024); NVIDIA, CUDA Programming Guide (2026); Verma and Vaidya, "Mastering LLM Techniques: Inference Optimization," NVIDIA (2023); NVIDIA, GeForce RTX 4090 and H200 specification pages; Lenovo, 100e Chromebook Gen 4 PSREF (2023); Mistral AI, model cards and "Introducing Mistral 3" (2025); MuggsOfCompSci, Lab 1 records and build spec (2026).
What It Takes to Run a Model, or: Where the Thinking Machine Keeps Its Stuff
This entry explains which parts of a computer a language model needs before it can do anything at all.
Storage and Memory
A server is a computer. It is not a glowing oracle in a mountain. It is built from the same main parts as the computers in a classroom: a central processing unit, a graphics processing unit, memory and storage. Storage is the part that keeps files, and it holds them even after the computer is turned off, like a closet. Memory is the part of a computer that holds the programs and data being used right now, and it is much faster than storage. Memory loses what it holds when the power goes off, which it accepts with great dignity. Before a program can run, the computer copies the program and its data from storage into memory.
For example, a language model's parameter file sits in storage while nothing is happening, and the computer has to load it into memory before the model can predict a single token, and the parameter file stays loaded in memory while the model runs. A parameter file that is only saved on a drive is not in memory yet, because nothing has copied it there. Owning a cookbook is not the same as cooking. A model needs enough memory to hold all of its parameters at once.
The CPU and the GPU
The CPU, or central processing unit, is the part of a computer whose main job is to run programs. It is built to finish one series of steps quickly, and it works on only a few tens of them at the same time. Think of a very fast, very focused chef.
A GPU, or graphics processing unit, is a part built to carry out thousands of simple calculations at the same time. Think of a stadium full of people who can each do one multiplication. For example, a model's prediction is made of very large groups of multiplications, and many of them can be done at the same time, which is exactly the kind of work a GPU is built for. A CPU can do the same arithmetic, but it works on far fewer calculations at once, so it produces tokens much more slowly. A GPU is not the part that runs the rest of the computer, because the operating system and the other programs run on the CPU. The stadium does not do the dishes.
Model Size and Lab 1
In Lab 1, the three panes used three models from Mistral: Ministral 3B, Ministral 14B and Mistral Large 3. Ministral 3B has about 3.4 billion parameters for language, Ministral 14B has about 13.5 billion, and Mistral Large 3 has 675 billion in total, a number best not thought about for too long. A bigger parameter file needs more memory to load. A model predicts one token at a time, and its parameters are read again for each new token. Mistral Large 3 uses about 41 billion of its parameters for each token, which is still more than either Ministral model has in total.
The seconds and tokens per second that students recorded in Lab 1 measured the whole trip through Mistral's service. Speed through a service also depends on the machines the provider uses and how it runs the model on them, none of which anyone tells you. For this reason, a slower pane fits with a bigger model, but the Lab 1 numbers alone do not show that size was the only cause.
Two Machines Side by Side
Figure 1 shows the inside of a server beside two spec cards. The first card describes a typical school Chromebook with 4 GB or 8 GB of memory and 32 GB or 64 GB of storage. Its GPU is built into the processor chip and shares the system memory with everything else on the computer, the way a younger sibling shares a room. The second card describes an example machine that can run a model, with an NVIDIA GeForce RTX 4090 GPU that has 24 GB of its own memory and with 96 GB of system memory. This machine is an example and not the class server, because the class models run on Mistral's servers, somewhere else entirely.
The model card for Ministral 14B says it fits in 24 GB of GPU memory, so it fits on the example machine's GPU and not on either Chromebook. Ministral 3B, the smallest class model, fits in 8 GB of GPU memory in the form Mistral publishes, which is all the memory the larger Chromebook has. The same model card says that a more compressed version needs less memory. A very small model could therefore load on a Chromebook, and the Chromebook would produce its tokens slowly, with visible effort. Mistral Large 3 needs far more, and its model card names a machine with eight NVIDIA H200 GPUs, each with 141 GB of memory.
Summary
A computer has to load a model from storage into memory before the model can run, and the memory must be large enough to hold all of its parameters. A GPU carries out the many simple calculations of prediction at the same time, while the CPU runs the rest of the computer and can run a model only slowly. A school Chromebook has enough memory for a very small model at most, and the class models need far more. Your Chromebook is fine. It is simply not a data center, and nobody expected it to be.
Sources: CompTIA, Tech+ FC0-U71 Exam Objectives (2023); Franchitti, Introduction to Computer Science, OpenStax (2024); NVIDIA, CUDA Programming Guide (2026); Verma and Vaidya, "Mastering LLM Techniques: Inference Optimization," NVIDIA (2023); NVIDIA, GeForce RTX 4090 and H200 specification pages; Lenovo, 100e Chromebook Gen 4 PSREF (2023); Mistral AI, model cards and "Introducing Mistral 3" (2025); MuggsOfCompSci, Lab 1 records and build spec (2026).
Each table decides: could it run the model, yes or no. The table agrees on one answer and names the one part the Chromebook is shortest on.
Chromebooks stay closed while your table talks. Sections 1 and 2 of the discussion form are already done. After you talk, finish Section 3 right here under My View. Submit before the exit ticket.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
This form has three sections. Section 1: answer today's Board Question. Section 2: after the reading. Section 3 opens later in class. Today's Board Question: Could the Chromebook in front of you run the model by itself? Yes or no, and what would it need.
CR 1. Board Question: Could the Chromebook in front of you run the model by itself? Yes or no, and what would it need. Give your answer and one reason.
Write your answer in your lab notebook, labeled CR 1.
MC 1. Think about your answer to the board question. Could the Chromebook in front of you run the model by itself? Yes or no, and what would it need. Which position is closest to yours now?
- Yes. Any computer can run any program if it is given enough time.
- No. It does not have enough memory to load the model.
- Only a very small model, and only slowly.
MC 2. (Statement 1) A file saved in storage is already in memory.
- Agree
- Disagree
MC 3. (Statement 2) A GPU helps run models because it does many simple calculations at the same time.
- Agree
- Disagree
MC 4. (Statement 3) A bigger model needs more memory to run.
- Agree
- Disagree
CR 2. (A different view) Write the view at your table that was most different from yours. If everyone agreed, write the strongest argument against your table's answer.
Write your answer in your lab notebook, labeled CR 2.
CR 3. Think about the Board Question again: Could the Chromebook in front of you run the model by itself? Yes or no, and what would it need. Do you have the same opinion as when you first answered the board question? If it is the same, what from the reading or table talk reaffirmed your opinion? If it is different, what from the reading or table talk changed your opinion?
Write your answer in your lab notebook, labeled CR 3.