Before the bell
You are responsible for all of this before the end of class.
- The discussion form in the I Do deck. All three sections. If you are here, this should be done.
- Your classwork. This form on the right. Section 1 before the exit ticket pops up; Section 2 past the floor.
- No Bench Basics today — this is a short or catch-up day.
- The exit ticket. It opens 12 minutes before the bell and lives on the last slide of this deck.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
Three Models, Three Sizes
This reading lists the three Lab 1 models and the memory each one needs to load.
The Three Models
In Lab 1, pane 1 was Ministral 3B, pane 2 was Ministral 14B, and pane 3 was Mistral Large 3. Ministral 3B has 3.4 billion parameters for language, and its model card says it fits in 8 GB of GPU memory. Ministral 14B has 13.5 billion parameters for language, and its model card says it fits in 24 GB of GPU memory. Mistral Large 3 has 675 billion parameters in total. Its model card says one form of it runs on a single machine with eight NVIDIA H200 GPUs, and each H200 has 141 GB of memory. That machine therefore has 1,128 GB of GPU memory, because 8 times 141 GB is 1,128 GB.
Deciding Whether a Model Fits
A model fits on a machine when the machine's memory is at least as large as the memory the model needs. Here, the GPU memory on a model card counts as the memory the model needs. The number that decides it is the memory, not the storage, because a parameter file in storage has not been loaded yet. For example, a typical school Chromebook has 4 GB or 8 GB of memory and 32 GB or 64 GB of storage. Its GPU has no memory of its own and shares the system memory with everything else that is running, including ChromeOS and the browser. Ministral 3B, the smallest of the three models, fits in 8 GB of GPU memory. The 4 GB Chromebook has less than 8 GB, so the model does not fit. The 8 GB Chromebook has exactly 8 GB, but ChromeOS uses part of that memory, so the model does not fit there. The 64 GB of storage does not change the answer, because the computer has to load the parameter file into memory before the model can run. Mistral Large 3 does not fit on the example machine in Figure 1, because the 24 GB on its RTX 4090 is far less than the 1,128 GB in the machine the model card names.
Sources: Mistral AI, model cards for Ministral 3 3B, Ministral 3 14B and Mistral Large 3 (2025); NVIDIA, H200 and GeForce RTX 4090 specification pages; Lenovo, 100e Chromebook Gen 4 PSREF (2023); Verma and Vaidya, "Mastering LLM Techniques: Inference Optimization," NVIDIA (2023); OpenStax, Introduction to Computer Science (2024); MuggsOfCompSci, Lab 1 records (2026).
Three Models, Three Sizes, One Small Chromebook
This entry lists the three Lab 1 models and the memory each one needs to load, which is more than you might hope.
The Three Models
In Lab 1, pane 1 was Ministral 3B, pane 2 was Ministral 14B, and pane 3 was Mistral Large 3. Ministral 3B has 3.4 billion parameters for language, and its model card says it fits in 8 GB of GPU memory. Ministral 14B has 13.5 billion parameters for language, and its model card says it fits in 24 GB of GPU memory. Mistral Large 3 has 675 billion parameters in total. Its model card says one form of it runs on a single machine with eight NVIDIA H200 GPUs, and each H200 has 141 GB of memory. That machine therefore has 1,128 GB of GPU memory, because 8 times 141 GB is 1,128 GB. This is not a machine you keep under a desk.
Deciding Whether a Model Fits
A model fits on a machine when the machine's memory is at least as large as the memory the model needs. It is the same rule as fitting a couch through a door. Here, the GPU memory on a model card counts as the memory the model needs. The number that decides it is the memory, not the storage, because a parameter file in storage has not been loaded yet. For example, a typical school Chromebook has 4 GB or 8 GB of memory and 32 GB or 64 GB of storage. Its GPU has no memory of its own and shares the system memory with everything else that is running, including ChromeOS and the browser. Ministral 3B, the smallest of the three models, fits in 8 GB of GPU memory. The 4 GB Chromebook has less than 8 GB, so the model does not fit. The 8 GB Chromebook has exactly 8 GB, but ChromeOS uses part of that memory, so the model does not fit there either. The 64 GB of storage does not change the answer, because the computer has to load the parameter file into memory before the model can run. Mistral Large 3 does not fit on the example machine in Figure 1, because the 24 GB on its RTX 4090 is far less than the 1,128 GB in the machine the model card names. Some couches are simply too big.
Sources: Mistral AI, model cards for Ministral 3 3B, Ministral 3 14B and Mistral Large 3 (2025); NVIDIA, H200 and GeForce RTX 4090 specification pages; Lenovo, 100e Chromebook Gen 4 PSREF (2023); Verma and Vaidya, "Mastering LLM Techniques: Inference Optimization," NVIDIA (2023); OpenStax, Introduction to Computer Science (2024); MuggsOfCompSci, Lab 1 records (2026).
What It Takes to Run a Model
This section explains which parts of a computer a language model needs to run.
Storage and Memory
A server is a computer, and it is built from the same main parts as the computers in a classroom. Those parts include a central processing unit, a graphics processing unit, memory and storage. Storage is the part that keeps files, and it holds them even after the computer is turned off. Memory is the part of a computer that holds the programs and data being used right now, and it is much faster than storage. Memory loses what it holds when the power goes off. Before a program can run, the computer copies the program and its data from storage into memory.
For example, a language model's parameter file sits in storage while nothing is happening, and the computer has to load it into memory before the model can predict a single token, and the parameter file stays loaded in memory while the model runs. A parameter file that is only saved on a drive is not in memory yet, because nothing has copied it there. A model needs enough memory to hold all of its parameters at once.
The CPU and the GPU
The CPU, or central processing unit, is the part of a computer whose main job is to run programs. It is built to finish one series of steps quickly, and it works on only a few tens of them at the same time.
A GPU, or graphics processing unit, is a part built to carry out thousands of simple calculations at the same time. For example, a model's prediction is made of very large groups of multiplications, and many of them can be done at the same time, which is the kind of work a GPU is built for. A CPU can do the same arithmetic, but it works on far fewer calculations at once, so it produces tokens much more slowly. A GPU is not the part that runs the rest of the computer, because the operating system and the other programs run on the CPU.
Model Size and Lab 1
In Lab 1, the three panes used three models from Mistral: Ministral 3B, Ministral 14B and Mistral Large 3. Ministral 3B has about 3.4 billion parameters for language, Ministral 14B has about 13.5 billion, and Mistral Large 3 has 675 billion in total. A bigger parameter file needs more memory to load. A model predicts one token at a time, and its parameters are read again for each new token. Mistral Large 3 uses about 41 billion of its parameters for each token, which is still more than either Ministral model has in total.
The seconds and tokens per second that students recorded in Lab 1 measured the whole trip through Mistral's service. Speed through a service also depends on the machines the provider uses and how it runs the model on them. For this reason, a slower pane fits with a bigger model, but the Lab 1 numbers alone do not show that size was the only cause.
Two Machines Side by Side
Figure 1 shows the inside of a server beside two spec cards. The first card describes a typical school Chromebook with 4 GB or 8 GB of memory and 32 GB or 64 GB of storage. Its GPU is built into the processor chip and shares the system memory with everything else on the computer. The second card describes an example machine that can run a model, with an NVIDIA GeForce RTX 4090 GPU that has 24 GB of its own memory and with 96 GB of system memory. This machine is an example and not the class server, because the class models run on Mistral's servers.
The model card for Ministral 14B says it fits in 24 GB of GPU memory, so it fits on the example machine's GPU and not on either Chromebook. Ministral 3B, the smallest class model, fits in 8 GB of GPU memory in the form Mistral publishes, which is all the memory the larger Chromebook has. The same model card says that a more compressed version needs less memory. A very small model could therefore load on a Chromebook, and the Chromebook would produce its tokens slowly. Mistral Large 3 needs far more, and its model card names a machine with eight NVIDIA H200 GPUs, each with 141 GB of memory.
Summary
A computer has to load a model from storage into memory before the model can run, and the memory must be large enough to hold all of its parameters. A GPU carries out the many simple calculations of prediction at the same time, while the CPU runs the rest of the computer and can run a model only slowly. A school Chromebook has enough memory for a very small model at most, and the class models need far more.
Sources: CompTIA, Tech+ FC0-U71 Exam Objectives (2023); Franchitti, Introduction to Computer Science, OpenStax (2024); NVIDIA, CUDA Programming Guide (2026); Verma and Vaidya, "Mastering LLM Techniques: Inference Optimization," NVIDIA (2023); NVIDIA, GeForce RTX 4090 and H200 specification pages; Lenovo, 100e Chromebook Gen 4 PSREF (2023); Mistral AI, model cards and "Introducing Mistral 3" (2025); MuggsOfCompSci, Lab 1 records and build spec (2026).
What It Takes to Run a Model, or: Where the Thinking Machine Keeps Its Stuff
This entry explains which parts of a computer a language model needs before it can do anything at all.
Storage and Memory
A server is a computer. It is not a glowing oracle in a mountain. It is built from the same main parts as the computers in a classroom: a central processing unit, a graphics processing unit, memory and storage. Storage is the part that keeps files, and it holds them even after the computer is turned off, like a closet. Memory is the part of a computer that holds the programs and data being used right now, and it is much faster than storage. Memory loses what it holds when the power goes off, which it accepts with great dignity. Before a program can run, the computer copies the program and its data from storage into memory.
For example, a language model's parameter file sits in storage while nothing is happening, and the computer has to load it into memory before the model can predict a single token, and the parameter file stays loaded in memory while the model runs. A parameter file that is only saved on a drive is not in memory yet, because nothing has copied it there. Owning a cookbook is not the same as cooking. A model needs enough memory to hold all of its parameters at once.
The CPU and the GPU
The CPU, or central processing unit, is the part of a computer whose main job is to run programs. It is built to finish one series of steps quickly, and it works on only a few tens of them at the same time. Think of a very fast, very focused chef.
A GPU, or graphics processing unit, is a part built to carry out thousands of simple calculations at the same time. Think of a stadium full of people who can each do one multiplication. For example, a model's prediction is made of very large groups of multiplications, and many of them can be done at the same time, which is exactly the kind of work a GPU is built for. A CPU can do the same arithmetic, but it works on far fewer calculations at once, so it produces tokens much more slowly. A GPU is not the part that runs the rest of the computer, because the operating system and the other programs run on the CPU. The stadium does not do the dishes.
Model Size and Lab 1
In Lab 1, the three panes used three models from Mistral: Ministral 3B, Ministral 14B and Mistral Large 3. Ministral 3B has about 3.4 billion parameters for language, Ministral 14B has about 13.5 billion, and Mistral Large 3 has 675 billion in total, a number best not thought about for too long. A bigger parameter file needs more memory to load. A model predicts one token at a time, and its parameters are read again for each new token. Mistral Large 3 uses about 41 billion of its parameters for each token, which is still more than either Ministral model has in total.
The seconds and tokens per second that students recorded in Lab 1 measured the whole trip through Mistral's service. Speed through a service also depends on the machines the provider uses and how it runs the model on them, none of which anyone tells you. For this reason, a slower pane fits with a bigger model, but the Lab 1 numbers alone do not show that size was the only cause.
Two Machines Side by Side
Figure 1 shows the inside of a server beside two spec cards. The first card describes a typical school Chromebook with 4 GB or 8 GB of memory and 32 GB or 64 GB of storage. Its GPU is built into the processor chip and shares the system memory with everything else on the computer, the way a younger sibling shares a room. The second card describes an example machine that can run a model, with an NVIDIA GeForce RTX 4090 GPU that has 24 GB of its own memory and with 96 GB of system memory. This machine is an example and not the class server, because the class models run on Mistral's servers, somewhere else entirely.
The model card for Ministral 14B says it fits in 24 GB of GPU memory, so it fits on the example machine's GPU and not on either Chromebook. Ministral 3B, the smallest class model, fits in 8 GB of GPU memory in the form Mistral publishes, which is all the memory the larger Chromebook has. The same model card says that a more compressed version needs less memory. A very small model could therefore load on a Chromebook, and the Chromebook would produce its tokens slowly, with visible effort. Mistral Large 3 needs far more, and its model card names a machine with eight NVIDIA H200 GPUs, each with 141 GB of memory.
Summary
A computer has to load a model from storage into memory before the model can run, and the memory must be large enough to hold all of its parameters. A GPU carries out the many simple calculations of prediction at the same time, while the CPU runs the rest of the computer and can run a model only slowly. A school Chromebook has enough memory for a very small model at most, and the class models need far more. Your Chromebook is fine. It is simply not a data center, and nobody expected it to be.
Sources: CompTIA, Tech+ FC0-U71 Exam Objectives (2023); Franchitti, Introduction to Computer Science, OpenStax (2024); NVIDIA, CUDA Programming Guide (2026); Verma and Vaidya, "Mastering LLM Techniques: Inference Optimization," NVIDIA (2023); NVIDIA, GeForce RTX 4090 and H200 specification pages; Lenovo, 100e Chromebook Gen 4 PSREF (2023); Mistral AI, model cards and "Introducing Mistral 3" (2025); MuggsOfCompSci, Lab 1 records and build spec (2026).
You Are the AI
Think of a song you have heard a hundred times by your favorite artist. If someone muted a few words in the middle of a verse, you would fill them in without thinking. You might not even notice they were gone. That is the small gap from Big picture 1.
Now take a new release by the same artist. You have never heard it, but you know how they write. You know the words they like, the way they rhyme, and their flow, which is the rhythm of the lines. Mute a few words, and you still do pretty well.
Next, try a song you have never heard by a different artist in the same genre. You do worse, but you still beat someone who has never listened to that genre, because you know how songs like this usually go.
Now go far outside your comfort zone, to a style of music you have never heard. You do not know the missing words. But here is the rule: you have to keep singing, and you have to sing with confidence. No stopping, no mumbling, and no admitting you are lost.
Here is the reveal. You are the AI. A language model picks each next word by probability, choosing a word that is likely to come next. It does that on the familiar song and on the strange one. On the strange song, many words are about equally likely, so the pick is closer to a guess. The voice never changes, though. Rung 4 comes out sounding just as sure as rung 1.
That is why confidence is not a sign of accuracy. An answer can sound certain and still be a guess. Andrej Karpathy's ISBN example from Big picture 1 is rung 4 in action. The model has learned that a number goes after "ISBN:", so it puts in digits that look reasonable. That number almost certainly does not exist. People call this a hallucination.
The song picture stretches in one place. The model does not store songs as recordings and play them back. It stores patterns in its parameters, and it builds an answer one word at a time from the prompt you type.
Answer the questions in Section 3 of your classwork.
Sources: Karpathy, Intro to Large Language Models (2023).
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
MC 1. (Pairing 1) The machine is a typical school Chromebook, with 4 GB or 8 GB of memory and 32 GB or 64 GB of storage. The model is Ministral 14B. Does the model fit in this machine's memory?
- Fits
- Does not fit
MC 2. (Pairing 2) The machine is the example machine in Figure 1, with an NVIDIA GeForce RTX 4090 GPU that has 24 GB of its own memory and 96 GB of system memory. The model is Ministral 3B. Does the model fit in this machine's GPU memory?
- Fits
- Does not fit
MC 3. (Pairing 3) The machine has eight NVIDIA H200 GPUs, each with 141 GB of memory. The model is Mistral Large 3, with 675 billion parameters. Does the model fit in this machine's GPU memory?
- Fits
- Does not fit
MC 4. (Pairing 4) The machine is the example machine in Figure 1, with 24 GB of GPU memory. The model is Mistral Large 3. Does the model fit in this machine's GPU memory?
- Fits
- Does not fit
MC 5. (Rule check) A model fits on a machine when the machine's GPU memory is at least as large as the memory the model needs. Is this the rule Reading 2 states?
- True of the rule
- Not the rule
CR 1. (Pairings 4 and 5) Give the two numbers that decide MC 4 and the two numbers that decide MC 5. Write each memory figure in GB.
Write your answer in your lab notebook, labeled CR 1.
The big picture so far (newest on top)
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
CR 2. Look back at the seconds and tokens per second you recorded for each pane in Lab 1. Which pane was slowest in your records? Explain it with one fact from today's reading, and name one other thing that can change speed through Mistral's service.
Write your answer in your lab notebook, labeled CR 2.
CR 3. In the extra reading, an AI answer sounds just as sure on rung 4 as on rung 1. If an answer sounds confident, what could you do to find out which rung it came from? Name one step and explain why it would work.
Write your answer in your lab notebook, labeled CR 3.
CR 4. When you paste an AI answer into an assignment without reading it, who did the singing? Explain your answer with one idea from the extra reading.
Write your answer in your lab notebook, labeled CR 4.
No bench today
There is no bench on this day. Finish your classwork and move to the exit ticket.
No bench today — the bench pop-up is off for this day.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
Sign out of Google when you finish on a shared computer, so the next student can sign in as themselves. Google then shows a "Redirect notice" page. Click the muggsofcompsci.net/cs1 link on it to come back.
This opens during your class period.