CS I · Tue Dec 1 · 17.2 · Do Now

Reporting a Problem

Terminology and concepts to look out for: issue · bug · steps to reproduce
Relevance to me: Every tool has problems, and the people who use it are usually the first to find them. Today you learn to write a problem down so that the builder can find it and fix it.
Board Question: A classmate tells you, "Code Coach is wrong." Is that enough for the builder to fix it? Yes or no, and name one more thing the builder would need to know.

Open the discussion form on the right and answer Section 1 from your first reading.

Announcements

Bugs, Issues and Steps to Reproduce

This section explains how to write down a problem in a tool so that the builder can make it happen again, find it and fix it.

Problems a Builder Missed

Code Coach has five written requirements, R1 to R5, and a test checks a tool against them. A test covers only the inputs that someone thought to try. People who use a tool often find problems that its builder missed.

A bug is a case where a program does something different from what its requirements say it should do. For example, R1 says, "Code Coach explains only the Java the student asked about." If a student asks about one line and Code Coach explains a different line, that is a bug, because the program broke a written requirement.

Not every problem a user reports is a bug, and a student might wish, for example, that Code Coach would give the answer to a bench job. The description of Code Coach says that it does not give the answer to a bench job, and R3 says that it does not write new code for the student. Wishing Code Coach would give the bench answer is a request, not a bug, because its requirements never promised that, and the builder decides whether to grant a request.

Writing a Problem Down as an Issue

An issue is a written report of a problem or a request, kept where the builders track their work, such as a repository. GitHub's documentation says that issues can track "ideas, feedback, tasks, or bugs." An issue stays open until someone closes it, for example when the bug is fixed or when the builders decide the work is not planned. Telling the builder about a problem in the hallway leaves no record, while an issue is written down and stays open until it is closed.

Steps to reproduce are the numbered actions that make the problem happen again. Mozilla's guide states, "Steps to reproduce are the most important part of any bug report." The same guide says that a bug a developer can reproduce is very likely to be fixed. A report that says "It broke when I used it" gives no steps, so the builder cannot make the problem happen again. After the steps, a report describes the expected result, which comes from the requirements, and the actual result, which is what happened instead. The report also says how often the problem happens, because a model's answer can change from try to try. Finally, a report needs a title that names the problem exactly. Mozilla gives "Software crashes" as an example of a poor summary, and a title such as "It's broken" has the same weakness, because neither one says what went wrong.

A Real Code Coach Bug

In October, the builder tested Code Coach with an 11-line segment of Java and asked what line 3 does. The segment and its real output are below, and line 3 is the if condition.

1int score = 17;
2int bonus = score % 5;
3if (score + bonus > 20)
4{
5    System.out.println("high");
6}
7else
8{
9    System.out.println("low");
10}
11System.out.println(score + bonus);

low
19

This request was built by a capture script, a small program that sends test requests to the class endpoint, and not by the bench. The script put a label line, "Java:", above the code, as the request's first four lines show.

Java:
int score = 17;
int bonus = score % 5;
if (score + bonus > 20)

Code Coach answered the question five times, and all five answers explained the line that stores bonus instead of line 3. The label line counted as line 1, so the third line of the request was the bonus line, and every line number was off by one. The builder concluded that the bug was in how the request was built, not in the model. Written as a report, the bug has a title and three parts. Figure 1 lays out the same report as a form.

Title: Asked about line 3, Code Coach explains line 2

Steps to reproduce
1. Use Code Coach's system prompt, version 2.
2. Put the label line "Java:" above the 11 lines of code.
3. Below the code, add the line "Question: What does line 3 do?"
4. Send the request to the class endpoint with the capture script.

Expected (from the requirements)
R1. Code Coach explains only the Java the student asked about.
Line 3 is the if condition, so the answer explains the if condition.

What happened
"Line 3 calculates the remainder when `score` (17) is divided by 5. It stores the result (2) in the variable `bonus`."
How often: 5 of 5 tries

Because the steps let the builder send the same request and get the same wrong line, the builder could test a change that sends the code as numbered lines, the way the bench shows them. With the same question, Code Coach explained line 3 in 5 of 5 tries. One answer began, "Line 3 checks if the sum of `score` and `bonus` is greater than 20." The bench now always sends the code to Code Coach as numbered lines, so a student cannot make this bug happen on today's bench.

Students who find a problem on a bench can report it in the bug report section of the Bench Basics credit code form. A report written there helps the builder most when it has the same parts as the Code Coach report.

Summary

A bug is a difference between what a program does and what its requirements say. An issue is a written report of a bug or a request, and it stays open until someone closes it. Steps to reproduce, what was expected and what happened give a builder what is needed to make a problem happen again and fix it.

Sources: Mozilla, Bug Writing Guidelines (n.d.); GitHub Docs, About issues (n.d.); GitHub Docs, Closing an issue (n.d.); Code Coach README and test captures (2026); course JDK runs (2026); Build spec Oct 26–Dec 18, §4.1 and §5 (2026).

I Do questions

notebook cue: issue
Why does a builder want a problem written down as an issue instead of hearing about it out loud?

Check
A problem told in the hallway leaves no record and can be forgotten. An issue is written down where the builders track their work, and it stays open until someone closes it, for example when the bug is fixed. See Writing a Problem Down as an Issue, paragraph 1.
notebook cue: bug
Asked what line 3 does, Code Coach explained line 2 instead. Is that a bug or a request? Name the requirement that decides it.

Check
A bug. Code Coach did something different from what its requirement R1 says: explain only the Java the student asked about. It explained line 2 when the question was about line 3, in 5 of 5 saved tries, until the request sent numbered lines. See the passage and Figure 1.
notebook cue: steps to reproduce
The example report in Figure 1 has a title and then three parts. Name the three parts.

Check
Steps to reproduce, expected (from the requirements) and what happened. The what-happened part quotes Code Coach's saved answer and also says how often it happened, 5 of 5 tries. See A Real Code Coach Bug and Figure 1.

Figure 1. The Code Coach report from the passage, laid out as a bug report form, with a title and three parts, and the last part saying how often the bug happened.

I Do questions

notebook cue: issue
Why does a builder want a problem written down as an issue instead of hearing about it out loud?

Check
A problem told in the hallway leaves no record and can be forgotten. An issue is written down where the builders track their work, and it stays open until someone closes it, for example when the bug is fixed. See Writing a Problem Down as an Issue, paragraph 1.
notebook cue: bug
Asked what line 3 does, Code Coach explained line 2 instead. Is that a bug or a request? Name the requirement that decides it.

Check
A bug. Code Coach did something different from what its requirement R1 says: explain only the Java the student asked about. It explained line 2 when the question was about line 3, in 5 of 5 saved tries, until the request sent numbered lines. See the passage and Figure 1.
notebook cue: steps to reproduce
The example report in Figure 1 has a title and then three parts. Name the three parts.

Check
Steps to reproduce, expected (from the requirements) and what happened. The what-happened part quotes Code Coach's saved answer and also says how often it happened, 5 of 5 tries. See A Real Code Coach Bug and Figure 1.

Bugs, Issues and Steps to Reproduce

This section explains how to write down a problem in a tool so that the builder can make it happen again, find it and fix it.

Problems a Builder Missed

Code Coach has five written requirements, R1 to R5, and a test checks a tool against them. A test covers only the inputs that someone thought to try. People who use a tool often find problems that its builder missed.

A bug is a case where a program does something different from what its requirements say it should do. For example, R1 says, "Code Coach explains only the Java the student asked about." If a student asks about one line and Code Coach explains a different line, that is a bug, because the program broke a written requirement.

Not every problem a user reports is a bug, and a student might wish, for example, that Code Coach would give the answer to a bench job. The description of Code Coach says that it does not give the answer to a bench job, and R3 says that it does not write new code for the student. Wishing Code Coach would give the bench answer is a request, not a bug, because its requirements never promised that, and the builder decides whether to grant a request.

Writing a Problem Down as an Issue

An issue is a written report of a problem or a request, kept where the builders track their work, such as a repository. GitHub's documentation says that issues can track "ideas, feedback, tasks, or bugs." An issue stays open until someone closes it, for example when the bug is fixed or when the builders decide the work is not planned. Telling the builder about a problem in the hallway leaves no record, while an issue is written down and stays open until it is closed.

Steps to reproduce are the numbered actions that make the problem happen again. Mozilla's guide states, "Steps to reproduce are the most important part of any bug report." The same guide says that a bug a developer can reproduce is very likely to be fixed. A report that says "It broke when I used it" gives no steps, so the builder cannot make the problem happen again. After the steps, a report describes the expected result, which comes from the requirements, and the actual result, which is what happened instead. The report also says how often the problem happens, because a model's answer can change from try to try. Finally, a report needs a title that names the problem exactly. Mozilla gives "Software crashes" as an example of a poor summary, and a title such as "It's broken" has the same weakness, because neither one says what went wrong.

A Real Code Coach Bug

In October, the builder tested Code Coach with an 11-line segment of Java and asked what line 3 does. The segment and its real output are below, and line 3 is the if condition.

1int score = 17;
2int bonus = score % 5;
3if (score + bonus > 20)
4{
5    System.out.println("high");
6}
7else
8{
9    System.out.println("low");
10}
11System.out.println(score + bonus);

low
19

This request was built by a capture script, a small program that sends test requests to the class endpoint, and not by the bench. The script put a label line, "Java:", above the code, as the request's first four lines show.

Java:
int score = 17;
int bonus = score % 5;
if (score + bonus > 20)

Code Coach answered the question five times, and all five answers explained the line that stores bonus instead of line 3. The label line counted as line 1, so the third line of the request was the bonus line, and every line number was off by one. The builder concluded that the bug was in how the request was built, not in the model. Written as a report, the bug has a title and three parts. Figure 1 lays out the same report as a form.

Title: Asked about line 3, Code Coach explains line 2

Steps to reproduce
1. Use Code Coach's system prompt, version 2.
2. Put the label line "Java:" above the 11 lines of code.
3. Below the code, add the line "Question: What does line 3 do?"
4. Send the request to the class endpoint with the capture script.

Expected (from the requirements)
R1. Code Coach explains only the Java the student asked about.
Line 3 is the if condition, so the answer explains the if condition.

What happened
"Line 3 calculates the remainder when `score` (17) is divided by 5. It stores the result (2) in the variable `bonus`."
How often: 5 of 5 tries

Because the steps let the builder send the same request and get the same wrong line, the builder could test a change that sends the code as numbered lines, the way the bench shows them. With the same question, Code Coach explained line 3 in 5 of 5 tries. One answer began, "Line 3 checks if the sum of `score` and `bonus` is greater than 20." The bench now always sends the code to Code Coach as numbered lines, so a student cannot make this bug happen on today's bench.

Students who find a problem on a bench can report it in the bug report section of the Bench Basics credit code form. A report written there helps the builder most when it has the same parts as the Code Coach report.

Summary

A bug is a difference between what a program does and what its requirements say. An issue is a written report of a bug or a request, and it stays open until someone closes it. Steps to reproduce, what was expected and what happened give a builder what is needed to make a problem happen again and fix it.

Sources: Mozilla, Bug Writing Guidelines (n.d.); GitHub Docs, About issues (n.d.); GitHub Docs, Closing an issue (n.d.); Code Coach README and test captures (2026); course JDK runs (2026); Build spec Oct 26–Dec 18, §4.1 and §5 (2026).

Table Talk

A classmate tells you, "Code Coach is wrong." Is that enough for the builder to fix it? Yes or no, and name one more thing the builder would need to know.

Each table decides yes or no, and lists what a builder needs to know before starting a fix.

Chromebooks stay closed while your table talks. Sections 1 and 2 of the discussion form are already done. After you talk, finish Section 3 right here under My View. Submit before the exit ticket.

Open the discussion form

This form has three sections. Section 1: answer today's Board Question. Section 2: after the reading. Section 3 opens later in class. Today's Board Question: A classmate tells you, "Code Coach is wrong." Is that enough for the builder to fix it? Yes or no, and name one more thing the builder would need to know.

CR 1. Board Question: A classmate tells you, "Code Coach is wrong." Is that enough for the builder to fix it? Yes or no, and name one more thing the builder would need to know. Give your answer and one reason.

Write your answer in your lab notebook, labeled CR 1.

MC 1. Think about your answer to the board question. A classmate tells you, "Code Coach is wrong." Is that enough for the builder to fix it? Yes or no, and name one more thing the builder would need to know. Which position is closest to yours now?

Yes. The builder can open Code Coach and look for the problem.
No. The builder needs the steps, what was expected and what happened.
Yes, if the classmate shows the builder in person.

MC 2. (Reading claim 1) A bug is a difference between what a program does and what its requirements say.

Agree
Disagree

MC 3. (Reading claim 2) A report should say what the user expected to happen.

Agree
Disagree

MC 4. (Reading claim 3) "It's broken" is a good title for a report.

Agree
Disagree

MC 5. (Reading claim 4) When the request had a label line above the code, Code Coach explained line 3 correctly in most of its tries.

Agree
Disagree

MC 6. (Reading claim 5) The Code Coach bug came from the way the request was built, not from the model.

Agree
Disagree

MC 7. (Reading claim 6) Mozilla's guide calls steps to reproduce the most important part of a bug report.

Agree
Disagree

MC 8. (Reading claim 7) A student who wants Code Coach to give the answer to a bench job is reporting a bug.

Agree
Disagree

MC 9. (Reading claim 8) In Figure 1, the what-happened part quotes the answer Code Coach actually gave.

Agree
Disagree

CR 2. (A different view) Write the view at your table that was most different from yours. If everyone agreed, write the strongest argument against your table's answer.

Write your answer in your lab notebook, labeled CR 2.

CR 3. Think about the Board Question again: A classmate tells you, "Code Coach is wrong." Is that enough for the builder to fix it? Yes or no, and name one more thing the builder would need to know. Do you have the same opinion as when you first answered the board question? If it is the same, what from the reading or table talk reaffirmed your opinion? If it is different, what from the reading or table talk changed your opinion?

Write your answer in your lab notebook, labeled CR 3.