Reliability
- Terminology and concepts to look out for: reliability · edge case · user test
- Relevance to me: Welcome back. Before the break we met Code Coach and checked what it says. This week is about using computers responsibly, and the first question is whether a program can be trusted to do its job.
- Board Question: Code Coach was built to explain short pieces of Java. Name two things a student could type into it that it was not built for.
Open the discussion form on the right and answer Section 1 from your first reading.
Reliability, Edge Cases and User Tests
This section reviews the class's earlier work and explains how a builder finds out whether a program can be trusted.
What the Class Has Learned So Far
In Week 13, the class learned that AI reaches people inside programs that people write, and Java is one of the languages they write them in. Java is compiled to bytecode, which runs on the Java Virtual Machine. In Week 14, the class learned that a specification lists requirements a person can test, and a test compares the output with the expected output. In Week 15, the class learned that files live at paths, a commit saves a version, and an error message is often where debugging starts. In Week 16, the class learned that a page is deployed to a host and reached by its address, and that Code Coach sends a request with its system prompt to a model on a server. The rest of this week asks how to use computers responsibly, starting with whether a program can be trusted to work.
What Reliability Means
Reliability means a program does what it is supposed to do, under the conditions it was built for, without failing. The College Board's AP Computer Science A course description uses the phrase "as expected under stated conditions without failure." Code Coach's conditions are written in its purpose: a student, a short piece of Java and a question about it.
A program that worked once, on the builder's own input, has not been shown to have reliability, because reliability means it works every time under the stated conditions. Code Coach has three test inputs, chosen before any testing: two lines that print text, `int x = 7 / 2;` followed by a line that prints `x`, and a sentence of plain English. Under version 2 of the system prompt, each ran twice, and all six runs met the requirements. Those results show that Code Coach does its job on the inputs its builder expected. They do not show what it does with anything else.
Code Coach's Edge Cases
An edge case is an input at the edge of what a program was built for, or just past it. For example, `int x = 7 / 2;` is one of Code Coach's planned test inputs, so it is a normal case. A piece of JavaScript pasted in by mistake is an edge case. Each answer below is quoted from runs saved on October 19, 2026. The first was JavaScript that looks like Java.
let x = 5; console.log(x);
Both runs on these lines gave the same answer.
This is not Java. Please paste a short piece of Java.
That answer meets R4, which says that when the input is not Java, Code Coach says so and asks for a Java snippet. The second edge case was Java that compiles but stops with an error when it runs.
1public class Main { 2 public static void main(String[] args) { 3 int x = 7 / 0; 4 System.out.println(x); 5 } 6}
Exception in thread "main" java.lang.ArithmeticException: / by zero at Main.main(Main.java:3)
Code Coach was asked about lines 3 and 4 in 5 runs, and in 4 of them it said the code was not Java or not valid Java. One run said, in full:
This is not Java. Please paste a short piece of Java.
That answer breaks R4, because R4 is for input that is not Java, and this input is Java. It also explains nothing, so it never says, as R2 asks, that the program stops on line 3 before storing or printing anything. Only one run explained the crash. The third edge case was Java with a missing semicolon.
1// This does not compile. 2public class Main { 3 public static void main(String[] args) { 4 int y = 4 5 System.out.println(y); 6 } 7}
Main.java:4: error: ';' expected
int y = 4
^
1 error
Code Coach was given lines 4 and 5 twice, and the second run named the problem.
This is not complete Java code because the first line is missing a semicolon. Please paste a short piece of Java so I can help you understand it.
Both runs meet R4, because code that the compiler rejects is not complete Java, but only the second run names the semicolon. Figure 1 shows all four inputs as one test table.
Why a User Test Finds More
A user test is a session in which a person who did not build the tool uses it for a real task while the builder watches and writes down what happens, without helping. The Nielsen Norman Group says the person running such a session watches the user "without accidentally influencing" what the user does. Every input so far came from the builder, who knew what Code Coach expected. Users type things builders never thought of, which is why a user test finds edge cases the builder's own tests miss.
The builder running the three test inputs again is a test. It is not a user test, because the builder already knows what the tool expects. Programmers make a program more reliable by testing it under many different conditions. A model's answer can also change from try to try, as the five runs on `7 / 0` show, so one good answer is not proof.
Summary
Reliability means a program does its job every time under the conditions it was built for. Code Coach met its requirements on its three planned test inputs, but it called real Java "not Java" or "not valid Java" in 4 of 5 runs on one edge case. A user test finds edge cases that the builder's own tests miss.
Sources: College Board, AP Computer Science A Course and Exam Description (2025); Moran, Usability (User) Testing 101, Nielsen Norman Group (2019); Code Coach README and captures, mistral-small-2603 (2026); course JDK 21 runs (2026); CS I passages, Oct 26 to Nov 20 (2026).
-
notebook cue: reliability
Code Coach met its requirements on all three of its test inputs. Does that show it is reliable? Say what else you would need to know.
Check
No. The three test inputs are the ones the builder expected, so they show only that Code Coach works on those. Reliability means it works every time under the stated conditions. You would also need to know what it does with edge cases, such as Java that crashes. You would also need to run it more than once, because a model's answer can change from try to try. See What Reliability Means, paragraph 2, and Code Coach's Edge Cases, where `7 / 0` drew "not Java" or "not valid Java" in 4 of 5 runs.
-
notebook cue: edge case
Which is an edge case for Code Coach: a two-line `println` snippet, or a line of Java with its semicolon missing? Explain your choice.
Check
The line with its semicolon missing is the edge case. A two-line `println` snippet is one of Code Coach's planned test inputs, so it is a normal case. Java that the compiler rejects sits just past the edge of what Code Coach was built for. See Code Coach's Edge Cases, the third edge case, and What Reliability Means, paragraph 2.
-
notebook cue: user test
During a user test, a classmate pastes a paragraph from a history reading into Code Coach. Why is that more useful to the builder than another test the builder planned?
Check
The builder already knows what Code Coach expects, so the planned tests come from the builder's own ideas. A classmate who did not build the tool types something the builder may never have tried. Watching what happens shows whether Code Coach still says the input is not Java and asks for Java, as R4 requires. See Why a User Test Finds More, paragraphs 1 and 2.
-
notebook cue: reliability
Code Coach met its requirements on all three of its test inputs. Does that show it is reliable? Say what else you would need to know.
Check
No. The three test inputs are the ones the builder expected, so they show only that Code Coach works on those. Reliability means it works every time under the stated conditions. You would also need to know what it does with edge cases, such as Java that crashes. You would also need to run it more than once, because a model's answer can change from try to try. See What Reliability Means, paragraph 2, and Code Coach's Edge Cases, where `7 / 0` drew "not Java" or "not valid Java" in 4 of 5 runs.
-
notebook cue: edge case
Which is an edge case for Code Coach: a two-line `println` snippet, or a line of Java with its semicolon missing? Explain your choice.
Check
The line with its semicolon missing is the edge case. A two-line `println` snippet is one of Code Coach's planned test inputs, so it is a normal case. Java that the compiler rejects sits just past the edge of what Code Coach was built for. See Code Coach's Edge Cases, the third edge case, and What Reliability Means, paragraph 2.
-
notebook cue: user test
During a user test, a classmate pastes a paragraph from a history reading into Code Coach. Why is that more useful to the builder than another test the builder planned?
Check
The builder already knows what Code Coach expects, so the planned tests come from the builder's own ideas. A classmate who did not build the tool types something the builder may never have tried. Watching what happens shows whether Code Coach still says the input is not Java and asks for Java, as R4 requires. See Why a User Test Finds More, paragraphs 1 and 2.
Reliability, Edge Cases and User Tests
This section reviews the class's earlier work and explains how a builder finds out whether a program can be trusted.
What the Class Has Learned So Far
In Week 13, the class learned that AI reaches people inside programs that people write, and Java is one of the languages they write them in. Java is compiled to bytecode, which runs on the Java Virtual Machine. In Week 14, the class learned that a specification lists requirements a person can test, and a test compares the output with the expected output. In Week 15, the class learned that files live at paths, a commit saves a version, and an error message is often where debugging starts. In Week 16, the class learned that a page is deployed to a host and reached by its address, and that Code Coach sends a request with its system prompt to a model on a server. The rest of this week asks how to use computers responsibly, starting with whether a program can be trusted to work.
What Reliability Means
Reliability means a program does what it is supposed to do, under the conditions it was built for, without failing. The College Board's AP Computer Science A course description uses the phrase "as expected under stated conditions without failure." Code Coach's conditions are written in its purpose: a student, a short piece of Java and a question about it.
A program that worked once, on the builder's own input, has not been shown to have reliability, because reliability means it works every time under the stated conditions. Code Coach has three test inputs, chosen before any testing: two lines that print text, `int x = 7 / 2;` followed by a line that prints `x`, and a sentence of plain English. Under version 2 of the system prompt, each ran twice, and all six runs met the requirements. Those results show that Code Coach does its job on the inputs its builder expected. They do not show what it does with anything else.
Code Coach's Edge Cases
An edge case is an input at the edge of what a program was built for, or just past it. For example, `int x = 7 / 2;` is one of Code Coach's planned test inputs, so it is a normal case. A piece of JavaScript pasted in by mistake is an edge case. Each answer below is quoted from runs saved on October 19, 2026. The first was JavaScript that looks like Java.
let x = 5; console.log(x);
Both runs on these lines gave the same answer.
This is not Java. Please paste a short piece of Java.
That answer meets R4, which says that when the input is not Java, Code Coach says so and asks for a Java snippet. The second edge case was Java that compiles but stops with an error when it runs.
1public class Main { 2 public static void main(String[] args) { 3 int x = 7 / 0; 4 System.out.println(x); 5 } 6}
Exception in thread "main" java.lang.ArithmeticException: / by zero at Main.main(Main.java:3)
Code Coach was asked about lines 3 and 4 in 5 runs, and in 4 of them it said the code was not Java or not valid Java. One run said, in full:
This is not Java. Please paste a short piece of Java.
That answer breaks R4, because R4 is for input that is not Java, and this input is Java. It also explains nothing, so it never says, as R2 asks, that the program stops on line 3 before storing or printing anything. Only one run explained the crash. The third edge case was Java with a missing semicolon.
1// This does not compile. 2public class Main { 3 public static void main(String[] args) { 4 int y = 4 5 System.out.println(y); 6 } 7}
Main.java:4: error: ';' expected
int y = 4
^
1 error
Code Coach was given lines 4 and 5 twice, and the second run named the problem.
This is not complete Java code because the first line is missing a semicolon. Please paste a short piece of Java so I can help you understand it.
Both runs meet R4, because code that the compiler rejects is not complete Java, but only the second run names the semicolon. Figure 1 shows all four inputs as one test table.
Why a User Test Finds More
A user test is a session in which a person who did not build the tool uses it for a real task while the builder watches and writes down what happens, without helping. The Nielsen Norman Group says the person running such a session watches the user "without accidentally influencing" what the user does. Every input so far came from the builder, who knew what Code Coach expected. Users type things builders never thought of, which is why a user test finds edge cases the builder's own tests miss.
The builder running the three test inputs again is a test. It is not a user test, because the builder already knows what the tool expects. Programmers make a program more reliable by testing it under many different conditions. A model's answer can also change from try to try, as the five runs on `7 / 0` show, so one good answer is not proof.
Summary
Reliability means a program does its job every time under the conditions it was built for. Code Coach met its requirements on its three planned test inputs, but it called real Java "not Java" or "not valid Java" in 4 of 5 runs on one edge case. A user test finds edge cases that the builder's own tests miss.
Sources: College Board, AP Computer Science A Course and Exam Description (2025); Moran, Usability (User) Testing 101, Nielsen Norman Group (2019); Code Coach README and captures, mistral-small-2603 (2026); course JDK 21 runs (2026); CS I passages, Oct 26 to Nov 20 (2026).
Each table lists its inputs and agrees on the one most likely to make Code Coach give a wrong or strange answer.
Chromebooks stay closed while your table talks. Sections 1 and 2 of the discussion form are already done. After you talk, finish Section 3 right here under My View. Submit before the exit ticket.
This form has three sections. Section 1: answer today's Board Question. Section 2: after the reading. Section 3 opens later in class. Today's Board Question: Code Coach was built to explain short pieces of Java. Name two things a student could type into it that it was not built for.
CR 1. Board Question: Code Coach was built to explain short pieces of Java. Name two things a student could type into it that it was not built for. Give your answer and one reason.
Write your answer in your lab notebook, labeled CR 1.
MC 1. Think about your answer to the board question. Code Coach was built to explain short pieces of Java. Name two things a student could type into it that it was not built for. Which position is closest to yours now?
- Nothing. If it explains Java correctly, it is ready for anyone.
- Inputs that are not Java, or Java that is broken, because students will type those too.
- Only long programs. Short inputs are all the same.
MC 2. (Reading claim 1) A program that passed every test its builder planned is reliable.
- Agree
- Disagree
MC 3. (Reading claim 2) An edge case is an input at the edge of what a program was built for, or just past it.
- Agree
- Disagree
MC 4. (Reading claim 3) A user test can find problems that the builder's own tests miss.
- Agree
- Disagree
MC 5. (Reading claim 4) In Figure 1, Code Coach said that `int x = 7 / 0;` is not Java, but the code is Java.
- Agree
- Disagree
MC 6. (Reading claim 5) The line `int x = 7 / 0;` does not compile.
- Agree
- Disagree
MC 7. (Reading claim 6) When JavaScript was pasted in, Code Coach met R4 in both runs.
- Agree
- Disagree
MC 8. (Reading claim 7) Code Coach gave the same answer every time it was asked about `int x = 7 / 0;`.
- Agree
- Disagree
MC 9. (Reading claim 8) When the builder runs the three test inputs again, that counts as a user test.
- Agree
- Disagree
CR 2. (A different view) Write the view at your table that was most different from yours. If everyone agreed, write the strongest argument against your table's answer.
Write your answer in your lab notebook, labeled CR 2.
CR 3. Think about the Board Question again: Code Coach was built to explain short pieces of Java. Name two things a student could type into it that it was not built for. Do you have the same opinion as when you first answered the board question? If it is the same, what from the reading or table talk reaffirmed your opinion? If it is different, what from the reading or table talk changed your opinion?
Write your answer in your lab notebook, labeled CR 3.