Statistical Tests of Data: The t Test by Pogil

Learning Objectives

Students should be able to:

Content

• Explain how the t value changes due to changes in the means, the standard deviations, and the number of data.
• Explain how the t test is used to make decisions regarding significance, and be able to use Excel's t test output to make such decisions.
• Develop an understanding of the t value's relationship to the probability of a significant difference.

Process

• Interpret tables of data. (Information processing)
• Draw conclusions about data sets. (Critical thinking)
• Express concepts in grammatically correct sentences. (Communication)

Prior knowledge

• Familiarity with common statistics terms such as mean, range, and standard deviation.
• Familiarity with simple cases of error propagation, especially sums and differences.

Further Reading

• Harris, D.C., Quantitative Chemical Analysis, 7th Edition, 2007 WH Freeman: USA, Harris, D.C. 2007. Quantitative Chemical Analysis, 7th Edition, pp. 59-65. New York: W.H. Freeman.
• Hecht, H.G. 1990. Mathematics in Chemistry: An Introduction to Modern Methods, pp. 210-255. Englewood Cliffs, NJ: Prentice Hall.

Authors

Carl Salter Statistical Tests of Data: The t Test POGIL WWW.POGIL.ORGScientists often compare sets of similar measurements. For example, an analytical chemist might compare two different methods for measuring the amount of carbon dioxide in air. Or a food chemist might measure dissolved carbon dioxide in two different brands of soda. Even when the comparison isn't explicit, analysts are often implicitly comparing their measurements against reference samples or accepted standards. Whatever the case, experimental scientists often must decide whether an observed difference is important and worthy of communication. Usually they will focus on the two most important statistics about a data set: its average X (or mean) and its standard deviation s. Together these are called the summary statistics of the set. Here are their formulas.

x =- 2×, i=1

S = ( i=1 E(X- x)2 n-1 1/2

Two sets of data can differ in their average values, or they can differ in their standard deviations. The scientists might judge either of these differences to be noteworthy.

Statisticians call this significance testing, and they have developed tests to judge the significance of a difference between summary statistics. What do they mean by a "significant difference"? A difference that is not the result of random error. Unfortunately it is not possible to determine with certainty whether a particular difference is caused by random error; however, statisticians have been able to develop a procedure that can compute this probability (based on certain assumptions). If we know the probability that the difference could be caused by random error, then we also know the probability that it is not caused by random error, so we will know the probability of a "significance difference." That is the best that the statistical test can do.

At this point, experimental scientists have to make a decision: how much leftover "random" probability are they willing to live with? With a brand-new result, it is unlikely that their initial measurements are decisive; the scientists will be betting that their limited set of data has really uncovered something "significant." Their professional reputations-perhaps even their jobs-could be at stake, so they will want to know the probability associated with their bet! In chemistry and physics, an experimental difference is usually labeled "significant" when the probability that it is not due to random error is greater than 95%. In other fields the threshold probability may be higher or lower than 95%. The threshold is never determined by statisticians; it is determined collectively by the experimentalists who work in a particular field.

There are two tests designed to test differences among summary statistics: the t test tests the difference of two averages, and the Ftest tests the difference of two standard deviations. This activity will examine the t test. Because there are three different variations of the t test, this activity is rather long, and so it has been divided into two parts. Part 1 develops an understanding of how the t test does its job, and Part 2 examines the three different types of t tests and how they can be performed in Excel. Another activity covers the Ftest.

Part 1: Fundamental Concepts

Probability

Consider this ... You put 19 white balls and 1 red ball, all the same size, into a large, black bag. You can't see the balls in the bag, but you can reach into the bag and hold them.

Key Questions

1. You reach into the bag, grab one ball, and remove it. What is the probability that you will withdraw a white ball? 19 out of 20, or 95%
2. If you could find someone who would take the bet, would you bet $50 that you could withdraw a white ball from the bag? (You can assume that you have $50.) Compare your answer with those of your group. Very low risk. It's an excellent bet!
3. For group discussion: Do you think that 95% probability is a reasonable threshold for accepting a new claim in chemistry or physics? The point here is to make students realize that results are not certain, and they have to accept some level of risk that the claim is false.
4. What areas of experimental science or engineering should have a threshold probability much higher than 95%? Record your group's best answer here. Medicine and communication come to mind. Ask the students if they would like their cell phone or television to work 19 out of 20 times!

Data Analysis

Consider this ... April and Betty each measure the absorbance of the same colored solution three times; each student has her own absorption spectrometer. (Absorbance is unitless.)

Set 1

April Betty 0.121 0.123 0.122 0.124 0.123 0.125 x 0.122 0.124 s 0.001 0.001

Key Questions

1. Compute the summary statistics for April and Betty's data and enter them in the table. The symbol Psionit represents the probability that the difference between April's data and Betty's data is significant, that is, that the difference is the result of some real difference in the sample or the measuring routine. P. insignif will represent the probability that the difference is the result of random error. (Note: Statisticians often use the symbol a for P. insignifi a=0.05 means P signif ::= 0.95)
2. What is the sum of P and P. ? Why? signif insignif Since there are only two alternatives, significant or insignificant, and they are mutually exclusive, the sum of their probabilities must be one.

Consider possible claims regarding April and Betty's data:

a. Betty's absorption data are higher than April's; therefore Betty's spectrometer must somehow be different from April's, making it give higher measurements than April's.

b. April's absorption data are lower than Betty's; therefore April must have somehow diluted her sample slightly before she made her measurements.

c. Betty and April's absorption data are a little different, but not by much, so maybe the difference is entirely random.

d. There's no real difference between April's and Betty's data; the difference is within the limit of performance of the two spectrometers.

1. Which claim(s) say that the difference in April and Betty's data is insignificant? c and d. Which claim(s) says that the difference in their data is significant and "real?" a and b.
2. If two people claim that a difference is real, do they have to agree on the cause of the difference? Explain. No; the reasons given in a and b are different. A real difference may have many possible causes.
3. Assume that P = 0.88 for Apsildataand/ouldBateyjudge the difference to be significance or signif insignificant? Which claims would be ruled out? Insignificant. a and b would be ruled out. c and d would still be possible.
4. Assume that P = . VOoQiTd we judge the difference to be significant or insignificant? Which claims would signif be ruled out? Significant. c and d are ruled out; a and b are still possible.

Consider the following possible sets of data that April and Betty might obtain. Set 1 is listed again for easy comparison. Determine the means and standard deviations by inspection. Divide the work up among your group.

Set 1 Data

April Betty 0.121 0.123 0.122 0.124 0.123 0.125 x 0.122 0.124 S 0.001 0.001

Set 2 Data

April Betty 0.120 0.124 0.121 0.125 0.122 0.122 0.122 0.126 0.123 0.123 x 0.121 0.125 x 0.122 0.122 S 0.001 0.001 S 0.001 0.001

Set 4 Data

April Betty 0.120 0.122 0.122 0.124 0.122 0.124 0.122 0.124 0.124 0.126 0.122 0.124 x 0.122 0.124 x 0.122 0.124 S 0.002 0.002 S 0 0

Set 6 Data

April Betty 0.121 0.123 0.121 0.123 0.122 0.124 0.122 0.124 0.123 0.125 0.123 0.125 x 0.122 0.124 S 0.001 0.001

Set 7 Data

April Betty 0.122 0.124 x S undefined undefined

Set 3 Data

April Betty 0.121 0.121Changing the difference in the means

1. a. Consider Set 2. Compared to Set 1, which summary statistics-the average, the standard deviation, or both-have changed? Both means are different; both std devs are the same. Compared to Set 1, what has happened to the value of P ? Has it increased or signif Explain your conclusion. deincreasing the difference in the means should increase the likelihood that the means are really different.
2. b. Consider Set 3. Compared to Set 1, which summary statistics have changed? Both means are different; both std devs are the same. Compared to Set 1, what has happened to the value of P ? Explain your conclusion. signif There is zero difference in the means, so there is no chance they are really different.
3. c. Based on your answers to the previous two questions, what should happen to P as the difference signif between the average values increases? Your answer should be a complete, grammatically correct sentence. As the difference in the means increases, Psignif must increase.

Changing the standard deviations

1. a. Consider Set 4. Compared to Set 1, which summary statistics have changed? The means are the same; the std devs have now increased. Compared to Set 1, what has happened to the value of P. ? Explain your conclusion. signif There is more overlapping data in the two sets, which suggests that the difference isn't real. Therefore Psignif has decreased.
2. b. Consider Set 5. Compared to Set 1, which summary statistics have changed? The means are the same; the std devs have now decreased to zero. Compared to Set 1, what has happened to the value of P ? Explain your conclusion. signif There is no variation in either data set, which makes the difference look very real. Psignif has increa sed.
3. c. Based on your answers to the previous two questions, what should happen to P as the standard signif deviation of the measurements increases? Your answer should be a complete, grammatically correct sentence. As the std dev of the measurements increases, Psignif should decrease.