Analyze change of pre-test and post-tes
by Dr. Alex Yu
I would like to compare the responses of a pre-test and a post-test.
The dependent t-test tells me the overall difference between two test scores.
But I want to examine the detail of the change--how people changed their choices after the treatment.
There are two ways to examine how people changed the selection--by table and by graph.
Although the procedure "proc freq; table var1 * var2;" can return a 2X2 table, I prefer to use the following tabular method:
proc summary nway; class pre_em1 post_em1; output out=temp1;
proc print data=temp1;
The above procedure would return the output as the following. We can tell that 33 people who chose 'b' switched to 'c' later, 51 people who chose 'c' in pre-test remains in 'c' in post-test ...etc.
OBS PRE_EM1 POST_EM1 _TYPE_ _FREQ_
1 b c 3 33
2 b d 3 1
3 c c 3 51
4 c d 3 2
5 d c 3 7
If you want to examine the data interactively, you may use the graphing function in SAS/Insight. In the bar chart when you click on any bar in the pre-test, the corresponding subjects will be highlighted in the post-test.
This Quicktime movie shows you how it works. The procedure of making an interactive bar chart is as the following:
- Import the data into a SAS library. Select Import from File and follow the on screen instruction. If the data have already been imported into a library, simply double-click the file icon. The dataset will be loaded into a temp library.
- Open Global/Analyze/Interactive Data Analysis.
- Select the dataset from the proper library.
- Select Histogram/Bar Chart from Analyse.
- Put one pre-test item and one post-test item into Y.
- Click on the button Method. Change "Other" Threshold to 0. Then SAS will show all values rather than collapsing some of them. Click OK to close it
- Click OK to plot the graph.
In addition, although a dependent t-test seems to be very straight-forward, you need to pay attention
to some possible pitfalls:
First, a ceiling effect would obscure the finding of the true difference. A ceiling effect is that
several respondents have mastered the subject matter before the treatment. For example, in the following table subject 1 achieved perfect scores in both pre-test and post-test. Although the difference between two tests is zero,
it doesn't mean that your treatment is ineffective.
To avoid the ceiling effect, you can simply exclude those "over-achievers.":
if pretest = 10 then delete;
proc means mean std n t prt; var diff;
Second, usually when you conduct a dependent t-test, you assume a directional hypothesis and apply a one-tailed test.
In other words, you expect an improvement after the treatment rather than just a difference that can go
either way. However, the default p-value of SAS is for a two-tailed test. Therefore, when the p-value is
0.10, which appears to be insignificant, the adjusted p-value for one-tailed test should be 0.05 (0.10/2)
Table of Contents