One may argue that when a high Cronbach Alpha indicates a high degree of internal consistency, the test or the survey must be uni-dimensional rather than multi-dimensional. Thus, there is no need to further investigate its subscales. This is a common misconception. Actually consistency and dimensionality must be assessed separately. The relationship between inconsistency and uni-dimensionality is illustrated in the figure on the left. Uni-dimensionalityis a subset of consistency. If a test is uni-dimensional, then it will show internal consistency. But if a test is internally consistent, it does not necessarily entail one construct (Gardner, 1995; 1996). Internal consistency is a form of reliability while dimensionality is associated with construct validity. Reliability is a necessary, but not a sufficient condition, for validity. This logic works like this: If I am a man, I must be a human. But if I am a human, I may not be a man (could be a woman). The logical fallacy that "if A then B; if B then A" is termed as "affirming the consequent" (Kelly, 1998). This fallacy often happens in the misinterpretation of Cronbach Alpha.

In the first scenerio, all nine sets have no overlapped area, and thus all nine items share no common variance. They are neither internally consistent nor uni-dimensional. In this situation, interpreting a low Alpha as the absence of uni-dimensionality is correct.

The second scenario is exactly opposite to the first one. It shows the presence of a high degree of internal consistency and uni-dimensionality because all items share common variance with each other. Interpreting a high alpha as an indication of the presence of one single construct could be accepted.

Unlike the above two situations, the last scenario is misleading. In Figure 6c, several items share variance with some others. In other words, subscales exist inside the instrument even though the Alpha is high and the instrument is internally consistent. Interpreting a high Alpha as a sign of uni-dimensionality is problematic.

A real life example is found in a sociological research concerning cognitive and affective attitudes toward sex. Originally, the cognitive scale and the affective scale are treated as two scales. Cronbach Alphas of the two scales are high (cognitive = .87, affective = .85) and thus it gave an illusion that the two scales represent two constructs only. However, factor analysis indicated that indeed there are two subscales in each scale. After spliting the two scales into four, Alphas were improved (see the following table).

Verify the number of factors

The original graphical output from SAS is too big to fit in a page and thus the above graph is a truncated version. The first two factors and variable 21 to variable 31 are not shown on the scree plot.

There are several ways to determine how many factors should be retained in a model. Scree plot and Kasier's criterion are two common ways. In a scree plot the number of factors is plotted against the eigenavlue. The cutoff is at the inflection point, where the slope has a sharp change. In this example the inflection point is indicated by red arrows. In Kasier's criterion, the cutoff is that "eigenavlue is equal to or greater than 1." In this example, the Kasier's criterion is depicted by a blue line.

Based upon the scree plot and the Kasier's criterion, seven subscales could be extracted from the data.

Correlation matrix

aware = sum(of a3 a12 a21 a23 a30);
info = sum(of a6 a14 a15 a26 a35);
person = sum(of a7 a13 a17 a28 a33);
manage = sum(of a4 a8 a16 a25 a34);
conseq = sum(of a1 a11 a19 a24 a32);
colla = sum(of a5 a10 a18 a27 a29);
refocus = sum(of a2 a9 a20 a22 a31);
proc corr; 
	var aware info person manage conseq colla refocus;
proc corr alpha nomiss; 
	var a1-a35;
proc corr alpha nomiss; 
	var a3 a12 a21 a23 a30;
proc corr alpha nomiss; 
	var a6 a14 a15 a26 a35;
proc corr alpha nomiss; 
	var a7 a13 a17 a28 a33;
proc corr alpha nomiss; 
	var a4 a8 a16 a25 a34;
proc corr alpha nomiss; 
	var a1 a11 a19 a24 a32;
proc corr alpha nomiss; 
	var a5 a10 a18 a27 a29;
proc corr alpha nomiss; 
	var a2 a9 a20 a22 a31;

It is important to note that a negative correlation is not a problem in data analysis. For example, if the instrument has a subscale measuring "individualism" and another subscale assessing "collaboration," it is expected that they should be opposite to each other conceptually and their relationship should be inverse.

On the other hand, a positive relationship is a problem because the concepts measured by the subscales are not distinct.

In this example, only two subcales, "management" and "collaboration" are independent and sustain an acceptable Cronbach coefficient Alpha.