Objectives
ognitive models of learning stress that learners are active
agents in constructing their own learning (Shaw, 1995; Biehler
& Snowman, 1986). Collaborative learning enhances the learning
process, allowing learners to engage in the construction of new
representations of knowledge and facilitating the transfer of
information from short-term to long-term memory (Hooper,
Temiyakam, & Williams, 1994; Alavi, 1994). Asynchronous and
synchronous communication provides users with opportunities to
collaborate across distances through such technologies as mailing
lists and chat sessions, respectively. It was hypothesized that
these modes of communication provide unique features conducive to
the development of a collaborative community of learners. The
benefits of Internet-based communication and collaboration are not
reflected solely in test performance. Behind the behavioral change
is an attitudinal change. Gain in test performance might be short
term; change in attitude could facilitate life-long learning (Yu,
1998). Therefore, this study employed chi-square analysis, data visualization, and regression modeling to investigate the use of asynchronous
(mailing lists) and synchronous (chat sessions) Internet-based
communication and its impact on teachers' attitude toward
collaboration, activity completion rate and test performance. In
addition, the impact of collaboration on activity completion rate
and teacher performance measured by objective tests was
investigated.
Theoretical Framework
Several theoretical perspectives suggest that peer interaction occurring in computer networked groups may be beneficial to learning. These perspectives include Vygotsky's (1978) and Piaget's (1963) emphasis on social construction of knowledge and Bandura's (1969) interpretation of peer modeling (King, 1989). From a social cognition point of view, when educators interact with peers they are faced with ideas, explanations and information inconsistent with prior knowledge and beliefs. Confronting these inconsistencies challenges one's current point of view; resolving such cognitive conflicts results in cognitive restructuring (Bearison, 1982). Group members presumably add detail to existing cognitive schemas, explore and correct misunderstandings, fill in gaps in their knowledge, and/or reorganize their knowledge structures (Bearison, 1982; Piaget, 1963; Vygotsky, 1978). During peer interaction, educators are exposed to new strategies, terminology, and ways of thinking about problems; through restructuring thinking, these experiences affect problem-solving behaviors as well as classroom instructional delivery. From a social modeling perspective, peer interaction provides opportunities for imitation of successful problem-solving behaviors. Thus changes in teachers' levels of competence take place as a result of peer interaction in small groups (Vygotsky, 1978).
Collaborative learning is the acquisition by individuals of knowledge, skills or attitudes as a result of group interaction, where that specific learning could not have been derived individually (Schrage, 1990). Computer-mediated collaboration provides a medium for educators to learn to engage in collaboration with other educators at a distance. Alavi (1994) observes that those involved in collaboration can monitor individual thinking, opinions and beliefs and provide feedback for clarification and change. In helping one another, learners may foster stronger elaborations between new and existing knowledge, facilitating the transfer of information from short-term to long-term memory (Hooper, Temiyakam, & Williams, 1994).
Internet-based communication provides both asynchronous and synchronous environments for collaboration. Synchronous communication (such as chat) allows interaction to occur at the same time among learners, as in classroom discussions: but learners in the Internet environment use a keyboard to type messages. Local Internet accounts enable learners to engage in collaborative chat sessions with distant peers, expanding the collaborative environment from a single classroom to classrooms around the world. Asynchronous communication (mailing lists) allows interaction to occur at different times and locations between two or more learners. Learners need not be present to receive information, and may communicate when ready. Learning occurring as a result of group interaction may take place in a virtual community and may be adjusted to student need, schedule and educational goals. Both of these forms of Internet-based communication expand learners' access to information, resources, and collaboration, and may increase learners' attitude toward collaboration as well as achievement performance. In this study, we are interested in both forms of Internet-based communication as they relate to learner attitude toward collaboration and the completion rate and performance on Internet skill tests.
Hypotheses
Based on the above, the following hypotheses were formed:
a. Use of Internet-based communication increases teachers' positive attitude toward collaboration on the Internet as measured by the Stages of Concern Instrument: Attitudes Toward the Internet (ATI; Hall, George, & Rutherford, 1977; Well and Anderson, 1997);
b. Use of Internet-based communication increases course completion rate;
c. Use of Internet-based communication increases test performance as measured by criterion-referenced Internet skills test;
d. The more positive teachers' attitudes toward collaboration, the higher the increase in teacher performance as measured by criterion-referenced Internet skills test; and
e. The more positive teachers' attitudes toward collaboration, the higher the course completion rate.
Methods
Participants
Participants in this study were one hundred sixty-one K-12 educators residing throughout a southwestern state. All participants took part in an interactive web-based course designed to enhance the collaborative teaching and learning communities for participating educators as they acquired Internet skills.
Treatment
The treatment was an interactive web-based course designed to enhance the collaborative teaching and learning communities for participating educators as they acquired Internet skills. Of particular interest were new outcomes made possible through use of advanced technologies in instruction. Therefore we looked at the engagement in Internet-based communication and increases in Internet skills.
Procedure
Participants were part of a large collaborative community (the course) as well as of smaller communities focused on establishing professional relationships and on technical and course help. These smaller communities were facilitated by a group leader trained in the use of Internet technologies and in how to facilitate Internet-based communication. Group leaders were not participants in the course. Participants had access to chats and were members of two course-related mailing lists, (a) course announcements, and (b) a group list headed by the group leader. Group leaders conducted at least one chat bimonthly and communicated with their participants via group mailing lists biweekly. Participation in chats was mandatory for group leaders and strongly encouraged for participants. Participants were encouraged to join other mailing lists within the course setting as well as outside of the course. The course itself maintained up to 25 mailing lists on various content-specific topics. Archives of eleven chat sessions and eleven mailing list groups were used in this investigation.
The Internet skills that participants were to learn focused on searching techniques and the use of email. Participants completed a pretest assessing their Internet skills in these areas. Following the pretest, participants completed tutorials specific to these skill areas, then completed a posttest.
Data collection
Two instruments were employed in this study. The first instrument was the Stages of Concern Instrument: Attitudes Toward the Internet (ATI; Hall, George, & Rutherford, 1977). This scale was developed to measure the evolving attitudes of learners exposed to an innovation. In this original scale, the variable word "innovation" may be substituted with the innovation of interest. Therefore, in a study regarding learner attitudes toward a new innovation, specifically the Internet, Wells and Anderson (1997) substituted the variable "innovation" with "Internet" to measure attitudes toward the Internet. Reliability of the instrument was reported as .91 (Reed, 1990), and discussed in detail by Wells and Anderson (1997). The second instrument was a performance test over the Internet skills learned that was developed by our research team. For both measurements, a pretest was administered prior to a web-based instruction program. Three months later a posttest was given. The interval was short in order to avoid the "maturation" effect in which subjects improve over time regardless of treatments (Cook & Campbell, 1979). All participants completed the pretest and 95 finished the posttest. One hundred and fifty-nine participants completed the ATI survey and fifty-six completed the post-survey.
A subscale of ATI was used to measure the attitude toward collaboration though the entire survey was administered to the participants. The survey was composed of 35 questions and could be divided into seven subscales, which indicated seven stages of concern with the Internet. The seven stages were awareness, informational, personal, management, consequence, collaboration, and refocusing (Wells & Anderson, 1997). Please see Table 1 for the items in the collaboration subscale.
Table 1.
Collaborative subscale in Stages of Concern Instrument: Attitude Toward Using the Internet
|
Question 5: |
I would like to help other faculty use the Internet. |
|
Question 10: |
I would like to work with present fellow workers and others who are using the Internet. |
|
Question 18: |
I would like to familiarize my colleagues about the Internet as I learn more about it and with it more. |
|
Question 27: |
I would like to coordinate my efforts in learning about the Internet with fellow workers. |
|
Question 29: |
I would like to know what other people are doing in relation to the Internet. |
Factor analysis confirmed the factor structure of the instrument. Based on the pre-survey data, the survey items could be grouped as eight latent variables according to the Kaiser criterion (eigenvalue as one or larger). By examining the inflection point in the scree plot, seven factors could extract most of the eigenvalues (Figure 1a). In the post-survey, Kaiser criterion also suggested eight factors while the scree plot implied five to seven factors (Figure 1b).
Figure 1 (a). Factor extraction by Kaiser criterion and the inflection point in the scree plot for pre-survey.
Figure 1(b). Factor extraction by Kaiser criterion and the inflection point in the scree plot for post-survey.
Based upon the pre-survey data, the estimated reliability of the entire ATI survey in terms of Cronbach coefficient Alpha was .86 whereas the Cronbach coefficient Alpha for the collaboration subscale was .87. In the ATI post-survey data, the overall Alpha was .89 while the collaboration subscale Alpha was .79.
The second instrument in this study was a criterion-reference performance test, which consisted of 30 items. Cronbach coefficient Alpha of the pretest was .45 whereas that of the posttest was .86. The low Alpha in the pretest, which was expected, implied that participants were unfamiliar with the subject matter and random guessing of answers led to inconsistency. In the posttest the high Alpha denoted that the response pattern was internally consistent.
Finally, use of mailing lists and Internet chat was logged as a measure of engagement in Internet-based communication as a collaborative tool. Originally, engagement in collaboration was defined by the number of words in email/chat and the total number of messages. However, these distributions were highly skewed and thus violated the assumptions of parametric tests. Therefore, observations were re-coded into four binary variables: (a) use of mailing lists, (b) use of chat sessions, (c) use of both mailing lists and chat sessions, and (d) use of neither mailing lists or chat sessions. For example, if the participant used mailing lists, the value of the first preceding variable was re-coded as "1" regardless of either the number of words in the posted message or the number of messages. If no messages were posted, the value was recorded as "0". The remaining variables were similarly re-coded.
Results
Descriptive statistics are reported in Table 2. Pearson Product Moment correlation coefficient between the Internet skills pretest and posttest scores was .13 whereas that of the ATI pre-survey and post-survey was .29.
Table 2
Descriptive statistics for Internet skills test and Attitude Toward the Internet.
Variable Pretest 12.83 3.31 161 Posttest 22.48 4.88 95 Difference score 9.12 5.45 95 Pre-survey 29.14 5.27 159 Post-survey 29.09 4.32 56 Difference score -.67 5.04 43
Results of the Kolmogorov-Smirnov Normality tests indicated that the distribution of the Internet skills pretest scores did not significantly depart from normality, w (174) = 97, p = .09. Similarly, neither did the difference between the pretest score and the posttest score, w (94) = .97, p = .14. The distribution of the posttest score was not normal, w (94) = .92, p = .0001. No outliers were found in the preceding distributions, which is displayed in Figure 2(a)-2(c).
Figure 2(a). Distribution of pretest scores on Internet skills.
Figure 2(b). Distribution of posttest scores on Internet skills.
Figure 2(c). Distribution of difference between Internet skills pretest and posttest scores.
Results of Kolmogorov-Smirnov tests indicated that both distributions of the collaboration pre-survey (w (158) = 85, p = .0001) and post-survey (w (55) = .93, p = .0032) departed from normality. Non-normality was not observed in the distribution of the difference between the pre-survey and the post-survey, w (42) = .95, p = .13. Again, no outliers were found in the preceding distributions, which is shown in Figure 3(a) ñ Figure 3(c).
Figure 3(a). Distribution of pre-survey on attitude toward collaboration.
Figure 3(b). Distribution of post-survey on attitude toward collaboration.
Figure 3(c). Distribution of difference between collaboration pre-survey and post-survey.
Prior to addressing the hypotheses, two one-tailed dependent t-tests were employed to examine whether improvement in teacher scores on the Internet skills test and the ATI collaboration subscale occurred after the treatment. The dependent t-test for the difference between the pretest and posttest yielded a significant difference, t (94) = 16.33, p = .0001. Given that the effect size was .03, the alpha level was .05, and the sample size was 95, the power for this t-test was .90, which indicated a high probability that the null hypothesis was correctly rejected.
On the average participants had a decrease in the positive attitude toward collaboration after the treatment, M = -0.67. A dependent t-test showed that this decrease was not significant, t (42) = -0.88, p = 0.38. This result might be due to the lack of statistical power. Given that the effect size was .03, the alpha level was .05, and the sample size was 43, the power for this t-test was .61, which indicated that a true significant difference might be undetected.
To test the hypothesis regarding the relationship between engagement in Internet-based communication and attitude toward collaboration (hypothesis a), exploratory data analysis instead of ANOVA was performed to examine the relationships. The data structure did not conform to the parametric assumptions of ANOVA such as normality and homogeneity of variances, and this sample size was not large enough to run a robust ANOVA test. In the exploratory data analysis, it was found that users of chat and mail tended to center around the mean of the difference between collaboration pre-survey and post-survey, as shown in Figure 4. Participants who engaged in chat sessions had less variability on the change score of ATI (-7.5 to 1.5) than the general test population. Likewise, participants who posted messages on mailing lists also centered around the mean (-4.5 to 4.5). Therefore this might indicate that participants who engaged in chat sessions or mailing lists have less change in their attitude toward collaboration than those in the general test population, maintaining scores close to the mean.
Figure 4. Linked observations between use of collaboration and changed scores in attitude toward collaboration
A Chi-square test of goodness of fit was applied to investigate whether engagement in Internet-based communication could increase the likelihood of completing the entire instructional module (hypothesis b). Likelihood ratio Chi-square was used because it is related to log-linear model and logistic regression model, and completion of module was a dichotomous variable. The Chi-square analysis yielded a significant result, Likelihood ratio C2 = 8.19, p = .042. The frequency of each cell is displayed in Table 3. Because one of the cells had no subjects and some cell sizes were very uneven, a Chi-square test might not be valid. To rectify this shortcoming, an exact test was performed using StatXact (SPSS Inc., 1999). Both asymptotic inference and exact inference yielded significant results. In the former the one-sided p-value was .02 whereas in the latter the one-sided p-value was .03. These results indicate that engaging in Internet-based communication increases the likelihood of completing Internet activities.
Table 3
Frequency table of module completion by use of Internet-based communication.
|
Frequency |
Chat |
|
Chat/mail |
None |
Total |
|
Did not complete the module |
7 |
5 |
0 |
54 |
66 |
|
Completed the module |
12 |
14 |
5 |
64 |
95 |
|
Total |
19 |
19 |
5 |
118 |
161 |
As the data structure did not conform to the parametric assumptions of ANOVA, a one-tailed two-sample independent t-test was applied to examine hypothesis (c), regarding whether engagement in Internet-based communication could lead to improvement as measured by the difference between the pretest and the posttest. Originally, use of Internet-based communication was classified into four groups, namely, use of chat, use of email, both, and none. For conducting a t-test, the first three groups were collapsed into one group. As a result, only two groups remained, namely, use of collaboration and no use of collaboration. The two-sample t-test did not yield a significant result, t(28) = .61, p = .54. Therefore, use of collaborative tools did not appear to have an impact on test performance.
Regression analysis was adopted to test hypothesis (d), which was concerned with the relationship between attitude toward collaboration and test performance. The assumption of random residual was checked by plotting predicted scores against residuals (see Figure 5a). The regression analysis disconfirmed use of Internet-based communication as a predictor to Internet skills test performance, R2 (42) = .0005, p = .88. The scatterplot of the two variables did not display a pattern (Figure 5b).
Figure 5(a). Residuals plotted against predicted posttest scores.
Figure 5(b). Scatterplot of attitude toward collaboration against posttest scores.
A logistic regression was employed to examine hypothesis (e), which addressed the relationship between activity completion and the attitude toward collaboration in the Internet. The logistic regression analysis did not yield a significant result, p = .60, odds ratio = .98, therefore scores on attitude toward collaboration did not predict activity completion rate.
Discussion
Collaboration provides a way for educators to form a virtual community of learners in which members can engage in individual thinking, share opinions and beliefs and provide one other with feedback for professional and personal growth and change. In the course studied, Internet-based communication allowed participants to engage in collaboration with others across the state. Although the Internet provides different types of environments for communication and collaboration among users, their effects are as yet unknown. In this study, it was hypothesized that asynchronous and synchronous Internet-based communication provides unique features that promote the development of collaborative communities. In addition, this environment would then impact participants' attitudes toward collaboration and performance on Internet skill tests.
Although in this study it was found that attitudes toward collaboration did not affect test performance, the data suggested a relationship between attitudes toward collaboration and use of Internet-based communication. It is encouraging that use of Internet-based communication increases the likelihood of completing the course activities, though no link by instructional theory was established between the two variables. It was speculated that those who used chat sessions and mailing lists were motivated by their peers, or self-motivated by their own engagement, but no measure of motivation was conducted in this study and thus the issue remains inconclusive.
The results do not substantiate the remaining hypotheses. Therefore it may be concluded that in this population, attitudes toward collaboration are little effected by engagement in an Internet-based course. In addition, using Internet-based communication such as chat sessions and mailing lists do not effect participant performance. Similarly, participants' attitudes toward collaboration do not predict performance on Internet skill tests.
Although not all results lead to positive conclusions, it is important to understand the aspects of Internet-based communication that may contribute to the development of collaboration among educators at a distance. Information about the characteristics of Internet-based communication and methods of implementing effective collaboration must be investigated to increase educators' effectiveness, to enhance achievement for all students and to promote systematic school improvement. This study provides a step toward such research.
Collaboration is not a context-free concept: its implementation is tied to specific types of communication provided by the medium and the specific application within the medium. As indicated in this paper, mailing lists and chat sessions represent two different modes of communication. Within these modes, there is further classification by application such as moderated and unmoderated groups. Findings in this study should not be over-generalized and different research studies for different modes and applications of Internet-based communication are recommended.
Due to the sample size and the distributional assumptions, variables in this study were examined in a pairwise manner. This research project serves as a preliminary study to identify relationships among variables. As relationships are identified, a coherent model specifying the inter-relationships among variables can be constructed. To achieve this goal, structural equation modeling with a large sample size will be employed.
References
Alavi, M. (1994). Computer-mediated collaborative learning: An empirical evaluation. MIS Quarterly, 18, 159-175.
Bandura, A. (1969). Principles of Behavior Modification. New York: Holt, Rinehart and Winston.
Bearison, D. J. (1982). New directions in studies of social interactions and cognitive growth. In F. C. Serafica (Ed.), Social-Cognitive Development in Context, (pp.173-254). New York: Guilford Press.
Biehler, R. F., & Snowman, J. (1986). Psychology Applied to Teaching. Sixth edition. Boston: Houghton Mifflin Co.
Cook, T. D. & Campbell, D. T. (1979). Quasi-experimentation: Design and Analysis Issues for Field Settings. Boston: Houghton Mifflin Company.
Hall, J.M., George, A.A., & Rutherford, W.L. (1977). Measuring the stages of concern about an innovation: A manual for use of the stages of concern questionnaire. Austin: Research and Development Center for Teacher Education, The University of Texas.
Hooper, S., Temiyakam, C., & Williams, M. D. (1994). The effects of cooperative learning and learner control on high-and average-ability students. Educational Technology, Research and Development, 41, 5-18.
King, A. (1989). Verbal interaction and problem solving within computer-assisted cooperative learning groups. Journal of Educational Computing Research, 5, 1-15.
Piaget, J. (1963). The Origins of Intelligence in Children. (M. Cook, Trans.) New York: Norton.
Pugach, M. C., & Johnson, L. J. (1995). Collaborative practitioners, Collaborative schools. Denver: Love Publishing Company.
Reed, W. M. (1990). The effect of computer-and-writing instruction on prospective English teachers' attitudes toward and perceived use of computers in writing instruction. Journal of Research on Computing in Education, 23(1), 3-27.
Schrage, M. (1990). Shared minds: The new technologies of collaboration. Random House.
Shaw, A. (1995). Social constructivism and the inner city. [On-line]. Available: http://el.www.media.mit.edu/people/acs/chapter1.html
SPSS, Inc. (1999). StatXact. [On-line].
Available: http://www.spss.com/software/StatXact/
Yu, C. (1998). An input-process-output structural framework for evaluating web-based instruction. [On-line]. Available: http://www.creative-wisdom.com/teaching/assessment/structural.html
Vygotsky, L. S. (1978). Mind in society: The development of higher psychological processes. (M. Cole, V. John-Steiner, S. Scribner & E. Souberman, Eds.). Cambridge: Harvard-University-Press.
Wells, J.G., & Anderson, D.K. (1997). Learners in a telecommunications course: Adoption, diffusion and stages of concern. Journal of Research on Computing in Education, 30(1), 83-106.
Navigation
|
 |