Introduction

As student enrollment in courses delivered online continues to increase across higher education (Sloan Consortium, 2004), it becomes necessary for professors who teach online to assess the effectiveness of their online teaching techniques.  Many faculty members utilize asynchronous discussions as a method of teaching and building learning communities within their online courses. Asynchronous discussions enable students to engage in class discourse at any time and in any place. They allow for students to create a learning community where they learn from each others’ perspectives. Additionally, asynchronous discussions permit all students in a class to have a voice, unlike face to face classes where time limitations, students who dominate discussions, and some students’ tendency to be introverted do not permit everyone to become involved in a discussion.  Asynchronous discussions facilitate reflection of course material by allowing students to consider their response before posting it and by allowing follow up on their own comments as others make relevant points. This last benefit encourages in-depth thorough discussions of a topic (Althaus, 1997; Baglione & Nastanski, 2007; Christopher, Thomas, & Tallent-Runnels, 2004; Hewitt, 2001).

Some attention has been given to building learning communities in online courses through the use of asynchronous discussions.  Much of this research deals with surveying students, and getting students’ perceptions regarding their learning experiences with online courses (Northrup, 2002; Swan, Shea, Fredericksen, Picket, Pelz, & Maher, 2000; Young & Norgard, 2006). Equally important to creating learning communities in online courses is the need to develop asynchronous discussions structured in such a way as to promote learning outcomes that are measurable and tangible.

Researchers have examined how participation in asynchronous discussions in both face to face classes and fully online classes contributes to students’ learning of course material. These researchers have examined grades on tests and assignments as well as, noting students’ perceptions about their own performance (Althaus, 1997; Picciano, 2002;  Steimberg, Ram, Nachmia and Eshel, 2004;  Wu & Hiltz, 2004). Althaus (1997) was interested in determining whether students’ participation in computer-mediated discussions within a face to face class would impact their learning of course material as measured by grades on tests and a class paper. He set up a list serve for students to discuss course topics through email correspondence. Participation was voluntary, and discussions were not graded.  Althaus found that those who volunteered to participate reported they felt they learned more by virtue of participating in the computer-mediated discussions.  After completing statistical analysis of the data, he also found that participants in the discussions had higher final exam grades than those in the class who simply participated in face to face discussions. 

Wu and Hiltz (2004) were interested in querying students regarding their perceptions of online discussions and their impact on learning in three classes where asynchronous discussions were added to regular face to face meetings. Over half of the 116 participants reported that they felt online discussions contributed to their quality of learning and that they learned from their peers during online discussions.

Steimberg, et al (2004) set up discussion groups to help students study for exams in a face to face course offered in 2004 and again in 2005.  They were interested in determining whether there would be a relation between the type and extent of discussion participation and test grades. The type of participation was categorized as students who wrote messages, those who only read messages and those who did not participate.  The extent of the discussion participation was measured by the number of messages students wrote and the number students viewed. They found that for the 2004 course, the group of students who were message writers had significantly higher test scores than the other two participation groups. For the 2004 course they also found a correlation between the number of messages written and test scores. They did not find a significant difference between the participation groups in the 2005 offering of the course, but they did find correlations between the number of discussion postings and exam grades.

Picciano (2002) was interested in how online discussions affect student performance in courses delivered completely online. Within a graduate online course, students had weekly discussion topics facilitated by peers within the class.  Students were not required to make a certain number of postings per week nor were the content of the postings graded.  Students were told that discussion posts would constitute their participation grade for the course. Picciano found a correlation between the number of  student discussion postings in a course and performance on exams; however, this correlation was not statistically significant.  He did find that students perceived greater quality and quantity of learning as a result of participating in the discussions.  He also broke the subjects into groups according to their level of participation: high, moderate or low.  He found no difference between the groups on the exams but did find a difference between the groups on scores for a written assignment. The high participation group had the highest written assignment scores. 

Within each of the studies discussed above (Althaus, 1997; Picciano, 2002; Steimberg et al, 2004; Wu and Hiltz, 2004) none of the researchers graded the content of the discussions, they simply quantified participation by counting the number of posts and type of interaction on the discussion boards.  Additionally, only Picciano studied discussion participation and its relationship to learning course content in a fully online course. The other researchers incorporated discussions in face to face courses, where additional instructional activities as well as discussions were present.  Finally, although the researchers cited above found that online discussions contributed to learning and perceived learning, there was little information about the content and structure of the discussions that may have contributed to learning. 

Other researchers, however, found that in order to promote learning in online courses, discussions must be planned and structured to enhance participation and meaningful interaction with materials related to the course objectives (Dennen, 2005: Vonderwell, Liang & Alderman, 2007). Dennen (2005) analyzed how asynchronous discussions were used in nine different courses. She found different types of activities impact student participation in asynchronous discussions.  Discussions that are not threaded do not promote participation. Discussions that are meaningful and relevant to students’ lives and allow for perspective sharing were considered to be productive. This type of discussion allows students to explain concepts to each other in terms that may be more relevant and understandable than the way the text or instructor explains concepts.  Similarly, Vonderwell, et al (2007) found that the way asynchronous discussions are structured is essential for successful learning. Students actively participated when discussions were threaded, when there was a grading rubric to guide participation, and when a learning community was established for students to learn from each others’ viewpoints and explanations.   

It is important for instructors who develop classes with asynchronous discussions to  incorporate basic learning principles into discussion activities so that students are engaging in practices grounded in learning theory, thus in the process maximizing their learning.  Within the present study, discussions were tailored so that students, if they participated fully, had to engage in long-term memory storage processes of meaningful learning, elaboration, and rehearsal in the form of distributed practice. 

According to Ormrod (2008, p. 222) what is most important regarding instruction is “how well it promotes effective storage processes.”  Asynchronous discussion activities can be created that enhance students’ engagement in storage activities.  These activities can be explained within the context of Atkinson and Shifrin’s (1968) dual store model of memory. The sensory register is where information is first received. If it is attended to, the information moves to short-term working memory. If it is not attended to, the information disappears. Once in working memory, information is processed in conjunction with information from long-term memory, and incorporated into long-term memory, or information is lost after about 20 seconds.  Long-term memory is theorized to have unlimited capacity and is the repository of much of what we know.  Cognitive processes associated with storing information into long-term memory include selection, internal organization, visual imagery, meaningful learning, elaboration and rehearsal (Ormrod, 2008). For purposes of the current study the latter three processes are the focus.

Meaningful learning can be defined as connecting new information to prior knowledge. This process is particularly effective when people connect new information to information about themselves.  Instructors can create meaningful learning activities by asking students to relate new information to their prior experiences and interests. Elaboration occurs when information is presented and learners fill in the details with assumptions, inferences and their own interpretations to create understanding. This process typically results in a deeper understanding of the material, but may at times lead to misunderstanding.  Having students define concepts in their own words is an example of elaboration. Finally, rehearsal in the form of distributed practice involves periodically practicing and reviewing material. Distributed practice enhances retention through repeated interaction with the material. Students process the information more frequently and thus continue to elaborate on the information for greater understanding (Ormrod, 2008).

The purpose of the present study was to expand on the previous  research cited above, and to determine if asynchronous discussions correlate with learning outcomes as measured by course exams when they are graded and structured to create learning opportunities for students to engage in meaningful learning, elaboration and rehearsal in the form of distributed practice.  Within several sections of an online class taught by this author and titled “Human Learning and its Application to Education” discussions were created to enhance storage activities.  Discussions were required and graded according to a grading rubric made available to the students at the start of each semester.  After several semesters of teaching this course online and touting the benefits of fully participating in discussions to the students, it occurred to this author that it was necessary to determine whether the discussion structure was actually contributing to students’ learning of the course objectives.

The instructor set up unthreaded discussions consisting of a multipart question for all students to answer.  Typically, unthreaded discussions are not recommended as research suggests they lead to one answer and do not encourage participation (Dennen, 2005; Vonderwell et al 2007).  However, these questions involved having students not only define concepts but apply them to their own experiences, thus making each post unique. Each weekly discussion question consisted of a query about a concept or several concepts where students had to demonstrate understanding and elaborate on the concept by explaining the concept in their own words. Typically the second part of the question required the students to apply the concept to an experience in their lives and to their future professional careers, thus creating opportunity for meaningful learning and additional elaboration. Once students answered the question, they were permitted to read their classmates’ response to the question and then they were required to respond in a substantive manner to four of their classmates’ posts. The response to classmates required students to engage in distributed practice by reviewing others’ responses and to elaborate by expanding on what their classmates said. Discussions occurred weekly throughout the semester, therefore, encouraging students to engage in additional distributed practice before each exam. 

For this study the researcher was interested in answering the following questions:

1. Would the grades on the discussions correlate with grades on course exams? 
 

2. Additionally, if students are divided into groups by their discussion grades, would there be a difference between groups on exams?  Would students with high discussion grades who are engaging in effective learning practices related to meaningful learning, elaboration and distributed practice have greater test grades than those with low discussion grades who, by virtue of their discussion grades are assumed not to be engaging in meaningful learning, elaboration and distributed practice?

Method

Participants

Two hundred and fifty undergraduate teacher education students enrolled in “Human Learning and its Applications to Education” were subjects for this study. Student grades were drawn from eight online sections of the course offered from the fall of 2005 through the spring of 2007.  For each section there was an average of 30 students enrolled.  Each of the online sections was taught by the same instructor and followed approximately the same schedule with similar course requirements. Each of the sections was delivered utilizing WebCT as the learning platform. 

Procedure

For six of the eight sections there were ten required discussion activities and three exams. For two of the eight sections there were nine required discussion activities and three exams. Students in each section were divided into two groups for the discussion to create smaller discussion groups of approximately 15 students.  

Discussions were based on material in the text and lecture notes that would be covered on the exams.  The following is an example of a typical discussion question:  

Define escape and avoidance.  Give an example from your own life of a time when you have engaged in escape and a time when you have engaged in avoidance.  How can you use your knowledge of these concepts in the classroom to help students who engage in these behaviors?

An example of a well written response to a discussion question was posted for students to use as a guide, prior to the first graded discussion.  Discussions were typically open for three days. Students were required to answer the initial question on the first day of the discussion period.  This requirement was based on this author’s experience with discussion.  When students are not given a timeline in which to post, the majority tend to wait until about an hour before the discussion closes to make all their posts. This approach does not allow for an active learning community. Students were asked not to read their classmates’ posts until after they made their initial post. Once the initial post was made students were free to read and respond to classmates’ posts. On average, students followed the basic requirements and had a total of five posts for each weekly discussion topic. 

For each discussion the instructor responded to each of the students’ initial posts with feedback regarding the accuracy of the content within the post. Typically, students received feedback regarding their posts within the same day they made their initial post. Thus students were provided information about whether they had an accurate understanding of the information before they went on to respond to their classmates. After the initial post was made, the instructor read the students’ responses to classmates but did not regularly participate in this part of the discussion. However, if students made inaccurate statements and classmates did not correct one another, the instructor did step in to clarify.  The instructor also responded when students directly asked the instructor for information or a comment.

Discussions were graded according to the ten point grading rubric shown in Table 1.  Students received points for accuracy, their ability to correctly apply the principles to their lives, and for their comments to at least four of their classmates’ posts. Each exam was multiple choice with a maximum score of 100 points.

Table 1. Discussion Grading Rubric

Results

To analyze the data, discussion grades leading up to each test were grouped, averaged, and correlated with the corresponding test. For example, the discussions that preceded the first exam were averaged and correlated to the first exam. This was done accordingly for all discussions and corresponding exams across the eight sections. Table 2 presents the means and standard deviations of exams and corresponding discussion sets.
 

Table 2. Means and Standard Deviations of Discussion Sets and Exams (n=250)

                                                                                            M                             SD

Exam I                                                                             78.53                         15.16

Discussion Set 1                                                               7.48                            2.09

Exam II                                                                            82.58                          15.71

Discussion Set 2                                                               8.24                            2.11

Exam III                                                                           87.02                         14.42

Discussion Set 3                                                               8.49                           2.36            

To determine correlations between discussion sets and corresponding exams, Pearson Correlation analyses were completed.  There were significant correlations between Discussion Set 1 and Exam 1, (r=.305, p< .01), Discussion Set 2 and Exam 2, (r=.259, p< .01), and Discussion Set 3 and Exam 3, (r=.161, p<.05). 

The author was also interested in whether there would be differences in test scores when students were divided into groups according to the overall average discussion grade for the course. Discussions across the entire semester were averaged for each student. Then, students were divided into the following three groups. Group 1 had average discussion grades of six or below. Group 2 had average discussion grades of seven and eight.  Group 3 had average discussion grades of 9 &10.  The rationale behind the division was that students in Group 1 who had average grades of six or below were not participating regularly, and when they did participate, they did not post the required elements of the discussion. Those who were in Group 2 tended to participate regularly but their initial post often contained errors and they did not discuss the topics beyond a superficial level. They typically responded to their classmates as required.  Those who were in Group 3 followed the discussion guidelines and consistently earned 9 or 10 points on each discussion. 

To determine if there were differences in test performance between the three groups of students on each of the three tests, an Analysis of Variance was undertaken.  A significant difference on scores for each of the three tests was found between the three groups of students, as shown in Table 3. 

Table 3.  Mean Test Scores and ANOVA F-values

A post hoc analysis was conducted using the Sheffe test of multiple comparisons between means in order to determine which of the three groups differed from one another. For each of the three tests there were significant differences between students’ test scores in Discussion Group 3 and the other two groups. In other words, across all three tests Discussion Group 3 outscored the other two discussion groups and the difference between the scores was statistically significant. Table 4 shows there were no significant differences on test scores between Discussion Group 1 and Discussion Group 2.  

Table 4. Scheffe Test of Multiple Comparison Between Groups.

* The mean difference is significant

Discussion

Overall the results of the study show support for the researchers’ practice of incorporating storage strategies within the structure of the discussion. There were positive correlations between discussion grades and the exams that were related to the discussion topics.  As the semester wore on however, the correlations between discussions and exams became weaker. This may be accounted for by the fact that the last exam covered fewer chapters than the previous two and had fewer discussion questions associated with it. Additionally, although discussions covered weekly content material that would be on the exams, it was not possible to have discussions over all the material covered on the exams. Thus correlations may have been stronger if discussions were numerous enough to cover all potential exam material.

In the present study, the positive correlations were further supported by the significant difference between discussion groups with regard to grades on tests. Those students in discussion Group 3 who actively participated in discussions throughout the semester had significantly higher test grades than those in the other two groups. This finding suggests that when students engage in discussions to the fullest extent required, they benefit in terms of learning the material, as reflected by their test grades.  

The findings from this study are similar to the studies cited above where discussions were not actively graded or required, nor was there much information about the structure of the discussions (Althaus, 1997; Picciano, 2002; Steimberg et al, 2004).  Consequently, this similarity begs the question of whether it matters to overall learning of course material if discussions are structured to maximize storage, if discussions are required, and if discussions are graded.  This author would argue that it does matter.  In the present study when the students were divided into groups, only Group 3 had significantly higher test grades. If participation alone is the key to successful learning through asynchronous discussions then Group 2 should have had higher test grades than Group 1 as well.  The difference between Group 3 and Group 2 was not in the number of discussion posts but in the depth and breadth of the posts.

As research demonstrates (Dennen, 2005; Rovai, 2002; Vonderwell et al 2007), it is good practice to engage students in structured activities where the expectations for the content of the discussions is clear and discussions are structured to enhance learning.  Students benefit when they are “forced” by virtue of the discussion structure to interact with the course content in such a way that they will learn the content.  Hence, instructors must find creative ways to incorporate basic learning principles and activities within their courses so students are actively engaged in learning through discussion. It is not enough to ask students to engage in discussion without having a structure by which they should engage.

Limitations

It appears that the practice of incorporating meaningful learning, elaboration, and distributed practice into the structure of the discussions plays a part in enhancing student learning of the course objectives within the present study. The current analysis however, did not include information regarding students overall GPA, nor was there a control group of students evenly matched who did not participate in online discussions. Thus, it can be suggested that providing opportunities for students to engage in structured discussions which incorporate the storage activities of elaboration, meaningful learning, and distributed practice, will enhance student learning of course objectives. However, it cannot be definitively stated that these practices make all the difference. It also can be argued that those students who benefitted from discussions would have done well no matter what assignments were created in the class, since other factors such as motivation and academic achievement level cannot be ruled out as being associated with test grades.  

Future Research

Future research may reinforce these findings by comparing student performance on exams in different sections of the same class over a semester. Discussions can be set up in one section so students have to engage in storage processes as described in this study, and in the other section students simply have to engage in a weekly, unstructured discussion for participation points. If the design of the discussion truly impacts learning, all things being equal among the classes, then the students in the class where discussions are required and structured to enhance storage should have higher exam grades than students in the class where discussions are not structured.        

References

Althaus, S. L. (1997).  Computer-mediated communication in the university classroom: An experiment with on-line discussions.  Communication Education, 46, 158-174.

Atkinson, R. C. & Shiffrin,  R. M. (1968).  Human memory:  A proposed system and its control processes.  In K. W. Spence & J. T. Spence (Eds.), The psychology of learning and motivation:  Advances in research and theory (Vol. 2). New York: Academic Press.

Baglione, S. L. & Nastanski, M. (2007). The superiority of online discussion:  Faculty Perceptions.  Quarterly Review of Distance Education, 8(2), 139-150.

Christopher, M. M. Thomas, J. A. & Tallent-Runnels, M. K. (2004).  Raising the bar:  Encouraging high level thinking in online discussion forums.  Roeper Review, 26(3).  166-171. 

Dennen, V. P. (2005).  From message posting to learning dialogues:  Factors affecting learner participation in asynchronous discussion.  Distance Education, 26(1), 127- 148.

Hewitt, J. (2001).  Beyond threaded discourse. International Journal of Educational      Telecommunications, 7(3), 207-221.

Northrup, P. T. (2002).  Online learners’ preferences for interaction. Quarterly Review of Distance Education, 32, 219-226.

Ormrod, J.E. (2008).  Human Learning. Upper Saddle River, NJ: Pearson Prentice Hall.

Picciano, A. G. (2002).  Beyond student perceptions:  Issues of interaction, presence,  and performance in an online course.  Journal of Asynchronous Learning Networks,   6(1), 21-40.

Rovai, A. (2002).  Building a sense of community at a distance.  International Review of Research in Open and Distance Learning, 3(1),

Steimberg, Y., Ram, J., Nachmia, R., & Eshel, A. (2006).  An online discussion for supporting students in preparation for a test.  Journal of Asynchronous Learning Networks, 10(4), 117-128.

Swan, K. Shea, P., Fredericksen, E., Pickett, A., Pelz, W., & Maher, G. (2000).  Building knowledge building communities:  Consistency, contact and communication in the virtual classroom.  Journal of Educational Computing Research, 23(4), 359-383.

Vonderwell, S., Liang X., & Alderman, K. (2007).  Asynchronous discussions and assessment in online learning.  Journal of Research on Technology in Education,  39(3), 309-328.

Wu , D., & Hiltz, S. R. (2004).  Predicting learning from asynchronous online discussions.  Journal of Asynchronous Learning Networks, 8 (2), 139-152.

Young, A. & Norgard, C. (2006). Assessing the quality of online courses from the students’ perspective.  Internet and Higher Education, 9, 107-115.