An Analysis of the History of Online Graduate-Level Courses
Taught by an Expert Instructor

Doris U. Bolliger
University of Wyoming
Laramie, WY USA
dorisbolliger@gmail.com

Oksana Wasilik
University of Wyoming
Laramie, WY USA
oksana@uwyo.edu
 

Introduction

Researchers have examined instructional theories and strategies in combination with emerging technology and have called for a re-examination of instructional design theories and pedagogy in distance education (Beldarrain, 2006; Wilson, 2002). Instructors need to examine practices in the design and delivery of online courses in order to incorporate new pedagogical perspectives and emerging technologies. As the number of online courses continues to grow, faculty members who teach online will continue to face challenges pertaining to efficiency and effectiveness. Because there is a lack of literature about the online course revision process, results of this study make a valuable contribution to the field of instructional design and technology because it gives us insights into how an expert utilizes course management system (CMS) resources, and how he approaches adaptation of instructional strategies and modification of content over time.

Literature Review

The number of universities offering online degree programs has grown rapidly, and the learner demand for online courses has increased. More than two thirds of all higher education institutions offered online courses and programs in 2007. In fall 2007, more than 3.9 million students were enrolled in an online course (Allen & Seaman, 2008).

More sophisticated, user-friendly technology tools have made the design of online courses easier for faculty (Koszalka & Ganesan, 2004; Rogerson-Revell, 2007). Many educational institutions utilize CMSs today. These technologies allow users to communicate, collaborate, and interact in both asynchronous and synchronous environments (Beldarrain, 2006). Online course developers must rethink course structure and choose available tools carefully (Kidney & Puckett, 2003). With the use of CMSs, instructors can link information and have the ability to archive and re-use learning objects—digital learning resources that can be used and re-used in instruction or training. The digital shift makes replication a possibility (Wilson, 2002).

Instructional design theories need to be applied in the creation and maintenance of the online learning environment. Designers need to utilize an instructional design model and instructional strategies must be selected (Smith & Ragan, 2005) in order to reach instructional objectives. These elements must be combined appropriately to support learners. Koszalka and Ganesan (2004) point out that CMSs provide developers with tools that can be used to inform, instruct, and interact. Moore and Kearsley (1996) identify student-to-student and instructor-to-student interaction as important types of interaction in the online environment. Designers need to create opportunities for interaction and collaboration for participants because students must be actively involved in their learning in order to minimize isolation (Shaw & Polovina, 1999).

Online learning can provide learners with a constructivist environment in which they are actively engaged and responsible for their own learning (Association for the Study of Higher Education, 2006). The inclusion of multiple ways to interact and methods of content delivery addresses different learning styles. Palloff and Pratt (2001) suggest the use of discussions, collaborative assignments, and links to external Web sites. They emphasize that “it is the pedagogy and not the technology that is critical to the success of the online course” (p. 153).

According to Berge (2005), instructor and student roles are changing in the online environment. Instructors have become guides and need to give up a certain level of control (Palloff & Pratt, 2001). Today’s adult learners are more autonomous and prefer to have a high level of control over their learning (Beldarrain, 2006); they want to apply newly acquired knowledge and wish to be involved with peers and the instructor.

The design of quality online courses that are rigorous and not merely a replication of textbook supplemental materials and their management can be a difficult and time consuming task. Boettcher (2004) estimates that it takes an experienced faculty member approximately 10 hours to develop one hour of online instruction or 450 hours for an online course. Once the content is developed and uploaded, however, faculty can make changes (Morgan, 2003).

Faculty members are concerned about their success and effectiveness in the online environment. Effective online teaching practices in higher education, according to Lewis and Adul-Hamid (2006), include facilitating dynamic instructor-to-student and student-to-student interaction; providing timely, continuous, high-quality, and individualized feedback; facilitating learning by setting clear expectations, restating goals, involving students actively through integration and reflection, and promoting self-directed learning strategies; and using good organizational skills in terms of content and time management.

Many individuals have addressed how to develop online courses, how to utilize instructional technologies, and how instructors can transition from the face-to-face to the online environment (Lee & Hirumi, 2004; Oblinger & Hawkins, 2006; Simmons, Jones, & Silver, 2004; Yang & Liu, 2007). Yet, there is a lack of literature on if and how online instructors maintain content and modify their strategies over time. There is a need to investigate this important topic.

Because many instructional technology faculty members have expertise in the design of online courses, use of distance learning technologies, and significant teaching and learning experience within the programs of instructional design and technology and distance education, we should examine how these faculty members have adapted their instructional strategies and modified course content in the online environment. The research questions are: (1) How does the expert utilize CMS resources and (2) How does he approach the adaptation of instructional strategies and modification of content over time?

Methodology

The research study was conducted at a western research university. Over 11,600 students were enrolled at this public institution in spring 2008. In 1999, the instructional technology department began offering graduate-level courses completely online by utilizing eCollege®, a CMS. All courses have been archived and the data is accessible to a system administrator.

Four online graduate-level courses were selected for analysis because they had been taught by the same instructor between 2000 and 2007. The instructor, a professor in instructional design and technology, is considered an expert. He has extensive experience with the design and delivery of online courses since 1993 with the use of different delivery tools. His expertise in the field of instructional design makes him an ideal candidate for the study of instructional practices in the design and delivery of online courses.

The following courses were selected: instructional design (ID), instructional technology (IT), distance learning technology (DE), and theory of change (TC). These courses had been taught several times during 15 semesters. The examination included how available tools in eCollege were used by an expert and how he modified the courses over time. Available tools in the CMS include a calendar, announcements, document sharing, web bibliography, threaded discussions, chat rooms, dropbox, gradebook, syllabus, and content items (course units and subunits). Levels of use by the expert of these features and course structure were recorded and compared.

In order to ensure a high inter-rater reliability between the researchers, an Excel template was created. It consists of three worksheets that include: (a) basic course statistics, participation requirements, and use for each tool; (b) course structure information such as the number of units, unit elements, file types, etc.; and (c) threaded discussion activity including the number of posts, the number of topics based on individual or group contributions, and required versus optional topics.

The template was developed by the researchers who analyzed one course each. Once the trial analysis was completed, they verified the data and reached a consensus on which elements to include in the analysis and on the process of data collection. The template was then modified based on agreements reached and used in the analysis of all selected courses. Furthermore, in order to ensure consistency, each course was analyzed by the same person. Once the process was complete, researchers collaboratively verified randomly selected data elements by revisiting archived courses; they examined collected data and evaluated trends in changes over time.

In order to support the rationale of selecting the professor and verify that he is a quality instructor, student course evaluations were analyzed to ascertain the levels of student satisfaction with the instructor and courses. Researchers only included evaluations between 2004 and 2007 due to the fact that the evaluation questionnaire had been revised significantly in 2004. Data are on an interval scale on which equal intervals are provided from an arbitrary origin. Items on a 5-point Likert-type scale are reported in descriptive measures such as frequencies and means. Qualitative data generated from open-ended questions were coded and summarized as explained by Dick, Carey, and Carey (2001).

Finally, the expert participated in a 60-minute interview conducted by one of the researchers. The semi-structured interview schedule included nine questions pertaining to instructor experience, awareness of trends, and rationale for tools and elements utilized in the courses. The data collected during the interview with the instructor were summarized in a narrative (Creswell, 1998; Flick, 1998) to give readers an in-depth description about the instructor’s background, perceptions, rationale for his use of instructional strategies and his utilization of tools to facilitate the students’ learning process. The instructor reviewed the narrative to ensure that his responses were represented and interpreted correctly.

Results

Similarities Between Courses

The number of enrolled students varied significantly by course or semester. All courses offered were structured by units. These units contained all instructional materials, each one beginning with an introduction or an instructional overview of the topic and assignments contained within the units. The content in all courses included readings, links, discussions, and assignments. File types included an integrated CMS text/multimedia tool, Microsoft Word documents, and images (either .jpg or .gif files). In addition, the courses included a Course Home section that consisted of informational material such as syllabus, announcements, student group assignments, and a general threaded discussion area.

The instructor chose to use the integrated template to create the syllabus for all four courses instead of downloading a Microsoft Word file. The dropbox, when used, was used for major assignments to be submitted in their final stages. Group chat rooms were created but were never used. The threaded discussion forum, gradebook, web bibliography, document sharing, and announcement tools were used in all courses; however, the levels of use varied considerably between courses and semesters.

Examination of Courses Over Time

Instructional design. The ID course was taught seven times, and the number of assignments changed over time (Table 1). In the first four semesters, the completion of seven or eight assignments was required. In the following semesters, the instructor included only one exam and four exercises (e.g., case studies). At the same time he began using the integrated examination tool. Over time, the instructor reduced the number of units from 13 to eight and, therefore, the course content changed. Fewer images and different file types such as Word, Portable Document, and video files were included in the first four semesters.

Table 1. Course Changes: Instructional Design

The number of threaded discussion topics varied greatly with time. The proportion of group discussion vs. individual participation increased during the last three semesters. The number of required participation topics changed over time, and the participation level of the instructor ranged from 2.20% to 8.62%.

Distance education technologies. The DE course was taught twice. Students had to complete five major assignments during both semesters, and the instructor provided feedback with the gradebook. In fall 2006, the instructor posted fewer announcements while the use of the document sharing tool through which students share their projects almost doubled compared to 2005. The chat room structure and calendar use remained unchanged; however, the instructor added a third dropbox in 2006.

The unit structure remained the same and changes to the course content were minor. The instructor added to the existing instructional content to provide additional information about a particular assignment, and he deleted two images and added two PDFs. The mean of student posts increased significantly even though the number of threaded discussion topics, their level of voluntary and required participation, and their structure in terms of small group and individual interaction remained unchanged (Table 2).

Table 2. Course Changes: Distance Education Technologies

Instructional technology. The IT course was taught twice in 2003. The number of assignments (one project and three exercises), units, and announcements did not change. The instructor did not use the calendar nor did he make changes to the unit structure. The number of images, PowerPoint, Portable Document (PDF) and audio files remained unchanged; however, the instructor increased the number of Word documents in fall 2003. During the first semester the instructor used the dropbox for one assignment but chose not to use it in the second semester. The gradebook was used during both semesters with more extensive comments provided in fall 2003.

The instructor reduced the number of discussion topics in the second semester. No group discussions were used and participation requirements remained the same. Instructor participation dropped from 6.6% to 1.5%; however, the mean for student posts remained stable (Table 3).

Table 3. Course Changes: Instructional Technology

Theory of change. The TC course was taught during four 6-week summer semesters. The assignments consisted of two case studies that students submitted through the dropbox in the final stages, and the participation grade was 50%. The calendar was included, and chat rooms had been created but they were not used. The instructor did not make any changes to the course structure. The course consisted of six units with introductions, readings, and discussions.

The course did not include many file types other than the integrated multimedia tool; however, the instructor added a few more Word and PDF documents over time. Interestingly, the instructor reduced the number of dropboxes. The gradebook was used to provide grades and feedback for all or some of the major assignments depending on the semester. The level of use of the web bibliography, document sharing tool, and announcements varied between semesters.

Overall, the number of discussion topics increased. The percentage of topics that required student participation remained stable. One major change in this course was that all discussions required individual student postings in the first semester the course was analyzed. In subsequent years, the majority of discussions were group based. The mean of student postings continually decreased over the four year period; the instructor participation rate fluctuated between 0.7% and 1.8% (Table 4).

Table 4.Course Changes: Theory of Change

Student Evaluations

All course ratings for the expert instructor remained high; the means of the Likert-scale ratings varied slightly between courses and semesters. Categories reported on the evaluation are: (a) resources provided, (b) communication, (c) faculty/student interaction, (d) assignments, (e) instructional materials and methods, (f) course outcomes, and (g) student involvement. Overall, students rated the instructor as effective with high ratings in all categories. The instructor received the highest ratings in the categories of resources provided and instructional materials and methods. These results show that students were satisfied with the content provided and instructional strategies employed in those courses.

The qualitative analysis reveals that most of the students’ positive comments related to course organization and content. Students enjoyed reading the textbooks and were pleased with the resources and links provided by the instructor. Assignments were described as challenging and applicable, and they appreciated the inclusion of images. Feedback and communication by the instructor was perceived as responsive and helpful. Whereas some students commented that they appreciated the instructor taking on the role of a facilitator and his use of student-led pedagogy, some students indicated they would have appreciated more feedback and instructor interaction.

Expert Interview

The expert participated in a 60-minute semi-structured interview. The interview was conducted to obtain information regarding his background in addition to understanding his rationale for changes he implemented. According to him, he began using the current CMS for online delivery in 1999. However, he has been involved in online course delivery since 1993 when he used a mixed-method approach using electronic mailing lists and other tools.

His favorite instructional strategies vary somewhat with the learning task. In a large campus-based course, for example, he uses lectures and relies mostly on the textbook because these courses tend to be information driven. His preference, however, is to use a seminar-oriented approach which includes instructional events within online units, thought exercises, discussions, and so forth. Rather than delivering content, he prefers to direct students to resources and to promote discussion among students in order to foster self-directed learning and self reliance.

His favorite CMS tools are asynchronous communication tools because they are not only convenient for students but allow time for reflection. In addition, they provide a written record for the instructor. According to him, “without threaded discussions, online learning would not exist.” Other tools that he considers valuable are sharing tools that all course participants can use to upload documents for everyone in the course to access, a Web bibliography where everyone can post links to Web resources, and an assignment submission tool. His least favorite tool is the chat room because it defeats the idea of online delivery due to inconvenience. Furthermore, chat rooms do not allow enough time for thought and reflection. Chat sessions are difficult to schedule, monitor, and keep up with. In addition, he questions how its use contributes to the acquisition of knowledge but states that its use might help with the building of a learning community.

His expectations for student participation focus on levels of engagement. Learners are expected to log into the course frequently throughout the week in order to read posts and respond to peers. He bases his evaluation of consistent engagement on the frequency and quality of postings instead of expecting a certain number of students’ postings. Expectations are based on traditional hours spent in and out of the classroom for each semester hour in which students are enrolled.

As previously mentioned, the average number of posts per student varies within some of the courses that were analyzed. He explains the variance by pointing out that there are some students who are lurkers and who do not post often. Students may spend a significant amount of time reading and reflecting on materials and posts throughout the week, but they may not contribute to discussions on a daily basis. The CMS has an area where instructors can access information regarding how much time students spend in the course, and he uses this management tool when he questions if a student is engaged. Other reasons may include that students feel uncomfortable, feel threatened, or may not have previous experience in online coursework. Another variance is the level of instructor participation in different semesters. This finding was not surprising to him because it is a reflection of his workload which varies throughout the semester due to balancing several responsibilities—teaching, advising, providing service, and conducting research in addition to carrying out administrative duties.

Many of the online courses are heavily text based. When asked if he would like to include more rich-media elements in his courses, his answer was yes. More specifically, he wants to include audio files because their inclusion holds promise—due to the fact that it is doable. In addition, he would like to include additional images. He would also like to include video files but he expressed concern about large file sizes, time requirements in order to produce quality videos, and protecting the privacy of students.

According to the instructor, his courses have remained fairly stable over the years, and he has made minor revisions to his courses. Changes he implemented include increasing the self-directedness of learners. He now provides a more structured environment by creating units within the course. Instructional events within units place more focus on the students and less on the instructor. He created a question and answer area in the form of a threaded discussion forum in order to incorporate informal conversations ― or what he terms hall talk.

The last interview question included a list of course changes that the researchers observed and the instructor’s reasons for implementing these changes. His reasons for decreasing the number of assignments included his workload and the realization that students do not necessarily need to be working on a major assignment or exercise every week in order to learn. In the past, he had included more chat sessions and whole-class asynchronous discussions. Then, he began assigning students to small groups to promote interaction and to help build a learning community. In small groups, students feel more comfortable participating and they do not duplicate responses as often as in larger discussions. He felt that students got lost in whole class discussions.

Reasons for the addition of Word and PDF documents include that these files are now more readily available than when he began delivering courses online. More journal articles are now available in full-text version and can be accessed easily. Articles authored by faculty members who teach courses can now be included in online readings which provide students with more information about the instructor and his or her work.

Limitations and Recommendations for Future Research

Several limitations need to be pointed out. First, only one instructor at one university was selected as a participant, and courses included in the analysis were taught by that instructor. Second, only one CMS was used by the instructor and, therefore, tools in eCollege only were examined in the study. Even though tools provided in eCollege are similar to other CMSs, it is possible that other features integrated in other CMSs could affect results. Third, as students participate in online courses, they may become for familiar with the CMS tools and their comfort levels may increase over time. However, in this particular program students are admitted every semester or individuals may enroll in courses as nondegree seeking students. Therefore, the reader should be careful about generalizing results of this study. Future research could be conducted in which online instructors from different geographical areas who use different CMSs and tools are included in a sample.

Discussion and Conclusion

Some of the changes over time implemented by the expert were motivated by nonpedagogical issues such as workload. However, several changes such as community-building and self-directed learning strategies were linked to pedagogical reasons aimed at improving online teaching and learning. In general, there was a trend of increasing the number of threaded discussion topics. The number of group-based and required discussion topics increased in most of the courses that were examined. Perhaps that is one of the reasons why the mean number of student postings increased in some of these courses. With the strategy of moving more and more from individual to group-based discussions in his courses, the instructor provides a consistent support structure throughout the semester and builds community within smaller groups in order to increase the comfort level of participants.

The expert encourages student participation and explicitly states in the syllabi what constitutes an acceptable level of participation. Several topics per week are provided and students have a choice in responding to questions that are most relevant to them. He also allows students to interact in informal ways in nonmandatory threaded discussion topics titled Humor or Q&A. These strategies have been identified as effective approaches of exemplary online instructors by Lewis and Abdul-Hamid (2006). According to Lee and Gibson (2003), computer-mediated interaction can promote self-directed learning ― an element valued by many adult learners (Knowles & Associates, 1984; Knowles, Holton, & Swanson, 1998).

In order to enhance text-based messages and support visual learners, the instructor included more images over time. Furthermore, he added files such as Word or PDF files to make content and files more accessible and user friendly. Internal files such as Word or PDF files are easier to download, read, and print than files created with the internal multimedia tool that can be used to create HTML files. The internal examination tool was added at a later time to enable automated assessment of students in order to provide learners with more timely feedback.

Maintaining online courses has proven to be time consuming for instructors. Results of the study support the notion that instructors can be successful in teaching online by making minor modifications to their courses over time. Making minor changes over time makes the task not only more manageable but may assist instructors in using a more reflective approach in the design of courses that are more engaging and more inclusive of diverse learners. This strategy also allows instructors to implement technology-based changes as new technologies become available, integrate newly available research-based information about teaching and learning in the online environment, and to adjust to changing learner expectations.

References

Allen, I. E., & Seaman, J. (2008). Staying the course: Online education in the United States, 2008. Needham, MA: The Sloan Consortium. Retrieved March 3, 2009, from http://www.sloan-c.org/publications/survey/pdf/staying_the_course.pdf

Association for the Study of Higher Education. (2006). Research on the factors of online learning. In ASHE, Higher education report, 32(1), pp. 37-87. Hoboken, NJ: Wiley & Sons. Retrieved July 25, 2007, from EBSCOHost database.

Beldarrain, Y. (2006). Distance education trends: Integrating new technologies to foster student interaction and collaboration. Distance Education, 27(2), 139-153.

Berge, Z. L. (2005). Taking the distance out of distance education. In G. Kearsley (Ed.), Online learning: Personal reflections on the transformation of education (pp. 12-25). Englewood Cliffs, NJ: Educational Technologies Publications.

Boettcher, J. V. (2004). Online course development: What does it cost? Syllabus. Retrieved May 2, 2008, from http://www.campustechnology.com/article.aspx?aid=39863

Creswell, J. W. (1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage.

Dick, W., Carey, L., & Carey, J. O. (2001). The systematic design of instruction (5th ed.). New York: Longman.

Flick, U. (1998). An introduction to qualitative research. Thousand Oaks, CA: Sage Publications Inc.

Kidney, G. W., & Puckett, E. G. (2003). Rediscovering first principles through online learning. Quarterly Review of Distance Education, 4(3), 203-212.

Knowles, M. S., & Associates. (1984). Andragogy in action: Applying modern principles of adult learning. San Francisco: Jossey-Bass.

Knowles, M. S., Holton, E. F., & Swanson, R. A. (1998). The adult learner: The definite classic in adult education and human resource development (5th ed.). Houston, TX: Gulf Professional Publishing.

Koszalka, T. A., & Ganesan, R. (2004). Designing online courses: A taxonomy to strategic use of features available in course management systems (CMS) in distance education. Distance Education, 25(2), 243-256.

Lee, J., & Gibson, C. C. (2003). Developing self-direction in an online course through computer-mediated interaction. American Journal of Distance Education, 17(3), 173-187.

Lee, J., & Hirumi, A. (2004, October). Analysis of essential skills and knowledge for teaching online. Paper presented at the annual meeting of the Association for Educational Communications and Technology, Chicago, IL. (ERIC Document Reproduction Services No. ED485021)

Lewis, C. C., & Abdul-Hamid, H. (2006). Implementing effective online teaching practices: Voices of exemplary faculty. Innovative Higher Education, 31(2), 83-98.

Moore, M. G., & Kearsley, G. (1996). Distance education: A systems view. Belmont, CA: Wadsworth Publishing Company.

Morgan, G. (2003). Faculty use of course management systems. Boulder, CO: Educause Center for Applied Research. Retrieved May 3, 2008, from http://www.educause.edu/ir/library/pdf/ers0302/rs/ers0302w.pdf

Oblinger, D. G., & Hawkins, B. L. (2006). The myth about online course development. Educause Review, 41(1), 14-15.

Palloff, R. M., & Pratt, K. (2001). Lessons from the cyberspace classroom: The realities of online teaching. San Francisco, CA: Jossey-Bass.

Rogerson-Revell, P. (2007). Directions in e-learning tools and technologies and their relevance to online distance language education. Open Learning, 22(1), 57-74.

Shaw, S., & Polovina, S. (1999). Practical experiences of, and lessons learnt from, Internet technologies in higher education. Educational Technology & Society, 2(3), 16-24. Retrieved July 28, 2008, from http://www.ifets.info/journals/2_3/stephen_shaw.pdf

Simmons, S., Jones, W., & Silver, S. (2004). Making the transition from face-to-face to cyberspace. TechTrends, 48(5), 50-85.

Smith, P. L., & Ragan, T. J. (2005). Instructional design (3rd ed.). Hoboken, NJ: Wiley.

Wilson, B. G. (2002). Trends and futures of education: Implications for distance education. Quarterly Review of Distance Education, 3(1), 91-103.

Yang, Z., & Liu, Q. (2007). Research and development of web-based virtual online classroom. Computers & Education, 48, 171-184.

Appendix: Combined Data

Note. S-Spring, Su-Summer, F-Fall; N/U-Not used, U-Used; N/S-not specified; MA-major assignments

This work is licensed under a
Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License