Introduction

Since commercially available Course Management Systems ( CMS) became widely available in the 1990s, a dialogue about their use has grown at a steady pace (Ullman & Rabinowitz, 2004). There are many resources that provide help in determining what features should be considered critical for a CMS (Hall, 2002; Ullman & Rabinowitz, 2004). Some shed light on specific features valued by the target users—faculty and students—as well as by administrators (Whitmyer, 2000; Jafari, McGee, & Carmean, 2006; Morgan, 2003). Other resources detail best practices in the process of selection and implementation, for example the lists of tips to facilitate both processes provided by The ELearning Guild (2006a, b). The dialogue even focuses on product comparisons by brand to allow informed decision-making for institutions of higher education (Marsh, et al. 1999; Whitmyer, 2000; Van de Pol, 2001; Jafari et al., 2006). However, little attention has been devoted to the impact of CMS upgrades and/or transitions from one product to another at the trench level—the impact on faculty grappling with repeated and constant evolution so inherent in the technology.

In an article on CMS change, Smart and Meyer (2005) conducted a study on faculty attitudes and perceptions about a conversion from one CMS product to another in the North Dakota University System. The study was not conducted on a large population of faculty nor based on experiences in their own courses. Instead, faculty provided input on the impact of the conversion of a training course used by the university’s IT group. However, two of Smart and Meyer’s conclusions correlate with the experiences chronicled in this article: “faculty—who would need to restore course content that did not convert intact and accurately—will face increased workloads and frustration should an institution choose to change from one CMS to another” (p. 69). Additionally, they conclude that “Faculty may not be the brakes to change that some make them out to be, but they are legitimately concerned about the time and effort needed to make the transition a success for their courses and students” (p. 70). It is for these very concerns that the authors address and offer resolutions.

In EDUCAUSE’s recent release of its “Top Ten IT Issues, 2008” (Allison, et al., May/June 2008), change management has made the list of institutional IT concerns for the first time. The atmosphere is ripe for focusing at least part of the dialogue on change management. One of its dimensions is centered on “developing a process for handling IT changes that are made on a regular basis (e.g., patches, upgrades, replacements) and that can be very disruptive if there is no change management process in place” (pp. 38, 40). This change can occur simply as part of the product’s evolving sophistication or when “institutions want a centralized system to assure uniformity of appearances and, if possible, to manage many other aspects of the course and overall administration” (Marsh, 1999). A sound management process or plan, then, becomes a good starting point for minimizing the impact of CMS changes on the population of key players, the faculty.

Whether to move to a new version or to a new product is not the issue because change is inherent in the use of technology (Finkelstein & Pittinsky, 2003; Sclater, 2008). Morgan (2003) reports a comment from one faculty participant in a study conducted at the University of Wisconsin System: “That we are still at the ‘Model T’ stage of CMS use.” The metaphor for the level and speed of change/growth is clear. Matthew Pittinsky, co-founder and chairman of Blackboard, Inc. and well-known leader in e-learning, looks forward to “booths at future Modern Language Association or American Psychological Association meetings” rich in new applications for faculty to integrate in their courses (Finkelstein & Pittinsky, 2003). The purpose of this article is to serve as an awareness-building overview of the situation facing faculty and as a guide for institutions considering or planning this type of change. This case study and sharing of lessons learned at the authors’ institution, Middle Tennessee State University, can be a starting point for facilitating and developing their own process management plan—it is all about managing the change effectively.

Background: The Dynamics of CMS Use and Change

The use of a course management system at Middle Tennessee State University (MTSU) has been evolving since the late 1990’s. At that time, having recognized the potential of the software, the campus’s Information Technology Division (ITD) began an investigation of various applications. Understanding that the choice should be faculty driven, ITD convened a faculty committee who reviewed three leading products. After a semester of review, the committee made its recommendation. The campus negotiated a contract, developed a training program, and hosted and supported the system. Although initial faculty use was minimal, a slow but steady period of growth followed as more faculty began to use the software each semester.

In September of 2000, the Tennessee Board of Regents (TBR) approved the Regents Online Degree Program (RODP). TBR is the sixth largest Board of Regents in the United States. It governs 47 colleges, universities, community colleges, and technical centers that offer degrees ranging from basic certificates to doctorates; its schools serve over 185,000 full- and part-time students. The RODP was designed to offer fully online degree programs and certificates, providing courses that are developed and delivered by faculty at all its institutions. The TBR staff developing the program chose a different CMS as the delivery mechanism for those RODP courses. This led to a dilemma for MTSU; faculty and students would now be using a different system for their RODP courses, and ITD support staff would be required to support two systems. As that was neither pedagogically sound nor financially supportable, MTSU’s ITD decided to no longer provide the initial CMS as an option and negotiated its own instance of the TBR-supported CMS. In their negotiations, TBR had written in the contract that all its institutions could purchase their own CMS license off a state contract.

The new CMS was introduced to the MTSU campus, and faculty using the old CMS were given a semester to make the transition. Naturally, the faculty were unhappy about this change because users had devoted a good deal of time and effort learning the original CMS. They now needed to learn a new CMS platform that was quite different from the one with which they were already familiar. This experience provided the campus with an important first lesson on the changing dynamics of CMS use, and campus support staff were determined to manage the use and transition process better in the future.

Throughout the next five years, MTSU faculty use of the CMS grew as more instructors adopted it for their on-ground, blended, and online courses. By 2005, an estimated 40% of faculty were using the system for some degree of Web course support. The number of fully online courses grew as well. During this time, the CMS had a version change and several upgrades. Each of these changes was managed by informing faculty in advance and conducting system training sessions and workshops.

In 2006, the TBR-negotiated contract with its CMS vendor reached its expiration date. Per state contract and purchasing law, a new CMS license required a process of calling for vendor proposals and a bidding procedure. Due to the introduction of new CMS platforms and confusion in the CMS market at the time (i.e., the pending Blackboard/WebCT merger), the TBR received permission for a one-year contract extension from the TN State Comptroller’s office prior to initiating the Request for Proposal (RFP) process. TBR academic and information technology administrators also decided to investigate a new installation model involving a state-wide instance (all TBR institutions using a single installation) hosted by the vendor.

To facilitate the RFP process, TBR developed a system-wide project plan. The Technology Enhanced Teaching and Learning Environment Committee (TETLE) was established under a TBR leadership team that was headed by the Vice Chancellor for Academic Affairs. This committee, with representation by the majority of TBR institutions, was charged with developing an RFP for the next CMS. Committee members first defined a number of criteria that were required, necessary, and desirable in a CMS and then began to develop a draft of the proposal template. After initial discussions, an RFP-writing subcommittee produced the final document.

The vendor process began with the release of a Request for Information (RFI), to which eight CMS vendors replied. After the Request for Proposals (RFP) was released, three vendors submitted proposals. An RFP evaluation subcommittee reviewed the functional components of the three proposals and determined that each met the minimal requirements. According to the state’s purchase/contract procedures, the financial component was determined separately by the purchasing office. The three vendors were invited to present their product at three vendor shows (one in each geographic area of the state); each institution was assigned a location and invited up to ten faculty to participate and provide feedback to a member of the evaluation subcommittee. Using a rubric that scored the various evaluation criteria and faculty feedback, the members of the evaluation subcommittee provided feedback to the TBR purchasing director and leadership team, who then reviewed the ranking from the subcommittee and the financial details before making a final recommendation. A contract with the winning vendor (a different company than the current CMS) was then negotiated to begin 1 January 2007.

Because the previous CMS contract was scheduled to end 12 December 2007, this provided for a one-year transition period. Each school was subsequently charged with developing and implementing a local conversion plan. The conversion, this time, required managing a much larger project – more faculty, more CMS complexity, more variety in the resources stored in the CMS, more complexity in the integration of the CMS with other campus resources, and a wider consideration of the CMS as a mission critical application for the university.

Background: MTSU’s Transition Process

In November 2007, academic and technology staff from all TBR institutions met to discuss issues and concerns involved with making the transition to the new CMS. As each school had historically managed its own resources, a centrally hosted and managed CMS presented numerous challenges due in part to the variety of support models employed at the institutions. To facilitate the transition process, a campus chair was appointed at each institution along with staff for system administration and campus trainer roles. At MTSU, the Dean of Continuing Education and Distance Learning (CEDL) was appointed by the Chief Academic Officer ( CAO) to function as campus chair. He relied on the ITD Director of Academic and Instructional Technology Services to manage the transition process as ITD had historically been responsible for system administration and faculty and student support for instructional technologies.

During previous conversions and upgrades, the support staff had learned the value of change management (at least somewhat). Thus, the approach centered on the intent to involve various stakeholders and gain consensus about the plan from the beginning. The transition process began with the appointment of a transition/conversion team which consisted of ITD staff, the president of the faculty senate, one faculty member from each of the university’s five colleges, and representatives from the library, Disabled Student Services, and CEDL. This group met December 2006 through May 2007 to determine and approve a transition plan and to discuss issues surrounding the conversion process. The team developed a timeline that detailed (1) the set up of test and production servers by the host vendor, (2) system administration and support staff training, (3) procedures for phasing out the previous CMS, (4) piloting of courses within the new CMS, and (5) the official launching of the new CMS. Simultaneous processes were determined to manage nightly batch uploading of enrollments/unenrollments into both systems until the final phase-out of the old system. The transition plan included:

The transition team also made decisions on CMS roles and permissions and whether all or selected courses would have a new CMS presence. The team decided to recommend that CMS courses be created for every credit course listed in the student information system (Banner Student). The team also decided that rather than moving old courses “en mass” to the new system, faculty would be encouraged and assisted to revise courses, clean out old materials, and move materials thoughtfully into the new CMS.

A large part of the plan involved a multi-approach communications model. Methods included a statewide TBR conversion wiki, an online repository of learning and conversion resources, periodic faculty emails, newsletters, handouts at large faculty meetings, and an open forum at a Faculty Senate meeting. The transition team also included periodic reports to senior academic and technology administrators and technology committees on campus.

Problems Encountered and Lessons Learned

As the transition program was implemented, ITD staff monitored the types of problems encountered, the help requests processed, and other issues. Problems encountered during the transition fell into two categories – technical and faculty issues. One of the technical issues was operating differences in the systems that caused files to display differently or that led to the need to redesign learning activities. Many course materials had not been created using interoperability standards which caused file corruptions upon transfer. In addition, there were upload issues that caused timeouts for large files and connectivity issues that led to a slowdown of processes and a difference in the way that various tools functioned in the new system. These technical issues required a good deal of troubleshooting time on the part of ITD staff and also generated a good deal of frustration on the part of the faculty members affected. The decision to minimize the bulk uploading of courses and to recommend a more individualized transfer methodology, we felt, proved correct as it made possible addressing these technical issues before student contact with the course. Had the transfer been done “en mass,” these problems may not have been discovered until course delivery, and faculty may not have had the skills to counter the resulting problems.

Given the many barriers that faculty typically encounter when working with instructional technologies (see Brinthaupt, Clayton, & Draude, 2009), it was not a surprise that there were several faculty-related issues that emerged in the transition process. One of the issues was the lack of compensation for the time required to convert courses. The university administration had decided at the beginning of the transition process that compensation was not available. In addition, faculty had to find time to revisit and redesign course materials and often lacked an understanding of instructional design. For some faculty, making the transition in a timely fashion was a low priority or a low-interest task. These faculty did not anticipate the time required to make their conversion and, as such, were frustrated when limited time remained.

Lessons learned by the transition team included new technical recommendations, the identification of “best practices,” and the need for an increased emphasis on communication. On the technical side, the differences in the way certain learning resources converted to the new system led to increased training and encouragement for staff and faculty to create materials based on interoperability standards (thus increasing cross platform compatibility). “Best practices” for online courses were incorporated into training materials and workshops. Support staff had found that communication was a key factor and, even though they had tried multiple communication methods, not all faculty and administrators were aware of the reasons for or ofthe processes of changing. Additional communication methods, such as an online learning community for CMS faculty users, have since been implemented.

Assessing the Success of the Transition Process

In order to assess faculty members’ perceptions of the transition process, the authors distributed a survey to all full-time instructors on campus. The survey assessed the extent to which instructors were informed about the need for the transition, whether they understood what they needed to do to make the transition, and their feelings about how the transition process had been managed.

Participants . All full-time faculty (N = 930) were sent the survey; however, we estimated that only approximately 50% of those faculty were engaged with the previous CMS and would complete the survey. Based on that estimate of regular faculty CMS users (N = 465), the authors received 105 responses (approximately 23% return rate). Of these, 6 respondents did not complete the CMS transition items, leaving a final sample of 99 instructors.

Materials and Procedure. The CMS transition survey consisted of two parts. The first part provided several items pertaining to the faculty member’s use of the previous and new CMS (1 = never, 5 = very often), whether or not they taught fully online and hybrid courses (yes/no), as well as gender, number of years teaching at the college level, and their college or division within the university. The second part of the survey consisted of eight 5-point, Likert-type items pertaining to perceptions and evaluation of the transition process (1 = strongly disagree, 5 = strongly agree). Table 1 provides the wording for these items.

The authors distributed the survey through a campus mass mailing at the beginning of the fall 2008 semester, one year after the occurrence of the transition to the new CMS. By distributing the survey at the one-year point in the transition process, they were able to assess its impact across the previous fall, spring, and summer terms. The final sample consisted of instructors who had returned the survey by approximately three weeks after the mass mailing.

Results and Discussion

Respondents (58% women, 41% men; years teaching: M = 15.87, SD = 9.35) were distributed approximately equally across the university’s five colleges. Respondents indicated moderate levels of usage of the previous CMS (M = 3.11, SD = 1.59) and the new one (M = 3.55, SD = 1.48). Thirty-three percent of the respondents reported teaching a fully online university course; 53% of respondents reported teaching a hybrid or blended university course. These data suggested that, while the return rate was low, the respondents represented a broad sampling of areas and previous experience and were representative of the university’s overall faculty. Finally, 69% of respondents reported that they had previously attended a training, workshop, or demonstration using the new CMS.

Table 1 shows the descriptive statistics for the faculty members’ responses to the transition process items. The authors conducted one-sample t-tests on these items, comparing the observed mean responses to the scale midpoint (3). As the table indicates, faculty members reported that they were well-informed about the transition, that they were frustrated with having to learn a new system, that the campus support staff was very helpful, that the time given for the transition was appropriate, that they were able to obtain help quickly when needed, and that, overall, their transition was a smooth one. Non-significant means suggested that faculty tended to not fully understand the need for the transition, but that they did not mind having to learn the new CMS.

Two additional sets of analyses were conducted. First, the authors compared those instructors who did and did not teach online on their perceptions of the transition process. These analyses indicated that online instructors rated themselves as more informed about the transition (t(94) = 2.20, p < .05, d = .05), saw the support staff as more helpful (t(95) = 3.71, p < .001, d = .13), and reported receiving help more quickly when they needed it (t(92) = 2.09, p < .05, d = .05). Second, the authors compared instructors who had and had not attended a new CMS training, workshop, or demonstration. These analyses showed that attendees felt more informed about what they needed to do for the transition (t(93) = 3.55, p < .001, d = .12), rated the support staff as more helpful (t(94) = 4.78, p < .001, d = .20), were more likely to feel that the time for the transition was appropriate (t(93) = 2.33, p < .05, d = .06), reported receiving help more quickly when they needed it (t(91) = 3.67, p < .001, d = .13), and felt that the transition was smoother (t(90) = 2.56, p < .05, d = .07).  

Table 1.Descriptive statistics for transition process survey items

Forward Move: Mentally Preparing for the Next Transition

Because of the nature of course management systems, there is literally no rest for the weary. With the emergence of new technologies, updates to existing programs, and changes in the needs and abilities of instructors, future CMS transitions are likely to be inevitable. By taking a careful approach to informing faculty and helping them make the transition, the authors hope to have begun to create a better understanding of the need (and inevitability) of this kind of transition. The data from the faculty survey suggest that there is still work to be done. For example, despite all the efforts of the transition team and its well-designed transition process, faculty still did not appear to sufficiently understand the need for making the transition to a new CMS, and, not surprisingly, they were frustrated with having to change. Since the survey results did suggest that faculty who attended workshops or training sessions had more favorable experiences with the transition process, this will continue to be a hallmark of the services provided on this campus.

As much as CMSs are likely to continue to evolve, how faculty, technology providers, and support staff communicate their needs and priorities must also continue to evolve. The challenge is to manage expectations, activities, and results. If a carefully constructed transition plan is implemented, campuses should be able to achieve smooth, or at least minimally disruptive, changes. In the present case, the authors believe that their campus was successful in meeting this challenge.

References

Allison, D.H., P.B. DeBlois, and the EDUCAUSE Current Issues Committee. (May/June 2008). Top 10 IT issues 2008. EDUCAUSE Review, pp. 37-61.

Angelo, J.M. (2006). New Lessons in course management. University Business: The Magazine for College and University Administrators. Retrieved May 2, 2008 from http://universitybusiness.ccsct.com/page.cfm?p=616 .

Brinthaupt, T. M., Clayton, M. A., & Draude, B. J. (2009). Barriers to and strategies for faculty integration of IT. In P. Rogers, G. Berg, J. Boettcher, C. Howard, L. Justice, & K Schenk (Eds.), Encyclopedia of Distance Learning (2 nd ed. pp. 138-145). Hershey, PA: IGI Global.

The ELearning Guild. (November 2006). 339 Tips on the Implementation of an LMS or LCMS. Retrieved April 28, 2008 from http://www.elearningguild.com/content.cfm?selection=doc.538.

---. (December 2006). Tips on the Selection of an LMS or LCMS. Retrieved June 15, 2008 from http://www.elearningguild.com/content.cfm?selection=doc.540.

Finkelstein, J., and M. Pittinsky. (2003). The Evolving role of course management systems providers in the transformation of education: An interview with Blackboard’s Matthew Pittinsky. The Technology Source. Retrieved April 28, 2008 from http://technologysource.org/article/evolving_role_of_course_management_system_providers_in_the_transformation_of_education/.

Hall, J. (2003). Assessing learning management systems. Chief Learning Officer. Retrieved April 28, 2008 from http://www.clomedia.com/content/templates/clo_feature.asp?articleid=91&zoneid=29 .

Jafari, A., P. McGee, and C. Carmean. (Jul/Aug 2006). Managing courses defining learning: What faculty, students, and administrators want. EDUCAUSE Review, Volume 41, Number 4, pp. 50-52, 13 pp.

Marsh, G. E., A. C. McFadden, and B. J. Price. (Fall 1999). An overview of online educational delivery applications. Online Journal of Distance Learning Administration, Volume II, Number II. Retrieved April 28, 2008 from http://www.westga.edu/!distance/marsh23.html.

Morgan, G. (May 2003). Faculty use of course management systems. ECAR Key Findings. EDUCAUSE Center for Applied Research, Volume 2. Retrieved May 20, 2008 from http://net.educause.edu/ir/library/pdf/ecar_so/ers/ers0302/ekf0302.pdf.

Sclater, N. (June 2008). Web 2.0, personal learning environments and the future of learning management systems. EDUCAUSE Center for Applied Research, Volume 2008, Issue 13. Retrieved June 27, 2008 from http://connect.educause.edu/Library/ECAR/Web20PersonalLearningEnvi/46952.

Smart, K. A., and K. A. Meyer. (2005). Changing course management systems: Lessons learned. EDUCAUSE Quarterly. 2: 68-70.

Ullman, C., and M. Rabinowitz. (Oct. 2004). Course management systems and the reinvention of instruction. T.H.E. Journal. Retrieved June 28, 2008 from http://www.thejournal.com/articles/17014.

Van de Pol, J. (May/June 2001). A Look at course management systems. IT Times. Retrieved June 15, 2008 from http://ittimes.ucdavis.edu/june2001/cms.html .

Whitmyer, C. (2000). Comparative features analysis of learning course management software. University of the Future, LLC. Retrieved April 28, 2008 from http://www.futureu.com/cmscomp/