MERLOT Journal of Online Learning and Teaching
Vol. 5, No. 2, June 2009


Developing an Online Program in Computer Information Systems
Using Quality Matters™ Standards

Lissa Pollacia

Jack Russell

Barbara Russell

CIS Dept., College of Business
Northwestern State University
Natchitoches, LA 71497 USA


The dramatic decline in the number of CIS majors since the year 2000 has focused attention on the need to develop new, innovative, and high quality CIS programs to attract more students. This paper describes a multi-disciplinary minor in CIS that has been recently developed for online delivery. To assure quality of the courses in the CIS minor, the authors participated in training to prepare for development of the new courses. These new courses had to meet the standards set out in the Quality Matters (QM) Rubric by MarylandOnline, Inc. An overview of the QM standards and Rubric and how it is used to measure the quality of online courses is described.

Keywords :Declining CIS enrollments, CIS minor, multi-disciplinary CIS minor, Quality Matters Rubric, QM Standards

The problem: declining CIS enrollments

All evidence indicates that the number of students who are electing to major in science, technology, engineering, and mathematics (STEM) disciplines has declined dramatically since the year 2000. Particularly hard hit are all computing-related fields, such as Computer Information Systems (CIS), Computer Science (CS), and Management Information Systems (MIS) (A Red Flag, 2006). After six years of steady declines, the number of new CS majors in Fall 2006 was only half of what it was in the Fall of 2000. The decrease in computing enrollments coupled with a large number of pending retirements is expected to result in a substantial shortage of skilled workers in the U.S. in the next few years (Bureau, 2008, Kessler, 2005). Figure 1 illustrates the research on Baccalaureate degree production from 1995 to 2007 (Vesgo, 2007).

Reasons for the decline

In a recent survey of faculty at U.S. universities that offer a bachelors degree in Computer Information Systems, Management Information Systems, or Information Technology (Lenox, Woratschek, & Davis, 2005), seventy-six percent (76.1%) of the respondents reported that enrollment had decreased in the past two years. When questioned about the reasons as to why they felt that enrollment had declined at their institutions, sixty-seven percent (67%) blamed the outsourcing of jobs, sixty percent (60%) stated the economy in general, sixty percent (60%) cited the dot com failure, forty percent (40%) indicated that the drop was cyclical in nature, and twenty-seven percent (27%) stated it was due to decline in students’ analytical abilities.

Figure 1 . BS Degree Production from 1995 to 2007

One of the authors (LP) own recent research has provided some information concerning the cause for the enrollment decline (Lomerson & Pollacia, 2005; Pollacia & Lomerson, 2006). In a survey of students in introductory computer literacy courses, many of the students reported that they did not choose a computer-related career due to inaccurate or insufficient information concerning computing careers. The results also show that students have a high level of dissatisfaction with their high school counseling experience, and received little or no guidance, particularly with respect to the computing profession. Other reasons cited were the lack of interest in a computer-related profession; they did not think they would like the work; they thought the work would be too hard; or they did not think that there were very many jobs available in this field. To sum it up: “This generation, which is noted for its pervasive use of technology, has little interest in, or awareness of, the occupations that drive the digital age” (Crampton, Walstrom, & Schambach, 2006).

The question facing many CIS programs is this: what measures can be taken to improve this situation? Most institutions are examining ways to provide course offerings to a wider audience of students. One way to do this is by introducing CIS minors or concentrations for students who are not CIS majors. It is believed that new and innovative curricula can offer immediate enrollment increases. Reaching a new audience is critical. Therefore development of an innovative minor that is delivered entirely online may help to boost enrollments (Patterson, 2005).

To counteract this decline, the authors’ CIS department decided to develop a new online, multi-disciplinary minor in CIS. The termmulti-disciplinary means that that focus is on the study of information technology through its application in more than two different fields. Elective courses may be any type of computer-centric course. Departments offering computer-centric courses may include art, computer science, English, journalism, biology, and so forth. Generally a required number of courses must be taken from the CIS department before the student takes elective courses from the other departments. However, the student has some flexibility when choosing electives, and therefore may tailor a program that suits his/her own interests.

Curriculum for the multi-disciplinary minor

In keeping with the rules of the Louisiana Board of Regents, it was determined that the CIS minor must consist of 6 classes (18 credit hours). After conducting an in-depth study of CIS minors at institutions in the United States (Pollacia, Lomerson, & Russell, 2008; Russell, J., Russell, B. & Pollacia, 2008) and an inventory of computer-centric courses from other departments at the authors’ university, a list of required and elective courses was composed. New courses were developed specifically for the minor that are significantly less technical than those in the CIS major, therefore a student of any major may enroll.  

The courses

The minor consists of six classes, two of which are required, and four electives. The two requirements are an introductory programming class, which uses a graphical language called Alice to create projects similar to simple video games. This course is very basic in nature, but gives students enough programming knowledge to proceed to the next level. The second required class is an introductory database class which will give students database skills using Microsoft Access. Upon completion of the course, students will be able to develop a small scale database application.

One interesting elective is a new class which will be taught by the university Library personnel called “Information Sources and Services”. This Freshman-level course was developed to teach students how to use and evaluate sources of information, copyright/plagiarism issues, as well as how to use the digital library.

The four elective classes can be chosen from the list below. Students can select according to their interests.

LIB 1030 – Information Sources and Services

CIS 3020 -- Web Page Development

CIS 3050 -- Multimedia Communication and Presentation

CIS 3070 -- Fundamentals of Systems Development

CIS 3100 -- Information Systems and Technology in Business

CIS 4030 -- E-Commerce Systems: Client-side Development

ART 1110 -- Graphic Communication I

ART 1120 -- Graphic Communication II

The Art classes are heavily computer-centered and web-oriented, therefore compliment the technical courses in web development. The minor is multi-disciplinary because required and elective courses are from the following distinct departments: University Library, CIS, Mathematics, and Art. Appendix 1 shows the course descriptions of the new courses.

Why online?

The three required courses, as well as several electives are offered online, so that a student may fulfill the minor requirements online.Why was the decision made to develop the entire program for online delivery? There are several factors that influenced this decision:

  • The authors’ university provides many courses and programs via the Internet, and is generally acknowledged to be the leader in digital and electronic education in Louisiana.
  • With declining enrollments, online delivery generally produces more FTE’s in each class. Whenever a course is scheduled for both online and face-to-face, students enroll in the online section and its enrollment reaches the maximum count before the traditional course.
  • The authors have been developing online courses since 1997, and thus have a great deal of experience in this area.
  • New tools and technology for developing online material now provides a rich and robust learning environment. Only in the past couple of years have new technologies, such as Audacity and Camtasia, become readily available and user-friendly so that suitable learning materials can be created.

Quality Matters ™ course development

Quality assurance is always a concern when dealing with online delivery. The university’s Electronic and Continuing Education and Distance Learning department provided the training to the selected faculty. This training involved several instructional design modules along with a number of technology training and media training modules. All of the training was founded on the Quality Matters ™ standards and rubric which provide a quality assurance and continuous improvement program for online courses (Lorenzetti, 2004; QM, 2006).

Quality matters ™

Quality Matters™ (QM) is a set of standards designed to provide the best practices in instructional design for courses that are delivered fully online or with a significant portion delivered online, i.e. hybrid courses (Lorenzetti, 2004). For this discussion, the term “online course” will be used to indicate both online and hybrid course. The focus of QM focus is to promote student learning, and is essentially a faculty-driven, peer review process. As described on its own web site (

Quality Matters is a faculty-centered, peer review process designed to certify the quality of online courses and online components. Sponsored by MarylandOnline, Inc, Quality Matters has generated widespread interest and received national recognition for its peer-based approach to quality assurance and continuous improvement in online education. Originating from a FIPSE grant, Quality Matters is now a self-supporting organization offering institutional subscriptions and a range of fee-based services including Quality Matters-managed course reviews and an array of trainings. Currently there are 204 subscribers in 38 states, with nine states adopting the QM standards statewide.

To meet QM standards, a course does not have to obtain 100% in the peer review; a level of 85% is considered to be of high quality. However, the development and peer review to enhance quality improvement should be a continuous process. Figure 2 illustrates this process.

Figure 2. Continuous improvement model for assuring the quality of online courses

QM focuses on course design, rather than on course delivery or the academic content of the course. A set of forty specific elements, distributed across eight broad standards, by which to evaluate the design of online and hybrid courses, have been identified. The eight broad standards are:

    • Course Overview and Introduction
    • Learning Objectives
    • Assessment and Measurement
    • Resources and Materials
    • Learner Engagement
    • Course Technology
    • Learner Support
    • Accessibility

The 40 standards as assigned different point values, ranging from three (3) points for essential, two (2) points for very important, to one (1) point for important elements. At the heart of the QM process is the Rubric, which provides a measurement for evaluating the online course quality. Table 1 shows the QM rubric for the first standard, Course Overview and Introduction. Similar rubrics are given for the other broad standards. There is a total of 80 possible points for the entire rubric, which is given in Appendix 2.

These standards provide a checklist for developing an online course. These elements may be included in the course syllabus or may be an activity in the course orientation. The process is applicable for course in which a course management system, such as Blackboard ™ or Web CT is utilized. After completion of the course development, the course is then reviewed by a qualified QM Peer Review team. The course must obtain a score of 85% or above to “pass” the peer review.

Course Overview and Introduction


1.1 Navigational instructions make the organization of the course easy to understand


1.2 A statement introduces the student to the course and to the structure of the student learning and, in the case of hybrid course, clarifies the relationship between the face-to-face and online components


1.3 Etiquette expectations with regard to discussions, email, and other forms of communication are stated clearly.


1.4 The self-introduction by the instructor is appropriate and available online.


1.5 Students are requested to introduce themselves to the class.


1.6 Minimum technology requirements, minimum student skills, and if applicable, prerequisite knowledge in the discipline are clearly stated.


Table 1. Specific review standards for “Course Overview and Introduction”

Instructional design and technology training

The training began in the Fall Semester of 2007, and continued through the Spring Semester of 2008. The content consisted of two components: Instructional Design training and Technology training. There were two levels of training that was required, described below.

The Level 1 training consisted of topics mainly to address faculty concerns about online delivery in general. The instructional design theories were covered, including Bloom’s Taxonomy (Bloom, 1956). It was emphasized that a quality course would utilize the various levels of learning outlined by Bloom. From the outset, the faculty learned that a new course would include 1) excellent navigational instructions to make the organization of the course easy to understand; 2) a statement introducing the student to the course and to the structure of the student learning activities; 3) etiquette expectations with regard to discussions, email etc.; 4) self-introduction by the instructor; required introduction of each student via discussion board, minimal technology requirements and minimum student skills needed; 5) learning objectives that describe learning outcomes that are measurable and consistent with the course-level objectives written in a clearly stated format so students could clearly understand the learning outcomes. These correspond to the standards in the first broad category.

The technology training covered many and various topics ranging from simple Windows file management, Adobe Photoshop, Microsoft Audicity (creating MP3 files), Photo Story (creating a picture guided story) and Camtasia (creating videos of lectures). The faculty learned how to create a story from PowerPoint slides using Photo Story software. In addition, Camtasia software was used to create both “voice over” presentations with either PowerPoint presentations or “voice over” with live demonstrations of software demonstrations. Some faculty learned how to develop podcasts with Camtasia that included Web Cam demonstrations to add a personal touch to the discussions. Faculty learned how to create banners to “brand” the course material, and how to incorporate Web 2.0 technologies into the course activities.

The Level 2 training focused on building course modules that were founded on quality learning outcomes using the Bloom’s Taxonomy. This training also focused on how to build rubrics and checklists. The faculty had to build example rubrics that would be used in their own classes. The courses were built with the assumption that they pass the QM Rubric. The new courses were submitted for review to a certified instructional design instructor. If the course did not meet expectations according to the rubric then the course was revised until it passed. Only courses that pass the expectations of this Rubric become a part of the CIS minor online courses.

Implementing QM ™ standards

This section demonstrates how the six standards for “Course Overview and Introduction”, Standards 1.1 through 1.6 were implemented for CIS 2070.

Standard 1.1 Navigational instructions make the organization of the course easy to understand. Figure 3 shows a screen shot of the home page. The course menu at the left makes it easy for the student to locate the major course elements.

Figure 3. Course menu and navigation elements

Standard 1.2
A statement introduces the student to the course and to the structure of the student learning and, in the case of hybrid course, clarifies the relationship between the face-to-face and online components. These items are addressed in a Welcome Message, andPhoto Story that explains important information about the textbook, and a document called Course Layout that explains the structure of the course. Figure 4 shows those elements of the course.

Figure 4. Statement of introduction and structure

Standard 1.3 Etiquette expectations with regard to discussions, email, and other forms of communication are stated clearly. The statement of etiquette expectations is one of several different policies required to adhere to QM standards. Due to the fact that several CIS courses were undergoing development, there was a need for all courses to have the same policies. Therefore CIS Departmental Course Policies and Procedures were developed to include all of those QM standard policies, shown in Figure 5, below.

Figure 5. QM policies

Standard 1.4 Self-introduction by the instructor is appropriate and available online.

Standard 1.5 Students are requested to introduce themselves to the class.

Figure 6. Introduction of instructor and students

Both of these standards are met by including an activity called ‘All About Me’, in which both the instructor and students introduce themselves to the rest of the class.

Standard 1.6 Minimum technology requirements, minimum student skills, and if applicable, prerequisite knowledge in the discipline are clearly stated. The minimum technology requirements are stated in the Course Syllabus.


Due to declining enrollments, the CIS department at the authors’ institution decided to develop service courses and a CIS minor. From the onset, the CIS minor was to be available online, and existing courses from other departments would be permitted as electives. At the same time, the Electronic and Continuing Education (ECE) offered training in Quality Matters ™ standards for online programs, and the CIS minor was chosen to be one of the programs to participate in this training program. The training proved to be invaluable to the success of the project.

Participating faculty ranged from those who were very experienced with online development to those with no experience. The training took place over one academic year (2 semesters) and resulted in six new courses that adhere to QM standards. This was very important to the CIS department, as the College of Business is AACSB accredited, and the QM standards provide quality assurance for the minor.

Experienced faculty were not expecting to learn much from the training, but were pleasantly surprised. New tools and technologies, such as Audacity and Camtasia, were used to produce course materials. Providing podcasts and videos results in a much richer learning experience for the student. The trainers for ECE have used some of our materials to illustrate best practices to new faculty who are undergoing the training this year.

From a “lessons learned” perspective, the development of an online class using the new technology for audio and video is a lengthy process requiring many hours. For example, the CIS 3070 Fundamentals of Software Development course was an arduous process requiring over 250 hours of development time. However, as a result of this effort, six quality online courses are ready and faculty began teaching the first four of these classes (LIB 1010, CIS 1010 Introduction to Programming, CIS 2070 Introduction to Database, and CIS 3050 Multimedia and Presentation) in the Fall semester 2008. The other courses will be offered in the Spring semester of 2009. Although, it is too soon to determine if the online CIS minor will increase enrollments, we will begin marketing the program to students and begin collecting data to determine if this endeavor is successful.

“A Red Flag in the Brain Game”. (2006, May). Business Week, May 1, 2006. Retrieved on 2/26/2008, from:

Bloom B. S. (1956) “Taxonomy of Educational Objectives, Handbook I: The Cognitive Domain”. New York: David McKay Co., Inc.

Bureau (2008) Occupational Outlook Handbook , 2008-09 Edition , Bureau of Labor Statistics, U.S. Department of Labor, retrieved from the Bureau of Labor Statistics web site on 2/6/2008:

Crampton, William J., Walstrom, Kent A., and Schambach, Thomas P. (2006, October) “Factors Influencing Major Selection by College of Business Students”, Issues in Information Systems, October 2006, Vol. VII(1), 2006, p. 226-230.

Kessler, Michelle (2005, May) Fewer Students Major in Computer: Some Fear Shortage of U.S. Workers, More Outsourcing. USAToday, retrieved on 7/12/2005, from:

Lenox, Terri L., Charles R. Woratschek, and Gary A. Davis (2005, October) “Exploring Declining CS/IS/IT Enrollments”, Proceedings of ISECON 2005, v22, Columbus, OH, Oct. 2005.

Lomerson, William and Pollacia, Lissa. (2005, October) “Declining CIS Enrollment: An Examination of Pre-College Factors”. Proceedings of ISECON 2005, v22, Columbus, OH, Oct. 2005.

Lorenzetti, Jennifer Patterson (August 2004) “Sophisticated Rigorous Assurance Program Designed for Your Institution”, Distance Education Report, 8/1/2004, Vol. 8, Issue 15, p 1-2.

Patterson, Laurie J. (2005, June) “Reaching a New Audience: Development of Computer-Centric Minor”, Proceedings 2005 ASCUE Conference, Myrtle Beach, South Carolina,

Pollacia, Lissa and Lomerson, William. (2006, October) “Analysis of Factors Affecting Declining CIS Enrollment”, Issues in Information Systems, October 2006, International Association for Computer Information Systems Vol. VII(1), 2006, p. 220-225.

Pollacia, Lissa, Lomerson, William and Russell, Jack. “CIS Minors in the U.S.: An Analysis of Existing Curricula”. Issues in Information Systems, International Association for Computer Information Systems Conference, October 2008, Vol. IX, No. 1-2, p. 241-248.

QM “Quality Matters Multi-Institutional Quality Assurance in Online Learning, Quality Matters Rubric for Online and Hybrid Courses”, Annotated FY 06/07, 2006 MarylandOnline, Inc.

Russell, Jack, Russell, Barbara, and Pollacia, Lissa. (2008, March) “Reversing the Decline of CIS Enrollment in Colleges and Universities by Creating Viable and Attractive Minors in CIS: A Statistical Study of CIS Minors at US Colleges and Universities”, Information Systems Education Journal.

Vesgo, Jay (2007, March) “Continued Drop in CS Bachelor’s Degree Production and Enrollments as the Number of New Majors Stabilizes”, Computing Research News, Vol. 19/No. 2, March 2007.

Appendix 1

New courses developed for CIS minor

LIB 1030 Information Sources and Services

The student will learn how to identify and evaluate information from various formats, sources and services using basic and advanced searching techniques. They will gain an understanding of the cultural and social contexts in which information is produced and disseminated. Issues of ethical and legal use of information, including copyright, intellectual property, and plagiarism will be explored within the context of current trends in academic plagiarism and media piracy. This is a foundational course. No prerequisites or co-requisites are required.

CIS 1010 Introduction to Programming Concepts

Students are introduced to the basic elements of developing programs by manipulating characters and objects in an interactive 3D world. Concepts addressed include components of programs, object oriented program elements, basic programming methods, and simple program design. Prerequisite: General familiarity with operating a computer and using basic features of a word processor program.

CIS 2070 Introduction to Database Systems

Introduction to the database concepts and software skills necessary for the design and development of a basic database system. This includes the tables, queries, forms, reports, and web pages within the database. Prerequisite: CIS 1010, CIS 1030 or CSC 1060.

CIS 3050 Multimedia Communication and Presentation

Emphasis on planning and delivering presentations enhanced by multimedia within a professional work environment. Concepts, design, and experience in developing multimedia presentations. Open to all majors. Prerequisite: Computer literacy or equivalent course.

CIS 3070 Fundamentals of System Development

A project-oriented study of the development of small business systems. Students will analyze, design and implement some application of a small business system. Students will learn how to design and write a proposal (requirements statement and business model). Students will develop a small database using MS Access and design and create the user windows interface. The application will be implemented using either MS Access Switchboard or MS Visual Basic.

Appendix 2

The QM Rubric


Review Standards




Course Overview

1.1 Instructions make clear how to get started and where to find various course components.





1.2 A statement introduces the student to the purpose of the course and to its components; in the case of a hybrid course, the statement clarifies the relationship between the face-to-face and online components.





1.3 Etiquette expectations (sometimes called “netiquette” for online discussions, email, and other forms of communication are stated clearly.





I.4 The self –introduction by the instructor is appropriate and available online.





1.5 Students are asked to introduce themselves to the class.





1.6 Minimum student preparation, and, if applicable, prerequisite knowledge in the discipline are clearly stated.





1.7 Minimum technical skills expected of the student are clearly stated.




Learning Objectives

2.1 The course learning objectives describe outcomes that are measurable.





2.2 The module/unit learning objectives describe outcomes that are measurable and consistent with the course-level objectives.





2.3 The learning objectives are stated clearly and written from the students’ perspective.





2.4 Instructions to students on how to meet the learning objectives are adequate and stated clearly





2.5 The learning objectives are appropriately designed for the level of the course.




Assessment and Measurement

3.1 The types of assessments selected measure the stated learning objective and are consistent with course activities and resources.





3.2 The course grading policy is stated clearly





3.3 Specific and descriptive criteria are provided for the evaluation of students’ work and participation.





3.4 The assessment instruments selected are sequenced, varied and appropriate to the content being assessed.





3.5 “Self Check” or practice types of assignments are provided for timely student feedback.




Resources and Materials

4.1 The instructional materials contribute to the achievement of the stated course and module/unit learning objectives.





4.2 The relationship between the instructional materials and the learning activities is clearly explained to the student.





4.3    The instructional materials have sufficient breadth, depth, and currency for the student to learn the subject;





4.4 All resources and materials used in the course are appropriately cited.




Learner Engagement

5.1 The learning activities promote the achievement of stated learning objectives.





5.2 Learning activities foster instructor-student, content-student, and if appropriate to this course, student-student interaction.





5.3 Clear standards are set for instructor response and availability.





5.4 The requirements for course interaction are clearly articulated.




Course Technology

6.1    The tools and media support the learning objectives, and are appropriately chosen to deliver the content of the course.





6.2   The tools and media support student engagement and guide the student to become an active learner





6.3  Navigation throughout the online components of the course is logical, consistent, and efficient.





6.4   Students have ready access to the technologies required in the course.





6.5    The course components are compatible with current standards for delivery modes.





6.6    Instructions on how to access resources at a distance are sufficient and easy to understand.





6.7    The course design takes full advantage of available tools and media.




Learner Support

7.1 The course instructions articulate or link to a clear description of the technical support offered.





7.2 Course instructions articulate or link to an explanation of how the institution’s academic support system can assist the student in effectively using the resources provided.





7.3    Course instructions articulate or link to an explanation of how the institution’s student support services can help students reach their educational goals.





7.4 Course instructions articulate or link to tutorials and resources that answer basic questions related to research, writing, technology, etc.





8.1    The course incorporates ADA standards and reflects conformance with institutional policy regarding accessibility in online and hybrid courses.





8.2 Course pages and course materials provide equivalent alternatives to auditory and visual content.





8.3 Course pages have links that are self-describing and meaningful.





8.4 The course ensures screen readability.




Manuscript received 28 Feb 2009; revision received 2 May 2009.

Creative Commons License

This work is licensed under a

Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License


Copyright © 2005-2009  MERLOT. All Rights Reserved.
Portions Copyright by MERLOT Community Members. Used with Permission.
ISSN: 1558-9528
Questions? Email:
Last Modified : 2009/6/15