As the sophistication of distance learning technology increases, there is a heightened concern that this increase in technology will actually reduce access. Students may need more sophisticated computers and learning resources and, therefore, some segments of society may be precluded from enrolling in these courses. Students in the CSULB- MPADL program are required to have a computer with Windows 2000 or newer, a sound card, broadband Internet access, and a headset and microphone. The only requirement that has presented an obstacle for some students in the program has been the broadband Internet connection. While a dial-up connection can be used, audio and video streams are facilitated with broadband connections. 

Concerns regarding academic dishonesty have also been cited as problems associated with distance learning programs (Grijavia, 2006). While it is true that it may be difficult to monitor academic dishonesty for timed examinations conducted online, random variation of examination questions can partially mitigate concerns over cheating and plagiarism. 

While many of these criticisms are certainly valid points to consider in developing distance learning programs, they may not necessarily preclude the increasing use of distance learning instructional modes in public affairs education. Perhaps the best source of data for measuring the validity of these concerns for the CSULB-MPADL program is the distance learning student population enrolled in the distance learning and traditional MPA programs at California State University, Long Beach.  

The MPA Programs 

Traditional MPA Program 

The Graduate Center for Public Administration and Policy (GCPPA) currently offers a traditional on-campus MPA program as well as a distance learning MPA program. The student populations enrolled in the programs are distinct, and it is rare that the GCPPA will allow an on-campus student to enroll in distance learning classes. The traditional on-campus program was established in 1973 and is NASPAA accredited. The traditional program also offers an Option in Public Works and Urban Affairs as well as certificates in areas such as Public Finance. The past five years have marked continued growth in the student population, and as of Fall 2005, there were 252 students enrolled in the traditional on-campus program and an additional 20 students in the Public Works and Urban Affairs Option Programs. The majority of traditional MPA students are female (58.7%), and 93.1% of students are in good standing. The remaining 6.9% are on probation and must maintain a 3.0 GPA to return to good academic standing. 

The curriculum consists of 36 units: 21 required units and 15 elective units. Required courses include an introductory course and courses in public budgeting, human resource management, organization theory, policy analysis, research methods, and a final directed research course that serves as a capstone for the program. Students must then choose five elective courses to complete their degrees. In Fall 2001, the GCPPA initiated a new portfolio graduation requirement to replace the existing comprehensive examination. In order to graduate, students must complete a four part portfolio, the cornerstone of which includes examples of their "best" work from all of their required courses. Since the inception of the portfolio requirement, graduation rates have increased from 50% to 67%, and the GCPPA graduates approximately 60-90 students per year from its traditional on-campus program. 

The Distance Learning Program 

In 1998, the GCPPA began a distance learning MPA program. The program is designed on a cohort model in which students begin the program as a cohort and progress through the program together. The sixth distance learning cohort began classes in Fall 2005 and will complete its studies in August, 2007. Cohort five is scheduled to complete coursework in August, 2006. This marks the first time that two cohorts are completing coursework concurrently. The program is designed to take 22 months to complete and consists of the same required courses as the traditional MPA program. Due to logistical necessity, distance learning students are not able to select their five elective courses. Rather, these "pre-selected" electives are constructed by the Distance Learning Director, and all students in the cohort take the same elective courses.  

The distance learning courses are offered in an accelerated format. Each course is six weeks in duration with a three hour on-campus meeting during the first week and a three hour on-campus meeting during the last week of classes. In addition to these two on-campus sessions, cohorts one through four also received instruction through one synchronous and one asynchronous session per week. The synchronous session consisted of a television broadcast utilizing fiber optics technology. Instructors broadcasted their lectures to various worksite locations throughout Los Angeles County. Students would share a microphone at each worksite that would allow limited communication with the instructor. This synchronous session would then be followed later in the week by an asynchronous session utilizing Blackboard. Students would participate in an asynchronous discussion board posting session. This afforded students the flexibility to complete assignments and postings during the week and did not mandate that they remain at their worksite for these asynchronous sessions.  

Beginning with cohort five in Fall 2004, the broadcast sessions were replaced with a synchronous session utilizing a computer assisted instructional platform (Centra Symposium). The new Symposium technology produced a virtual classroom where students have real-time interaction with the instructor. Each student can communicate with the professor and his/her classmates through audio or text chat. The Symposium platform allows students to indicate their desire to speak and then allows the instructor to open student microphones to facilitate discussion. Symposium is currently being used in both cohorts five and six.  

Student enrollment has increased steadily in the program over the past three years. Cohorts five and six currently have 25 and 20 students respectively, which represent cumulatively a 350% increase over cohort four.  

The demographic profile of the students in the distance learning program is relatively similar to the profile of on-campus students. The average distance learning student is 37.2 years old and holds an undergraduate degree in the social sciences. Specifically, 43.3% of students majored in a social science discipline. The second most frequent undergraduate major is business (23.3%) followed by liberal arts and engineering (16.7% each). As expected, they enter the program with above average undergraduate point averages (Mean = 3.27), and only two students hold another advanced degree. The student population is evenly distributed on the gender variable, and the average years between earning an undergraduate degree and entering the MPA program is 7 years, 10 months. Many of the students have used this time to rise to management level positions in government agencies. All of the students are currently employed in full-time positions with either government agencies or non-profit organizations. 46.7% hold management positions.   Most (46.7%) work for county agencies and departments; 33.3% work for city governments or city government organizations; 6.7% are federal employees; and 13.3% are employed in non-governmental/non-profit organizations.  

Method 

As discussed earlier, the purpose of this paper is to compare the satisfaction of students enrolled in the MPA-DL with the satisfaction of their counterparts in the traditional on-campus MPA program. One of the most vexing problems in distance learning research is the lack of comparability between the courses offered in distance learning and traditional MPA programs. While many programs have distance learning MPA programs and compile satisfaction indicators for students enrolled in these programs, it is usually not possible to compare equivalent classes across the two student populations. The CSULB MPA-DL program’s curriculum is nearly identical to the curriculum offered to traditional on-campus students. Specifically, the core courses are identical and are often taught by the same instructors. This study explores the student satisfaction scores for four (4) core courses that are offered in both the distance learning and on-campus programs. These courses include: an introductory/foundations course; a course in public budgeting and finance; a course in research methods; and a policy analysis course.  

To ensure comparability of content, the syllabus for each of these courses was examined to assess several factors that could threaten the comparability of the courses. First, the syllabi were examined to determine the number and type of assignments used. Each of these classes utilized essay based examinations, practical exercises, and a portfolio assignment that required students to integrate salient aspects of the course into a practical assignment. The length of the assignments was equivalent between all four classes. Second, the syllabi were examined to determine the amount of reading required of students. Each course required two textbooks, and the weekly required reading was approximately 100 pages. Third, to determine the style of content delivery, the researcher observed and/or participated in the distance learning versions of three of the four courses. Each course was a  primarily lecture based course that afforded ample opportunity for student participation. The comparability was also assured through informal interviews with the instructors to determine their assessment of the equivalency of the distance learning and on-campus versions of the course. The primary differences between the distance learning and on-campus class offerings included the medium of delivery and the accelerated nature of the distance learning class. While the on-campus courses encompassed 15 weeks of instruction, distance learning courses involved only 6 weeks of direct instruction. Given the demographic equivalency of the distance learning and on-campus student populations and the equivalency of the course material and instructors,  it was highly likely that any differences in student satisfaction between distance learning and on-campus students was attributable to the method of instruction native to distance learning or the instructor’s use of distance learning technology. 

Once the equivalency of the courses was established, the distance learning sections of the course were compared to two on-campus sections of the same course. Each of the courses was offered in the same semester by the same instructor. Data were collected from student evaluation of instruction surveys that were routinely distributed during the last two weeks of each class. The surveys used in the study were conducted in the Spring and Fall semesters, 2005. The survey questions are listed in Appendix A. The first eight questions were university required questions and appeared on both the distance learning and on-campus surveys. These questions measured student satisfaction of various aspects of the course including clarity of course objectives, consistency of grading, usefulness of assignments, reasonability of instructor expectations, instructor preparation, instructor effectiveness in presenting content, instructor availability, and a measure of the overall effectiveness of the instructor. The optional questions (normally questions 9-13) differed between the distance learning and on-campus evaluations. Question 9 on the distance learning survey measured the effectiveness of the Symposium software used in the weekly synchronous, interactive sessions. Question 10 measured student satisfaction of the Blackboard software that was used in the weekly asynchronous sessions. Question 11 assessed the usefulness of the course in improving a student’s understanding of concepts in the field. This question appeared as Question 9 on the on-campus survey. Question 12 measured the students’ assessment of the instructor’s knowledge of the course subject. This question appeared as Question 10 on the on-campus survey. All of the questions with the exception of the question regarding overall teaching effectiveness (Question 8) were scored on a Likert scale with a score of 5 indicating “Strongly Agree” and a score of 1 indicating “Strongly Disagree”. Student satisfaction with the overall teaching effectiveness of the instructor is also measured with a Likert scale with a score of 5 indicating “Most Effective” and a score of 1 indicating “Least Effective”.  

The surveys were not required, and students were permitted to opt out of submitting the survey. The on-campus student response rate was 82% (124 out of 151 possible respondents completed the survey). The distance learning student response rate was 44% (42 out of 96 possible respondents completed the survey). This difference in response rates was primarily attributable to the method of survey administration. For on-campus courses, instructors distributed the hard copy of the survey during the last two weeks of classes. According to university policy, instructors may not be present while evaluations are completed, and a proctor seals and signs the envelope before returning it to the Public Policy Office. Beginning in Fall 2004, distance learning students completed their student evaluation surveys on-line. The Distance Learning Director created the evaluation form using the survey function in Blackboard. Students then logged into the Blackboard site to complete their survey during the two weeks between courses. The self-directed nature of completing the distance learning surveys certainly contributed to the lower response rates.  

Caveat 

Obviously, given the relatively small sample sizes reflected in Tables 1-8, statistical significance was difficult to achieve. The data were analyzed with descriptive statistics as well as t-tests for independent sample means with unequal variance. While the tables note the levels of statistical significance of the t-values, caution should be used in asserting statistical significance given these sample sizes. These results should be viewed as exploratory in nature and represent an initial glimpse into the satisfaction levels of distance learning and on-campus students enrolled in equivalent courses. No attempt was made in this study to measure learning outcomes. The focus was merely to identify any important differences in the satisfaction levels of students enrolled in these two programs.  

Results 

Table 1 presents the mean scores and standard deviations for Courses A and B (an introductory level on-campus class) and the introductory level distance learning course. These courses were taught by the same instructor during the same semester. The mean satisfaction levels of distance learning students are higher than students in Course A for every question with the exception of Question 4. However, these differences are not statistically significant. On-campus student satisfaction with the reasonableness of the instructor’s expectations is marginally higher than distance learning student satisfaction. Other than this one exception, distance learning student satisfaction is marginally higher than on-campus satisfaction. However, the differences are rather small. The only significant difference in satisfaction levels concerns Question 3. Distance learning students express much larger satisfaction levels with the usefulness of assignments than their on-campus counterparts.  

The same distance learning class offering is then compared to another section of the introductory level on-campus class from the same semester (Course B) in Table 1. Again, the distance learning students express higher satisfaction levels than on-campus students for eight out of the ten questions. The exceptions to this pattern are question 2 (satisfaction with the consistency of instructor grading standards) and question 5 (satisfaction with instructor preparation). However, as was the case with Course A, the differences are all small, and distance learning students are at least as satisfied as their on-campus counterparts.  

Table 2 shows the student satisfaction means for two on-campus research methods courses (Courses C and D) as compared to a distance learning research methods course. The instructor for this course is different than the instructor for Courses A and B. For this instructor, student satisfaction scores are lower among distance learning students for all of the questions on each survey with the exception of Question 9 in Table 2 and the differences are minor except for questions 3 and 10. For Course D, Table 2 shows that on-campus satisfaction levels are higher for every question with the exception of question 9.  However, for Course D, the differences between the satisfaction levels of students in this course and the distance learning course are significant for questions 3, 5, 7,8, and 10. The only discernable pattern for the research methods courses is the higher satisfaction levels for on-campus students regarding the usefulness of assignments and instructor knowledge. On-campus student satisfaction in these two aspects is significantly higher than distance learning student satisfaction.
 

Table 1: Mean Student Satisfaction Responses of two On-Campus Introductory MPA Courses and an Introductory Distance Learning MPA Course

*NS = Difference is Not Statistically Significant at the p ≤ 0.05 level.
Note: Standard deviations appear in parentheses.

Note: The full text of the survey questions may be found in Appendix A.

Table 2: Mean Student Satisfaction Responses of two On-Campus Research Methods MPA Courses and a Research Methods  Distance Learning MPA Course