MERLOT Journal of Online Learning and Teaching
Vol. 4, No. 3, September 2008

Student Perceptions of Web-based Instruction: A Comparative Analysis

 

Dana V. Tesone
University of Central Florida
Rosen College of Hospitality Management
Orlando, FL 32819 USA
dtesone@mail.ucf.edu


Peter
Ricci

Florida Atlantic University
Barry Kaye College of Business
Boca Raton, FL 33431 USA
pricci1@fau.edu

 

Abstract

The article presents the findings of a recent longitudinal study that compares student perception of instruction scores between online and regular classes. Participants in the study were campus-based traditional undergraduate students enrolled at a large public U.S. university. The findings suggest that student perceptions of course participation are influenced by matters of convenience. The article concludes with relevant implications for educators who design and teach web-based courses to traditional undergraduate students.

Keywords: distance learning, student evaluations, web-based instruction, online learning, instructor evaluations

 

Introduction

Early adopters of web-based online learning methods in the middle 1990s contended that the delivery method would be restricted to outreach programs for those learners who possessed barriers to attending traditional education and training programs (Lyttinen & Rose, 2003). Since that time, the use of electronic formats to provide alternative course delivery methods continues to become a mainstream practice within many universities and training programs. According to the United States Department of Education, 62 percent  of public and not-for-profit private institutions offered classes via distributed (distance or online) learning as of the 2004-2005 academic year (U.S. Department of Education, 2006). For this reason, a number of learners, professors, and trainers continue to embrace web-based delivery methods at varying levels. The current online learning landscape includes both traditional and non-traditional learners, with traditional learners defined by the authors as the 18-22-year-old fulltime student cohort.

Over recent years, thousands of campuses have experimented to some extent with distributed learning methods using electronic communication tools for course delivery (Schneider & Germann, 1999; Kalamatianou & McClean, 2003). Today, a majority of mainstream training and education institutions intend to use online learning environments to provide some level of course delivery. The extent to which non-traditional classroom alternatives are used is referred to as online learning environment (OLE) modalities (Slate, 2001). Modalities range from traditional courses using web enhanced features, to reduced class time (mixed mode) methods, to fully web-based course versions. Some professors are teaching courses entirely online to learner enrollments that include traditional campus-based students who may be new to the experience of a fully online educational environment. These “traditional student” versus “fully online leaner” comparisons of instructional effectiveness form the theme of this paper.

The paper presents empirical findings comparing student perceptions of instruction (SPI) (also known as student perceptions of teaching [SPOT]) between students enrolled in regular face-to-face classes with those enrolled in fully web-based online learning environments (OLEs). The study was conducted at a large, public university located in the southeastern United States. Survey respondents provided SPI scores near the completion of management courses offered for major and non-major students through a hospitality management college. All of the participants resided near the main campus of the university. The paper identifies areas from the related literature as well as the methodology employed by the study. It concludes with implications for educators concerning a current trend in traditional student perceptions of instruction in OLE courses that is not widely represented in the existing literature.

Historical Perspectives of Online Learning

Innovators from training programs in the areas of engineering and computer science began working with distance learning initiatives during the 1980s through the use of telecommunication interfaces requiring some level of technical proficiency on the part of each user (Gibson & Herrera, 1999).  Early adopting educators in the fields of business and nursing were attracted to web-based distributed education modalities during the proliferation of public Internet access that began in the middle 1990s (Lytinen & Rose, 2003).  Today, most traditional training and education institutions intend to use online learning environments (OLEs) to provide some level of course delivery. This has also sparked the interests of researchers in the discipline.

A vast body of distance learning literature has emerged over recent years (Schrum, 2002).  This could suggest that these programs have achieved “mainstream” status in many academic institutions.  A number of studies address issues such as faculty compensation, incentives, course evaluation, and pedagogy in online learning environments (Wardrope, 2001).  It is commonly reported that the shift from traditional teaching to OLE requires ‘change’ on the part of the instructor.  It is likely that the concept of change may serve as both a catalyst and a cause of resistance on the part of educators based on varied perceptions concerning OLE instruction. One study suggests that online education has created a new paradigm, which has fostered new modes of learning that involve innovative views of pedagogy and requires internal change on the part of participating faculty members (Rahman, 2001).

Journal articles addressing the issues of pedagogy and OLE began to appear in the management literature during the latter 1990s (Rimmington, 1999). It was predicted that the frequency of such contributions would continue to escalate into the new millennium (Cho, 2000). Indeed, this does seem to be the situation that has occurred over recent years.

Online Learning Environments (OLE)

At the end of 2001, Symonds reported in Business Week that nearly half of the major 4,000 colleges and universities in the U.S. were offering online classes to augment their more traditional classes. The U. S. Department of Education (1999) predicted that total college enrollment would increase to 15.3 million students by the end of 2001, a finding which was rather accurate. An earlier study found that one-third of U. S. higher education institutions offered distance learning courses, with another 25 percent reporting the intention to add these to the curricula (Russell, 1995; “College Enrollment Surge,” 2001). The report goes on to note that 45 percent of these courses were offered by four-year public institutions, 39 percent by public two-year institutions, and 16 percent by private four-year institutions. It was suggested that the number of  institutions offering bachelor degrees would reach 1.4 million, a 13 percent increase, and that graduate enrollments would increase by 12 percent by 2011 (Russell; “College Enrollment Surge”). The proliferation of online course offerings is already impacting the learning strategies of current students who appear to be enrolling in online courses more frequently than those who came before them. It is possible that students choose the OLE alternative over traditional classes offered by institutions for reasons related to convenience. As evidenced in the authors’ findings, the OLE course sections always seemed to fill up prior to the traditional course offerings when the same subject was offered in both options. This observation is consistent with a number of OLE studies that address the issue of convenience as a major factor influencing the intention of students to enroll in these classes. This further suggests that student satisfaction rankings of instruction might be based on perceptions of convenience experienced throughout a given online course.

OLE Studies and Convenience Factors

A number of studies concerning aspects of the OLE distribution method exist within the literature. One study addressed the anytime/anywhere aspects of asynchronous OLE courses by identifying the key drivers of student convenience, flexibility, and stability associated with well designed classes (Chang, Hung, Keh, Chang & Shih, 2005). Convenience is also addressed in an article that presents the findings of a study conducted among practicing engineers seeking online continuing education courses to maintain job skills and employability (Kariya, 2003). The report continues by appropriately noting the requirements of diligent study, hard work, and motivation on the part of online learners. There are instances in which enrolling students overlook these requirements when choosing courses and select based solely upon convenience factors. Another study of working professionals noted that both synchronous (real-time) and asynchronous courses still meet with convenience criteria, in that such courses eliminate travel time and expenses as well as time away from the office (Anonymous, 2001). Yet, another study with working professionals notes a criticality associated with online learning environments by suggesting that individuals are willing to trade-off certain learning outcomes for the sake of online convenience (Farris, Haskins, & Yemen, 2003).

The convenience and financial savings aspects of OLE course selection are also cited in a study that took place within a professional association to provide arbitrator continuing education credits (Slate, 2005). The association has been providing online learning programs for members since 2001 and has trained more than 2,500 individuals during that time.

While a number of incidents within the literature address OLE initiatives among working professionals, there are entries that include programs provided for traditional learners within universities. A study conducted at Ball State University collected feedback from students and instructors within a master’s level program and found that the largest advantage cited for choosing OLE courses was the convenience factor that precluded the need to travel to campus during the typical school day (Flowers & Cotton, 2003). Researchers from a business college collected data concerning the popularity of OLE as a viable choice of student enrollments and found that the majority (68%) of favorable responses came from non-traditional (>25 years of age) learners who considered themselves to be highly proficient in the use of technology (Smith & Rupp, 2004). Benbunan-Fich and Hitz (2003) found that a comparison of student perceptions of various course modalities did not yield significant differences with the exception of convenience factors that rated highest for online courses.

Methodology/Design

Student participants had their student perceptions of instruction (SPI) evaluation scores compared over a period of four academic years beginning in 2002 and ending in 2006. Students were assumed to have randomly chosen between face-to-face course offerings and/or online learning options. The mix of these two types of course offerings was varied across semesters by time of day taught, day of week taught, and on campus or off campus location. With the number of specific, individual course sections (twenty sections in total) and variety in times of day and days of week taught, it was an assumption by the authors that students were randomly placed into each course section type. This assumption is further strengthened by the fact that during certain semester periods one section would fill quite quickly and a second, alternatively-taught section of the course was added at the last moment during the student registration process. In essence, students were not able in many cases to pick their preferred method of instruction in advance of having to register for these required courses.

To further strengthen consistency of comparisons, every section was taught by the same instructor presenting identical material from course to course in equal proportions and in identical styles of presentation in each course section. Hence, the instructional methods were controlled by the instructor for consistency throughout the experimental period. Lastly, all sections enrolled 45 students each and every semester. Thus, instructor workload and teacher-to-student ratio was also controlled. These course offerings were all required for hospitality and tourism management students during the time period under investigation (2002-2006).

Mean scores of professorial teaching evaluations were analyzed through a comparison of groups during each semester involved in the study. These semesters included: spring and fall, 2002, spring and fall, 2003, spring and fall, 2004, spring and fall, 2005, and, lastly, spring and fall, 2006. During any particular semester, each course section was independent of the other. Students enrolled in one section could not be simultaneously enrolled in another; hence, membership in a course section implies mutual exclusivity of any student participant. Since two groups’ mean scores were investigated, the independent samples t-test was utilized to compare the means of the two sample groups for each particular semester of interest as well as for the overall time period. Numerical functions and analyzes were performed using SPSS® (2007).

The Study

In the spring of 2002 the first group of participants began the period of examination. Each course section enrolled 45 students and ten separate course sections were under investigation for a total of 450 participants. All participants were seniors enrolled in a hospitality management program within a large, public southeastern institution. The course under examination was required for all seniors prior to graduation. Indeed, all participants were senior-level standing at the time of participation with ninety percent taking the course in their final semester prior to matriculating with a baccalaureate degree. Participants were all of traditional college age with ages ranging from twenty years to twenty-three years. All participants lived within a thirty-mile range of the classroom location. As such, this study’s participants utilized distance learning in a somewhat close proximity to the actual classroom compared to a larger geographic national or international dispersion. For detailed participant information refer to Table 1.

 

Table 1 . Student Participant Descriptive Information

Descriptive Information (n=450)
Frequency
Percent    
Currently senior-level standing 
450     
100%
Age

20 years

180
40%
21 years
225
50%
22 years
27
6%
23 years
18
4%
Gender

Male

171
38%

Female

279
62%
Minority/Non-minority

Non-minority

386
86%

Minority

63
14%

Asian-American

13
2.9%

African American

5
1.1%

Hispanic

45
10.0%
Employment

Employed in the hospitality industry

315
70%

Hold supervisory-level position or above

23
5.1%

Hold non-supervisory level position  

292
64.9%
Residence

On campus

180
40%

Off-Campus within 30 miles 

270
60%

Off-campus greater than 30 miles 

0
0%

 

Each semester the authors coordinated with the on-campus course scheduler. Since this particular course is required for all graduating seniors, the course sections filled to capacity each and every semester. As such, the course scheduler would only open one section – traditional face-to-face method or online learning environment/distance learning. He would alternate which section to open first based upon the semester. Once that section was filled, the next section would open. Thus, students were forced into a course regardless of method of instruction to randomize the process of course enrollment preventing students from choosing a preferred method over the two. With 90% of the participants in their very last semester prior to graduation, these participants had no other choice but to enroll in the section which was open at the time of registration whether or not the teaching method was the one they personally preferred.

In total, 450 students participated in the process with 10 course sections each filled to capacity with 45 students per section. Prior to the semester’s completion, the university generates a “student perceptions of instruction” (SPI) survey form to all registered students in all courses campus-wide. Those registered in a traditional face-to-face course will complete the survey in pencil and paper format whereas those taking a fully online course will complete the survey in its online format. Historical reports indicate that over 45% of students regularly complete these student perceptions of instruction (SPI) surveys. Further, this particular college of hospitality and tourism management has a historical reporting record of over 65% for its majors. This study indicated similar results with over 67% of enrolled students completing the end-of-semester student perception of instruction surveys during this 5-year, 10-semester longitudinal study.

This particular university has a 16-item student perceptions of instruction (SPI) survey with the last item deemed the most important to professors, administrators, and those evaluating the professor’s effectiveness in the classroom. The 16 item statements are: 1) feedback concerning your performance in this course was, 2) the instructor’s interest in your learning was, 3) use of class time, 4) the instructor’s overall organization of the course was, 5) continuity from one class meeting to the next was, 6) the pace of the course was, 7) the instructor’s assessment of your progress in the course was, 8) the texts and supplemental learning materials used in the course were, 9) description of course objectives and assignments, 10) communication of ideas and information, 11) expression of expectations for performance, 12) availability to assist students in or outside of class, 13) respect and concern for students, 14) stimulation of interest in the course, 15) facilitation of learning, and, 16) overall assessment of instructor. Each student could respond using a five-point Likert-type scale with a 5.0 indicating excellent, a 4.0 indicating very good, a 3.0 indicating good, a 2.0 indicating fair, and, lastly, a 1.0 indicating poor. Many faculty members at this university and others using this type of student perceptions of instruction (SPI) survey, rely heavily on their average score in item 16 which refers to their students’ overall assessment of them as an instructor.

Data Analysis and Results

Data were analyzed using The Statistical Package for the Social Sciences (SPSS®, 2007). Two groups were compared: students enrolled in course sections utilizing a traditional, face-to-face teaching method and students enrolled in course sections utilizing an online learning environment (OLE) teaching method. The two groups are independent of each other. Student enrollment in one course section implies mutual exclusivity of the student participant as he/she could not be simultaneously enrolled in the one other course section available during the particular term of instruction. With two groups compared, the independent samples t test was utilized to compare the mean score ratings of professor rating between the two groups. No statistically significant differences between mean professor ratings depending on method of instruction were found on any single statement during any of the ten semesters (see Table 2). Further, no statistically significant difference in professor ratings by teaching method was found overall (see Table 3). Thus, the overall finding can be reported as follows. An independent samples t test was performed comparing the mean professor rating scores of live face-to-face course sections in years 2002 through 2006 with fully online learning environment (OLE) sections taught during the same time period. One section of the face-to-face format and one section of the fully online section were taught each semester for a period of 10 semesters beginning in the fall of 2002 and ending with the fall of 2006. The t test found no statistically significant difference in mean professor ratings based upon method of instruction between live course sections (m = 4.24, sd  = .36) and online learning environment (OLE) course sections (m = 4.29, sd = .35), t(318) = -1.318, p  = .189.

Discussion

While statistical testing did not show significant differences among students concerning perceptions of instruction between courses offered in each modality, anecdotal evidence concerning convenience factors associated with OLE versions emerged within the student comments section of each survey. Table 2.  Student perceptions of instruction (SPI) mean professor rating comparisons by method of instruction (face-to-face [F2F] versus fully online [OLE]) over ten semester period spring, 2002 through fall, 2006

Statement on SPI inventory

        F2F

               OLE

 

 

 

 

m

sd

m

sd

  t

df

p

 

 

 

 

 

 

 

 

Feedback concerning your performance in this course

4.24

0.43

4.34

0.29

-0.602

18

0.554

The instructor’s interest in your learning was

4.25

0.27

4.31

0.39

-0.422

18

0.678

Use of class time was

4.24

0.37

4.33

0.35

-0.581

18

0.568

The instructor’s overall organization of the course was

4.26

0.4

4.38

0.28

-0.779

18

0.446

Continuity from one class meeting to the next was

4.29

0.31

4.36

0.27

-0.492

18

0.629

 

 

 

 

 

 

 

 

The pace of the course was

4.24

0.38

4.35

0.31

-0.738

18

0.47

The instructor’s assessment of your progress in the course was

4.21

0.39

4.3

0.36

-0.544

18

0.593

The texts/supplemental learning materials used in the course were

3.92

0.45

4.04

0.44

-0.611

18

0.549

Description of course objectives and assignments

4.2

0.4

4.3

0.31

-0.653

18

0.522

Communication of ideas and information

4.33

0.39

4.31

0.43

0.158

18

0.876

 

 

 

 

 

 

 

 

Expression of expectations for performance

4.25

0.44

4.35

0.33

-0.598

18

0.557

Availability to assist students in or outside of class

4.26

0.28

4.23

0.41

0.199

18

0.845

Respect and concern for students

4.31

0.39

4.36

0.37

-0.315

18

0.757

Stimulation of interest in the course

4.21

0.31

4.23

0.44

-0.1

18

0.921

Facilitation of learning

4.28

0.31

4.24

0.37

0.203

18

0.841

Overall assessment of instructor

4.32

0.36

4.33

0.34

-0.051

18

0.96

 

Table 3. Overall comparison of mean professor ratings between all Face-to-face (F2F) course sections with all Online Learning Environment (OLE) course sections 2002-2006*

        Face- to-face

             OLE

 

t

 

df

 

p

m

sd

m

sd

4.24

0.36

4.29

0.35

-1.318

318

0.189


Student comments were not compulsory for instrument administration and a small number of respondents took the time to provide narrative statements. Student narrative comments were in response to open ended questions concerning aspects of the course considered by students to be most favorable and least favorable. Favorable comments were focused on the method of instruction for courses in both modalities, while unfavorable comments noted costs associated with textbooks and materials. Students enrolled in the asynchronous OLE sections of the course also included favorable comments concerning convenience factors in the areas of time and travel flexibility.

As noted previously, the authors exercised care to provide control factors for the purpose of investigating comparisons between groups. The only differences between the modalities existed in the pedagogical aspects of instructional design and delivery, which are supported by studies from the OLE literature. The instructor employed a ‘sage on the stage’ manner of experiential learning for the face-to-face course interaction. In contrast, the same instructor provided a learner-centered facilitation of peer oriented experiential learning in the OLE versions of the course. The effectiveness of each pedagogical approach is evident in the overall favorable ratings of same-content course sections provided in both modalities.

The main limitation associated with this study is that it tracked the SPI scores for a single instructor. Future studies may collect data from institutions in which larger numbers of educators teach in both OLE and traditional modes over long periods of time. For control purposes it would be important to track single instructors teaching multiple sections of the same course in both modalities. Larger samples, however, would permit researchers to compare aggregate comparisons.

Implications for Educators

Summative evaluations of student perceptions of instruction have been the norm within higher learning institutions for decades. Most educators record and compare these scores over a number of years. Such comparisons are particularly informative for instructors teaching in both traditional and OLE formats. Most institutions offer training in the pedagogical and technological aspects of OLE course delivery. This provides evidence that successful OLE instructional methods vary to some degree from those employed in traditional classroom environments.

The authors have observed that the majority of educators continue to prefer the face-to-face method of instruction. In fact, they have noted less than desirable outcomes among colleagues who have experimented with OLE delivery methods, only finding them to have returned to the traditional classroom setting. Younger colleagues, however, appear to be experimenting with blended learning or mixed mode approaches of instruction. These courses provide reduced classroom seat time with OLE assignments. The methodology employed in this study provides applications for these individuals to statistically compare student perceptions from these courses with their traditional counterparts. The main implication of the study is that educators teaching in multiple modalities possess the means to compare student perceptions of instruction between sections.


References

Anonymous. (2001, January). Professionally speaking: Flexible learning. Mechanical Engineering, 1(31), 1-3.

Benbunan-Fich, R., & Hitz, S. R. (2003, December). Mediators of the  effectiveness of online courses. IEEE Transactions on Professional Communication, 46(4), 298-302.

Chang, F. C., Hung, L. P. Keh, H. C., Chang, W. C., & Shih, T. K. (2005,  July/September). Design and implementation of a SCORM-based courseware system using influence diagram. International Journal of  Distance Education Technologies, 3(3), 82-94.

Cho, W. (2000). Just-in-time education: Tools for hospitality managers of the future? International Journal of Contemporary Hospitality Management, 12(1), 31-40.

College enrollment surge to continue through 2011. (2001, August 24). Student Aid News, 28(17), 1.

Colley, R., & Volkan, A.  Teaching workloads of finance program leaders and faculty and criteria for granting load relief.  Journal of American Academy of Business, Cambridge.  4(1/2).  84-89.

Farris, P. W., Haskins, M. E., & Yemen, G. (2003). Executive education programs go back to school. Journal of Management Development, 22(9/10), 784-802.

Flowers, J., & Cotton, S. (2003, September). Master of Arts in career and technical education—now 100%. Tech Directions, 63(2), 22-28. 

Gibson, J. W., & Herrera, J. M. (1999, January). How to go from classroom based to online delivery in eighteen months or less: A case study in online development. T.H.E. Journal (Technological Horizons in Education), 26, 57-60.

Kalamatianou, A. G.,  & McClean, S. (2003, December). The perpetual student: Modeling duration of undergraduate studies based on lifetime-type education data. Lifetime Data Analysis, 9(4), 311-13.

Kariya, S. (2003, May). Online education expands and evolves. IEEE Spectrum,    40(5), 49-54.

Lytinen, K., & Rose, G. M. (2003, December). The disruptive nature of information technology innovations: The case of Internet computing in systems development organizations. MIS Quarterly, 27(4), 557-563.

Rahman, M. (2001, Winter). Faculty recruitment strategies for online programs. Online Journal of  Distance Learning Administration, IV (IV). Retrieved  January 20, 2007 from http://www.westga.edu/~distance/ojdla/winter44/rahman44.html

Rimmington, M. (1999).Vocational education: Challenges for hospitality management in the new millennium. International Journal of   Contemporary Hospitality Management, 11 (4), 186-191.

 Russell, C. (1995). The official guide to the American marketplace: The real fact about how rich, well educated, healthy, family-oriented, hard working, and diverse we are (2nd ed.). New York: New Strategist Publications.

Schneider, S. P., & Germann, C. G. (1999, Winter). Technical communication on the Web: A profile of learners and learning environments. Technical Communication Quarterly, 8(1), 37-48.

Schrum, L. (2002, July). Dimension and strategies for online success: Voices from experienced educators. Journal of Asynchronous Learning Networks. 6(1). Retrieved February 3, 2008 from http://www.sloan-c.org/publications/jaln/v6n1/v6n1_schrum.asp

Slate, W. K., II. (2001, September 1). Settling claims on the Internet: Dispute  prevention and resolution in e-Commerce. Vital Speeches of the Day, 67(22), 684-688.

Slate, W. K., II. (2005, August-October). Technology is good. Dispute Resolution Journal, 60(3), 1.

Smith, A. D., & Rupp, W. T. (2004). Managerial implications of computer-based  online/face-to-face business education: A case study. Online Information Review, 28(2), 100-107.

SPSS® (2007). The Statistical Package for the Social Sciences (Version 15.0) [Computer software]. Chicago: SPSS, Inc.

Symonds, W. C. (2001, December 3). Giving it the old online try: More people are clicking their way to degrees, creating huge opportunities for the education industry. Business Week, 3760, 76,

U.S. Department of Education. (1999), Distance education in higher education institutions: Incidence, audiences, and plans to expand. Washington, D.C.; U.S. Department of Education.

U.S. Department of Education (2006). 2004 integrated postsecondary education data system (IPEDS). Washington, D.C.; U.S. Department of Education National Center for Education Statistics.   

Wardrope, W. J. (2001). A communication-based response to distance learning in business communication. Business Communication Quarterly, 64, 92-97.

 


Manuscript received 27 May 2008; revision received 23 Jul 2008.

Creative Commons License

This work is licensed under a

Creative Commons Attribution-NonCommercial-ShareAlike 2.5 License


   
Copyright © 2005-2008  MERLOT. All Rights Reserved.
Portions Copyright by MERLOT Community Members. Used with Permission.
ISSN: 1558-9528
Questions? Email: jolteditor@merlot.org
Last Modified : 2008/9/15