Introduction

The advancement in technology has been revolutionizing the way educators teach and students learn (Wells, Fieger & Lange, 2005). In the last decade or so, educational trends have been changing rapidly with a movement towards web-based instruction and “blended” instruction, where web-based instruction replaces components of face-to-face instruction. One example of this revolution is the development of learning management systems (LMS), course management systems (CMS), and virtual learning environments (VLE) that facilitate teaching and learning outside the physical classroom. Often, these terms are used interchangeably in designating the same tools or software.

For the purposes of this study, a learning management system (LMS) is a software environment that enables the management and delivery of learning content and resources to students.  It provides an opportunity to maintain interaction between the instructor and students and to assess the students by providing immediate feedback on the online quizzes. Most LMS are web-based to facilitate "anytime, anywhere" access to learning content and administration. Common LMS in higher education fall under two broad categories: Commercial systems such as Blackboard, WebCT, eCollege, and Desire2Learn; Open source code products such as Moodle, Sakai, Segue, and Coursework.

These learning environments may be used to totally replace face-to-face (F2F) teaching in a physical classroom, partially replace F2F teaching (“blended”) or only supplement existing F2F teaching (Arbaugh & Duray, 2002). The LMS can enhance learning through efficient access to learning materials, by the provision of immediate feedback to students through on-line assessment (Breen, Cohen, & Chang, 2003), and by improved communication between students and instructors through discussion forums and email (Beard & Harper, 2002).

What is Blackboard?

Blackboard is one of the leading commercial LMS (or CMS) products used in North America and Europe (Munoz & Duzer, 2005) and is the most widely-adopted learning management system among U.S. postsecondary institutions. Blackboard provides a password-protected environment and has administration tools that make teaching online easier (Lowe, 2003).

Recently, researchers have compared learning management systems based on their functionality, user-friendliness, and cost. Some of the key features evaluated in a learning management system are its usability, availability, security, stability, interoperability, and scalability (Hall, 2003). This study explores the use and value of these features.

Computer Literacy

Computer literacy has been defined as "an understanding of computer characteristics, capabilities, and applications, as well as an ability to implement this knowledge in the skillful, productive use of computer applications suitable to individual roles in society" (Simonson, Mauere, Montag-Toradi, & Whitaker, 1987, p. 233). Gupta (2006) defines it as the individual’s ability to operate a computer system. This includes having a basic understanding of the file management processes such as formatting a disk and how to save, copy, delete, open, and print documents. It also involves using computer applications software to perform personal or job-related tasks, using web browsers and search engines online, and being able to email.

Computer literacy is a fundamental part of the undergraduate curriculum in this decade. It is as basic to undergraduate students as the course work in the core curriculum (Dugger, Shelli, Lisa & Crystal, 2003). Students educated in computer literacy use the computer skills in most of the other courses in their discipline. Previous studies have found that computer competency is essential to both academic and career achievement (Davis, 1999).

According to Lankshear and Knobel (2003), some of the new computer literacies include electronic gaming, synchronous and asynchronous communication, Web logs (“blogs”), Web pages, and multimedia text production. Andrews (2004) researched the new literacies in computer technologies including the environment in which students learn how to read and write with multiple modalities (graphics, animations, video, audio, hyperlinks, and print). It has also been noted that in higher education, LMS skills are listed as one of the categories of computer literacy, along with general computer and file management skills (Office of Distance Learning, Old Dominion University, 2004; Testing, Evaluation and Research Services, Wayne State University, 2007). 

The Computer Literacy Course

Within this study, the usefulness of an LMS in content delivery and its value in learning desired computing skills was researched in an undergraduate computer literacy course offered by the Educational Technology program at a large southwestern U.S. university. In this course, students are given a solid introduction to computers and the software applications they will use in their professional and personal lives. The instruction features illustrated lectures, in-class discussions, on-line research and discussion, student-generated information, demonstrations, and hands-on lab activities. The class met face-to-face each week, in a blended delivery format. The Blackboard LMS was used mainly for content delivery and for online discussions and quizzes. The course is designed such that the LMS was not used for collaborative activities among the students, except for the online group discussions. This particular learning management system was chosen as it was available through the university, although its usage was not mandated.

This is a general studies required course, and students from different majors campus wide were enrolled. The course has two areas of concentration: theoretical knowledge of computers, and knowing how to use computer applications for productivity, problem solving, and data analysis. It is a semester-long course and the students meet once a week for three hours.

Purpose

The goal of the present study was to assess the usefulness of an LMS for content delivery and to determine its impact on the development of computing skills in the context of a computer literacy course at a large southwestern university. The main questions that were answered as a result of the survey that was administered as part of this study were:

1.      What are the skills that students achieve from a computer literacy course?

2.      How useful are the different features of the LMS?

3.      What are the different areas in which the LMS is helpful?

4.      How do instructors rate Blackboard based on different characteristics of a learning management system?

Method

Participants

The participants were the students and instructors of a multi-section computer literacy course.  One hundred and forty-five students enrolled in the computer literacy course responded to a survey uploaded through the LMS. Seven instructors responded to an email survey about the usefulness of the Blackboard as an LMS. Out of 145 students, 38 of them were male and 107 were female. There were 13 freshmen, 63 sophomores, 46 juniors and 23 seniors. 120 students who participated in the survey were in the age group of 24-32.

Table 1 shows that the users were spread out in all four categories based on Blackboard usage, with a greater part having used the Blackboard in more than one class.  Based on computer proficiency, the majority of the participants perceived themselves to be fairly knowledgeable in using the different computer applications.

                 Table 1. Blackboard Usage and Computer Proficiency of Participants 

Material

An online survey was distributed to find the usefulness of the LMS in content delivery and how it has helped the students in learning computing skills. The students were surveyed within Blackboard and the instructors were surveyed over email. The survey had 26 questions that included five categories 1) Demographic questions, 2) Computer Proficiency, 3) Usefulness of Blackboard Features, 4) Areas in which Blackboard is helpful, and 5) Learning Management System characteristics. The students’ survey did not include the fifth category, which was Learning Management System Characteristics. The instructors’ survey did not include the first two categories, which were demographics and computer proficiency.

Results

One hundred and forty-five students and seven instructors responded to the survey. The respondents rated the questions on a four-point Likert scale, which varied based on the question. The weighted mean of the responses obtained from students and instructors were analyzed.

Usefulness of Blackboard Features

Nine features regarding use of Blackboard were listed in the survey and the students and instructors were asked to rate on a four point Likert scale for level of usefulness (1=Not Useful, 2= Somewhat useful, 3= Useful, 4=Very useful). Table 2 gives the student and instructor ratings on the usefulness of the LMS features.

The students rated the assignments feature to be the most useful feature in Blackboard (M=3.51), followed by Gradebook (M=3.48). Instructors rated Course Documents and Gradebook to be the most useful features (M=4.00). The feature that was rated to be the least useful by the students and instructors was Virtual Classroom (M=1.80). Instructors had rated Announcements and Communication higher than the students. 

Table 2. Usefulness of Blackboard Features

 Areas where Blackboard was helpful

The fourth category of questions on the survey dealt with the areas in which Blackboard was helpful. Seven areas in which Blackboard could have been helpful were listed in the survey and the students and instructors were asked to rate on a four point Likert scale for level of helpfulness (1=Not helpful, 2= Somewhat helpful, 3= Helpful and 4=Very helpful). Table 3 gives us the student and instructor ratings on the helpfulness of Blackboard in different areas.

Immediate feedback on the online quizzes was rated to be the most helpful area by both the students (M = 3.67) and the instructors (M=3.71).  This immediate feedback helps students to recognize their mistakes at the time they are made and also helps them to correct their misconceptions in learning. The next most helpful feature was considered to be the ability to access the course materials at all times (M = 3.43). Using an LMS helped students to become more comfortable with technology and was rated helpful by students as (M= 3.18) and by instructors as (M=3.29). Both students and instructors agreed that the experience gave them an opportunity to practice file management skills (download, save and open files) and it was rated helpful by both students  (M=3.16) and instructors (M=3.00). With the exception of collaborative work (M = 2.73), all areas were given ratings of above 3.0. Collaborative work was rated as the lowest by the students and instructors who thought that they were not benefited by using Blackboard for group work. 

   Table 3. Areas where Blackboard was helpful

Characteristics of a Learning Management System

Instructor ratings were evaluated on six characteristics of a learning management system. The six characteristics were availability, scalability, security, usability, interoperability and stability (Hall, 2003). Instructors rated these characteristics on a four point Likert scale that varied for each characteristic. For example, the characteristic availability was rated as (1=Not available, 2= Somewhat available, 3= Available and 4=Highly available) and scalability was rated as (1=Not Scalable, 2= Somewhat scalable, 3= Scalable and 4=Highly scalable).  Table 4 provides the mean ratings of instructors on the different characteristics of Blackboard as a learning management system.

   Table 4. Instructor ratings on different characteristics of a Learning Management System 

Availability, scalability and security were rated the highest (M=3.57) in issues for instructors. Stability of the environment was rated the lowest (M=2.71), followed by interoperability (M=2.86). Due to heavy usage load there were days that students had trouble accessing the system during the first week of classes.

The survey included an open-ended question that asked the participants what was positive about using an LMS in this course and what could be improved. The four most frequent responses were (1) accessing course documents at anytime from anyplace (n=52), (2) option to check grades anytime (n=26), (3) online quizzes with immediate feedback (n=16), and (4) Blackboard’s Digital Drop Box to turn in assignments (n=15). The most frequent response for what could be improved was (1) unavailability of the system due to downtime (n= 22). It is noteworthy that 25 participants mentioned that there was nothing they disliked about the experience.

Discussion

The goal of the present study was to explore the usefulness of an LMS in content delivery and assess its impact on student learning of computing skills while enrolled in a computer literacy course.

Computer Proficiency

The rapid pace of technological advances in the computer industry has forced businesses to reorganize, to acquire the latest systems, and demand a computer-literate workforce (Porter & Miller, 1985).  Computer literacy courses help students become computer proficient. In this study, using Word documents (M=3.57) and emailing (M=3.52) were rated as the top two skills that the students learned from the computer literacy course. This objective aligned with Ndahi and Gupta’s study (2000), which found that the most required computer skill was word processing. Using Blackboard (M=3.2) was rated as the third highest skill learned and was rated higher than learning to create Web pages or  spreadsheets.  These results aligned with the work of Drennan, Kennedy, and Pisarski (2005), which found that using Blackboard helped to improve pre-service teachers self-reported computer skills and their confidence in using technology.

Usefulness of Blackboard Features

Students and instructors reported that assignments, gradebook and course documents were the most useful Blackboard features. Access to the course material, assignments and gradebook at any time and any place is an important feature of an LMS. This remains consistent with Drennan et al. (2005) who found that the key attribute to student satisfaction was positive perceptions of technology in terms of access and use of online flexible learning material. Recent advances in using learning management systems for online and blended course delivery has made flexible learning and teaching possible.

Teng and Allen (2005) found that students liked the feature of an online gradebook because they can find out where they are in the class right after the teacher posts the grades, and where they need to improve. Texley and Adelstein (2006) found that the online gradebook is handy and convenient, and encouraged students to login every day to check for missing assignments. In this study, the instructors rated the online gradebook to be the most useful feature of the LMS. Texley and Adelstein (2006) found that even though entering grades punctually can sometimes seem overwhelming to teachers who tend to procrastinate, using an online gradebook helped them to see immediate and significant results in classroom management and achievement.

Features Seen as Helpful

The most helpful feature was the availability of immediate feedback in online quizzes. This quantitative result was reinforced in the open-ended question where students mentioned that immediate feedback on quizzes was valuable. Feedback provided increases the amount of correct information remembered from the target material (Kulhavy, Yekovich & Dyer, 1979).  Feedback also facilitates criterion performance as it corrects the inaccurate information obtained during instruction, and has little effect on correct responses where the learner has correct understanding of the text information (Kulhavy & Anderson, 1972).

Accessing the material at all times was rated as the second most helpful element of Blackboard and was reinforced again in this category. This is consistent with research that has found that flexibility of time and place is a major advantage in online courses (Burke, 1996; Laaser, 1998; Zhang, Perris & Young, 2005).

Getting comfortable in use of technology was rated the third most helpful aspect of LMS use. Using an LMS also taught students file management skills. This was rated as the fourth most helpful area. In the usefulness category, group pages and virtual classroom were rated the least useful features. This aligned with the helpfulness category where the computer literacy students rated that for collaborative work, Blackboard was least helpful.

Conclusion

Based on the positive responses of both the students and instructors, this survey found that using the Blackboard LMS in a computer literacy course was useful both in teaching the course and also in helping the students develop computing skills. Combining the use of a learning management system along with the teaching of basic computer applications in the face-to-face classroom contributed to the enhancement of computer literacy and technology skills. The survey responses from the students and instructors provide evidence that many learning outcomes can be enhanced by the presence of a learning management system. The findings also showed that the participants found Blackboard to be an effective learning management system. It served as a vehicle for the students to become more familiar with technology and access the course material from “any where, any time” in a digital format.

References

Andrews, R. (Ed.) (2004). The Impact of ICT on Literacy Education. London: Routledge Falmer.

Arbaugh, J. B., & Duray, R. (2002). Technological and Structural Characteristics, Student Learning and Satisfaction with Web-based Courses. Management Learning, 33, 3, 331-347.

Beard, L. A., & Harper, C. (2002). Student Perceptions of Online Versus On Campus Instruction. Education, 122, 4, 658-663.

Breen, L., Cohen, L., & Chang, P. (2003). Teaching and learning online for the first time: Student and coordinator perspectives. Paper presented at the Partners in Learning: 12th Annual Teaching Learning Forum, Edith Cowan University, Perth.

Burke, J. (1996). Using e-mail to teach: expanding the reach of BI. Research Strategies 14, 36-43.

Davis, P.  (1999). How undergraduates learn computer skills: results of a survey and focus group: THE Journal, 26, 9, 68-71.

Drennan, J., Kennedy, J., & Pisarski, A. (2005) Factors Affecting Student Attitudes Toward Flexible Online Learning in Management Education. The Journal of Educational Research, 98, 6, 331-8

Dugger, W. E., Meade, S. D., Delany, L. & Nichols,C. (2003) Advancing Excellence in Technological Literacy. Phi Delta Kappan, 8, 4, 316.

Gupta, G. K. (2006). Computer Literacy: Essential in Today’s Computer-Centric World. ACM SIGCSE Bulletin, 38, 2, 115-119

Hall, J. (2003). Assessing Learning Management Systems. Retrieved August 26, from http://www.clomedia.com/content/templates/clo_feature.asp?articleid=91&zoneid=29

Isakson, C. (2005) Presentation Skills, Powerpoint and Beyond. The Education Digest, 71, 1, 79-80

Kulhavy, R. W., & Anderson, R. C. (1972). Delayed retention effect with multiple-choice tests. Journal of Educational Psyschology, 63, 505-512.

Kulhavy, R. W., Yekovich, F. R., & Dyer, J. W. (1979). Feedback and content review in programmed instruction. Contemporary Educational Psychology, 4, 91-98.

Laaser, W (1998). Virtual seminars-second best or neiv quality of distance teaching. Paper presented at Shanghai International Open and Distance Education Symposium, Shanghai, China

Lankshear, C. & Knobel, M. (2003). New Literacies: Changing Knowledge and Classroom Learning. Buckingham: Open University Press.

Lowe, C. (2003) Open Source Weblog CMS's: An Alternative to Blackboard. Retrieved August 26, 2006 from http://cyberdash.com/node/view/83

McComb, M. (1983). Augmenting a group discussion course with computer-mediated communication in a small college setting. Interpersonal Computing and Technology, 1, 3

Munoz, K. D., & Van Duzer, J. (2005). Blackboard vs. Moodle: A comparison of satisfaction with online teaching and learning tools. Unpublished raw data. Retrieved October 14, 2005, from http://www.humboldt.edu/-jdvl/moodle/all.htm .

Ndahi, H. & Gupta, A. (2000). Computer literacy for workforce development: Reading improvement, 37, 1, 39-44.

Office of Distance Learning, Old Dominion University.  (2004) Computer Literacy Survey, Old Dominion University. Retreived May 8 from http://www.clt.odu.edu/oso/index.php?src=pe_comp_lit

Porter, M.E. & Miller, V.E. (1985). How information gives you competitive advantage. Harvard Business Review, 63, 4, 149-160.

Simonson, M.R., Maurer, M., Montag-Torardi, M. & Whitaker, M. (1987). Development of a standardized test of computer literacy and a computer anxiety index. Journal of Educational Computing Research, 3, 2, 231-247.

Teng, Y. & Allen, J. (2005). Using Blackboard in an Educational Psychology Course to Increase Preservice Teachers' Skills and Confidence in Technology Integration. Journal of Interactive Online Learning, 3, 4, 1-12

Testing, Evaluation and Research Services, Wayne State University. (2007) Computer Literacy Competency Examination. Wayne State University. Retrieved on May 7 from http://www.testing.wayne.edu/complit_Testdates.pdf

Texley, J. & Adelstein, D. (2006). A Platform to Stand On. The Science Teacher, 73, 7, 30-32

Wells P., Fieger, P., & de Lange, P. (2005). Integrating a virtual learning environment into a second year accounting course: Determinants of overall student perception . Proceedings of the AAA Annual Meeting, San Francisco.

Zhang, W., Perris, K., & Young, L. (2005). Online tutorial support in open and distance learning: students perceptions. British Journal of Educational Technology, 36, 5, 789-804