Introduction

Course management systems (CMS) such as Blackboard (Bb), ANGEL, Sakai, Moodle, etc. are becoming standard on many higher education campuses. Use of CMSs has fostered expectations from both faculty and students that access to library services and materials will be provided in this new learning environment.  CMSs have not addressed those expectations. As early as 2004, Buehler observed that “Course management software (CMS) or courseware products, such as Prometheus, FirstClass, Blackboard and WebCT, do not include the Library as an essential curricular component in their design” (p. 75). Bell & Krasulski (2004) further observe that “By design, courseware systems offer no presence for the academic library and faculty members can unknowingly create an information environment for their students that largely ignores library resources” (p.85). The result is that the return on university investments in both library resources and course management systems are minimized.  (Cohen, 2002, p. 12; Council on Library and Information Resources (CLIR), 2002, #7).  However, libraries immediately recognized the problem and are moving quickly to create methods of integrating their services and resources into CMSs to meet faculty and student expectations. This article reports a recent survey of the Harold B. Lee Library’s (HBLL) integration in Bb courses. Results confirm that the integration was helpful to students completing research papers or projects.

CMS use is growing on many campuses. Data from the 2001 Campus Computing Project indicates  “approximately one-fifth (20.6 percent) of all college courses now use course management tools, up from 14.7 percent in 2000.” (Campus Computing Project, 2001). Morgan (2003) found that “Course management systems are an increasingly important part of academic systems in higher education…[and] the numbers of faculty using these systems is increasing rapidly…”    More recently, the 2006 ECAR Study of Undergraduates and Information Technology found that of the 28,724 students surveyed, “Three quarters of these undergraduates use course management systems, most of them using it several times a week or more ” and “Three-quarters (75.6 percent) of those who use them are overall, positive or very positive about them” (Katz, p 5). Hybrid courses, where the CMS supplements the face-to-face course, are common on many campuses as are totally online courses and distance education courses. A 2006 Gartner survey found that

                                                In the Gartner 2002 higher education learning survey, a little more than 30% of the courses were hybrid; in 2004 it grew to almost 45%. In 2006, overall, more than 57% were hybrid…Many institutions in our 2006 survey indicated that many traditionally taught courses were required to use CMSs to post course content, for assessment and course discussion (Zastrocky, 2007).

Literature Review

Since 2000, articles have been written advocating the need for library integration into course management systems and reporting integration of specific library resources or services such as E-Reserve and information literacy materials, into CMSs. An editorial in Online Libraries & Microcomputers (2001) asserts that “With all of the hype surrounding new courseware, however, there appears to be a missing element—the library.” Cohen (2002) points out that libraries provide access to numerous electronic resources representing an enormous institutional investment, and goes on to say that “Librarians spend much time selecting resources from the Internet and organizing…information that is best for student use. Integrating course-management software with the library’s digital offerings is essential for getting the maximum value from the institutional investments of both money and expertise” (p.12). It is essential that libraries discover ways to integrate their services and resources into CMSs to support student requirements for appropriate and credible resources in their coursework (Shank & Dewald, 2003, p. 38). As late as 2005, Gibbons claimed that “To remain relevant, academic libraries must go where the students and faculty are.  More to the point, libraries need to be where the learning is happening, even if this is the virtual environment of a CMS” (p.12).

Examples of integration of library services and resources into CMSs are found in the literature. Integrations reported thus far fall into two main areas—Information Literacy (Getty, Burd, Burns & Piele, 2000;  Giles, 2004; Cox, 2002; Lenholt, Costello & Stryker, 2003; Roberts, 2003 (Corrado & Mouliason, 2006 ) and E-Reserve (Cubbage, 2003); Ziegenfuss & McCloskey, (2004); Bell & Krasulski, (2004); Holobar, (2006); Sander, van Vuren & dePlessis, (2006) ). A recent search by this author retrieved a few articles reporting on broader integration of library resources into CMSs.  Articles that report the evaluation and results of integration are scarce in the literature. Sander, et al. (2006) found that their Library LIVE integration resulted in student awareness of “what quality information is electronically available.” Students “view research as a means to an end” so employing a goal based structure, Collard & Tempelman-Kluit (2006) postulated that students would use the resources as a “means” to complete assignments, “the end” (p. 58).  Their results showed high use of library resources integrated into Bb courses correlated with “busier and slower research periods of the year” (p.63). In a more recent article, Hightower, Rawl & Schutt (2008) found that faculty are interested in providing library resources via their CMS and that usage picked up when a library module was promoted during instruction sessions.

History of BYUs use of Blackboard

Brigham Young University (BYU)  is a Doctoral/Research Extensive institution with approximately 30,242 undergraduate students, 1484 graduate students and 1600 faculty. BYU adopted Blackboard as the campus CMS in 2000. Use of Bb picked up momentum gradually, beginning with fifty courses in 2002 and increasing to 1350 courses in winter of 2003. Faculty implementation of Bb varied based on personal preferences and comfort with the technology. Some faculty moved their entire course content to Bb; others used a few features such as announcements, the grade book and the assignments folder. Whatever features faculty use, a large number employ Bb as an adjunct to their existing course. A 2002 survey revealed that eighty-one percent of BYU students have at least one Bb course and seventy-eight percent of students prefer that their professors use Bb (Waddoups, 2002). Further,

In the winter 2005 semester, only 14% of instructors surveyed were not using Bb at all, and 53% were using Bb for multiple courses. …only 5% of students surveyed during winter semester reported not using Bb in any of their courses… On the average, students are enrolled between three to four Bb courses each semester.  (West & Kennedy, 2005, p. 39).

Student and Faculty Library Expectations in Blackboard

Soon after Bb was adopted, it became apparent that both students and faculty were expecting a library presence in Bb courses. During a break out session of the Annual University Conference, a panel of students and instructors discussed their experiences using Bb. Students on the panel wanted the library’s online resources to be available in their courses. Faculty expressed similar feelings in a recent evaluation by the Center for Instructional Design (CID) of the Bb implementation. In that evaluation West & Kennedy (2005) reported that “Most instructors want their Bb course to be a ‘one-stop shopping’ experience for their students where they can get everything they need in one place….” (p.1.)  Knowledge of student and faculty expectations provided a significant opportunity to integrate library support in Bb courses.

Faculty were the focus of the library’s initial integration efforts. First, the library produced and distributed a brochure outlining library services available in Bb. Librarians handed out the brochure in a booth at the annual Technology Info Fair as they demonstrated adding library resources to courses. Second, the library participated in CID sponsored seminars for faculty interested in teaching with Bb. Faculty, subject librarians and CID technical personnel worked together during the seminar. Subject librarians assisted the faculty to choose the most appropriate resources for inclusion in their course and CID technical personnel taught them how to construct the HTML links inside their Bb course. Third, the Distributed Learning Services Librarian developed a short tutorial on constructing links to library resources that was linked to the Faculty Resources tab in Blackboard. Two things became very evident from these efforts--faculty were not experts at creating or maintaining library links and thought it took too much time to do so—occupying valuable time they could be using to improve their courses.

During the same time, the library also created a student information module on the “My Institution” page in Bb courses. The module contained generic links to library materials and services, that librarians thought would be helpful to any student taking a Bb course. While this was a scatter-shot technique, it had the advantage of linking students to the library within the course, regardless of which course they were enrolled in. Students no longer had to leave their Bb course to login to the library Web page. The library felt that students had easier access to library resources and service by providing these links. An apparent disadvantage was that students had to return to the “My Institution” page to access the links.

An upgrade to a newer version of Blackboard provided an enhanced opportunity for integrating course specific library resources into Bb courses.  Building on Shank and Dewald’s (2003) concept of “macro-level” vs. “micro-level” integration, a Library Services tab was added to the “My Institution” page. The tab is highly visible and easier to access because it remains on the top frame of the page no matter where a student is in the course. The tab consisted of two modules, one with links to finding materials, services, and research help (macro-level integration) (see Figure 1). 

 Figure 1. Contents of Library Services tab.

Further, an application programming interface (API) technology was used to construct a “Building Block”, making it possible for the library to integrate subject specific resources into Bb courses (micro-level integration). After logging into a Bb course and clicking the Library Services tab, the building block presents a list of all the courses in which the student is currently enrolled. The course title, listed under the Course Research Page module, is a link to the subject page for the course (see Figure 2).

Figure 2. Example of subject page.

The subject page contains a federated search tool, links to appropriate databases and research guides, and contact information for the subject librarian. An Instant Messaging application was added later, but was not available as part of this study. The building block required no learning curve or implementation time from the faculty.  Students benefit by being able to access library resources without having to leave their course environment and by being able to access library resources for all their courses, regardless of the course they are logged in to at the time.

How The Building Block Works

Programmers and librarians collaborated on the building block. After a project team determined the functionality of the building block, a programmer from Library Information Systems (LIS) worked closely with the both campus Bb engineer and Bb corporate support personnel to build and test it. Existing, carefully prepared, subject pages created by librarians were linked to the corresponding departments in the university so that students in that department would get the appropriate page for their course. Only one subject page can be linked to a department, so for departments like history where there are several possibilities, the building block displays a page of all the subjects and the student chooses which page to use (see Figure 3). While this is not the ideal situation, nevertheless, students are linked to a page where they can access resources for their course. The building block was successfully deployed in all Bb courses during the summer term of 2005.

 Figure 3. List of all subject pages.

Marketing and Publicity

Providing automatic integration of library resources and subject pages in Bb courses was a big step forward. However, the library knew from previous experience that merely providing the link and hoping that students would find it was not enough. Holobar (2006) corroborates this finding “As the online learning environment has become simultaneously more integrated and compartmentalized, simply making resources available on library websites isn’t enough” (p.70).  Therefore, the library embarked on a publicity campaign to inform students and faculty about the library link in Bb courses. The library knew that “Faculty’s encouragement and guidance decide to a large extent the resources that online students seek out in support of their course work” (Cahoy & Moyo, 2005,  p. 2). MacDonald & vanDuinkerken (2005) state, “If professors are aware of the service, they can recommend it to their students” ( p. 33). Marketing focused on the desire articulated by both students and faculty for Bb to provide a “one-stop shopping” experience for students.

The library targeted faculty  in two ways. First, the library hosted a booth and a scheduled session at the university’s Teachnology Expo. This event, jointly sponsored by The Center for Instructional Design (CID), the Faculty Center and the Library, consisted of organizational booths, faculty presentations, a working area with computer stations and scheduled fifty minute sessions on various models for effective use of technology in teaching. The author and other librarians demonstrated the improved library integration at the booth and in a session.  Second, a short piece in the Faculty Library Guide, published and disseminated by the library’s Communications Officer, also explained the integration.  The guide emphasized the benefits of library integration to students in the courses and highlighted the fact that the integration required no time or effort from faculty—it just showed up automatically in their courses.

Concurrently, the library Multimedia Production Unit also worked on a short promotional video to increase student awareness of the library integration and a module on the “My Institution” page of Bb advertised the enhanced library services, and the fact that they were embedded in the course.  The promotional video, with screenshots and background music, demonstrated to students that the library tab would lead them to the resources for all their Bb courses in the same place, thus saving them valuable time. The video played on a large plasma screen at the entrance to the library for a full year.

Assessment of Library Integration

A year after the integration occurred, the library assessed the effectiveness of student experience and the integration strategy.   A survey was used to evaluate student experience and to discover  if students found the page helpful in their course work. The library wanted to know if students felt that their papers, projects, etc. were improved because they used the resources on the course page.  Additional objectives for the survey were to determine: (1) how much students were using the course pages, (2) how students learned about the library integration, and (3) how hard or easy the pages were to use.  Finally, the library wanted their suggestions for improvements.

Survey Methodology

The library partnered with CID for this survey because they routinely survey students twice a year about their experience with Bb courses. Originally the author planned to add the questions about the CRP to their survey; however, CID personnel decided this approach made it too long and that the library questions did not fit the purpose and scope. Consequently, the library evaluation was sent out separately. Since the survey went out under the auspices of CID, Institutional Review Board approval was not necessary.

A ten-question survey was constructed (see Appendix) using Qualtrics software. The first three questions asked about how participants used, learned about and accessed the CRP. Next, four questions asked about the page itself. Finally, three questions solicited suggestions for improving the page. The first seven questions were a mix of multiple-choice and yes-no answers with the opportunity for further explanation and the remainder were open-ended.

The survey review process involved four steps. First, the library’s Process Improvement Officer reviewed the questions. Second, it was sent to several other library faculty members to review the clarity of the questions. Third, it was vetted with CID student employees from four different disciplines.  Fourth, the CID Quality Assurance team took the survey in preview mode to test the timing and accuracy of the survey.

Once constructed, CID used the student database to select a random sample of students. Those selected were emailed an invitation to participate with their reply serving as consent to participate. The survey was sent to 600 students and of those, ninety-four responded. For the purposes of this evaluation, and based on prior experience, the library did not want survey recipients to answer the questions about the page if they had not used it. Therefore, the first question on the survey asked if the student had used the CRP.  A screenshot of the page was included so that those who may have used it, but did not remember what it was named, could still answer the question. If the answer to the first question was “No” the software prevented them from answering any further questions. On the advice of the Quality Assurance personnel at CID and because it was nearing finals, CID opted to close the survey after receiving thirty completed surveys where the recipient indicated they had used the CRP. While this was a small sample size, it should be noted that the administration of the survey was handled by CID and therefore subject to their determination of an adequate response pool. 

Results and Discussion 

The responses to the survey questions reported in this article represent only the thirty respondents who indicated that they had used the page. Percentages were rounded to the nearest whole percent. The library hoped to determine if there were correlations between a respondent’s class standing and their answers, but it was impossible due to the fact that over half, fifty-seven percent of the respondents were seniors.  The remaining forty-three percent were almost evenly divided among freshmen, sophomores, juniors or those who did not indicate their class standing. The first question asked if respondents had used the CRP. Of the ninety-four survey respondents, thirty-two percent (30) of respondents had used the CRP while sixty-eight percent (64) had not. This seemed to indicate that in spite of the marketing and publicity efforts, a large percentage of students were not aware of the existence of the CRP.

The second question asked respondents how they learned about the CRP. The two most common ways students learned about the CRP were from their instructor, sixty percent and from a library instruction session, fifty-seven percent (see Figure 4).Percentages for this question total more than 100% because respondents were asked to check all the ways they learned about the course. The responses substantiated the fact that learning about library resources from an instructor, whether in a course or during a library instruction session, was one of the most successful ways to increase student awareness of library integration in Bb courses.

 Figure 4. How students learned about the CRP.

The third question asked respondents how they typically accessed the CRP once they knew about it. The predominate method of accessing the page, reported by seventy-seven percent of respondents, was accessing the page by logging into their Bb course (see Figure 5).This figure was obtained by combining the responses for those who said they logged on to their Bb course and those who marked  “Other”,  but specified that they accessed it via the library website or through the university website.  Both the library and the university websites provide access to Bb courses, but no direct access to the CRP, therefore the supposition was that the respondents logged on to either the library or the university website and then into their Bb course where they accessed the page.

Figure 5. Method of accessing the CRP.

Question three was of particular interest to the library because of a significant drop in use statistics beginning and continuing through 2006.  One supposition was that students were book marking the page and were not counted in our statistical tracking tool because it only counted those who logged onto the page via their Bb course. However, the findings did not support this conjecture. After the survey was completed, the programmer discovered that the required data needed to link the CRP to the correct course had not been received for almost a year. Consequently, all students were being linked to the Subjects A-Z page where they were required to choose which CRP to use. This may have been a factor in the declining use of the CRPs.  The error has since been corrected and current statistics show that usage is again on the rise.

The next four questions asked about the use of the page itself. Question four asked about the ease of use of the page. A majority of the respondents, eighty-seven percent, answered that the page was “Very easy”, “Easy” or “OK” to use (see Figure 6).  Thirteen percent said it was “Hard to use”.  Because the survey had been previously taken by four student employees from CID with no problems, definitions of these terms did not seem necessary. Explanations from those who responded that it was “Hard to use” included experiencing technical problems, feeling that the amount of information available made it hard to know where to start, and not knowing what the links were for or which one to use.

Question five asked respondents if the page helped them locate resources for their paper/project. Ninety-three percent responded “Yes” and six percent said “No”.  Those who responded “No” commented that they couldn’t figure out how to use the resources. Confusion about which sources to use and not being able to figure out how to use the resources could have been a function of the way our federated search product works or perhaps unfamiliarity with searching online databases.

Figure 6. Ease of use of CRP

Question six was designed to elicit the respondents opinion of whether using the CRP improved their paper/project. Eighty-seven percent responded “Yes” and thirteen percent said “No”. Reasons given for the paper/project being better included: more sources, helped find sources, got results and sources were organized (see Figure 7). Reasons given by respondents that their paper or project wasn’t better were: difficulty in finding appropriate sources, easier to use the library homepage and used Google rather than the CRP. The library provides high-quality resources for students but also wants to know if the resources students use contribute to better papers. These responses indicated that a large majority of students felt that using the CRP improved their paper/project.

Question seven asked if respondents would recommend the CRP to others. Ninety-three percent said “Yes” and seven percent said “No”.  When asked why they would recommend it to other students, several responded that they found it helpful and felt that other students would, too. Another group of respondents indicated that the CRP was a good place to find resources, that  it was a convenient way to find resources, that the resources were good, accurate sources and that it’s easy [to use] (see Figure 8). Of respondents who answered no, one said “I haven't used enough to remember it very often” ; another reported a poor experience with page, and  a third said “because there are more resources available through the library website.” The majority of positive responses to these questions leads us to believe that the CRP contains specific resources that respondents found helpful for their course and that they found the page valuable.

When the survey was analyzed, the library wanted to know if the respondents who said the page was “Hard to use” also responded that the CRP did not help them find resources for their paper/project, that their paper/project was not better because they use the CRP or that they would not recommend the CRP to others. No significant correlations of questions five through seven with the ease of use response in question four were found.

The final three survey questions were open ended and focused on respondent opinions about the CRP.  Question eight elicited suggestions for improvements to the page which fell into four main categories: 1) twenty-six percent had no suggestions for improvement; 2) six percent said it was “good”; 3) thirteen percent provided no response to the question; and 4) fifty-five percent, slightly over half, suggested making the page easier to use, providing instructions on how to use the page, and promoting the page (see Figure 9).

Suggestions for making the page simpler to use included clarifying database features, making it easier to navigate and search through, and help choosing from among the various options.  Providing use instructions for the page was not considered in the initial integration plan since students, presumably, used these identical pages from the library website as demonstrated in the responses to question nine. In the aggregate, the responses indicate that the subject pages require further fine-tuning or that the library needs to provide some use instructions in order to address student needs. Based on the answers to this question combined with the number of respondents who had not used the page, promotion is also an area that needs attention.

Question nine asked if respondents had discovered anything on the page that they were previously unaware of. The majority of respondents, (seventy percent), said they had not discovered anything new on the page. Of those who had, seventeen percent responded “everything”, “databases, tutorials, resources”, and “I don’t know.” The remaining thirteen percent respondents said “Yes”, but did not indicate what they had discovered.  The fact that seventy percent of respondents did not discover anything new on the CRS, indicated some familiarity with the page and with the resources included on the page. This finding was encouraging because it indicated that students had previous knowledge about CRPs and that only five of the thirty respondents were totally unfamiliar with the resources.

The final open ended question, number ten, invited respondents to make any other comments about the CRP (see Table 1). A total of fifty-three percent had no comments or responded “Nice” or “Thanks”.  Seventeen percent did not provide a response to the question.  The remaining thirty percent provided comments that generally fell into two categories:  (1) features they liked about the CRP and (2) problems they experienced with the page or specific suggestions for improvements.

Conclusion

Overall, the results showed that most respondents surveyed were not aware of the CRP. For those respondents that were, the page was easy to use, it helped them locate resources for their paper or project, they felt their paper/project was better because they used the page and they would recommend the page to other students. Upgrading the ease of use, providing use instructions and better promotion were the main suggestions for improving the page. Final comments about the page mirror these same concerns along with providing positive feedback on some features of the page. An online tutorial and improved promotion that focuses on those who were not aware of the page are two areas the library will begin to address.

For those who were aware of the CRP integration and used it, the library met student and faculty expectations for resource availability inside their Bb courses. Integrating the library into Bb made it a “one-stop shopping experience” for students. At the same time, it freed the library from relying on faculty to add library content to their courses and freed the faculty from the burden of creating and maintaining links to library services and resources. On the other hand, the number of students who were not aware of the page also means that the library needs to redouble its efforts to make faculty aware of the CRP so that they can recommend its use to students.  

Further research should investigate whether faculty think student papers/projects are better as a result of the CRP integration and student use of the page. Another area for continued investigation is whether faculty are amenable to embedding a subject librarian in their course to provide answers to library related questions on discussion boards, through course assignments or other means. Subject librarians could provide proactive help during times of heavy research projects or respond to students who find the page difficult to use. Becoming more deeply embedded in Bb courses requires an effort by librarians to reach out to faculty and establish strong collaborative relationships with them. As Shank and Dewald (2003) observe, if the library is successfully integrated into courseware, relationships and collaborative ties with faculty will be strengthened. Finally, “Course management software, with all its bells and whistles, should …offer an enhanced learning environment in which students, faculty and librarians collaborate in the construction of knowledge” (Ziegenfuss & McClosky, 2004, p. 95).  Knowing students perceptions of the CRP integration is a first step in that direction.

References

Buehler, M. A. (2004). Where is the library in course management software? Journal of Library Administration, 41(1/2), 75-84.

Bell, S. J. & Krasulski, M. J.  (2004). Electronic reserves, library databases and courseware: A complementary relationship. Journal of Interlibrary Loan, Document Delivery & Electronic Reserve, 15(1), 75-85.

Cahoy, E. and Moyo, L. (2005). Faculty perspectives on e-learners’ library research needs. Journal of Library & Information Services in Distance Learning, 2(4), 1-17.

Council on Library and Information Resources (CLIR). 2002. Bulletin Number 7. Retrieved February 8, 2008,  from http://www.clir.org/pubs/cliringhouse/house07.html

Cohen, D.  (2002). Course-management software: Where’s the library? Educause Review, 3(37), 12-13.

Campus Computing Project. (2001). The 2001 National Survey of Information Technology in US Higher Education  Retrieved on April 2, 2008 from, http://www.campuscomputing.net/summaries/2001/index.html

Collard, S. & Tempelman-Kluit, N. (2006). The other way in: Goal based library content through CMS. Internet Reference Services Quarterly, 11(4), 55-68.

Corrado, E.M. & Mouliason, H.L. (2006). Integrating RSS feeds of new books into the campus course management system. Computers in Libraries, 26(9), 6-9, 61-64.

Course management software: Where is the library? (2001). In Online Libraries & Microcomputers, 19(10), 1-2. Retrieved January 29, 2008 from EBSCO database (5426288).

Cox, C. (2002). Becoming part of the course: Using Blackboard to extend one-shot library instruction. College & Research Libraries News, 63(1), 11-13, 39.

Cubbage, C.  (2003).Electronic reserves and Blackboard’s course management system. Journal of Interlibrary Loan, Document Delivery & Information Supply, 13(4), 21-32.

Getty, N.K., Burd, B., Burns, S.K. & Piele, L. (2000). Using courseware to delivery library instruction via the web: Four examples. Reference Services Review, 28(4), 349-359.

Gibbons, S. (2005). Integration of libraries and course-management systems. Library Technology Reports, 41(3), 12-20.

Giles, K.L. (2004). Reflections on a privilege: Becoming part of the course through a collaboration on Blackboard. College & Research Libraries News, 65(5).  Retrieved January 1, 2007 from, http://www.ala.org/ala/acrl/acrlpubs/crlnews/backissues2004/may04/reflectionprivilege.cfm

Hightower,  B., Rawl, C.,  & Schutt, M.  (2008). Collaborations for delivering the library to students through WebCT. Reference Services Review, 35(4), 541-551.

Holobar, J.C. (2006). Electronic reserves and course management software: A collaborative model for large institutions. Journal of Interlibrary Loan, Document Delivery & Electronic Reserve, 16(4), 65-71.

Katz, R. N. (2006). ECAR Study of Undergraduates and Information Technology. Retrieved January 2, 2007 from,  http://www.educause.edu/ir/library/pdf/EKF/EKF0607.pdf

Lenholt, R., Costello, B.  & Stryker, J. (2003). Utilizing Blackboard to provide library instruction: Uploading MS Word handouts with links to course specific resources. Reference Services Review, 31(3), 211-218.

MacDonald, K. & vanDuinkerken, W.  (2005). Distance education and virtual reference: Implementing a marketing plan at Texas A&M University. Journal of Library & Information Services in Distance Learning, 2(4), 29-40.

Morgan, Glenda.  2003. Faculty use of course management systems. ECAR, 2003. Retrieved  July 27, 2005,  from http://www.educause.edu/ir/library/pdf/ers0302/rs/ers0302w.pdf

Roberts, G. (2003). The yin and yang of integrating TILT with Blackboard. Computers in Libraries, 23(8), 10-12, 54-56.

Sander, H., van Vuren, A. & dePlessis, T. (2006). LIBRARY LIVE: Embedding and contextualizing information resources in the virtual learning environment. IATUL Annual Conference Proceedings, 16, 1-9. Retrieved December 27, 2007 from Library, Information Science & Technology Abstracts database (26225962).

Shank, J.D. & Dewald, N.H.  (2003). Establishing our presence in courseware: Adding library services to the virtual classroom. Information Technology and Libraries, 22(1), 38-43

Waddoups, G. (2002). Blackboard Vital Signs Survey. Brigham Young University. Provo, UT: Center for Instructional Design

West, R. and Kennedy, M.  (2005). Evaluation of Brigham Young University’s Blackboard Implementation and Its Effects on the Instructional Experience. Brigham Young University. Provo, UT: Center for Instructional Design.

Zastrocky, Michael, Marti Harris, Jan-Martin Lowendahl. (2007). E-Learning for higher education: Course Management Systems. Gartner Industry Research ID # G00125522. Published 5 January 2007.  Retrieved February 7, 2007, from: http://www.gartner.com

Ziegenfuss, D. H. & McCloskey, J. M.  (2004). A consideration of Docutek’s electronic reserve system in an university’s courseware environment. Journal of Interlibrary Loan, Document Delivery & Electronic Reserve, 15(1), 87-97. 

APPENDIX

1-Did you use the Course Research Page under the Library Services tab?

   Yes         No

   (If you answered “No”, please do not continue with the survey.)

---------------------------------------------------------------------------------------------------

2-Check each of  the ways you learned about this page.

   Course Syllabus

   Instructor informed me

   Announcement on Bb Welcome page

   Saw it on the plasma screen in the library

   Clicked the Library Services Tab in my course

   Someone at the library told me about it

   I learned about it in my library instruction class

   Classmate told me about it

   Other, please explain ___________________________________________________

3- After visiting the page the first time, how did you access it subsequently?

   I bookmarked the page

   I logged on to my Blackboard course

   Other, please explain ______________________

4-Was this page

   Very easy to use        Easy to use            OK            

   Hard to use, please explain_______________________________________

5-Did the page help you identify sources for your research project/paper?

   Yes        No 

   If No, was it because:

     The resources were not specific to my topic

     The resources appeared (seemed) to be appropriate, but didn’t contain the information I needed

     I couldn’t figure out how to use the resources

     Other, please explain___________________________________

6-Do you feel your research project/paper was better because you used the page?

   Yes        No

   Please explain why or why not_______________________________________________

7-Would you recommend it to other students?

   Yes         No

   Please explain why or why not ________________________________________

8-What are your  suggestions for making the Course Research page more useful?

9-Please tell us about anything you found on the page that you were previously unaware of.

10- Is there anything else you’d like to tell us about this page?

 NOTE: Questions for the survey were adapted with permission from the Oregon State University Libraries Interactive Course Assignment Project.