The expansion of online learning throughout the world has generated interest from within the education profession for the identification of effective teaching practices for Web-based instruction. One particular facet of interest from the field can be found in the identification of those attributes specific to faculty roles and competencies associated with effective online course facilitation. Over the past two decades, investigators have identified and validated lists of roles and competencies as viewed by administrators, faculty, and students. In one such study, Bailie (2006) sought to determine whether a consensus in the ranking of a previously identified list of competencies could be reached between online faculty and students. While the panel of experienced online faculty and learners involved in the 2006 study did demonstrate a consensus toward the criticality of a list of online faculty competencies (to be reviewed in this paper), the potential for change in pedagogical approach and instructional strategy of a field that continues to emerge warrants an updated examination of whether a consensus is still possible. This interest is shared by others that have also investigated this facet of education for more than the past decade (Davis, Naughton, & Rothwell 2004; Egan & Akdere 2005; Varvel 2007).

The purpose of this sequel study was to revisit the Bailie (2006) investigation to determine whether a consensus of the ranking of competencies associated with effective online faculty instruction as perceived by two informed constituent groups, practicing online faculty and online students, could be re-established. The main research question to be addressed in this study was: Can a newly seated group of online faculty and students establish a consensus concerning the criticality of online teaching competencies demonstrated by faculty. The intended result of this investigation is to determine whether those competencies that were previously agreed upon by distance education participants could still be met with consensus, when collectively viewed by those who are clearly the most involved in the endeavor: online faculty and online students.

Review of Related Literature

• Over the past decade, a number of investigations have examined the question of distance learning faculty competency. Such studies have spanned multiple academic systems that represent many regions of the world, including the continents of North and South America, Europe, Asia, and Australia. The contemporary literature details investigations of online faculty competency, including those completed by Abdulla (2004), Bailie (2006), Blackerby (2002), Egan & Akdere (2005), Simon (2002), Varvel (2007), and Williams (2000). A review of these investigations establishes that further examination of the various attributes and characteristics of Web-based teaching competencies should be routinely considered so that the findings can be integrated into selection and training opportunities for future online practitioners.
  
  
• Three investigations included in the literature became the focal point of the study by Bailie (2006). The three investigations included Thach (1994), Williams (2000), and Abdulla (2004). Thach (1994) conducted a study to identify the roles, outputs, and competencies necessary at a time when education at a distance consisted of delivery through correspondence, telecourses, and video recording or conferencing (as it predated online instruction), while the other two focused on Web-based instruction. Each of these examinations offered recommendations for further study on the changing role of distance learning faculty competencies.
  
  
• Participants in the Thach (1994) study identified 11 roles and 51 competencies demonstrated by effective distance learning faculty. Thach’s primary conclusions were that the competencies identified were not of equal importance across roles and interpersonal communication skills were determined to be the most critical competency (by frequency) across all roles.
  
  
• Participants in the Williams (2000) study identified 13 roles and 30 competencies demonstrated by effective online learning faculty. Williams’ primary conclusion was that the importance of roles and competencies varies depending on the distance education environment. Williams also concluded that a Web-based Delphi approach was appropriate for studying competencies in the field of distance education and argued that ongoing changes associated with technology advancement and the novelty of the field required ongoing investigation and further research.
  
  
• Participants in the Abdulla (2004) study identified 4 roles and 28 competencies demonstrated by effective online learning faculty. Abdulla’s primary conclusions were that students rated the online instructor’s intellectual role as being the most important, and that effective communication skills are necessary to encourage student interaction. When compared to the perceptions of the professional experts in the studies of Thach (1994) and Williams (2000), Abdulla (2004) identified differences in the perceptions of students regarding the most important competencies of online instructors.
  
  
• With an interest in furthering the validation of the competencies regarded as important to the distance education profession, the Bailie (2006) investigation focused on the examination of online faculty competencies as perceived by both faculty teaching undergraduate students online and undergraduate students who were enrolled at a single university in the U.S. Midwest. Through a comparison of the outcomes of previously completed studies that centered on the independent perspectives of these distinct groups (Abdulla, 2004; Thach, 1994; Williams, 2000), a list of competencies was mutually acknowledged, thereby demonstrating that a consensus view between the groups was indeed possible (see Table 1). The implication of this investigation was that a multidisciplinary constituency group, consisting of online faculty and students, realized consensus in the selection of competencies that had been identified in previous studies.
  

Bailie (2006) recommended that additional investigations should be conducted concluding that “as Web-based instructional delivery proliferates…, further study of how faculty competency is influenced by the changes in technology is strongly encouraged. Also noted was that “a follow-up study should be completed at 5- and 10-year intervals to consider whether the identified competency list has changed as a result of advancements in pedagogical approach, practitioner and consumer interests, and online staff development initiatives” (p. 53). Other investigators reporting studies in the area of online faculty competency have consistently offered similar recommendations (Williams, 2000; Abdulla, 2004; Davis, Naughton, & Rothwell, 2004; Egan & Akdere, 2005; Varvel, 2007).

Egan & Akdere (2005) concluded that the identification of distance learning roles and competencies should be a result of interchanges between key stakeholders, theory and research, and current and emerging technologies. They suggested that an ongoing exchange would lead to clarification and development of key competencies that, in turn, could affect training and development in formal and informal educational settings. They went on to note that, “because of the dynamic nature of distance education, the aforementioned exchange must be ongoing and continuous. Failure to engage in stakeholder exchange regarding the description of needed competencies and clarification regarding related training and development could lead to stagnation in the field and misaligned delivery of formal and informal education” (p. 90).

According to Davis, Naughton, and Rothwell (2004), the examination of roles and competencies serves both as a basis for the clarification of a profession and as an update regarding newer practices. The results of such examinations have influenced related policy, formal training and certification programs, curriculum content, and individual approaches to professional development, and have provided comparisons of changes over time (as cited in Egan & Akdere, 2005). Varvel (2007) also suggested that technologies, incoming faculty, student needs, and curriculum are always changing, so as time passes and technologies change, the competencies can and should be updated.

Table 1. Comparison of Final Competency Rankings

Rank	Thach (1994)	Williams (2000)	Abdulla (2004)	Bailie (2006)
1	Interpersonal communication skills	Collaboration/ teamwork skills	Content knowledge	Content knowledge
2	Planning skills	Basic terminology knowledge	Facilitation skills	Feedback skills
3	Collaboration/ teamwork skills	Interpersonal communication skills	Organizational skills	Interpersonal communication skills
4	English proficiency	English proficiency	Planning skills	Organizational skills
5	Writing skills	Knowledge of distance learning	English proficiency	Knowledge of distance learning
6	Organizational skills	Writing skills	Presentation skills	Presentation skills
7	Feedback skills	Questioning skills	Interpersonal communication skills	Collaborative learning skills
8	Knowledge of distance learning	Collaborative learning skills	Learning styles and theory	English proficiency
9	Basic terminology knowledge	Adult learning theory	Teaching strategies and models	Learning styles and theory
10	Technology access	Knowledge of support service	Skills with Internet tools	Skills with Internet tools

This sequel investigation sought to reaffirm the competencies associated with effective online undergraduate faculty instruction as perceived by two informed constituent groups--practicing online faculty and online students. The results of this investigation should help to authenticate the competencies that have previously been considered by distance education practitioners and participants to be most prevalent in effective online instructors as collectively viewed by those who are clearly the most involved in the endeavor--the faculty and students.

Participants

In preparation for the study, a formal review by the Institutional Review Board (IRB) of the participating institution was conducted to further assure that the proposed research design was appropriate, that associated risks to participants were minimized, expected benefits were maximized, and that consent procedures, confidentiality provisions and safeguarding of data were adequately addressed. Subsequently, a heterogeneous panel of experienced online students and faculty was seated to assist in the determination of online faculty competencies specific to the university level. Participants for this study were not randomly selected given the investigative design relied on the involvement of informed participants. Toward this end, a student sample demonstrating a minimum equivalency of one year of experience in an online program, and faculty having conducted a minimum of five online course deliveries were considered eligible candidates for this investigation. Based on this qualification profile, representatives of the administrative staff of the online campus nominated faculty members that met the candidate profile. In addition, administrators asked faculty to post an announcement describing the study in each of their courses. This announcement served as an invitation to students meeting the designated qualification profile to participate. Enrollment in the study was closed when the target sample of 30 individuals representing an equally distributed heterogeneous group of consenting adult online faculty and students (15:15) had been qualified. Ultimately however, only 26 of the invited candidates (13:13) advanced to successfully complete the prescribed protocol necessary to be seated as a panelist.

Method

As with the Bailie (2006) investigation, a modified Delphi technique was used in this study to define the perceptions of current online faculty and students regarding online faculty competencies. Since its development in the 1950s, the Delphi method has been widely used to inform technology, education, and other fields (Cornish, 1977). The selection of a Delphi approach has been, in general, based on the desire to understand a narrowly defined issue. In the case of studies that focus on competencies, the Delphi approach is used to clarify, update established competency lists, as well as to support related future development (Egan & Akdere, 2005). The Delphi technique employs an open-ended questionnaire that is given to a panel of informed participants to solicit specific information about an explicit content area. Subsequent probes allow panelists to rate the relative importance of items. Through a series of inquiries, the process is designed to determine whether a consensus of informed opinion between panelists is possible.

The modified Delphi technique is quite similar to the Delphi in approach (i.e. a series of inquiries of experts) and intent (i.e., to discover whether the panel can arrive at a consensus). However, the primary difference is that the first round of the modified Delphi begins with panelists being offered a set of pre-determined items. These items are drawn from various sources including the findings of previous studies, reviews of contemporary literature, or interviews with content experts. The foremost benefit of the modified Delphi technique is that it provides a recognized foundation from previous work (Custer, Scarcella, & Stewart, 1999).

The instrumentation created for the Bailie (2006) investigation remained basically the same for the follow up study. The design team previously seated for the 2006 study had recommended that a list of competency definitions be included in the instrument. In response, the definitions created by Williams (2000) were included once permission for their use was granted by the author.

The first Delphi probe was constructed to solicit the 15 leading competencies perceived by the current panel as most important, or critical, to an effective online instructor. The panel was presented with the list of 20 primary competencies used in the 2006 study, as gleaned from the studies of Thach (1994), Williams (2000), and Abdulla (2004), including a definition of each. Individual participants were asked to reduce the original list of 20 competencies to a list of 15 competencies that they perceived as being the most important. They were encouraged to add to or delete from the existing entries. The objective of this round was to identify the 15 leading competencies that were perceived by the current experts as the most important competencies to be demonstrated by an online instructor. A comment section was provided for participants to “write in” additional competencies that were not included in the established list. The frequency of selection for each item from the combined panels were calculated and ranked.

The Delphi Round 2 survey was constructed based on the leading competencies identified by the combined panel in the first round survey. From this close-ended list, participants were asked to determine the criticality of the 15 competencies that were identified during the first round by assigning a value from a 4-point Likert scale of importance that ranged from 1 point representing “very important” to 4 points for “unimportant.” As with similar Delphi studies, the interquartile range (IQR) was selected to determine the level of consensus. The IQR, which is the difference between the 1st quartile and 3rd quartile of the data set, is calculated by applying a formula of IQR = Q3 - Q1 (Williams, 2000). The IQR was used to determine the level of consensus and was calculated by first establishing the 1st and 3rd quartiles using the following formulas: Q1 = (n+1)/4 and Q3 = 3(n+1)/4. Adopting the approach used in the previous studies, consensus was defined as being achieved when an IQR of 1 or less was attained.

Results

The optimal outcome of the Delphi technique is the establishment of a consensus among participants. In this investigation, the survey was designed to determine whether a consensus relative to the importance of specific online faculty competencies could be established between two invested groups of participants. Table 2 presents the resulting frequency distribution from Delphi Round 1 in which the 26 participants were asked to select 15 entries from the list of 20 primary competencies used in previous studies. Added to this list were the four additional competencies offered by participants, including one from a faculty member and three from student members.

 

Table 2. Descriptive Statistics of Combined Group Ratings, Delphi Round 1

Competency	f	Response %
Feedback skills	26	100
Content knowledge	25	96.2
Organizational skills	25	96.2
Interpersonal communication skills	23	88.5
Facilitation skills	22	84.6
English proficiency	22	84.6
Questioning skills	22	84.6
Skills with Internet tools	21	80.8
Planning skills	20	76.9
Writing skills	20	76.9
Skills in collaborative learning	19	73.1
Knowledge of distance learning	18	69.2
Adult learning theory	18	69.2
Teaching strategies and models	17	65.4
Learning styles and theory	16	61.5
Email efficiency	1	n/a
Classroom assessment technique	1	n/a
Multicultural competence	1	n/a
Student engagement techniques	1	n/a

Note. n/a = added as a write in by participants.

The second round directed the panels to rate the criticality of the 19 competencies identified in round one. It should be noted that two panelists, one faculty and one student, elected to not participate in round two. Table 3 presents the median, standard deviation, and IQR calculations from Delphi Round 2. Because a complete consensus was established, a third round was not necessary.

As individual groups, the Delphi Round 2 data denotes that the online faculty panel as a whole rated all 15 competencies as important, with a mean score range of 1.0 to 2.25. However, the standard deviation indicated that the majority of the responses were somewhat loosely clustered around the mean. An absolute consensus on the importance of four competencies; content knowledge, feedback skills, interpersonal communication skills, and student engagement techniques, was indicated by an IQR of 0. A strong consensus within the faculty panel was recognized with an IQR of 1 for 13 of the competencies. Within the faculty group, there was one competency that fell outside the IQR of 1, as Learning Style and Theory resulted in an IQR of 2 (albeit rated as important by all faculty participants).

Table 3. Descriptive Statistics of Combined Panel Ratings, Delphi Round 2

Competency	M	Mdn	SD	IQR
Feedback skills	1.08	1	0.283	0
Content knowledge	1.17	1	0.381	0
Organizational skills	1.54	1	0.721	1
Interpersonal communication skills	1.13	1	0.338	0
Facilitation skills	1.33	1	0.637	1
English proficiency	1.38	1	0.495	1
Questioning skills	1.25	1	0.433	1
Skills with Internet tools	1.17	1	0.381	0
Planning skills	1.67	1.5	0.762	1
Writing skills	1.25	1	0.443	1
Skills in collaborative learning	1.33	1	0.637	1
Knowledge of distance learning	2.17	2	0.702	1
Adult learning theory	1.79	2	0.658	1
Teaching strategies and models	1.67	1.5	0.762	1
Learning styles and theory	1.83	2	0.702	1
Email efficiency	1.46	1	0.588	1
Classroom assessment technique	1.71	2	0.588	1
Multicultural competence	1.71	2	0.690	1
Student engagement techniques	1.17	1	0.381	0

Note. IQR = interquartile range.

The online student panel as a whole also identified each of the competencies to be of importance. The mean scores for the student group ranged between 1.0 and 2.0. In this case, the standard deviation indicated that the responses were rather tightly clustered around the mean. Absolute consensus was realized in six of the fifteen competencies for this grouping based on the IQR of 0, with an IQR of 1 for the remaining thirteen.

For the combined group response of both faculty and students regarding the perceived importance of the competencies, the mean values indicated that all of the 19 competencies were perceived as important, although the standard deviation for Round 2 demonstrated a loosely clustered grouping around the mean for the competencies. In Delphi Round 2, the standard deviation was tightest for the four competencies where the IQR was 0: (a) feedback skills, SD = 0.283; (b) interpersonal communication, SD = 0.338; (c) student engagement techniques, SD = 0.381; and (d) content knowledge, SD = 0.381. Because the IQR for all 15 competencies was either 0 or 1 (as seen in Table 3), a solid consensus was demonstrated within the combined group, thereby validating that a longitudinal consensus was possible.

Discussion

A number of competency investigations have been included in the literature, conducted with an interest toward the identification of distinct characteristics common to effective online faculty. However, the focus of these investigations has overwhelmingly emphasized the perceptions of faculty and administrators of distance education programs. In the interest of furthering the analysis of the competencies regarded as important to the online teaching profession, this follow up investigation focused on the examination of online faculty competencies as perceived by both faculty teaching undergraduate students online and undergraduate students who were enrolled at a single university. Specifically, the study considered the question of whether such a group could arrive at a consensus on the criticality of online teaching competencies.

A two round modified Delphi probe was conducted in this study. The goal of the first round was to reduce a list of 20 previously identified competencies for effective distance learning faculty to a list of 15. During this round, an additional four competencies were introduced by the panel. The panel was able to easily accomplish this task. The Delphi 2 survey found that participants rated all 19 of the competencies as being at some level of importance (i.e., the criticality), and a solid consensus within the combined group was established when measured against the IQR.

Comparisons of the leading competencies identified for online instruction and the differing modalities from previous investigations support that the outcomes of this study are representative of those contained in the literature. This investigation confirmed that when a modified Delphi approach was instituted, competencies that were identified by similar research studies over the past decade continue to be regarded by vested constituents as important. Finally, it was shown that a consensus between what online faculty and online students perceived as important instructional competencies continues to be possible.

The intended outcome of this collaborative examination was a further validation of a recognized set of online faculty competency data, leading to a greater appreciation for the influence that certain attributes have on student satisfaction. Although technological advancements and support for the changing distance learning environment served to augment a new generation of educational activity, ultimately high-quality instruction will continue to be based on an array of distinct competencies attributed solely to instructors and their application of sound pedagogical practice. The constant transformation of technology would appear to make the classification of competencies held by online educators an interminable process, whereas emerging generations of technology will impact how the online faculty member most effectively apply their skills. The results of this study suggest that, at least over the short spans of five years, competencies previously identified as being critical continue to be held in high regard. It should come as no surprise to the active online practitioner, that among the highest regarded competencies are those directly associated with instructional immediacy. Through the timely validation of competencies, it is hoped that the outcomes of investigations such as this will contribute to the evolving list of key instructional attributes associated with effective online instruction.

References

Abdulla, A. G. (2004). Distance learning students’ perceptions of the online instructor roles and competencies. Dissertation Abstracts International, 51(07), 2409A.

Bailie, J. L. (2006). Effective distance education competencies as perceived by online university faculty and students. Available from ProQuest Dissertations and Theses database. (UMI No. 3406623)

Blackerby, C. B. (2002). A forecast of competencies in implementing professional development programs for teaching faculty of community colleges participating in the virtual college.

Dissertation Abstracts International, 63(07), 2507A. (UMI No. 3060768)

Cornish, E. (1977). The study of the future. Washington, DC: World Future Society.

Custer, R., Scarcella, J., & Stewart, B. (1999). The modified Delphi technique: A rotational modification. Journal of Vocational and Technical Education, 15(2), 50-58.

Davis, P., Naughton, J. & Rothwell, W. (2004). New roles and new competencies for the professional. T+D, 58(4), 26-36.

Egan, T. M., & Akdere, M. (2005). Clarifying distance education roles and competencies: Exploring similarities and differences between professional and student-practitioner perspectives. The American Journal of Distance Education, 19(2), 87-103.

Simon, J. K. (2002). Perceptions of Web-based instruction experts regarding the roles, outputs, and competencies needed in the field of Web-based instruction: A model for training and development. Dissertation Abstracts International, 63(12), 4247A.

Thach, E. C. (1994). Perceptions of distance education experts regarding the roles, outputs and competencies needed in the field of distance education. Dissertation Abstracts International, 55(10), 3166A. (UMI No. 9506728)

Varvel, V.E. (2007). Master Online Teacher Competencies. Online Journal of Distance Learning Administration, 10(1), Spring 2007 . Retrieved from: h ttp://www.westga.edu/~distance/ojdla/spring101/varvel101.htm

Williams, P. E. (2000). Defining distance education roles and competencies for higher education institutions: A computer-mediated Delphi study. Dissertation Abstracts International, 61(04), 1325A.