Selecting Delivery Systems and Media to Facilitate Blended Learning: A Systematic Process based on Skill Level, Content Stability, Cost and Instructional Strategy

Atsusi Hirumi
University of Central Florida
Orlando, FL USA
atsusi.hirumi@ucf.edu

George Bradford
University of South Australia
george.bradford@unisa.edu.au

Leanne Rutherford
United States Army Intelligence Center for Excellence
leanne.rutherford@us.army.mil

Abstract Like others, the US Army, is examining the use of emerging instructional technologies, such as but not limited to blended learning (BL) to optimize training. The problem is there are neither established formulas nor published algorithms for determining which aspects of a course to put online and to administer face-to-face (f2f) to facilitate blended learning. To address the problem, a team of Instructional Designers examined existing training and formulated, tested, refined and transferred a process for analyzing and nominating specific aspects of military intelligence (MI) coursework for either residential (f2f) and/or distributed (online) delivery. The resulting process consisted of three basic stages that addressed five primary questions: Stage I – Basic Instructional Mode Selection. “What aspects of a course should be nominated for blended learning, conventional classroom learning, or distance learning?” Stage IIa – Primary Distance Learning Delivery System Selection. “For those aspects nominated for Distance Learning, what primary system should be used to deliver the contents?” Stage IIb – Primary Face-to-Face Instructional Setting Selection. “For those aspects nominated for Face-to-Face instruction, what primary settings should be used to deliver the contents?” Stage IIIa – Instructional Strategy Selection. “What instructional strategy should be used to design and deliver the coursework?” Stage IIIb – Media and Communication Tool Selection. “What specific media and telecommunications tools should be used to facilitate Distance Learning, Face-to-Face instruction, and Blended Learning coursework?” The process seeks to optimize training by addressing (a) basic skills and knowledge through the use of distributed learning technologies, and (b) higher order thinking skills in conventional, face-to-face classroom settings. Three additional factors were considered for making media selection decisions including cost, stability and instructional strategy. Keywords: Media Selection, Blended Learning, Hybrid Training,

Introduction

Since 9/11, training requirements for United States Army Intelligence (AI), like other branches of the military, have increased exponentially. However, the time available to prepare MI soldiers for active duty has either decreased or remained the same. To address this challenge, the US Army Intelligence Center of Excellence (USAICoE) turned to technology-based training solutions, including but not limited to blended learning.

Blended learning (BL) combines the engaging benefits of traditional instructor led training with the advantages brought by the use of a variety of technologies to create optimum learning programs (Alvarez, 2005). Seen as a natural evolution of instruction methods, the approach has been found to enhance student learning and reduce attrition rates in higher education (Dzuiban, Hartman, & Moskal, 2004) and is replacing e-learning in corporate settings (Bersin & Associates, 2003).

The problem is there are no universally recognized formulas, algorithms or ratios for blending the design of a course or training program. Different models and case studies have been published for facilitating BL (c.f., Bonk & Graham, 2006), but it appears that there are no replicable guidelines for determining which specific aspects of a course to put online and what to administer face-to-face (f2f) to facilitate BL. Indeed, every organization approaches BL in their own particular way.

To facilitate BL at USAICoE, a team of five experts in Instructional Design and Technology developed a systematic process for analyzing MI coursework and nominating specific aspects of MI coursework for f2f, distributed and blended learning. The analyses included critical examinations of existing course information, lesson plans and lesson materials, and discussions with USAICoE personnel. The resulting process was established, tested and refined as the team conducted the analyses in three iterative phases. Initially, team members analyzed one course together to first establish the BL media selection process. Team members than analyzed four courses individually to test the process and discussed results and shared insights to improve the process. Four additional courses were analyzed by individual team members to refine the process. Throughout the development, testing and refinement of the process, two members of the team reviewed all analyses for consistency across courses. Staff members at USAICoE were then trained to apply the process, make additional refinements, and facilitate transfer.

The resulting process consisted of three basic stages that addressed five primary questions, including:

• Stage I – Basic Instructional Mode Selection. “What aspects of a course should be nominated for blended learning (BL), conventional classroom learning (f2f), or distance learning (dL)?”
  
  
• Stage IIa – Primary dL Delivery System Selection. “For those aspects nominated for dL, what primary system should be used to deliver the contents?”
  
  
• Stage IIb – Primary F2F Instructional Setting Selection. “For those aspects nominated for f2f instruction, what primary settings should be used to deliver the contents?”
  
  
• Stage IIIa – Instructional Strategy Selection. “What instructional strategy should be used to design and deliver the coursework?”
  
  
• Stage IIIb – Media and Communication Tool Selection. “What specific media and telecommunications tools should be used to facilitate dL, f2f and BL coursework?”

This article documents the application of the BL Media Selection Process to enable readers to analyze and formulate f2f, dL and BL nominations for their own training programs and coursework. First, we list seven assumptions that guided the development, implementation, testing and refinement of the BL media selection process. We then describe each stage, detailing the application of the process, including charts that illustrate the flow of media selection questions and decisions, and a template for reporting the analyses of coursework. We conclude by discussing the qualification and preparation of those who may be asked to apply the process.

Assumptions

The development, testing and refinement of the BL media selection process were based on the following assumptions:

1. “Official” time will be allocated for both active and reserve soldiers to complete dL components of a course, whether that is prior to or during the course.
   
   
2. Learning may be viewed hierarchically. High-order skills involving application, analysis, synthesis and evaluation, often require knowledge and comprehension of basic facts, concepts, principles and procedures.
   
   
3. dL technologies may be used to facilitate learning of higher-order thinking skills. However, USAICOE currently does not have the resources necessary to assign an expert/instructor to facilitate dL lessons. Thus, for this initiative, dL nominations are self-instructional.
   
   
4. When produced, dL nominations will be created as sharable content objects (SCO’s) based on SCORM standards currently in use by the DoD.
   
   
5. Instructional design is more important (accounting for greater variance in learner attitudes and performance) than instructional media. In other words, lesson and course design affects learner attitudes and performance more than that use of different media (Clark, 1994; Salomon, 1979).
   
   
6. There is no one "best" media for teaching and learning and not all media can facilitate specific instructional methods with equal cost and/or effectiveness.
   
   
7. Individuals tasked with conducting course analyses for BL (hereby referred to as BL course designers or BL designers) have working knowledge of the systematic approach to training (SAT) and key principles of instructional design.

Systematic BL Media Selection Process

Flowcharts illustrate Stages I-III, including guiding questions and key decisions for formulating f2f or dL course nominations. In short, Stage I helps determine the basic instructional mode: In other words, whether lessons or parts of lessons within a course should be facilitated in residence (f2f) or online (dL). Stage II flowcharts help determine the primary instructional delivery system for dL and the primary instructional setting for f2f lessons. Stage III flowcharts define the instructional strategy, specific media (e.g., audio, video, text or graphics) and telecommunication tools (e.g., electronic bulletin boards, podcasts, static or interactive web pages) that should be used to facilitate learning in either a dL or f2f environment. After Stage III questions and decisions have been made, course designers are encouraged to compile all the results into a course analysis report, described at the end of this section of the article.

Two aspects of the BL media selection process posited by this study require qualification before describing and applying the process. First, Stage I of the media selection process nominates lessons or portions of lessons that address higher-order thinking skills for f2f training (rather than dL). As such, dL lessons that address higher-order thinking skills do not follow the logical sequence of decisions posed in process. Although it is assumed that dL technologies may be used to facilitate learning of higher-order thinking skills, the fundamental approach taken in this initiative focuses on optimizing residential training by addressing basic skills and knowledge in a dL format and higher order thinking skills and complex procedures in a f2f format. Thus, when analyzing courses containing dL lessons that address higher-order thinking skills, BL designers have at least three choices, they may decide (a) not to analyze the dL lessons and note why in related course analysis reports, (b) skip Stage I and Stage II of the BL media selection process and begin the analysis of the dL lesson(s) at Stage IIIa to examine the instructional strategy applied in the lesson(s) and assess the appropriateness of using dL technologies to facilitate the lesson, or (c) skip Stages I, II and IIIa of the process and begin the analysis of the dL lesson(s) with Stage IIIb to assess appropriateness of using dL technologies to facilitate that lesson.

The second qualification is that alternative taxonomies may be used to distinguish higher and lower order skills. For this study, Bloom’s (1956) original taxonomy for classifying learning objectives was applied due to the familiarity and current use of the taxonomy for designing and delivering instruction at USAICoE. However, we recognize a number of viable taxonomies for classifying learning outcomes, including but not necessarily limited to the newer version of Bloom’s taxonomy published by Anderson & Krathwohl (2001) as well as Gagne’s (1985) Taxonomy of Learning Outcomes. The criteria used to distinguish lower from higher order skills for this study using Bloom’s original taxonomy (but may also be used with other taxonomies) is that the application of lower order skills typically leads to one correct answer/solution or there is only one correct way to apply the skill. The application of higher-order skills, in contrast, may result in multiple correct answers. Higher order skills may also be applied in different ways, yet still lead to a “correct” answer.

Stage I –Basic Instructional Mode (f2f or dL) Selection

Stage I of the overall media selection process answers the question, “What aspects of a course and related lessons should be nominated for blended learning (BL), conventional classroom learning (f2f), or distance learning (dL)?” Figure 1 depicts the sequence of key questions and decisions to be made during Stage I. In short, Stage I helps determine the basic instructional mode (BL, f2f or dL) that should be used to facilitate a lesson or parts of lesson within a course.

Stage I begins with the question, “Does the terminal learning objective (TLO) for a lesson address higher- skills (i.e., analysis, synthesis or evaluation according to Bloom’s taxonomy) or job-tasks that require the application of complex procedures and/or advanced psychomotor skills?” Armed with a curriculum map and related course information (e.g., course maps and lessons plans), BL course designers identify and classify the terminal learning objective (TLO) for a given lesson according to Bloom’s (1956) Taxonomy of Educational Objectives.

Figure 1. Flowchart depicting key questions and decisions to be made during Stage I
of Blended Learning Media Selection Process

If the answer to the question (1.1) is, “Yes, the TLO does addresses higher order skills or complex job-tasks,” then course designers go to Question 1.2. If the answer is “No, the TLO addresses lower order skills (i.e., knowledge, comprehension, or application according to Bloom’s taxonomy)” then BL course designers go to Question 1.3.

Question 1.2 asks, “Do any enabling objectives (ELOs) address higher order skills or complex job tasks?” In other words, do soldiers need to master any higher order skills or complex job tasks within a lesson to achieve the terminal learning objectives? To answer Question 1.2, BL designers identify and classify related enabling objectives according to Bloom’s taxonomy. If enabling objectives are not specified in lesson plans, BL designers should review related course materials to determine if any higher order skills or complex job tasks must be mastered or applied to achieve the TLO. At this point, BL course designers should also analyze related lesson materials and determine if the instructor or any aspects of the lesson spend significant time (e.g. >15-30min) covering fundamental skills. If parts of the lesson do address fundamental skills and/or knowledge, those parts should be forwarded to Question 1.3 and be considered for dL nomination. If the answer to Question 1.2 is, “Yes, ELOs address higher order skills or complex job-tasks,” then BL course designers nominate the entire lesson for f2f training and move to “B” (Stage II – Primary f2f Instructional Delivery Setting Selection). If the answer to question 1.2 is “No, ELOs do not address higher order skills or complex job-tasks,” then BL course designers go to Question 1.3.

Question 1.3 asks, “Do assessments of [specified] lesson objectives require expert review or observation?” In other words, is an expert necessary to properly assess students’ achievement of the objectives or can course developers readily program a computer or computer application to make valid and reliable assessments of students’ work? If the answer to Question 1.3 is, “Yes, assessments require expert evaluation or observation,” then the lesson is nominated for f2f training and course designers go to “B” on the flowchart. If the answer is, “No, assessments do not require expert evaluation or observation,” for the lesson (or portions of the lesson under analysis) then BL course designers go to Question 1.4.

Question 1.4 asks, “Do [any] instructional events and activities require f2f interactions?” In other words, do soldiers need to interact with the instructor or with other soldiers in a face-to-face setting to complete any instructional events, activities or assignments to achieve specified lesson objectives? If the answer to Question 1.4 is, “Yes, the instructional events and activities within the lesson require f2f interactions with instructor and/or other soldiers,” then the lesson is nominated for f2f training and course designers go to “B.” If the answer is, “No, the instructional events and activities within the lesson do NOT require f2f interactions with instructor and/or other soldiers” then course designers nominate the lesson or portion of the lesson under analysis for dL and go to “A” (Stage IIa – Primary DL Instructional Delivery System Selection).

Stage IIa –Primary DL Instructional Delivery System Selection

Stage II of the BL media selection process is separated into two processes or flowcharts, depicted in Figures 2A (for dL nominations) and 2B (for f2f nominations). Figure 2A depicts the key questions to be answered during Stage II for a lesson or portions of a lesson that is nominated for dL.

Stage IIa of the overall BL media selection process starts with Question 2.1, “Does [the] lesson or [any lesson] events require live synchronous interactions?” Although similar to Question 1.4 (in Stage I), this question focuses on the need for live (real-time) synchronous interactions that may be mediated by technology, rather than if face-to-face interactions are necessary. There are situations when live f2f interactions are not necessary yet live synchronous interactions are warranted. For example, when diagnosing the writing of relatively complex intelligence reports with an individual or small group, f2f interactions may not be necessary, but immediate feedback for questions and concerns that may be mediated through the use of interactive television or audio-conferencing may be desired. If the answer to Question 2.1 is “No, no live synchronous interactions are necessary,” BL course designers go to Question 2.2. If the answer to Question 2.1 is, “Yes, live synchronous interactions are necessary,” BL course designers go to Question 2.3.

Questions 2.2 and 2.3 both ask whether a visual is required to facilitate the lesson or parts of the lesson. In other words, are text, still or motion graphics, or video necessary to facilitate the lesson or parts of the lesson? If the answer to Question 2.2 is “No,” then BL course designers should nominate the lesson or the parts of the lesson under analysis for Pre-recorded audio lectures or Podcasts as the primary dL instructional delivery system and go to “C” (Stage IIIa Instructional Strategy Selection). If the answer to Question 2.2 is “Yes,” then BL course designers should consider nominating the lesson or the parts of the lesson under analysis for recorded desktop video conferencing, recorded webcasts, recorded TV broadcasts, Tegrity or other web-based training delivery systems stored in a learning management (LMS) or learning content management system (LCMS) as the primary dL instructional delivery system and go to “C” (Stage IIIa Instructional Strategy Selection). If the answer to Question 2.3 is “No,” then BL course designers should nominate the lesson or the parts of the lesson under analysis for Audio Conferencing as the primary dL instructional delivery system and go to “C” (Stage IIIa Instructional Strategy Selection). If the answer to Question 2.3 is “Yes,” then BL course designers should consider nominating the lesson or the parts of the lesson under analysis for interactive television, desktop video conference or webcasts as the primary dL instructional delivery system and go to “C” (Stage IIIa Instructional Strategy Selection).

Note: If BL designers intend to use recorded audio or video to deliver course contents, they must be mindful of the file size that will be transmitted across the network and the number of recipients to receive each file. Such calculations should force BL designers to consider the bandwidth of the whole network: you can have really great throughput for 95% of the route, but if the last 5% is slow (e.g., when using dial-up or similar connections), the slowest transmission becomes the effective throughput for the whole route. BL designers need to consider how to get the volume through, such as by sending smaller, timed and sequenced packets. Calculating file size and the number of recipients is critical because it's simple to record synchronous sessions but if the files cannot be transmitted, the system will fail and the cost of reconfiguration may be exorbitant.

Figure 2A. Flowchart depicting key questions and decisions made during Stage II of
Media Selection Process for dL nominations

 

Stage IIb – Primary Instructional Setting Selection for f2f Nominations

Figure 2B depicts the sequence of key questions and decisions to be made during Stage II of the overall media selection process for a lesson or portions of a lesson nominated for f2f training. It helps BL course designers determine if the f2f session should be facilitated in a conventional classroom, in a simulated field or in an actual field setting.

While it is recognized that f2f military training may also occur in a range (primary when live ammunition is being used), discussions with USAICOE representatives revealed that no military intelligence training requires a range so the option was not included Stage IIb and in the overall BL media selection process.

Stage II for f2f training begins with Question 2.4 which asks, “Is the training scenario-based?” In other words, does the training require the use of scenarios to achieve specified lesson objectives? If the answer to Question 2.4 is “Yes,” then BL course designers go to Question 2.5. If the answer to Question 2.4 is “No,” BL course designers go to Question 2.7.

Question 2.5 asks whether the lesson or a portion of the lesson includes collective tasks. In other words, do multiple groups need to work together to complete assignments and activities and achieve specified lesson objectives? If the answer to Question 2.5 is “Yes,” then BL course designers go to Question 2.6. If the answer to Question 2.5 is “No,” BL course designers go to Question 2.7.

Figure 2B. Flowchart depicting key questions and decisions made during Stage II of
Media Selection Process for f2f Training Nominations

Question 2.6 asks if an authentic environment is required to complete lesson assignments and activities and achieve specified lesson objectives. In other words, do soldiers need to complete job tasks in an environment that is basically the same as they would experience during active duty? If the answer to Question 2.6 is “Yes,” then BL course designers should nominate the lesson or the portion of the lesson for field training, plan the field training exercise (FTX) and go to “C.” If the answer to Question 2.6 is “No,” BL course designers should go to Question 2.7.

Question 2.7 asks if a simulated field environment is required to complete lesson assignments and activities and achieve specified lesson objectives. In other words, do soldiers need to complete job tasks in an environment that imitates what they would experience during active duty? If the answer to Question 2.7 is “Yes,” then BL course designers should nominate the lesson or the portion of the lesson for simulated field training (STX), plan the STX and then go to “C.” If the answer to Question 2.6 is “No,” BL course designers should nominate the lesson or portion of the lesson under analysis for conventional classroom instruction, plan for conventional classroom training and go to “C.”

Stage IIIa – Instructional Strategy Selection (for dL and f2f nominations)

After BL course designers complete Stage II of the BL media selection process, they should define the instructional strategy for each lesson. Stage IIIa defines the instructional strategy that, in turn, guides the design and sequencing of instructional events and activities for each lesson of a course. The design and sequencing of instructional events then drives the selection of specific media and communication tools during the next stage of the overall media selection process (Stage IIIb).

If an appropriate instructional strategy has already been specified for the lesson, and the nature and sequencing of related instructional events (e.g., Lecture (LE), Discussion (CO), Practical Exercise (PE), Demonstration (DM), or Test Exercise (TE)) have been documented in lesson plans, the BL course designers may simply refer to the instructional strategy while they complete the next stage of the overall process (Stage IIIb). If an instructional strategy has yet to be specified or if designers are considering revisions to the instructional strategy, they should proceed through Stage IIIa.

Figure 3A. Flowchart depicting key questions and decisions made during Stage IIIa
of the overall BL Media Selection Process

Stage IIIa begins with the definition of an instructional strategy. To define an appropriate instructional strategy, BL course designers should analyze the learning goals and objectives specified for a lesson and determine if, in general, a learner-center, experiential, conventional teacher-directed or other alternative approaches are warranted. For example, if the terminal learning objective (TLO) for a lesson addresses higher-order thinking skills (e.g., analysis, synthesis or evaluation), than a learner-centered approach may be most appropriate for facilitating achievement of the TLO (e.g., Problem-Based Learning, the 5E Instructional Model). In contrast, if the terminal learning objective (TLO) for a lesson addresses lower-order thinking skills (e.g., knowledge, comprehension or application), than a conventional, teacher-directed approach may be the most efficient method for facilitating achievement of the TLO (e.g., Gagne’s Nine Events of Instruction). Course designers must also consider the need for practical experience. If soldiers need to see, practice and/or otherwise experience the application to specified skills in real or simulated real-life context, then an experiential instructional strategy should be considered for the lesson.

BL course designers may choose from a variety of instructional strategies (e.g., problem-based learning, 5E instructional model) that are grounded in learning and instructional research and theory. Refer to Hirumi (2006, 2002a, 2002b) for further details and information on selecting and applying alternative instructional strategies grounded in research and theory. The application of grounded instructional strategies determines the design and sequencing of instructional events, including the standard types of events currently specified in lesson plans (i.e., LE, PE, CO, DM, TE and TR). After defining a strategy, BL course designers may answers questions 3.1-3.5 for each instructional event prescribed by the strategy. BL designers then compile their results to generate a lesson plan (LP) using TRADOC’s/USAICoE’s standard lesson planning format.

It is also important to note that alternative instructional events (other than the standard LE, PE, CO, DM, TE and TR) that were found in a few lesson plans (such as panel discussions, scenarios, and case studies) have been integrated into Stage IIIa flowchart under the standard event categories. For example, case studies and scenarios may be considered as types of practical exercises (PE) and panel discussion may be considered as a form of discussion (DE). After generating a lesson plan, BL course designers answer Question 3.6, “Have media and [telecommunication] tools already been selected?” If yes, the media selection process is complete and BL designers go on to verify their results with course managers and instructors. If no, BL designers go to Stage IIIb to select specific media and communication tools for each lesson.

Stage IIIb – Media and Communication Tool Selection

Stage IIIb helps BL course designers select the specific media and telecommunication tools that should be used to facilitate the f2f or dL lessons. Figure 3B illustrates the sequence of questions and decisions for selecting specific media and telecommunications tools.

Stage IIIb begins with Question 3.1, “Does the training support materials need to be changed or updated frequently?” If significant changes to course support materials (e.g., LPs, PE, MS PowerPoint TM slides, pdf documents) must be updated frequently (once a year or more), the answer to Question 3.1 is, “Yes,” and BL designers go select from a range of Low Cost media and telecommunication tools (listed in Figure 3A). If the answer to Question 3.1 is, “No,” BL designers further delineate the stability of the course materials by answering Question 3.2.

If course materials are considered very stable (with no significant changes predicted for more than 3+ years), then the course materials are considered “More” stable and BL course designers may choose from a complete list of media and telecommunications tools that range from High Development Costs for Very Stable Content to Low Development Costs for Changing Content. If significant changes to course materials are forecasted every 1-3 years, then the answer to Question 3.2 is “Less” stable and BL designers may choose from a range of High to Low cost media and tools for Stable and Changing content information and course materials.

As noted earlier under Assumptions, there is no one "best" media for teaching and learning and not all media can facilitate specific instructional methods with equal cost and/or effectiveness. Thus, the selection of specific media and communication tools from the list of viable options derived from answering questions posited in Stage IIIb is left to the discretion of BL course designers based on the design of lessons, the availability of human, technical and financial resources for their specific context. After identifying viable media and tool options based on Stage IIIb, there are additional questions that BL course designers may answer to help narrow the options and guide the selection of specific media and tools, including:

1. What is the nature of the educational experience? Do learners need to see graphics, motion video, listen to audio, read text? Would animations and/or simulations facilitate learning? If so, are they worth the time and cost?
   
   
2. Are f2f interactions planned? If so, when? What other interactions (student-student, student-instructor, student-content, student-community) are planned during the lesson?
   
   
3. Where are the learners located and how readily can they access the training? Are there soldiers in remote sites that need access? What media and telecommunication are currently available for use?
   
   
4. What kind of budget do you have? Is there time and resources to acquire additional media and communications tools? Are human resources available to provide technical and pedagogical support to use of planned media and tools?

Again, the degree of specificity that BL course designers prescribe media and tools for each dL and f2f nomination is left to the discretion of the designers based on their authority to dictate use and context.

Figure 3B. Flowchart depicting key questions and decisions made during Stage III
of Media Selection Process for dL Nominations Note: Audio and video files included
in lessons based on Stage IIIb of the BL media selection process are designed to be streamed.

Compilation, Comparison and Verification of Results

After analyzing all lessons within a course, BL course designers are encouraged to compile their results into a Course Analysis Report that includes:

1. Current Course Description
   
   
2. Blended, f2f and dL Nominations,
   
   
3. Related Sharable Content Objects (SCOs),
   
   
4. Constraints and Recommendations,
   
   
5. References, and
   
   
6. Appendices

At the beginning of the Course Analysis report, BL designers may provide a short course description based on the available course materials and the initial input gained from course managers, instructors and educational specialists : (a) to provide an overview of the scope and sequence of the course, including throughput, current status, and resources used for the analysis, for readers who may not be familiar with the course, and (b) to demonstrate good working knowledge of the course and add validity to the analysis for readers familiar with the course.

The second section of the report lists recommended Blended, f2f and dL nominations and provides a short description of each nomination. Three categories of nominations were defined: (a) Blended for lessons broken down into f2f and dL components, (b) f2f for entire lessons nominated for f2f delivery only, and (c) dL for entire lessons to be delivered using dL technologies only. The short description notes the prescribed learning outcomes, learner assessments, instructional strategy, and media/tool options and refers readers to the Lesson Analysis Report provided in the Appendix of the Course Analysis that presents further details and information for each nomination. The information necessary to compile Section 2 as well as all proceeding sections of the Course Analysis Report should be readily available in the Results section of each Lesson Analysis.

The section on Related Sharable Content Objects (SCOs) identifies SCOs that may be (re)used within a course or in other MI courses to increase return on investment. For this initiative, a SCO is defined as the smallest unit of instruction that contains at least an objective, a learning event or activity and an assessment, and it is assumed that each dL nomination will be produced as a SCO based on the latest SCORM standards that have been generated and adopted for use by the DoD. As BL designers prepare this section of the Course Analysis Report, they should compare the dL nominations made within the course as well as compare the dL nominations made in the course to the dL nominations made in other courses to identify areas of duplication and list related SCOs.

The section of Constraints and Recommendations lists the limitations faced during the analysis of each course, including (a) the lack of or discrepancies found in the course materials and information used for each analysis, and/or (b) design issues noted during each analysis, such as, but not limited to the lack of measurable enabling objectives, incongruent assessment method or inconsistencies found been LPs and actual course materials.

After BL course designers prepare an initial draft of the Course Analysis Report, they should ask Course Managers, Instructors and Educational Specialists to verify and provide feedback on the contents of the report to ensure accuracy and garner additional buy-in for proposed nominations. BL designers should then analyze the feedback and make revisions if necessary. If time permits, BL designers may want to gather input on prescribed blended, f2f and dL nominations immediately after Stage I of the overall course analysis and media selection procedure has been completed.

Qualifications and Preparation

To conduct proper course analyses for blended learning, BL course designers must have working knowledge of the systematic approach to training (SAT) and key principles of instructional design, including:

1. The use of outputs resulting from design tasks as inputs for subsequent tasks to ensure alignment between learning objectives, learner assessments and instructional strategy;
   
   
2. The selection and application of instructional strategies grounded in learning and instructional theories and research;
   
   
3. The classification of targeted learning outcomes based on Bloom’s taxonomy of educational outcomes, or other published taxonomies, such as Gagne (1985);
   
   
4. The use of targeted learning and performance outcomes as the foundation for course and lesson design and delivery; and
   
   
5. The use of conventional criterion referenced tests, as well as performance checklists and assessment rubrics based on targeted learning outcomes.

The BL designers should also be familiar with alternative media and telecommunication tools that may be used to facilitate learning in both f2f and dL environments, as well as the specific media and telecommunication tools available at USAICoE.

In preparation for course analysis, BL course designers should (a) acquire all of the instructional materials that have been prepared for the course, including lesson plans (LPs), worksheets, crosswalks, MS PowerPoint presentations, readings, and practical exercises, (b) generate a Course Curriculum Map based on the instructional materials to get an overview of the course by compiling terminal and enabling learning objectives, related tasks, assessment methods, and instructional events/activities that are associated with each lesson, and (c) gather input and buy-in for blended learning by interviewing and discussing key points with course managers, instructors and educational specialists.

Conclusions

This article presented a process for analyzing coursework, determining which aspects of a course to deliver online and to administer f2f, and selecting media to facilitate blended learning. Although it is recognized that distance and distributed learning (dL) technologies may be applied to facilitate the learning of higher-order skills, it was assumed that the organization delivering the training did not have the resources necessary to assign an expert/instructor to facilitate dL lessons. The process seeks to optimize training by addressing (a) basic skills and knowledge through the use of distributed learning (dL) technologies, and (b) higher order thinking skills in conventional, face-to-face (f2f) classroom settings. In other words, dL is recommended to facilitate knowledge, comprehension and the application of verbal information, basic concepts, and simple procedures so that more time can be spent during scheduled f2f sessions on the analysis, synthesis and evaluation of relatively complex information under varying conditions. Questions and concerns regarding cost, stability of the content information, and instructional strategy were also considered and were identified as key factors that drove formulation and application of the BL media selection process. Together, Stages I-III of the process, along with recommendations for compiling results, provide organizations with a replicable process for analyzing course contents for BL and determining which aspects of a course to put online and to administer f2f that may be repeated and refined over time.

Acknowledgements .

We would like to acknowledge and give special thanks to Dr. Rhonda Atkinson from Valencia Community College, and Anna Reese and Wendy Cydrus, graduate students at the University of Central Florida during the time of this study, who served as an integral part of the team that formulated, tested and refined the media selection process as well as David Gerloff and CACI, Inc. for endorsing the project, and to the men and women at the United States Army Intelligence Center of Excellence (USAICoE) at Fort Huachuca for their input and time contributed to the project and to our country.

References

Alvarez, S. (2005). Blended learning solutions. In B. Hoffman (Ed.), Encyclopedia of Educational Technology. Retrieved October 13, 2009, from http://coe.sdsu.edu/eet/Articles/blendedlearning/start.htm

Anderson, L.W. & Krathwohl, D.R. (Eds.). (2001). A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives. New York: Longman.

Bershin, J. (2004). The blended book of learning. San Francisco: Pfeiffer.

Bonk, C. J. & Graham, C. R. (eds.) (2006). The Handbook of Blended Learning: Global Perspectives, Local Designs. Hoboken, NJ : John Wiley & Sons.

Bloom, B.S. (Ed.). (1956) Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: Longman.

Clark, R. E. (1994). Media will never influence learning. Educational Technology Research and Development, 42(2), 21-29.

Gagne, R.M. (1985). The conditions of learning (4th ed.). New York: Holt, Rinehart, and Winston.

Hirumi, A. (2006). A framework for analyzing and designing e-learning interactions. In C. Juwah (ed.). Interactivity and Interactions in Distance and Online Education (pp. 46-72). London, UK. Kogan Page.

Hirumi, A. (2002a). A framework for analyzing, designing and sequencing planned e-learning interactions. Quarterly Review of Distance Education, 3(2), 141-160.

Hirumi, A. (2002b). The design and sequencing of e-learning interactions: A grounded approach. International Journal on E-Learning, 1(1), 19-27.

Salomon, G. (1979). Media and symbol systems as related to cognition and learning. Journal of Educational Psychology, 71(2), 131-148. doi:DOI: 10.1037/0022-0663.71.2.131