Improving Undergraduate Sociology Students' Presentation Skills through Reflective Learning in an Online Learning Environment

Natalia V. Smirnova
Instructor of English as a Second Language
Department of Foreign Languages
National Research University Higher School of Economics
Saint Petersburg RUSSIA
smirnovan@hse.ru

Irina V. Nuzha
Associate Professor of Teaching English as a Second Language
Department of Foreign Languages
National Research University Higher School of Economics
Saint Petersburg RUSSIA
inuzha@hse.ru

Abstract The increasing growth of online learning creates opportunities for language learners to improve their academic proficiency. However, quite often, foreign language educators claim that an online learning environment is not suitable for developing students' oral speaking proficiency. This paper is a case study of teaching academic presentation skills using a blended learning format. It takes a closer look at the existing web-based presentation skills pedagogy, discussing the challenges, perspectives of development, and methods of improving online academic presentation teaching through students' reflective learning practices. It is suggested that reflective learning stages applied to an online course design and proper face-to-face instruction mode foster students' academic oral proficiency and makes online learning highly effective. Keywords: blended learning, learning management system (LMS), reflection, pedagogical model, self-assessment, business speaking skills, oral presentation skills

Introduction

Presentation skills have gained attention both in higher education and among business trainers, as they constitute one of the core competencies of a professional (Linardopoulos, 2010; Raybould & Sheedy, 2005). Effective communication skills are a sign of professionalism and should be addressed as a key competency by educational institutions (De Grez, Valcke, & Roozen, 2009). Oral presentations are seen as one of the most important forms of public speaking (Kenkel, 2011).

Courses aimed at teaching oral presentation skills are traditionally conducted in a face-to-face environment, despite the remarkable growth of online learning that has occurred in higher education. Many researchers highlight an increased demand for online programs in the field of higher education though few institutions are ready to offer this mode to their students (Allen & Seaman, 2010).

This case study describes how the Department of Foreign Languages at the Saint Petersburg Campus of the National Research University Higher School of Economics used a learning management system (LMS) to improve students' academic presentation skills by developing their reflective learning skills. Results showed students successfully managed to improve this academic skill, though a number of challenges and limitations were met.

The Challenges in Delivering Communication Skills Training Online

Despite the fact that developing oral presentation skills are hardly ever a part of the online curricula, some researchers have been investigating the potential of web-based learning environments to foster oral communication skills and overcome a number of obstacles. There is a widespread belief that speaking skills can only be mastered in an environment where there is face-to-face interaction. Synchronous online learning solutions in the form of a virtual classroom could be a good alternative to the traditional face-to-face modality (Hrastinski, 2008a). As research shows, within synchronous online sessions, immediate feedback and the sense of community help learners to overcome frustration and isolation they might experience while communicating asynchronously (Hrastinski, 2008a).

Most studies focus solely on providing online communication skills trainings in business education. Business students increasingly demand synchronous delivery modes, as they are already used to studying various subjects online (Linardopoulos, 2010). Despite positive feedback on conducting courses in this style and students’ learning progress (Kuzma, 2011; Linardopoulos, 2010), synchronous online learning has some significant limitations.

One of the major drawbacks of this style is that students have to be tied to a fixed schedule and location where they must be online at the same time. Little control over the pace of instruction and inability to review the content are often mentioned among other key weaknesses of synchronous online courses. Although numerous studies prove that synchronous communication increases learners' psychological arousal (Kock, 2005) and motivation (Robert & Dennis, 2005), its key limitation is the lack of reflection on the process and outcomes of learning.

The key feature of asynchronous online learning, however, is its flexibility. It provides learners with information and training at any time they need it. Its elastic nature allows for more thoughtful, elaborate participation on the learner's end, as the access to the content is not limited. Students have more time to process the information of the course and develop their self-study, self-reflection, and critical thinking skills. In his analysis of the nature of collaboration within asynchronous and synchronous online seminars, Hrastinski (2008b) argues that asynchronous communication is more content related as it is focused on cognitive participation rather than personal participation. Talalakina (2012) adds that "indeed, an asynchronous mode of the discussion leaves room for more self-reflection and a more careful choice of language means" (p. 85). The increase of audio and video content added to traditional tools used within an LMS will also help minimize the effects of major limitations of asynchronous online learning – the feelings of boredom, isolation, and motivation loss (Ice, Curtis, Phillips, & Wells, 2007).

The majority of public speaking courses include online components as well as traditional face-to-face classroom interaction (Böhme, 2009). The present study supports the idea that blended learning is the most beneficial means of teaching. Fully online courses tend to neglect one of the most important components of a public speech: the audience (Böhme, 2009). Therefore, classroom-based seminars are seen as a way to provide students with an opportunity to experience real-time communication and focus on its extra-linguistic features (body language, interactive skills, eye contact).

Kuzma (2011) notes another two factors that can make online learning hard to implement are technological complexities and human capabilities. The first factor is usually associated with complicated technological procedures of incorporating information and communication technologies into the learning process. Kuzma highlights that staff training and information technology department support lead to successful implementation of the course, and research by Kenkel (2011) supports the notion that this problem can be solved by using proven technology such as the University intranet.

The University's eFront LMS was used as a platform for the delivery of the oral presentation course. This system is widely used by the University staff and students, and enables the distribution of course materials, the tracking of learning progress, and the management of the teaching–learning process. It is capable of handling large media files and functions as a video-hosting service.

Human capabilities are another concern that is often referred to as the attribute of online instruction (Kuzma, 2011). Students need real-time contact, immediate feedback, and peer and instructor support (Kuzma, 2011), although in the case of a blended learning course, this gap is bridged with the help of a teacher's facilitative support throughout the course and face-to-face personal contact.

Teaching Presentation Skills: Proven Pedagogy and Recommendations

Academics and practitioners often emphasize the importance of feedback (Mitchell & Bakewell, 1995), the role of peer assessment (Rust, Price, & O'Donovan, 2003), and self-assessment (Ross, 2006) to improve learning development. However, little research highlights the impact of these tools regarding academic presentation skills.

Peer and tutor feedback play an important role, though researchers argue about whether they can substitute for each other because of their strong correlation (Campbell, Mothersbaugh, Brammer, & Taylor, 2001) or if only the combination of both types will lead to significantly improved performance (Mitchell & Bakewell, 1995).

While there is not enough evidence to reasonably support one of the approaches, the present study is in line with the idea that developing students' self-assessment skills is beneficial to their learning. Researchers often highlight the importance of assessment practices in teaching and learning (Kenkel, 2011). Ross (2006) states that self-assessment is the ability of an individual to identify personal weaknesses and strengths with the aim to increase learning performance. Assessment outlines are often designed, as students need to know clear guidelines and criteria on which their work will be assessed (Andrade, 2005) and should become active participants of the process (Stiggins, 2001).

Almost every institution develops its own rubric to evaluate students' performance in acquiring presentation skills. However analysis of research conducted in online training of communication skills and teaching oral presentations mostly focuses on a business style format. A business presentation is usually evaluated in accordance with the following criteria: structure (introduction, conclusion, and organization), vocal qualities, extra linguistic features (eye contact, body language, appearance of the speaker), and visuals (Kenkel, 2011).

Little research highlights the nature of evaluation criteria for an academic presentation. One study states that academic presentation skills incorporate academic reading, writing, and presenting (Lytaeva & Talalakina, 2011). In other words, students who undertake a course in academic oral communication should first of all acquire the skills of academic critical reading, as at this level they learn how to select relevant information and sources for their research project and critically interpret information. At the next stage, academic writing, students acquire information-processing, note-taking, synthesis, and summarization skills. Having mastered the first two stages, students can learn to present their research results orally.

While the issue of the hierarchy of skills raises some concerns, the present study supports the idea that academic presentation skills tend to be different from business ones. By comparing the existing presentation teaching approaches and assessment rubrics serving academic and business purposes, the results shown in Table 1 were obtained. As can be seen from Table 1, it is not possible to claim that academic presentations are more complicated by nature than business ones. Both formats include substantial challenges for students. Linguistic competency plays a more crucial role in the academic field and involves a number of advanced sub-skills such as critical thinking, academic reading, and academic writing. The main purpose of business presentations and academic presentations is financial return and stimulation of research discussion, respectively. Developing business presentation skills tends to focus on delivery skills, namely effective and persuasive communication strategies and extra linguistic components, while the development of academic presentation skills is mostly based on the linguistic component and research work that was done before the actual public performance.

Table 1. Academic versus business presentation features

Thus, successful acquisition of academic presentation skills depends on the ability of students to possess a reflective learning strategy. Some researchers suggest that reflective learning provides the best outcomes for students' academic presentations. Wrenn and Wrenn (2009) argue that "experience must be followed by reflective thought and an internal processing that links the experience with previous learning, transforming the learner's previous understanding in some manner. Learning, therefore, takes place within a cycle that includes action, reflection, and application" (p. 260).

Kenkel (2011) also suggests reflection skills are necessary for successful acquisition of oral communication skills at the stage of the presentation's evaluation. Overall, academics and practitioners focus mainly on the important role of self-reflection in evaluating presentation delivery skills, ignoring the potential benefits it can bring if reflective learning is encouraged throughout the whole learning process. In this study, the potential of reflective learning within a web-based learning environment was investigated.

Reflective Learning Model

The main aim of the present study was to investigate the potential of "reflection tools" incorporated into the LMS-based component of the course aimed at developing academic presentation competency and teaching students a learning strategy. The study applies the model proposed by Kolb (1984), building upon the earlier work of Dewey (1938), Piaget (1972), and Lewin (1951). According to this holistic perspective, "learning is the process whereby knowledge is created through the transformation of experience" (Kolb, 1984, p. 38). The cycle consists of four stages, which can be entered at any point, but all stages should be followed in sequence in order to ensure successful learning (see Figure 1).

Figure 1. Kolb's (1984) experiential learning cycle

According to Kolb's (1984) theory, having an experience is insufficient to progress in studies. It is necessary to reflect on the experience, gain understanding of the concepts engaged in the experience, and later test the concepts in a new situation. This is the way to continuously transform information into knowledge and for practical skills to be applied in a number of meaningful situations.

General principles underpinning every stage of the cycle in relation to the LMS course on developing academic presentation skills are presented in the following paragraph, while more specific "hands-on" implications are presented in the next one.

As a general rule, at the "concrete experience," or the "doing" stage, learners process and analyze the e-course content, focusing on the relevant information. They read texts and watch videos in order to primarily build theoretical grounds for further developing practical skills. However, a widely shared misconception concerning most LMS courses is that providing students with information on the key components of the subject is often considered enough to prepare them for an effective independent production stage. Being informed does not equate to being knowledgeable, as knowledge is gained through reflective analysis of materials with respect to a specific learning situation causing concrete goals and activities.

This point brings us to the next stage, which is believed to be the most difficult one, and as a result, is often excluded from online curricula. At this stage of observation and reflection, students are supposed to reflect on the content and possible ways to apply the new information to their learning environment including goals, style of learning, preliminary concepts, and assumptions they might have on the subject, and anticipate problems they might encounter. As experience shows, students usually fail to perform effectively on this stage and therefore need some tools to get insights into the process of observation and reflection. Additionally, learners might need to ensure that they fully comprehended the content and that their cognitive participation was a success. For this purpose, self-check questions and exercises are incorporated into the course design.

At the next stage, students are to form key concepts underpinning their own idea of an academic presentation. They work individually selecting academic sources, appropriate materials, visuals, and linguistic resources, and design an optimal structure of their presentations.

Finally students come to the active experimentation stage at which they deliver their own presentations. At this stage, learners apply their knowledge and the practical skills they gained in the course of processing and reflecting on the new information such as building new concepts and planning their activities in relation to a new actual real-life situation. The mechanics of using an LMS technology, self-assessment, and self-reflection components are included in the next sections.

Project Implementation

In 2011, a project was undertaken to develop an online, LMS-based component on academic presentation as part of an English for Specific Academic Purposes course targeted at Russian students for whom English was a foreign language. This enabled the course to be delivered in a blended learning format, with a focus on reflective learning strategies (Table 2). The main objectives of the project were: (1) to help students master academic presentation skills; (2) to develop a reflective learning strategy specific to foreign language acquisition; and (3) to develop self-assessment potential. The project consisted of two stages: creating an LMS-based course that would help to improve academic speaking sub-skills in the reflective learning style and conducting an analytical evaluation of the reflection-based course.

Table 2. Blended learning course components

The course was run during the Fall and Winter 2012 semesters for 20 junior students majoring in sociology. The students' language proficiency level was intermediate–high, according to their regular second-year English language end-of-year assessment. Students were instructed in the classroom about the new course and all the requirements.

Each LMS-based study session had a focus on a particular academic presenting sub-skill. In total, there were six classroom-based seminars. During this stage, students were required to prepare their first presentation, on the topic "Socialization throughout the life cycle." Leading up to the presentation, the course provided them with reading, video, and vocabulary materials introducing students to the issue, equipping them with the appropriate linguistic/extra-linguistic tools, and stimulating further sub-skills practice (see Table 3).

Table 3. Reflective learning tasks within the LMS-based course component

Every week, students worked individually in their personal LMS space, where they read the relevant information, studied examples, and carried out the tasks to practice the necessary sub-skills. Having completed the last task within this learning environment, each student was transitioned with the help of proper instructions to the reflective learning style (Table 3). After the first stage of the project, students had to continue their work on another four topics.

To illustrate, consider the second lesson, which was aimed at developing the sub-skill of selecting proper academic sources for the presentation. University students often complain that they are not able to find relevant material for their topic. They first look through the given texts (from different sources such as journals, an online report, or an essay) and are encouraged to contrast them and decide if they would use them for an academic presentation. Secondly, at the reflection stage they are instructed to critically think and analyze all possible criteria for selecting a credible academic text. Then students are provided with further links to other assessment rubrics. Their task was to compile their own rubric for future use in the academic context (see Appendix B). Having defined all the key concepts, learners actively experiment by finding relevant sources for their presentation, and checking their academic status. If a student is successful in mastering this sub-skill, he/she moves to the next stage. However, if any problem arises the teacher can resort the student back to the first stage of experience so that the learner completes the full cycle for maximum learning.

Results and Discussion

Upon the evaluation of the academic presentation skills course, it can be concluded that the outcome was positive. Academic presentation assessment results for the group of 20 students are presented in Table 4. As displayed in the table, all students surpassed the 6.5 band level (which indicates a "good" level of language use) and the majority demonstrated sufficient academic success. The number of students who attained the 8.5 (indicating "advanced" language use) and 9.5 ("proficient" language use) bands almost doubled, which also represents substantial learning progress.

Table 4. Academic presentation skills assessment results

In order to assess student perceptions of the online learning component into their English for Specific Academic Purposes course, surveys were conducted at the end of the course. The survey included Likert-scale and true/false questions, as well as open-ended questions. Students were surveyed at the last seminar at the end of the course and were submitted anonymously. There were 20 students who participated in the survey. Both qualitative and quantitative responses clearly indicate that they had a valuable and productive blended learning experience.

The quantitative results enforced by the qualitative feedback highlighting a number of important findings of the blended learning style. First, about 89% of the respondents indicated that the offered style of study is more effective compared to a classroom setting (Figure 2). When asked about the main benefits of the new style, students noted convenience in terms of time and place, ability to concentrate on a sub-skill, and a lot of independent practice. Furthermore, about 82% of students found that the blended learning method provides more opportunities for acquiring new knowledge, sub-skills, and successful learning habits (see Figure 3).

Figure 2. Students' perceptions of blended learning, based on their experience in the academic presentation course

Figure 3. Students' perceptions of their learning in the academic presentation course

Additionally, the vast majority of respondents (95%) admitted substantial improvement of their academic presentation competency (see Figure 4). Students noted the key factor that leads to their success was clear instructions in LMS supplied with a variety of learning sources. Students emphasized that they not only mastered the academic presentation delivery skills but also learned how to start preparing a presentation, how to critically select information, how to effectively prepare a draft, and how to critically assess their own learning and progress.

Figure 4. Students' perceptions of the improvement in their academic presentation skills as a result of undertaking the course

When asked about the self-assessment skill, 87% of the students indicated its benefits (see Figure 5). They emphasized that they now better understand the academic presentation assessment rubric provided by the teacher (Appendix A), though they'd rather use their own criteria for assessment sub-skills which feature clearer explanations and language (see Appendix B).

Figure 5. Students' perceptions of whether self-assessment helped them improve their academic presentation skills

Another question was about the reflective learning strategy and its benefits. Ninety-six percent of respondents clearly indicated that they understood what self-reflection means and its role in their studies (see Figure 6). The LMS-based course structure and sequence of tasks was new for them, but it was described as "an effective, clear, and natural path to follow when you need to prepare a presentation." Students explained that now they always try to correct their mistakes, are more willing to ask questions not only about the linguistic side of the course but also how to better organize their studies, how to effectively manage their time, and what other strategies they can apply to their learning.

Figure 6. Students' perceptions of whether self-reflection on their learning habits, mistakes, and progress helped them in the English for Specific Academic Purposes course

Finally, it is worth noting that all students realized the difference between business and academic presentations and their main features. They emphasized that now they can be more flexible and effective when preparing various presentation assignments throughout their studies. They noted, for example, the business style is more suitable for a case study seminar, while the academic style is the best option to present their course paper results.

Overall, although the blended learning mode was merely a first experience for the students, it proved to be mostly beneficial. Students demonstrated far greater satisfaction with their learning progress, development of relevant study skills (autonomous and reflective learning), and their ability to transfer reflective learning experience to other areas of their studies.

Despite the positive feedback, some students expressed concerns including technological problems, lack of previous online study experience, and little room for an audience interaction. These factors should be taken into account while delivering the LMS-based course.

Technological barriers can be overcome by proper selection of the online tools. The choice depends on the teacher's preferences and reliability of the technology used. The problem with the lack of blended learning experience could be solved by providing students with relevant instructions and self-study guides, and the time allocated for the course should be extended in order to let students undertake another stage in Kolb's (1984) cycle. Interaction with a live audience is an academic presenting sub-skill, so teachers should incorporate this sub-skill into a classroom setting. Suggested ways to incorporate it into learning is to arrange classroom-based sessions or to have videoconferencing sessions. The choice depends on the learners' needs and preferences.

Conclusion and Future Research

This paper has presented a case study of an LMS-based course in oral academic speaking delivered in a blended learning mode. The results clearly indicate that the course can be more beneficial than face-to-face traditional classroom instruction alone in terms of skills, workload, and academic rigor. It is suggested that apart from supplying students with a formal version of an assessment rubric, teachers would benefit from helping students to develop their own, simplified version of the criteria to follow. This will encourage students to critically consider all the assessment areas and learn how to apply them not only at the final delivery stage of a presentation but rather throughout the whole learning process.

Notwithstanding its success, it is clear that the online version of the course requires teachers to possess specific skills of pedagogical design of course materials and the style of instruction which should be based on the reflective learning principles. Students also need time to adapt to the new style of learning at the first stage of the course.

Future research should be done with a more diverse group of learners, which includes not only junior students but also freshmen and sophomore students in order to compare the learning progress and any encountered problems. The nature of an academic presentation in comparison with a business style presentation should be taken into account and further investigated. Moreover, it should be studied if the blended style of learning can become a platform for developing reflective learning style not only in the area of English for Specific Academic Purposes but in other disciplines as well.

References

Allen, I. E., & Seaman, J. (2010). Learning on demand: Online education in the United States, 2009. Babson Park, MA: Babson Survey Research Group. Retrieved from http://www.sloanconsortium.org/publications/survey/pdf/learningondemand.pdf

Andrade, H. G. (2005). Teaching with rubrics: The good, the bad, and the ugly. College Teaching, 53(1), 27-31. doi:10.3200/CTCH.53.1.27-31

Böhme, K. (2009). Web-based rhetorical training – a virtual impossibility? Problems and perspectives of improving public speaking skills in virtual learning environments. Journal of Education, Informatics, and Cybernetics, 1(1), 1-6. Retrieved from http://www.journaleic.com/article/download/3460/2595

Campbell, K. S., Mothersbaugh, D. L., Brammer, C., & Taylor, T. (2001). Peer versus self assessment of oral business presentation performance. Business Communication Quarterly, 64(3), 23-40. doi:10.1177/108056990106400303

De Grez, L., Valcke, M., & Roozen, I. (2009). The impact of an innovative instructional intervention on the acquisition of oral presentation skills in higher education. Computers & Education, 53(1), 112-120. doi:10.1016/j.compedu.2009.01.005

Dewey, J. (1938). Experience and education. New York, NY: Kappa Delta Pi.

Hrastinski, S. (2008a). Asynchronous and synchronous e-learning. EDUCAUSE Quarterly, 31(4), 51-55. Retrieved from http://www.educause.edu/ero/article/asynchronous-and-synchronous-e-learning

Hrastinski, S. (2008b). The potential of synchronous communication to enhance participation in online discussions: A case study of two e-learning courses. Information & Management, 45(7), 499-506. doi:10.1016/j.im.2008.07.005

Ice, P., Curtis, R., Phillips, P., & Wells, J. (2007). Using asynchronous audio feedback to enhance teaching presence and students' sense of community. Journal of Asynchronous Learning Networks, 11(2), 3-25. Retrieved from http://www.sloanconsortium.org/sites/default/files/v11n2_ice_0.pdf

Kenkel, C. S. (2011). Teaching presentation skills in online business communication courses. MERLOT Journal of Online Learning and Teaching, 7(3), 412-418. Retrieved from https://jolt.merlot.org/vol7no3/kenkel_0911.htm

Kock, N. (2005). Media richness or media naturalness? The evolution of our biological communication apparatus and its influence on our behavior toward e-communication tools. IEEE Transactions on Professional Communication, 48(2), 117-130. doi:10.1109/TPC.2005.849649

Kolb, D. A. (1984). Experiential learning: Experience as the source of learning and development. Englewood Cliffs, NJ: Prentice Hall.

Kuzma, J. (2011). Using online technology to enhance student presentation skills. Worcester Journal of Learning and Teaching, 5. Retrieved from http://www.worc.ac.uk/adpu/documents/WJLTIssue5ReportsJKuzma.pdf

Lewin, K. (1951). Field theory in social science: Selected theoretical papers. New York, NY: Harper & Row.

Linardopoulos, N. (2010). Teaching and learning public speaking online. MERLOT Journal of Online Learning and Teaching, 6(1), 198-209. Retrieved from https://jolt.merlot.org/vol6no1/linardopoulos_0310.htm

Lytaeva, M. A., & Talalakina, E. V. (2011). Academic skills: Susnost', model', praktika [Academic skills: Nature, model, experience]. Journal of Educational Studies, 4, 178-201. Retrieved from http://vo.hse.ru/attachment.aspx?Id=790

Mitchell, V.-W., & Bakewell, C. (1995). Learning without doing: Enhancing oral presentation skills through peer review. Management Learning, 26(3), 353-366. doi:10.1177/1350507695263005

Piaget, J. (1972). The psychology of the child. New York, NY: Basic Books.

Raybould, J., & Sheedy, V. (2005). Are graduates equipped with the right skills in the employability stakes? Industrial and Commercial Training, 37(5), 259-263. doi:10.1108/00197850510609694

Robert, L. P., & Dennis, A. R. (2005). Paradox of richness: A cognitive model of media choice. IEEE Transactions on Professional Communication, 48(1), 10-21. doi:10.1109/TPC.2004.843292

Ross, J. A. (2006). The reliability, validity, and utility of self-assessment. Practical Assessment, Research & Evaluation, 11. Retrieved from http://www.pareonline.net/pdf/v11n10.pdf

Rust, C., Price, M., & O'Donovan, B. (2003). Improving students' learning by developing their understanding of assessment criteria and processes. Assessment & Evaluation in Higher Education, 28(2), 147-164. doi:10.1080/02602930301671

Stiggins, R. J. (2001). Student-involved classroom assessment (3rd ed.). Upper Saddle River, NJ: Merrill/Prentice Hall.

Talalakina, E. V. (2012). Video-conference debates: A platform for critical thinking within foreign language acquisition. In M. Helfert, M. J. Martins, & J. Cordeiro (Eds.), Proceedings of the Fourth International Conference on Computer Supported Education (CSEDU 2012) (Vol. 2, pp. 88-93). Setúbal, Portugal: Institute for Systems and Technologies of Information, Control and Communication.

Wrenn, J., & Wrenn, B. (2009). Enhancing learning by integrating theory and practice. International Journal of Teaching and Learning in Higher Education, 21(2), 258-265. Retrieved from http://www.isetl.org/ijtlhe/pdf/IJTLHE727.pdf

Appendix A: Academic Presentation Assessment Rubric (Sample Selected Bands)

Band	Content		Delivery				Linguistic Features
	Coherence and structure	Use of academic sources	Timing	Use of PowerPoint	Expressiveness	Interactive skills	Grammar	Lexis
10: High Distinction	Coherent, extremely sophisticated signaling, framing, linking, analysis of good depth	Original sources, skillful use of 3-4 resources	7 min, perfect timing	Excellent design, creative, appropriate graphics, no mistakes	Extremely confident, enthusiastic, very fluent, highlighting, appropriate body language, good eye contact	Perfectly able to comprehend, express complex ideas easily, competent answering of questions	Accurate, complex, and varied sentence structures	Competent use of a very wide range of vocabulary
8: Distinction	Cohesive, good signaling, framing, linking, analysis of adequate depth	Original sources, good use of 3-4 resources	7 min, good timing	Good design, creative enough, appropriate graphics, a few mistakes	Quite confident, enthusiastic, quite fluent, highlighting, good body language, maintains eye contact	Able to comprehend and respond effectively, competent discussion, good comprehension and answering of questions	Reasonably accurate, sometimes complex, and different sentence structures	Good use of vocabulary relevant to the topic
7: Merit	Good structuring, clear signaling, correct, natural framing, linking, satisfactory analysis	Original sources of low level, good use of 3 resources	Almost good timing (slightly shorter or longer)	Simple design, some good graphics, a few mistakes	Seems to be confident, not very fluent or rushing, poor highlighting, often good body language, keeps eye contact	Good level of comprehension, quite competent discussion, correct answering of questions	Almost accurate, good, and correct sentence structures	Good use of basic vocabulary relevant to the topic
5: Pass	Satisfactory structuring, unsystematic and fragmented signaling, framing, linking, no analysis, mostly description	Original sources of low level, problematic use of < 3 resources	Runs too short or long	Simple design, simple graphics, many mistakes	Not confident and hesitant, not fluent or rushing, no highlighting, no body language, very rare eye contact	Able to comprehend but simple or brief answers, some hesitancy	Many inaccuracies, mostly simple sentence structures	Only basic terms, not able to rephrase, often inappropriate usage

Appendix B: Excerpt from a Student's Personal Assessment Grid

Acknowledgment

The study reported in this paper was supported by a 2011-2012 grant from the National Research University Higher School of Economics Educational Innovations Program.