Teamwork and Team Performance in Online Simulations: The Business Strategy Game

Steve Jenner
California State University Dominguez Hills
College of Business Administration & Public Policy
Carson, CA 90747 USA
sjenner@csudh.edu

Meng Zhao
California State University Dominguez Hills
Carson, CA 90747 USA

 Tom H. Foote
Colorado Technical University
Colorado Springs, CO 80907 USA

Introduction

Substantial research focuses on the importance of teamwork in business management (Hackman & Wageman, 1995; Katzenbach & Smith, 2003; Wageman, 1995; Wageman & Baker, 1999). Most individuals work in a business environment that requires cooperative team efforts to successfully reach business goals. This makes the ability to work in teams a critical skill set. The recent trend towards leaner, flatter organizations, with fewer levels of hierarchy, places even greater emphasis on the need for cross-functional teamwork (Ancona, 1999). Furthermore, as working in teams gains in importance, increased diversity within the workplace creates new pressures for employees who are struggling to make teamwork effective. As a result, students need to develop skills to help them navigate the new teamwork culture that permeates industry.

Practitioners and university faculty in the field of business management generally agree that students need to develop teamwork skills. Business professors routinely require teamwork assignments in their courses. However, the challenge of assessing student learning of teamwork skills in these assignments is formidable. Managing and controlling these teamwork assignments requires teachers to develop feedback control systems utilizing multiple forms of evidence as the basis for continuous improvement in student learning.

Online Teamwork

The performance of face-to-face and online virtual teams is similar in general, especially if members are experienced at teamwork (Parks & Sanna, 1999). Online teams take longer to exchange information, but people are less concerned about criticism of their ideas or about diversity in gender, age, and race (Levi, 2007). This could lead to higher performance for online teams. However, face-to-face meetings are very important for newly created teams in order to become acquainted and develop trust, and set team goals, roles, and norms (Hertel, Geister, & Konradt, 2005). It is also harder to motivate team members and control social loafing or “free riders” who don’t do their fair share of the work with online teams. These factors could lead to higher performance for on-campus teams.

In using team assignments, self-reported team effectiveness and peer evaluations are common assessment tools, along with final reviews of oral and written reports on team projects. The team assignment in this study was the Business Strategy Game (BSG). The BSG is a computer simulation of a competitive athletic footwear industry. Participants form teams and enter inputs on ten decision screens (sales forecast, marketing plan, production schedule, financial decisions, etc.). The simulation processes the inputs of all the teams and generates a set of results in the form of reports (game-to-date performance scores, competitor intelligence, etc.). For more details, go to bsg-online.com..

Since market inception, t he Business Strategy Game has been used at more than 500 schools in over 25 countries and played by approximately 500,000 students. In the BSG, overall team performance is measured by the final overall team score calculated by the computer simulation model.

In this article, we investigated the impact on the performance of student teams of the course delivery mode and the instructor’s decisions about how to use the BSG. Specifically, we looked for relationships between the BSG performance scores for each team and the following variables: 1) delivery mode (online vs. on-campus), 2) team size, 3) class meeting frequency, 4) class discussion time for the BSG project and 5) class size. Our theoretical framework is shown in Figure 1, which posits that the institution’s and instructor’s teaching decisions such as team size, class meeting frequency, BSG project discussion time, and class size affect the student team BSG scores. However, we argue that these relationships are moderated by the delivery mode. The theoretical framework and its proposed hypotheses are further developed in the next section, followed by statistical tests, results, conclusion and discussion.

Figure 1 . Teaching Decisions and Team Performance

Hypothesis Development

Class Size

The class size-performance nexus has received considerable attention in the literature over the past half century (McConnell & Sosin, 1984; Shane, 1961, Simmons, 1959). Much of the literature on student learning and class size suggests that more might be learned in smaller classes. Students in smaller classes are more likely to get personalized attention and support from their instructor, and their individual learning needs are more likely to be addressed (Glass & Smith, 1979). Following this stream, a number of studies have demonstrated a negative association between class size and performance (Glass & Smith, 1980). The latest study by Bandiera, Larcinese, and Rasul (forthcoming) showed that there was robust evidence of a negative class size effect — on average, larger classes reduce student academic achievement as measured by test scores. Thus, we propose that a smaller class size will lead to better student team performance.

H1: Larger class size will have a significant and negative impact on student team performance.

Class Meeting Frequency

The most challenging and also the most essential part of teaching a business management class is the transfer of tacit knowledge (Nonaka, 1994), which involves complex concepts, ideas, subjects, and problem-solving skills. Informal communication channels such as face-to-face meetings have been found to be more useful in transmitting highly complex subject matter and more capable of conveying a variety of information (i.e., the channel “richness”) (Daft & Lengel, 1986; Tushman, 1978).

Therefore, more frequent class meetings devoted to a team assignment may lead to better informal communication channels, which will in turn facilitate more effective transfer of tacit knowledge and thus increase student team performance.

H2: Class meeting frequency will have a significant and positive impact on student team performance.

Team Size

Numerous studies have been conducted on team size and its relationship to performance and to various factors such as team spirit, individual and team attitudes, and interaction among team members (Cummings et. al. 1974, Grofman et. al. 1984, Helms and Wyskida 1984, Wagner et al. 1987). To date, the findings suggest that smaller teams allow for closer relationships among members, a deeper knowledge of the members and a better sense of the whole picture at any given time (Cohen et al. 1992). On the other hand, there may be a preference on the part of some team members to participate in large teams in order to avoid an "intimate" environment, achieve greater anonymity and have the security of knowing that there are more people to do the work required of the group (Cohen et al. 1992). In a marketing simulation with teams of two, three, and four, team performance varied by team size, with performance significantly higher for the four-person teams (Cosse, Ashworth, and Weisenberger 1999). Yetton and Bottger (1983) noticed performance gains as group size increased from three to five members. Gains declined as team size increased with no noticeable improvement in performance observed for groups larger than five. Referring to other classroom studies, Rosser (1998) stated that team size should generally be between three and six members. Larger teams have more diverse viewpoints which may lead to better decisions and higher team performance, although they may experience greater challenges with scheduling meetings and poor participation (Cooper, Robinson, & Ball, 2003). Therefore, within the range of team size between three and six, we propose that:

H3: Team Size will have a significant and positive effect on student team performance.

BSG Project Discussion

In contrast to teacher-centered didactic approaches, in which the teacher transmits a fixed body of knowledge to learners, dialogue has been described as learner-centered and participatory, "guided by a spirit of discovery" in which participants develop greater understanding, insight, and sensitivity (Burbules, 1993, p. 8). According to Burbules (1993), dialogue involves our ability to reason, "especially our ability to solve problems, to think sensibly toward conclusions, to weigh competing considerations and to choose reasonable courses of action" (p. 11). In teaching business management, it is very common for instructors to allocate considerable class time for student discussion, especially on complex issues and tasks such as the BSG project. Therefore, a critical question is whether the allocation of the discussion time indeed helps student learning and thus improves their performance. A recent study by Smith et al (2009) showed that class discussions, and especially peer discussions, help students learn more than listening to lectures by instructors.

H4: BSG project discussion time has a significant and positive effect on student team performance.

Delivery Mode: On-Campus or Online

The literature on the effectiveness of on-campus versus online instruction delivery is mixed (Benoit et al 2006). Some studies suggested that online delivery has advantages over on-campus delivery (e.g. Maki & Maki 2002; Twigg 2003). Some found that on-campus delivery was preferable (e.g., Wang & Newlin 2000; Waschull 2001). Other studies were inconclusive (e.g., Botsch & Botsch 2001; Hiltz 1993). According to Clark, “there are no learning benefits to be gained from employing any specific medium to deliver instruction” (Clark, 1983 & 1994).

The BSG is played online whether the course is delivered on-campus or online. The transfer of learning about strategic management to the BSG can be the same or similar regardless of the delivery mode. Therefore, we propose that the delivery mode will not directly impact team performance.

The Moderating Effects of Delivery Mode

There have been many studies and debates on the delivery mode’s direct impact on student performance (for an integrated review, see Benoit et al 2006). However, the delivery mode’s indirect effects on student performance are largely overlooked. The indirect effects here refer to the moderating effects of delivery mode on other variables that affect student performance. In this study, we focus on the delivery mode’s moderating effects on a few institution and instructor decision variables such as class size, class meeting frequency, team size, and BSG project discussion (see Figure 1). We posit that, under different delivery modes, these decision variables have different impacts on student team performance.

It was noted that the use of various forms of multimedia also appears to play a substantial role in the relationship between class size and student performance (Karakaya, Ainscough, & Chopoorian, 2001). In Drago and Peltier’s study (2004), while the traditional courses showed significant and negative associations across all building blocks as well as overall effectiveness (even within the narrow range and relatively small classes analyzed) the online courses showed only two significant but positive relationships. Using the online delivery mode, students receive instruction in an environment which does not have the physical limits of time and space of the on-campus delivery mode. The resources available to the students in the online environment need not be reduced as the class size increases. The increased number of students in the class will increase the diversity of the students, and the interactions among students, which will have a positive effect on student performance. Therefore, we propose that:

H5: Delivery mode will have both an indirect or moderating impact on team performance, and a direct impact on team performance.

H6(a): Using the online delivery mode, the class size will not have a significant effect on student performance.

H6(b): Using the on-campus delivery mode, the class size will have a significant and negative effect on student team performance. (This is the same as H1).

Using the online delivery mode, students have greater freedom to decide the time and the frequency of reviewing the instructions and meeting with their teammates. On the other hand, in the on-campus delivery mode, the frequency of class meetings is determined by the class schedule and the teacher. Thus, meetings are not a true reflection of the students’ efforts to learn and communicate in the class. Therefore, we propose that:

H7(a): Using the online delivery mode, the meeting frequency will have a significant and positive impact on student team performance. The effect will be greater than that under on-campus delivery mode.

H7(b): Using the on-campus delivery mode, the meeting frequency will have a significant and positive impact on student team performance. (This is the same as H2). The effect will be smaller than when using the online delivery mode.

In the online environment, the communication between teammates is primarily by emails and online chat, which may limit the “richness” of the communication (Daf and Lengel 1986; Tushman 1978). As such, there are greater difficulties for teammates exchanging complex ideas, concepts, and pooling their resources (Nonaka 1994). Consequently, a virtual online student team is limited in their ability to take advantage of the increased diversity and brain power as the team size increases. Therefore, we propose that:

H8(a): Using the online delivery mode, team size will not have a significant effect on student team performance.

H8(b): Using the on-campus delivery mode, team size will have a significant and positive effect on student team performance. (This is the same as H3).

Online student discussion promises to enhance reflective critical thinking (Mitchell, 2003; Wells, 1999). Online student discussion is better suited to critical reflection than oral discussion is because it has greater permanence, takes more time to produce and is under greater control of readers once they receive it (Wells, 1999; Windschitl, 1998; Beach & Lundell, 1998). However, despite the promise of online student discussion, research findings are mixed about whether it actually produces greater critically reflective thinking among students. Some studies supported this notion (Harrington & Hathaway, 1994; Harrington & Quinn-Leering, 1996; Davidson-Shivers, Muilenburg, & Tanner, 2001; Salmon, 2002). However, results from other studies call into question the potential of online student discussion with regard to critical reflection (Levin, He, & Robbins, 2006; Angeli, Valanides, & Bonk, 2003). Dodson (2000) found that students were as likely to take a monological stance as a dialogical one; their purpose for posting was to transmit information rather than to explore ideas together or debate and negotiate meaning. In the online environment, students have greater freedom to choose the actual length and intensity of their discussions on their BSG project, with limited influence from their instructors. In the on-campus environment, the length and intensity of the discussions are primarily determined by their instructors. Therefore, we propose that:

H9(a): Using the online delivery mode, team project discussion time will have a significant and positive effect on student performance. The effect will be much smaller than that under the on-campus delivery mode.

H9(b): Using the on-campus delivery mode, project discussion time will have a significant and positive effect on student team performance. The effect will be much larger than under the online delivery mode. (This is the same as H4).

Methodology

Sample and Procedure

The survey (see Appendix 1 Survey Instrument and Statistical Analysis) was administered to 137 instructors with a total of 6,632 teams at universities using the Business Strategy Game (BSG) at the end of the Fall, 2007 semester. Working with the BSG Administrators, the authors developed a short survey. The survey was sent out and the results were collected by the BSG Administrators confidentially to ensure that the authors did not have access to the respondents’ names and contact information. The survey announcement was e-mailed to about 400 BSG instructors known by the Business Strategy Game administrators to be using their product in the Fall of 2007. The message contained a link to a web page containing the survey. A total of 137 instructors responded (34%) and these instructors had a total of 6,632 students. We assume that the sample of survey responders and non-responders did not differ significantly on any relevant characteristics (e.g., age, gender, rank, or experience with the game).

Results

Descriptive Statistics

The descriptive statistics for the key variables are shown in Table 1. The measurement instrument of the key variables and their coding are provided in Appendix 1 of this paper. In addition, Online is a dummy variable, which has only two levels (for online delivery mode, online=1; and for on-campus delivery mode, online=0). In the dataset, there are 29.1% with online=1, and 79.9% online=0.

Table 1. Descriptive Statistics for Key Variables

Statistical Analysis

We tested our theoretical framework and its associated hypotheses using a multiple regression analysis. The criterion/dependent variable in the regression model is team performance (i.e. GTD score). The results are shown in Table 2. The R square for the regression is 5%, with F value = 32.573 (d.f. = 9, 5558), p value = .000. The R square is small but statistically significant at the 1% confidence level. The small R Square was expected, however, as individual and team efforts, aptitudes, and competence were expected to play a much more important role in deciding performance than the characteristics of the teaching method. Furthermore, from a technical standpoint, not including student aptitude-related variables in our model does not pose a serious methodological problem, as the primary purpose of the model is to test our theoretical hypotheses, not make predictions. Because the missing variables pertain to each individual student’s aptitude/competence, they are not correlated with the current regression model’s independent variables, which are decided by the institutions and instructors, not the students. Therefore, the omitted variable bias in our OLS regression estimation is close to zero, even the estimated errors might be increased. Given that our sample size is sufficiently large, the increased estimated errors are sufficiently alleviated. (for a further discussion of the omitted variable bias, see pages 89-94 in Wooldridge, 2003).

Therefore, the estimation of our regression model’s parameters is not biased due to the missing variables.

Table 2. Parameter Estimates from Multiple Regression Analysis.

Note: * indicate the coefficient estimates are significant at 95% confidence level.

Using the variable Online (i.e. Delivery Mode) as a dummy variable (Online=1 for online delivery mode, and Online=0 for on-campus delivery mode), we can convert the results in Table 1 into a comparison between online vs. on-campus delivery modes (see Table 3).

Table 3 Comparison between Online vs. On-Campus Delivery Mode

Based on the results in Table 2 and Table 3, we summarize our findings as follows:

On-Campus Delivery Mode

• The coefficient estimate for Class Size (see Table 3) is statistically significant and negative, and thus supports H1 and H6(b). That is, using the on-campus delivery mode, an increase in class size is correlated with lower student team performance.
• The coefficient of Meet Freq is not statistically significant (See Table 3). Therefore, H2 and H7(b) are not supported. That is, using the on-campus delivery mode, an increase in class meeting frequency was not found to have an impact on student team performance. Even though the finding does not support our hypothesis, it is consistent with the rationale behind our hypothesis development. As we explained in our hypotheses development section, the frequency of class meetings is not voluntary and does not necessarily reflect student’s effort in learning and communicating in the class. Therefore, it was expected that the effect of Meet Freq would be smaller or not significant.
• The coefficient of Team Size is statistically significant, and positive (See Table 3). Therefore, H3 and H8(b) are supported. That is, using the on-campus delivery mode, an increase in team size is associated with higher student team performance.
• The coefficient of Discuss is statistically significant and positive (See Table 3). Therefore, H4 and H8(b) are supported. That is, using the on-campus delivery mode, an increase in class time for team discussion will lead to higher student team performance.

Online Delivery Mode

• The coefficient of the variable Online is statistically significant and negative (See Table 2). Therefore, H5 is supported. That is, the online delivery mode is associated with a poorer student team performance than the on-campus delivery mode is, if the delivery model’s moderating effects (i.e. interacting effects) on other variables’ relationship to student team performance are not considered.
• The coefficient of the Class Size is statistically insignificant (See Table 3), and thus, H6(a) is supported. That is, using the online delivery mode, an increase in class size did not have an impact on student team performance. The finding is consistent with our theoretical rationale, which argued that the class size effect in the online environment will not be negative compared with using the on-campus delivery mode (i.e. H6(b)).
• The coefficient of the Meet Freq is statistically significant, positive, and significantly greater than that under on-campus delivery mode (See Table 3). Therefore, H7(a) is supported. That is, using the online delivery mode, an increase in the frequency of student meetings was associated with higher student team performance.
• The coefficient of Team Size is statistically insignificant (See Table 3), and thus H8(a) is supported. That is, with online delivery, an increase in the team size did not have an impact on student team performance.
• The coefficient of the variable Discuss is statistically significant and positive, but much smaller than the coefficient for on-campus delivery (See Table 3). Therefore, H9(a) is supported.

Discussion

Is online teaching and learning less effective than on-campus delivery? When considering the effectiveness of a team activity, our research suggests that it may depend on the effect of other variables which can be controlled by teachers and institutions. The design of courses and assignments can increase the effectiveness of the online delivery mode. More class meetings, more class time for student teams to discuss their projects, and larger team size (up to a maximum of 6) can make online delivery more effective in terms of team performance. Using on-campus delivery, larger class size can sacrifice student team performance; for online delivery, class size did not matter. It should be noted that these findings pertain only to performance on a team activity.

Is team performance a reliable measure of teamwork? Is it possible to achieve a high team score without high team performance? The BSG Team Scores may not reliably measure teamwork or team performance in spite of the claims of many instructors who use this assignment in courses on strategic management.

However, this team score may not reflect a real team effort. A dominant star player can lead a team to high performance. The remaining teammates can simply stand back and let the star take over, free riding on the leader’s brilliant performance.

Requiring teams to submit Decision Records detailing how the members worked interdependently toward shared goals is a promising approach to measuring teamwork. Decision Records make individual team members accountable for their weekly contributions to the team effort. In one case, Decision Records revealed that the winning BSG team was dominated by one member with a powerful understanding of how to win within the context of the BSG software model. (See the single-leader unit in Ap pendix 2 Decision Records of Two Teams. )

Teachers who choose to assign team assignments need to do a much better job of managing such assignments. Too often they rely on self-reported team effectiveness and peer evaluations as assessment tools. Evaluations of oral and written reports on team projects are also commonly used for grading, but they may be the result of dividing up the task without working as a team.

More direct observation of teams at work would also be helpful in assessing the effectiveness of teamwork behaviors. For example, the online instructor could monitor e-mails and online chats between team members to assess the team’s shared goals and interdependent behavior (e.g., building on each other’s ideas in discussions). Other team member behaviors to observe include making commitments to perform tasks, doing his or her fair share of the work, producing work of acceptable quality and actively participating in team discussions and decision making (Levi, 2001). Besides relying on instructor observations, executive judges could lead class team meetings to assess teamwork.

Other evidence of individual efforts could include quizzes on the teamwork assignment (e.g., knowledge of the BSG rules and how to interpret the reported results) and website activity logs tracking the frequency and amount of time spent by each team member.

Conclusion

The design of courses and assignments can increase the effectiveness of the online delivery mode. More class meetings, more class time for student teams to discuss their projects, and larger team size (up to a maximum of 6) can make online delivery more effective in terms of team performance. Using on-campus delivery, larger class size can sacrifice student team performance; for online delivery, class size did not matter. Thus, the delivery mode’s indirect effects on student performance were demonstrated.

Compared to the traditional on-campus delivery mode, teaching online had a negative impact on a student team’s performance in the BSG. Online teaching’s direct effect (that is the main effect) on a student team’s performance was negative. That is, teams of students in online classes had lower performance than students in traditional on-campus classes.

Requiring teams to submit Decision Records detailing how the members worked interdependently toward shared goals is a promising approach to measuring teamwork. Decision Records make individual team members accountable for their weekly contributions to the team effort .