The Impact of a Web-based Homework Tool in University Algebra Courses on Student Learning and Strategies

Angie Hodge
Departments of Mathematics and Teacher Education
North Dakota State University
Fargo, ND 58105 USA
Angela.Hodge@ndsu.edu

Jennifer C. Richardson
Purdue University
West Lafayette, IN 47907 USA
jennrich@purdue.edu

Cindy S. York
Purdue University
West Lafayette, IN 47907 USA
cyork@purdue.edu

Background

A growing trend in university mathematics courses is the implementation of web-based homework in place of traditional paper-based homework. For instance, at one large Midwestern university, the college algebra courses have shifted to an online format for most homework assignments. Studies have been conducted on the use of web-based homework in physics courses (Bonham, Beichner, & Deardorff, 2001; Dufresne, Mestre, Hart, & Rath, 2002; Pascarella, 2004; Toback, Mershin, & Nazimova, 2005), chemistry courses (Cole & Todd, 2003; Freasier, Collins, & Newitt, 2003; Penn, Nedeff, & Gozdzik, 2000), and even in mathematics ( Bliwise, 2005; Hauk, Powers, Safer, & Segalla, 2005; Hirsch & Weibel, 2001; Zerr, 2007). Despite the number of studies conducted, few have moved beyond the examination of “equivalency” of the methods. For example, Demirci (2007) and Hauk and Segalla (2005) both looked at student perceptions related to a web-based homework tool with the Demirci (2007) study also examining student perceptions of student learning through the use of web-based homework. Picciano (2002) noted, “Student perceptions of their learning may be as good as other measures because these perceptions may be the catalysts for continuing to pursue coursework and other learning opportunities” (p. 22).

The increased use of technology in university classrooms can be both a motivator and an inhibitor for students and instructors. In the case of the College Algebra course in this study, the use of web-based homework freed the instructors from grading homework. Previously, instructors would only grade one or two problems and check the rest of the homework assignment for completion versus correctness.

Graded homework had a positive effect on student learning versus homework without feedback (Walberg, Paschal, & Weinstein, 1985). Thus, instructors should use technology for the efficiency, enhancement, and effectiveness of lessons (Newby, Stepich, Lehman, & Russell, 2006).

The College Algebra web-based homework tool provides an immediate correct/incorrect response for every problem. This tool also provides the opportunity to get every homework problem graded (with a correct or incorrect response). The problems are either multiple-choice or require only the solution to the problem, however, there are no comments provided or helpful suggestions from the tool on how to correctly solve an incorrect problem. If students get a problem incorrect, they have the opportunity to try a comparable problem multiple times until they get it correct. Even so, there is the question of whether or not the web-based homework enhances the students’ learning or their motivation to learn mathematics. Bonham, Deardorff, and Beichner (2003) believed physics “students like the immediate feedback [of web-based homework] and being able to resubmit assignments, whereas their instructors like not having to grade student work manually” (p. 1053).

The tool in this study (iLrn) also offers a randomization of homework problems, meaning that when students logon they might not receive the same problems as the previous logon, preventing the sharing of answers between students. All problems cover the same mathematical concepts and are aligned with the course textbook. It was found in our pilot study (York, Hodge, & Richardson, 2008) that this randomization caused frustration for some of the students. For example, a student may have difficulties with a homework question, logoff to go find help, and when they logon again, a different question is posed. Although the homework problems entail similar steps to complete, the students may just want to get the original question correct.

Purpose of Study

Due to the considerable use of web-based homework in courses such as the College Algebra course, it is necessary to investigate its effectiveness on student learning as it relates to course content. Additionally, there is need to examine student perceptions of the web-based homework on their learning of mathematics. Therefore, the purpose of this study was to examine the degree to which web-based homework, specifically for one college algebra course, affected student motivation and perceptions of learning. Two research questions were posed:

• Did student characteristics (age, gender, ethnicity, academic level, expected grade, previously used web-based homework tool, previously taken course) or other factors (frequency of completing web-based homework, ease of navigation of web-based tool, motivation to complete homework using web-based versus traditional paper/pencil methods) impact their perceptions of the use of web-based homework tool affecting their learning?
  
• Did the use of the web-based homework tool influence students’ learning strategies or motivation to complete their homework?
  
• Did the web-based homework tool tend to encourage or discourage students in terms of completing their homework?

Method

Research Design

This study investigated the students’ motivation and perceptions of learning in relation to the use of a web-based homework tool. The study utilized the Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich, Smith, Garcia, & McKeachie, 1991), which is based on “a general cognitive view of motivation and learning strategies” (p. 3). The MSLQ framework embraces three components that affect learners’ achievement-related behaviors and attitudes related to seeking help and learning from peers: (a) the attitudes or beliefs learners have of themselves and their capabilities (self-efficacy), (b) the value a student places on the task at hand, and (c) the emotional response of the student to the task (Pintrich & De Groot, 1990).

Quantitative methods (descriptive statistics, correlations, and multiple regressions) were used to examine students’ perceptions of the factors that influenced their mathematics learning. Surveys were utilized to inform the researchers about students’ perceptions of the factors that influenced their mathematics learning and related motivation for a web-based homework tool in a College Algebra course. Prior to this study, a pilot survey was conducted that informed the researchers on the final survey design (York, Hodge, & Richardson, 2008). Eleven of the items were adopted from the Motivated Strategies for Learning Questionnaire (MSLQ) (Pintrich et al., 1991). Items were rated on a Likert-type 1-5 scale following the standard evaluation scale familiar to the students, (1= “strongly disagree”, 3= “undecided,” and 5= “strongly agree”). Based on feedback from professional colleagues knowledgeable in the field, the researchers identified and modified questions that were unclear in the pilot study survey. The final survey items are included in the manuscript (see Appendix).

Two of the MSLQ learning strategy subscales were used in this survey: Peer Learning (PL) and Help Seeking (HS), as well as one MSLQ motivation subscale, Control of Learning Beliefs. According to the MSLQ developers, “The MSLQ is a self report instrument designed to assess college students’ motivational orientations and their use of different learning strategies for a college course” (Pintrich et al., 1991, p. 3). Peer Learning and Help Seeking were categorized as assessing students’ learning strategies, particularly “students’ use of different cognitive and metacognitive strategies” (p. 3). The Control of Learning Beliefs expectancy subscale, which “refers to students’ beliefs that their efforts to learn will result in positive outcomes,” was employed to compare students’ responses based upon their perceived control of learning (p. 12).

Participants

In the fall of 2007, data were collected from students enrolled in a college algebra course (N=1945), with 1394 students responding to the survey for a participation rate of 70.3%. During the data cleaning process, several students were eliminated after indicating they had never “done” the online homework (n=2) or for non-response to the question (n=34) so that only participants that claimed to have used the web-based homework tool at some point in the course were included. The final sample used for this research was 1333 participants, and of those students 77 (5.8%) had previously taken the course and 1125 (84.4%) had previously used some form of web-based homework in mathematics. 1277 students (95.8%) were between 18-22 years of age with 593 male and 708 female students and 32 unidentified. In addition, most students indicated they were Caucasian (80.1%) with 5.6% indicating African American, 5.0% Asian, 3.4% Hispanic and 2.7% “Other.” Details on age, academic level, expected grades, and frequency of homework completion are found in Table 1.

Data Collection

Thirty teaching assistants (instructors) distributed surveys in their individual classroom sections for the College Algebra course, which contained 20-40 students each, in a total of 53 sections. The students were informed the surveys were voluntary, anonymous feedback about the course and the web-based homework and would be used to improve the course.

Validity / Reliability

The team of researchers developed the survey with help from the College Algebra course coordinator and information obtained in a pilot study (York, Hodge, & Richardson , 2008). The pilot survey was developed after reviewing similar surveys in the literature (e.g., Bonham et al, 2003; Cole & Todd, 2003; Demirci, 2007; Hauk & Segalla, 2005; Klein, Monico, Williams, & Dwyer, 2006; Morote & Pritchard, 2004; Pascarella, 2004; Zerr, 2007). Moreover, expert reviewers, including the course coordinator for the College Algebra class, provided suggestions for improvement. The final survey incorporated these changes, including wording and specific details to assure the items were relevant to web-based homework users. These steps were incorporated to ensure face validity of the instrument.

Also, as previously mentioned, the study incorporated several of the subscales from the MSLQ, specifically the Control of Learning Beliefs (a motivational subscale) (alpha=.68) and the Peer Learning and Help-Seeking subscales (learning strategies subscales; alpha=.76 and .52 respectively) (Pintrich et al., 1991). The MSLQ has been validated by the developers, including a confirmatory factor analysis; the analysis indicated factor validity for each of the subscales (Pintrich et al., 1991).

Table 1. P articipant Demographics Reported in Percents

Age (n =1308)		Academic Level (n =1305)		Expected Grades (n =1297)		Frequency did Homework (n =1333)
Under18	1.1	Freshman	86.5	A	34.0	Always	60.8
18-22	95.8	Sophomore	8.0	B	36.1	Most of the time	32.9
23-27	1.1	Junior	2.3	C	16.3	Half of the time	4.9
28-35	<1	Senior	1.0	D	2.0	Once in a while	1.4
36-42	0	Graduate	<1	F	<1
43-50	<1			Not Sure	8.6

Data Analysis

Initial data analysis was descriptive and included frequencies for the demographic responses and means and standard deviations for each of the individual factors included on the survey. A multiple regression was conducted to determine if any of the participants’ demographic or individual factors were predictors of their “increased mathematical understanding.” Additionally , correlational research was conducted to examine potential relationships between the variables (learning and motivation subscales) and the web-based tool (Gall, Borg, & Gall, 1996). The quantitative data analysis was calculated using Statistical Package for the Social Sciences (SPSS). SPSS is a computer software package that assists in the analysis of statistical data.

Results

For the first research question, A multiple regression analysis was run to examine the relationship between students’ perception that using the web-based homework would increase their mathematical understanding more so than with traditional paper/pencil methods (dependent variable), expected course grade, previous use of web-based homework in mathematics, ease of navigation of the web-based homework tool, frequency with which homework was completed, and demographic items (independent variables). The demographic data included age, ethnicity, gender, previously taken course, and academic level. No violations were found in the assumptions of normality, linearity, and homoscedasticity of residuals.

The results of the regression model were found to be significant, F (24, 1051) = 38.953, p<.001. The multiple correlation coefficient was .686 indicating that 47% of total variance of “increased mathematical understanding” could be accounted for by expected course grade, previous use of web-based homework in mathematics, ease of navigation of the web-based homework tool, frequency with which homework was completed, and demographic items. This information is presented in Table 2.

The majority of the variables entered into the regression model were not significant including the demographic items (age, gender, academic level, and ethnicity) and if students had previously taken the course. Significant predictors of the model included expected course grade (grade=B), if students had previously used a web-based homework tool, students were motivated to complete more homework when using the web-based tool, and the ease of the web-based tool navigation. Further examination of the part correlation coefficients and the related squared values tell us that (1)13% of the total variance can be uniquely explained by students’ motivation to complete more homework with the web-based tool, and (2) approximately 5% of the total variance can be uniquely explained by the ease of navigation for the web-based tool. While students who expected a grade=B or higher and students who had previously used a web-based tool were also significant predictors in the model each accounted for less than 1% of the unique contribution to the variance. While the researchers anticipated additional variables would prove to be significant within the regression model, the findings do tell us that the web-based tool plays an important role in students’ motivation to complete more homework, possibly because of the immediate feedback (simple as that feedback may be) as a means to increase their mathematical understanding. Not surprisingly, the ease of navigation of the tool also plays an important role; if the tool is easy to use and not cumbersome, then it is more likely to be used and more often.

Table 2 . Significant Predictors of the Regression Model ( n=1076)

	Corr. w/ increased mathematical understanding	t	Sig	Beta	Part Corr.	Part Corr. 2
Expected grade (B)	.082	-2.188	.029	-.057	-.039	.002
Used web-based homework tool previously	.033	2.880	.004	.073	.065	.004
Motivated to complete more homework when using web-based tool than traditional methods	.635	16.057	.001	.464	.360	.130
Ease of web-based tool navigation	.563	9.452	.001	.269	.212	.045

*significant at the p<.001 level

For the second research question, determining if the web-based homework tool contributed to students’ study habits or learning strategies, the three MSLQ subscales were used: Peer Learning, Help Seeking (learning strategies), and Control of Learning Beliefs (motivation). The means and standard deviations of student responses to the individual MSLQ subscale questions and the three subscale scores are provided in Table 3. Next, the three subscale scores were correlated with the question “I was motivated to complete more homework when using the web-based homework system than traditional methods.” Results are reported in Table 4.

The Peer Learning subscale question means ranged from 2.35 to 3.12 on a 5-point scale (1= “strongly disagree”, 3= “undecided,” and 5= “strongly agree”) and the mean for the subscale total was 2.829. Means for the Help Seeking subscale question ranged from 2.72 to 3.61 on a 5-point scale and the mean for the subscale total was 3.204. The Control of Learning Beliefs subscale question means ranged from 3.17 to 4.13 on a 5-point scale and the mean for the subscale total was 3.678. While the data does not provide a simple answer to this research question based on the subscales as a whole, individual questions allowed us to gain some insight. For example, within the Peer Learning subscale the majority (66%) tended to disagree/strongly disagree that they set aside time to discuss the course material with a group of students from the class. Additionally, within the Help Seeking subscale, 64% of students indicated that they would ask the instructor to clarify concepts that they did not understand well, while 57% indicated they agreed/strongly agreed that they would ask another student for help when they could not understand the material in this course. It is also clear that students needed further clarification with regards to the tool (e.g., different problems, same structure).

Table 3 . Student Score Means and Standard Deviations for MSLQ Subscales Peer Learning, Help Seeking, and Control of Learning Beliefs

Items	N	M	SD
Peer Learning Subscale Scores	1305	2.829	0.945
When studying for this course, I often try to explain the material to a classmate or a friend.	1331	3.12	1.199
I try to work with other students from this class to complete the course assignments.	1329	3.02	1.306
When studying for this course, I often set aside time to discuss the course material with a group of students from the class.	1311	2.35	1.081
Help Seeking Subscale Scores	1304	3.204	0.725
Even if I have trouble learning the material in this class, I try to do the work on my own without help from anyone. (Reversed)	1319	2.72	1.198
I ask the instructor to clarify concepts I don’t understand well.	1326	3.61	0.989
When I can’t understand the material in this course, I ask another student in this class for help.	1329	3.36	1.160
I try to identify students in this class whom I can ask for help if necessary.	1329	3.15	1.127
Control of Learning Beliefs Subscale Scores	1319	3.678	0.776
If I study in appropriate ways, then I will be able to learn the material in this course.	1327	4.03	0.834
It is my own fault if I don’t learn the material in this course	1330	3.58	1.090
If I try hard enough, then I will understand the course material.	1331	3.94	0.878
If I don’t understand the course material, it is because I didn’t try hard enough.	1327	3.17	1.151

The correlation results show us that there is a significant correlation between the PL subscale and the HS subscale r (1283) =.534, p<.01. Given that these two MSLQ subscales measure similar constructs (student learning strategies in college courses) one would expect there to be a positive, significant correlation between them. More interesting to note is the positive correlations between all 3 MSLQ subscales and the item that asked students to indicate if they were motivated more to complete homework using web-based homework system over traditional methods. As Table 4 indicates, there is a significant correlation between the “motivated” item and the Peer Learning subscale score r (1300)=.126, p<.01, the “motivated” item and the Help Seeking subscale score r (1297)=.094, p<.01, and the "motivated” item and the Control of Learning Beliefs subscale score r (1309)=.232, p<.01. The significant correlations provide us some additional impressions. For example, students who felt more motivated to complete their homework using the web-based system “were also more likely to acknowledge the need for help and seek out assistance from others (peers or instructors)” (Pintrich et al., 2001, p. 29), which accounted for 88% of the variance.

Table 4. Pearson Product-Moment Correlations for “Motivated to Complete More Homework Using Web-based tool” and the Subscales for Peer Learning, Help Seeking and Control of Learning Beliefs

	1	2	3	4
Peer Learning subscale	__	.543*	.007	.126*
Help Seeking subscale		__	.004	.094*
Control of Learning Beliefs subscale			__	.232*
Motivated to Complete More Homework Using Web-based Tool				__

*p <.01

The authors also found that students who felt more motivated to complete their homework using the web-based system were also more likely to have a higher Control of Learning Beliefs score. This tells us that “students who felt that their efforts to learn would result in positive outcomes and that those outcomes are contingent on their own efforts (not instructors)” (p. 12) were more likely to indicate they were motivated to use the web-based tool; this accounted for 5% of the variance. Finally, students who felt more motivated to complete their homework using the web-based system were also more likely to believe “that collaborating with peers would have a positive effect on achievement through clarification and gaining new insights” (p. 28), but only 2% of the variance is accounted for by this relationship.

The second part of the second research question, whether students were encouraged or discouraged to complete the homework in relation to it being online, was focused on understanding whether or not students were more likely to complete their homework if it was online versus it being completed in a more traditional fashion. Survey items related to this question provided data that suggested most students were not discouraged by the web-based homework. Table 5 illustrates the results.

Table 5. Student perceptions of web-based homework tool (n=1326)

Item
	0%	25%	50%	75%	100%
What portion of the homework for the course should be completed using the web-based homework system?	35%	29%	17%	10%	9%
	SA	A	U	D	SD
I would take another course knowing web-based homework is required.	16%	39%	22%	11%	8%
	SA	A	U	D	SD
I was motivated to complete more homework when using the web-based homework system than traditional paper/pencil methods.	27%	32%	14%	13%	11%

As Table 5 shows, over half (55%) of the participants claimed they would take another course that required web-based homework to complete the assignments while only 19% of the students answered that they would not take a course that required them to do web-based homework. However, when asked what portion of the homework for the course should be completed using the web-based homework system, 9% answered that all of the homework should be web-based while 64% indicated that none or very little (only 25% of all homework should be web-based) should be completed using the web-based tool. While these data conflict, it would seem to indicate that while students don’t mind using the web-based homework tool (e.g., it would not deter them from taking a course) they would also like the opportunity to complete homework using more traditional methods.

In summary, it would appear from the correlations and related variance that students who already possessed effective study habits and learning strategies were more likely to view the web-based homework as being beneficial to them. This was especially true for students whose subscale sores indicated they were able to identify when they needed help and whom to go to for assistance as well as students who believed that their own efforts would have positive results (e.g., completing homework would help them learn). As for the question related to whether the web-based tool encouraged or discouraged students from completing the homework, the data indicate that students were motivated to complete more homework using the web-based tool, yet would prefer not all homework was set up in this manner. In addition, it would not deter them from taking future courses using the same or a similar web-based tool. In the end, it would appear that students were more encouraged than discouraged to complete their homework using the web-based tool. However, given other issues such as technology glitches in pedagogical practices, instructors should be cautious about the level of homework being assigned in this format with this tool in its current state.

Discussion

This study investigated students’ motivation and perceptions of learning in relation to the use of a web-based homework tool in one College Algebra course. Specifically, it examined learners’ achievement-related behaviors and attitudes about seeking help and learning from peers. It probed student demographic characteristics and whether those factors had any impact on their perceptions regarding the web-based homework tool. This study also explored whether the tool encouraged or discouraged students to complete their homework.

The two research questions comprise a component of learner motivation. The results indicate that students’ perceptions varied widely about web-based homework. About one-third of the students surveyed felt that the web-based homework did increase their mathematical understanding more so than with traditional paper-based methods. These results are consistent with our theoretical framework that the value of a task can affect learners’ achievement-related behaviors and attitudes related to seeking help and learning from peers (Pintrich & De Groot, 1990 ). Based on the results, it can be noted that the web-based tool plays an important role in students’ motivation to complete more homework, possibly because of the immediate feedback as a means to increase their mathematical understanding. The ease of navigation of the tool also contributed to the use of the tool or a higher motivation to use it; the easier the tool is to use, the more it is likely to be used.

The results suggest that students were motivated to complete more homework using the web-based tool than with traditional paper-based methods. However, this could be contributed in part to the fact that there were strict deadlines as well as points assigned for every question being correct, rather than only one or two questions graded when done with the traditional paper-based method. The web-based homework tool, in this case, made teaching more efficient due to less manual grading time. Perhaps a web-based tool “help sheet” would cut down on the time students spend trying to “figure out” what format the tool would accept. In addition, students need to understand that the problem does not change each time they login to the program; the numbers might change, but the basic problem is the same.

The web-base tool has the ability to provide instantaneous feedback to the student about the correctness of their work and allows them to resubmit until it is correct. Unfortunately, the virtues of immediate feedback were not appreciated by all students. Perhaps this could be remedied if the teaching assistants (instructors) were to point out the purpose of the web-based homework tool to the students with the first assignment. This would be true for any web-based homework tool. In addition, students need to be told to use their book for examples, as there are none in the web-based homework tool. This might help students to connect the homework to the book and class work rather than think of it as a separate entity.

This study provided evidence web-based homework such as the iLrn tool used by this College Algebra class may be beneficial, in the eyes of the students, as a way to receive instantaneous feedback on their homework. Of the students surveyed, 57% indicated they would seek helps from peers and 64% from an instructor, but the data did not discern if this was a change from other courses or as a result of having the web-based tool. This perception could be further investigated through examination of the number of students seeking help from the “mathematics help room.” Is it because they do not understand that the mathematical concept is the same each time they login to do a problem, or is it technical difficulties with the tool and how to enter in responses? Are they seeking help solely to get the correct answer to the question or to understand the mathematical concept being assessed by the web-based homework tool?

Another extension of this research would be to include more types of mathematics courses beyond College Algebra. While these results offered valuable insight into using a web-based homework tool, it was limited to college algebra. Other mathematics courses also use similar web-based homework tools, and to study them could provide much needed information in other mathematics content areas. Any questions that seek to understand the web-based homework tool and its role in the mathematics classroom are critical at this time as technology becomes more fully integrated into coursework. With the continued increase in use of web-based homework systems, it is critical for instructors to understand the benefits and drawbacks of such systems for student learning. This article provides student perceptions of one web-based homework tool and open doors to additional research on this topic.