User experience and motivation with engineering design challenges in general chemistry laboratory
Innovation and Education volume 3, Article number: 9 (2021)
Our career-forward approach to general chemistry laboratory for engineers involves the use of design challenges (DCs), an innovation that employs authentic professional context and practice to transform traditional tasks into developmentally appropriate career experiences. These challenges are scaled-down engineering problems related to the US National Academy of Engineering’s Grand Challenges that engage students in collaborative problem solving via the modeling process. With task features aligned with professional engineering practice, DCs are hypothesized to support student motivation for the task as well as for the profession. As an evaluation of our curriculum design process, we use expectancy–value theory to test our hypotheses by investigating the association between students’ task value beliefs and self-confidence with their user experience, gender and URM status. Using stepwise multiple regression analysis, the results reveal that students find value in completing a DC (F(5,2430) = 534.96, p < .001) and are self-confident (F(8,2427) = 154.86, p < .001) when they feel like an engineer, are satisfied, perceive collaboration, are provided help from a teaching assistant, and the tasks are not too difficult. We highlight that although female and URM students felt less self-confidence in completing a DC, these feelings were moderated by their perceptions of feeling like an engineer and collaboration in the learning process (F(10,2425) = 127.06, p < .001). When female students felt like they were engineers (gender x feel like an engineer), their self-confidence increased (β = .288) and when URM students perceived tasks as collaborative (URM status x collaboration), their self-confidence increased (β = .302). Given the lack of representation for certain groups in engineering, this study suggests that providing an opportunity for collaboration and promoting a sense of professional identity afford a more inclusive learning experience.
The persistence of undergraduate students, especially during their first two years in a science, technology, engineering and mathematics (STEM) program is a critical issue (Brainard & Carlin, 1998; Weston, 2019), particularly for students who identify as female or a member of an ethnic minority that is underrepresented (URM) (Cech et al., 2011; Robinson et al., 2018, 2019). For example, a 2013 study from the US Department of Education found that roughly one-quarter of the large random sample of college students who declared a STEM major had changed to a non-STEM major after six years (Chen & Soldner, 2013). In this group who were leaving STEM, women changed at a higher rate than men and both African-American and Hispanic students changed at rates higher than other groups. These results are consistent with other studies that have been completed at a national scale (Weston, 2019).
The culture of introductory STEM courses is notoriously unwelcoming, a situation that intimidates and challenges first- and second-year students, resulting in a learning climate that acts as a barrier to their persistence (Gasiewski et al., 2012; Marra et al., 2012; Perez et al., 2014). For female and URM students, this is characterized as the chilly climate, which leads to feelings of cynicism and emotional discouragement (Krapp & Prenzel, 2011). As an introductory course that predominantly serves different STEM majors, general chemistry is among the courses that exemplify this situation (Harris et al., 2020). Better understanding how to support students from the broadest range of STEM majors during this time period is an important point of focus that transcends STEM disciplines, requiring instructors and researchers who are focused on interventions and innovative programs. For our interest, i.e., engineering majors taking general chemistry, this would imply a laboratory curriculum that better aligns the learning experiences with the professional practice of engineering.
Targeted curriculum interventions are one potential way to address this issue of persistence (Beier et al., 2018; Henderson et al., 2011). Successful interventions typically involve an emphasis on students using the practices of science or engineering, such as inquiry (Schoffstall & Gaddis, 2007; Wheeler et al., 2017), modeling (Brewe, 2008; Hester et al., 2018) or ways of thinking (Talanquer & Pollard, 2010). In our case, the focus is a career-forward laboratory curriculum for general chemistry that addresses the learning climate barrier for engineering majors. This approach emphasizes student experiences that are developmentally appropriate approximations of the work of professionals where active problem solving requires career-authentic knowledge and practice. Thus, engineering majors learn chemistry using the ways that professional engineers use chemistry in their daily work. This implies that from the very beginning of a degree program, students learn the requisite knowledge, skills and dispositions as situated in professional practice (Johri & Olds, 2011). As the embodiment of their goal in choosing a particular major, this situation affords a more adaptive alignment of their views of themselves with that of their long-term career goal (Verdin & Godwin, 2015). Such an approach should align a student's beliefs with their long-term career goal, which is hypothesized to support the professional identity building process, and thus should improve persistence (Chachra et al., 2008; Chen et al., 2021; Osborne & Walker, 2006). Based upon the synthesis provided by Trede et al. (2012), we defined professional identity as the recognition of similarity with members and sense of membership with a profession based upon knowledge, values, motivations, sets of skills and ways of being. This study focused on task value beliefs and self-confidence as key motivational constructs in the development of professional identity.
Our laboratory curriculum requires student teams to complete design challenges (DCs), which are contextualized problems and methods that are unique to the practice of professional engineers that involve chemistry concepts (Authors, 2018). These challenges are scaled-down engineering problems related to the US National Academy of Engineering’s Grand Challenges (NAE, 2008). For example, students are tasked with building a model of a solar thermal reservoir and using it to collect data for recommending an optimum mixture of materials. Using a coffee-cup calorimeter of their own construction, they measure the specific heat of different materials and use these results coupled with cost to recommend an efficient, economical and accessible thermal energy storage medium. In addition to providing real-world context, the approach forecasts the professional practice of various engineering careers. Thus, students who are predominantly freshmen (i.e., first-year) will experience a developmentally appropriate form of professional engineering work as a learning strategy for the domain of chemistry. Accordingly, the curriculum is designed to build and maintain student motivation for their declared engineering major by helping them understand the practice and career work of different professional engineers (Harackiewicz et al., 2016; Walton & Cohen, 2007). Additionally, formalized teamwork and an emphasis on universal/global engineering issues are considerations that are intended to promote the sensitivities and interest of historically marginalized students (Driscoll et al., 2008; Estrada et al., 2018; Su & Rounds, 2015). With task features aligned with professional engineering practice, DCs are thus hypothesized to support student motivation for the task as well as for the profession (Beier et al., 2018).
Learning experience design (LXD) (Schmidt et al., 2020) has defined our approach to developing the potential of this curriculum, which has involved optimizing the learning experience based upon continuous feedback from participants (Authors, 2019, 2020). LXD is a sub-area of user experience design (Earnshaw et al., 2018), which defines successful design as achieving high-quality engagement and effectiveness by prioritizing the relationships among perceptions, feelings and behaviors for participants. Accordingly, we collected key user experience data after each course meeting using a short survey that itself has been optimized for ease of completion by students while providing key information about our model for both instructors and instructional designers. We define learner experience as the extent to which students can use the materials to achieve the course goals, which was further operationalized as students’ perceptions of the level of difficulty, degree of effort for successfully completing the task (henceforth, degree of effort), satisfaction, collaboration, frequency of help provided by the teaching assistant and the degree to which the participants felt like an engineer after completing a DC. To this point, this data has been used for continuous just-in-time evaluations and incremental modifications to the DCs (Authors, 2019, 2020), but the process across DCs has not been formally assessed.
The objective of this study was to use expectancy–value theory (EVT) to assess the assumptions of our LXD process by investigating the association between students’ task value beliefs and self-confidence with their user experience, gender and URM status (Fig. 1)(Eccles & Wigfield, 1995; Wigfield & Eccles, 2000). Such a point of focus is often understudied during the development of interventions (NRC, 2012), and for us it represented an important departure from formative to summative evaluation. Accordingly, the following served as our research questions:
For engineering majors completing DCs as general chemistry laboratory,
What relationship exists between their perceived value of these tasks, self-confidence, user experience, gender and URM status?
What effect does user experience have on the magnitude and direction of the relationship between task value beliefs (and self-confidence) with gender or URM status?
Theoretical framework and related literature
Expectancy–value theory posits that individuals’ motivation for a task is influenced by their expectations of success, or expectancy beliefs, in combination with their views about the value of the task, or subjective task value beliefs (Eccles & Wigfield, 1995; Wigfield & Eccles, 2000). Taken together, these two sets of beliefs dictate productive engagement with a task and subsequently what a student might derive from the experience. However, each set of beliefs does not have equal influence on outcomes. Expectancy beliefs are more predictive of academic achievement and performance (Jones et al., 2010; Rosenzweig et al., 2019), while task value beliefs are better predictors of future course taking or retention (Bong, 2001; Harackiewicz et al., 2014). Expectancy and task value beliefs are recognized as being composed of different types that are assessed using various constructs (Hulleman et al., 2016).
Expectancy beliefs are internal judgements about how one anticipates their performance with a domain-specific task (i.e., “Can I do this?”). These judgements apply and are measured for both the immediate or current context as well as for those in the future. When measured for the current context (i.e., expectancy for success), this belief represents an individual’s perceived capability for learning concepts and performing skills that are specific to a domain (Rosenzweig et al., 2019). Self-efficacy, self-concept and self-confidence are commonly operationalized variables for measuring expectancy beliefs (Hulleman et al., 2016). Self-efficacy—an individual’s perceived capability for learning concepts and performing skills (Bandura et al., 1999)—is widely recognized as the strongest predictor of first-year retention when controlling for high school GPA, ACT/SAT and socioeconomic status (Robbins et al., 2004). For this study, expectancy beliefs were operationalized as self-confidence, a participant’s perspective on their ability to engage and complete a task successfully.
Eccles and colleagues have most consistently argued that task value beliefs are composed of three components: intrinsic, attainment and utility value (Eccles, 2005; Eccles & Wigfield, 2002, 2020). Intrinsic value or intrinsic interest is the perception of enjoyment or interest that an individual anticipates from performing an activity or completing a task (i.e., “Will I enjoy doing this?”). Intrinsic interest has been shown to be a key predictor of positive student experiences when learning in a laboratory context (Barrie et al., 2015). Attainment value refers to the perception of a task’s importance relative to an individual’s self-concept (i.e., “Is this an important thing to be doing?”) and utility value refers to the perceived usefulness of a task in relation to the individual’s short- and long-term goals (i.e., “Is this related to my goals?”).
Task value beliefs are influenced by expectancy beliefs, such as self-confidence, but also by affective reactions and memories that are derived from experience (Eccles & Wigfield, 2020). Thus, these beliefs are defined as subjective, or based upon an individual interpretation of experience (i.e., learner experience). When the level of difficulty and amount of required effort for a task matches expectations, students are more likely to find value and experience high expectancy (Eccles, 1983; Walton et al., 2015). However, perceiving the level of difficulty as low can lead to an inflated estimate of one’s personal ability (Eccles & Wigfield, 2002; Pajares, 1996). Meaningful interactions with other students (i.e., collaboration) as well as positive interactions with an instructor or teaching assistant, such as receiving help, produces positive task value judgments (Patrick et al., 2000; Walton & Cohen, 2007). Connecting career aspirations, such as being an engineer or feeling like an engineer, with other identities, such as racial, ethnic or gender identity, improves both expectancy and task value beliefs (Oyserman et al., 2006). The lower rates of retention for female and URM undergraduate students in STEM implies that they are collectively having different experiences from their male and non-URM counterparts (Brown et al., 2015; Gaspard et al., 2015, 2017; Kang et al., 2019; Miyake et al., 2010; Thoman et al., 2015; Walton & Cohen, 2007).
Intentional curriculum or instructional approaches that target a change in some dimension of a student’s task value beliefs are recognized as task value interventions, and a host of different varieties have shown to be successful, and in many cases, rather simple to implement (Harackiewicz & Priniski, 2018). For example, Walton and Cohen (2011) found that having students read research about the commonality of social adversity among first-year STEM majors and then writing and speaking about it resulted in positive outcomes for African-American students. Focusing on self-worth and identity as a STEM major, this personal value intervention has the potential to address a lack of self-confidence, which is a major factor in retaining underrepresented students (Goodman & Cunningham, 2002). Utility value interventions, those that promote the usefulness of a task for a student’s goals have shown particular promise for supporting URM students (Brown et al., 2015; Thoman et al., 2015). In these cases, the more explicit the value is communicated, the greater the potential impact on students (Canning & Harackiewicz, 2015; Curry et al., 2019).
With an explicit focus on the professional practice of engineers, the DC’s of the laboratory curriculum studied here should align with the goals of first-year engineering majors and generate positive task value beliefs. We anticipated intrinsic interest value because they chose to major in engineering as well as utility value due to the practices addressing their long-term career goal (Cech et al., 2011; Hulleman et al., 2017; Jones et al., 2010). Therefore, we anticipated positive relationships among satisfaction, collaboration and feeling like an engineer with task value beliefs. Task value beliefs should also be strongly related to self-confidence (Keller, 2010). Since task value beliefs are also influenced by gender and URM status (Brown et al., 2015; Miyake et al., 2010; Thoman et al., 2015), we further anticipated a positive influence for previous experience and negative influence for gender and URM status on task value judgements.
This task-based study involved the use of a correlational methodology to evaluate the research questions. The context was two separate instances of a first-semester general chemistry laboratory course for engineers at a large public research university in the USA. This course was required for students majoring in environmental, chemical, bioengineering, biomedical, material science and nuclear engineering, but was also an option for other engineering majors and included the same core topics of a traditional first-semester chemistry laboratory such as physical and chemical properties, kinetics and colligative properties. For the course, student teams completed a series of four DCs that integrated engineering design principles and context with chemistry content in three phases that corresponded to weeks in the course (Authors, 2018). Within each phase, teams were tasked to perform quality management duties to foster communication and collaboration. Each DC culminated with the teams creating a formal response to a client (Beier et al., 2018), such as a technical memo for a government agency that was requesting an independent proposal or evaluation.
We used a purposeful sampling method to invite all students enrolled in a special section of the course to participate in the study at the beginning of Fall 2018 and Fall 2019. A total of 250 students (92%) provided informed consent for this university-approved study involving human subjects and became participants (UF IRB201600944). The distribution of the sample with information about gender and URM status is reported in Table 1.
Maximizing the potential of a learning intervention through evidence-based iterative design is achieved by focusing on enhancing satisfaction and learning outcomes through improvements in four principle attributes: usability, learnability, accessibility and desirability (Tullis & Albert, 2013). Some of these attributes can be measured directly, but others are assessed as learner perceptions of the experience using task-based metrics (Sauro, 2018). In this case, assessment of the learning experience included level of difficulty, degree of effort, satisfaction, collaboration, frequency of help provided by a teaching assistant and the degree to which the participants felt like an engineer. These variables were assessed using two different survey types, a one-time demographic survey given at the beginning of the course and multiple repeated instances of a post-laboratory experience survey that participants completed before leaving the laboratory. All data were collected with personal information that identified participants (i.e., not anonymously), which allowed for verification and non-duplication of responses. Once the data were verified and collated, personal information was anonymized for analysis. The post-laboratory surveys, which were completed after each phase of a DC for a total of ten potential responses per participant, were collated as 2436 data points (97.5% response).
The demographic survey included items on gender and ethnic identities. The item for gender included five response options (including “prefer not to say”). The majority of the participants identified as either male or female, hence we only included those two levels for the gender variable. The item for ethnicity included six options. Participants who identified with ethnic groups recognized as underrepresented were categorized as URM, and those who identified as either White or Asian were categorized as non-URM. This distinction was based upon the definition used by the U S National Science Foundation (NSF, n.d.).
The post-laboratory experience survey consisted of ten items, each with a five-point Likert-type response scale. The survey was segregated into three parts: 1) user experience (six items), 2) task value (three items) and 3) self-confidence (one item) (Table 2). User experience at the task level is commonly assessed with single-item metrics and research in this genre has established such measures to be as good, or in some cases better than, more extensive and complicated instruments (Sauro & Dumas, 2009). Level of difficulty is a measure of how complicated the participant perceived the task, while the degree of effort indicates how much work they perceived as necessary for successfully completing the task. Satisfaction is a measure of how well the experience met the participant’s expectations and is a nearly universal measurement of user or customer attitudes (McColl-Kennedy & Schneider, 2000). Collaboration indicates the task’s success in supporting participants working together productively within the task, an important engineering practice. Feeling like an engineer indicates the degree to which the task supports the intended professional role playing and thus matches the participant’s academic and career goals. Help from a teaching assistant indicates the necessity of this person as an integral part of task success and also measures a participant’s perceived capacity for completing the task independently. Finally, self-confidence is a participant’s perspective on their ability to engage and complete a task successfully and is a key indicator of problems within the execution of the task. The task value and self-confidence items were derived from Schukajlow et al., (2012) and modified to reference “this design challenge.”
Based on our assumption that a student’s experience is related to their motivation for a DC, we used the six-user experience and two-demographic variables (i.e., gender and URM status) as independent variables with task value and self-confidence as dependent variables for analysis. Since task value was measured as a multi-item construct with an existing instrument, we used confirmatory factor analysis (CFA) to affirm the internal structure of the scale using a single-factor structure and maximum likelihood estimation with the factor loading for the first item set to 1. The fit statistics provided in Table 3 indicate good model fit given that comparative fit index (CFI) and Tucker-Lewis index (TLI) are above 0.90 and standardized root mean square residual (SRMR) < 0.08 (Bentler & Bonett, 1980). The standardized factor loadings indicate a strong relationship between the items of enjoyment (0.807), importance (0.758) and interest (0.888) with the construct of task value (Cronbach's alpha = 0.85). The CFA and reliability statistics were run using the statistical software R version 4.0.1 (R Core Team, 2020).
For the first research question, we conducted a correlation analysis using Pearson’s correlation coefficient to identify which independent variables had significant associations or correlation with the task value and self-confidence constructs and with each other. This test enabled us to identify which variables were likely to be predictors for task value and self-confidence. We used the following correlation cut points for interpretations: |.20| and below (small, weak), |.21| to |.49| (medium, moderate), and above |.5| (large, strong) (Cohen, 1988).
The stepwise multiple regression analysis conducted for the second research question was exploratory by nature, intending to build the best-fit model that could predict task value and self-confidence using the user experience and demographic variables as independent variables. Informed by the results of our correlation analysis, a stepwise multiple regression analysis was conducted to test the significance of the strength of each independent variable in predicting both task value and self-confidence (Tabachnick & Fidell, 2012). The assumptions for independence of observations, linearity, normality, homoscedasticity and multicollinearity were assessed prior to analysis and were found to be appropriate. In conducting a stepwise regression analysis, variables were introduced into the model sequentially utilizing both correlation strength and its significance as a predictor to inform the order in which variables were added into the model. Each time a variable was added, a removal test of the least significant predictor was automatically conducted until a final model with only significant predictors was obtained. When the probability value was less than or equal to 0.05, the variable was kept and if the probability value was greater than 0.05, the variable was removed.
We sought to identify if either or both, gender and URM status, moderated or altered the strength of the causal relationships between the independent and dependent variables. We therefore created interaction terms, with each interaction being a product term between a demographic and user experience variable (e.g., gender x task difficulty). Interaction terms indicate a contingency relationship where one variable is moderated by another and is a statistical technique used as part of multiple regression analysis (Aguinis & Gottfredson, 2010). A similar stepwise regression analysis was conducted with interaction terms included. All regression and correlation analyses were run in SPSS version 28 (IBM Corp., 2019). The significance level of alpha < 0.05 was used for all regression analyses.
During both semesters, the participants were provided with the same versions of the DCs, but were instructed by different teaching assistants and experienced other randomized factors, such as time of day. Before analyzing the results for each research question, we used a Chi-square test of proportions to evaluate any differences between groups. Our results indicated that groups from each semester did not differ in the proportion of males and females (χ2(N = 250) = 0.020, p = 0.889), non-URM and URM students (χ2(N = 250) = 0.20, p = 0.887) or by proportions of majors (χ2(N = 250) = 17.681, p = 0.343). These results justified our combining semesters into one large group for analysis.
The results of the correlational analyses showed that among various user experience variables, perceptions of satisfaction (r(2436) = 0.590, p < 0.01), collaboration (r(2436) = 0.593, p < 0.01) and feeling like an engineer (r(2436) = 0.631, p < 0.01) showed significantly strong, positive correlations with task value (Table 4). Perceptions of satisfaction (r(2436) = 0.419, p < 0.01) and collaboration (r(2436) = 0.416, p < 0.01) showed significantly moderate, positive correlations with self-confidence. Among the demographic variables, gender (r(2436) = -0.107, p < 0.01) was found to be significantly correlated to self-confidence with a low and negative correlation coefficient, while URM status (r(2436) = − 0.053, p < 0.009) showed a low, significant correlation with task value and less than one percent of the variation in this relationship.
The results of the stepwise regression revealed that among the user experience variables tested, participants’ perception of feeling like an engineer, satisfaction, collaboration, help from the teaching assistant and difficulty were found to be significant predictors for task value, F(5, 2430) = 534.96, p = 0.004, adjusted R2 = 0.524. Gender and URM status were not statistically significant predictors of task value in the regression model, thus were removed. See Table 5 for details on each regression model, where only significant predictors are included. Students found task value when they felt like an engineer (0.334 standard deviation increase in task value for every one standard deviation increase in feeling like an engineer), were satisfied (0.265 standard deviation increase for every one standard deviation increase in satisfaction), worked collaboratively (0.199 standard deviation increase for every one standard deviation increase collaboration) and were provided help from the teaching assistant (0.088 standard deviation increase for every one standard deviation increase in help from the teaching assistant). The more students interpreted a DC as difficult, the less value they found (0.042 standard deviation decrease in task value for every one standard deviation increase in difficulty), though the coefficient indicates that this is a small effect. The best-fit model explained 52.4% of the variance for task value, which is considered a large effect (Cohen, 1988).
For self-confidence, the full regression model of perception of satisfaction, difficulty, collaboration, help from the teaching assistant, feel like an engineer, gender, URM status and effort was found to be statistically significant, F(8, 2427) = 154.86, p = 0.042, adjusted R2 = 0.338. Among the variables in the model, difficulty (β = − 0.244), gender (β = − 0.107), URM status (β = -0.068) and effort (β = -0.036) have negative standardized regression coefficients. Generally, students had self-confidence when they were satisfied (0.150 standard deviation increase in self-confidence for every one standard deviation increase in satisfaction), worked collaboratively (0.149 standard deviation increase for every one standard deviation increase in collaboration), were provided help from the teaching assistant (0.159 standard deviation increase for every one standard deviation increase in help from the teaching assistant) and felt like an engineer (0.148 standard deviation increase for every one standard deviation increase in feeling like an engineer). They lost self-confidence for completing a DC when the task was too difficult (0.244 standard deviation decrease for every one standard deviation increase in difficulty) and required a lot of effort (0.036 standard deviation decrease for every one standard deviation increase in effort). Females were less self-confident than males that they could complete a DC (β = − 0.107) and URM students were less self-confident than non-URM students (β = − 0.068). By explaining 33.8% of the variance for self-confidence, this model also indicates a large effect.
The results of the stepwise regression for moderators revealed that gender and URM status were significant predictors for self-confidence, F(10, 2425) = 127.06, p = 0.005, adjusted R2 = 0.344 and increased the variance explained to 34.4%. Thus, the relationship between self-confidence and the independent variables of satisfaction, difficulty, collaboration, help from a teaching assistant, feel like an engineer and effort changed in the presence of either gender or URM status. As a single variable, gender had a significant negative coefficient in the model when all other variables were kept constant (β = − 0.257), indicating that female students were less self-confident. However, this situation changed dramatically, essentially reversing direction and magnitude when gender was moderated with feeling like an engineer (gender x feel like an engineer), indicating that female participants were in fact more self-confident when they felt like an engineer (0.288 standard deviation increase for every one standard deviation increase in feeling like an engineer). This encouraging result suggests that female students gained self-confidence when they were made to feel like engineers as part of the experience. Effort also had a moderating effect with gender (gender x effort; β = − 0.135), essentially decreasing the negative effect on self-confidence of gender alone. Females perceived a need for more effort than males in order to be self-confident in completing a DC.
A similar situation and intriguing result was documented when URM status was used as a moderating variable. As a single variable, URM status had a significant negative coefficient in the model when all other variables were kept constant (β = − 0.369), indicating that URM students were significantly less self-confident than their peers. When URM status was moderated with collaboration (URM status x collaboration), self-confidence effectively changed from a negative relationship to a positive one, indicating that URM participants were more self-confident when they collaborated (0.302 standard deviation increase for every one standard deviation increase in collaboration).
In exploring the relationships among user experience and task value for this career-forward curriculum, participants found value in the DC tasks when they felt like an engineer and were satisfied, but also when they felt that they were provided opportunities to work collaboratively and had access to a teaching assistant for help. Alternatively, when participants felt that the task was too difficult, their perceptions of value suffered. This finding supports our hypothesis that these materials would generate intrinsic interest value due to the alignment between their goals of doing well and becoming an engineer and the explicit focus on the professional practice of engineers (Cech et al., 2011; Hulleman et al., 2017; Jones et al., 2010). This result further supports prior research showing a strong relationship between task value beliefs and self-confidence (Keller, 2010). The negative relationship between level of difficulty and task value should be interpreted cautiously, particularly for curriculum or teaching design implications, as this small effect could be more attributed to the participants’ stage in development than an influence of the curriculum or instruction.
Self-confidence was a problem for female students, but this situation was rectified when they perceived themselves as being an engineer during the curriculum activities. This finding further supports the influence of gender on task value beliefs (Brown et al., 2015; Miyake et al., 2010), but also the potential of a career-forward approach as one that can address gender-based barriers to persistence. For these students, feeling like an engineer translated to a boost in self-confidence that would have otherwise been a barrier. Not only does such an approach directly illustrates the utility of the activity for the students’ goals (Harackiewicz & Priniski, 2018), it also affirms their identities with the domain (Harackiewicz et al., 2014, 2016; Miyake et al., 2010).
The situation for URM students was similar, but subtly different than that for female students. URM students were also less self-confident in completing career-based tasks, but when they perceived collaboration in the learning process, this changed a negative perception to a positive consequence and essentially removed what would have been a barrier to their persistence. This finding further affirms the potential of a career-forward approach as a strategy for promoting inclusion and equity. Aside from utility value, perceived collaboration within career-forward curriculum activities may be promoting a view of engineering as a profession that is more communal than solitary, thus providing opportunities for interpersonal relationships, which is a theme that seems to resonate with URM students (Brown et al., 2015; Smith et al., 2014; Thoman et al., 2015).
Given the lack of representation for students who identify as female and/or URM in the profession of engineering, this study makes an important contribution to our understanding of how to support them during their undergraduate degree programs, particularly during the first two years when they are the most vulnerable to exclusion. Providing developmentally appropriate tasks that align with professional practice will lessen a known gender-based self-confidence barrier. Achieving the same result for URM students will also require the addition of teamwork or collaboration, otherwise the opposite effect can be anticipated because the self-confidence barrier is different for this student population. By applying these suggestions with our innovative career-forward curriculum as an example, practitioners and researchers can better design inclusive and equitable learning environments that serve the student body as well as the profession.
Limitations and future research
The results reported here are limited by a number of factors and our use of single survey items for measuring the user experience is likely the most impactful. In addition, the carryover effect, an important form of bias any time that participants are assessed at multiple points in time, was not assessed and likely had an influence on the results. However, we attempted to minimize this influence with our messaging to participants and with the timing of surveys, both in the duration of time between instances as well as having participants complete them immediately following the experience. Future research using crossover or changeover designs could more rigorously address this issue. A more comprehensive understanding of motivation, especially over a longer duration, could be provided if both expectancy and task value beliefs were assessed with more rigorous instrumentation. Finally, the regression analysis was based upon an assumed linear relationship among the variables and future research should investigate other potential best-fit models.
The persistence of undergraduate students, particularly those who identify as female or URM, is a critical issue for all STEM disciplines. Systematically assessing student’s programmatic experiences, especially in light of our emerging understanding for how to support URM students, is one means to addressing this systemic problem. Career-forward approaches to curriculum, such as our use of design challenges to re-envision general chemistry laboratory for engineers, is shown to address factors related to persistence when implemented in introductory courses. As the embodiment of a student’s goal in choosing a particular major, this approach affords an adaptive alignment via its explicit nature, which productively builds their task value beliefs and self-confidence.
Availability of data and material
Raw data are available upon request from the corresponding author.
Aguinis, H., & Gottfredson, R. K. (2010). Best-practice recommendations for estimating interaction effects using moderated multiple regression. Journal of Organizational Behavior, 31(6), 776–786. https://doi.org/10.1002/job.686
Bandura, A., Freeman, W. H., & Lightsey, R. (1999). Self-efficacy: the exercise of control. Journal of Cognitive Psychotherapy, 13(2), 158–166. https://doi.org/10.1891/0889-83220.127.116.11
Barrie, S. C., Bucat, R. B., Buntine, M. A., Burke da Silva, K., Crisp, G. T., George, A. V., Jamie, I. M., Kable, S. H., Lim, K. F., Pyke, S. M., Read, J. R., Sharma, M. D., & Yeung, A. (2015). Development, evaluation and use of a student experience survey in undergraduate science laboratories: The advancing science by enhancing learning in the laboratory student laboratory learning experience survey. International Journal of Science Education, 37(11), 1795–1814. https://doi.org/10.1080/09500693.2015.1052585
Beier, M. E., Kim, M. H., Saterbak, A., Leautaud, V., Bishnoi, S., & Gilberto, J. M. (2018). The effect of authentic project-based learning on attitudes and career aspirations in STEM. Journal of Research in Science Teaching. https://doi.org/10.1002/tea.21465
Bentler, P. M., & Bonett, D. G. (1980). Significance tests and goodness of fit in the analysis of covariance structures. Psychological Bulletin, 88(3), 588–606. https://doi.org/10.1037/0033-2909.88.3.588
Bong, M. (2001). Role of self-efficacy and task-value in predicting college students’ course performance and future enrollment intentions. Contemporary Educational Psychology, 26(4), 553–570. https://doi.org/10.1006/ceps.2000.1048
Brainard, S. G., & Carlin, L. (1998). A six-year longitudinal study of undergraduate women in engineering and science*. Journal of Engineering Education, 87(4), 369–375. https://doi.org/10.1002/j.2168-9830.1998.tb00367.x
Brewe, E. (2008). Modeling theory applied: Modeling Instruction in introductory physics. American Journal of Physics, 76(12), 1155–1160. https://doi.org/10.1119/1.2983148
Brown, E. R., Smith, J. L., Thoman, D. B., Allen, J. M., & Muragishi, G. (2015). From Bench to Bedside: A communal utility value intervention to enhance students’ biomedical science motivation. Journal of Educational Psychology, 107(4), 1116–1135. https://doi.org/10.1037/edu0000033
Canning, E. A., & Harackiewicz, J. M. (2015). Teach It, Don’t Preach It: The differential effects of directly-communicated and self-generated utility value information. Motivation Science, 1(1), 47–71. https://doi.org/10.1037/mot0000015
Cech, E., Rubineau, B., Silbey, S., & Seron, C. (2011). Professional role confidence and gendered persistence in engineering. American Sociological Review, 76(5), 641–666. https://doi.org/10.1177/0003122411420815
Chachra, D., Kilgore, D., Loshbaugh, H., McCain, J., & Chen, H. L. (2008). Being and becoming: Gender and identity formation of engineering students. 13.250.1–13.250.20. https://peer.asee.org/3597
Chen, S., Binning, K. R., Manke, K. J., Brady, S. T., McGreevy, E. M., Betancur, L., Limeri, L. B., & Kaufmann, N. (2021). Am I a science person? A strong science identity bolsters minority students’ sense of belonging and performance in college. Personality and Social Psychology Bulletin, 47(4), 593–606. https://doi.org/10.1177/0146167220936480
Chen, X., & Soldner, M. (2013). STEM attrition: College students’ paths into and out of STEM fields (Statistical Analysis Report (SAR) NCES 2014–001). U. S. Department of Education. https://nces.ed.gov/pubs2014/2014001rev.pdf
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates. https://doi.org/10.4324/9780203771587
Crippen, K. J., Imperial, L., Wu, C.-Y., Korolev, M., Brucat, P., & Payne, C. (2018). General chemistry laboratory as situated engineering design. Proceedings of the American Society for Engineering Education. American Society for Engineering Education (ASEE) Annual Conference & Exposition, Salt Lake City, UT. https://doi.org/10.18260/1-2--29965
Curry, K. W., Spencer, D., Pesout, O., & Pigford, K. (2019). Utility value interventions in a college biology lab: The impact on motivation. Journal of Research in Science Teaching. https://doi.org/10.1002/tea.21592
Driscoll, D. M., Zawojewski, J. S., & Stahura, J. (2008). Learning from first-year engineering student surveys. In J. S. Zawojewski, H. A. Diefes-Dux, & K. J. Bowman (Eds.), Models and Modeling in Engineering Education (pp. 11–136). Sense Publishers.
Earnshaw, Y., Tawfik, A. A., & Schmidt, M. (2018). User Experience Design. In R. E. West (Ed.), Foundations of Learning and Instructional Design Technology: The Past, Present, and Future of Learning and Instructional Design Technology. EdTech Books. Retrieved from https://edtechbooks.org/lidtfoundations.
Eccles, J. (1983). Expectancies, values and academic behaviors. In J. T. Spence (Ed.), Psychological and sociological approaches (pp. 75–146). Freeman.
Eccles, J. S., & Wigfield, A. (1995). In the Mind of the Actor: The Structure of Adolescents’ Achievement Task Values and Expectancy-Related Beliefs. Personality and Social Psychology Bulletin, 21(3), 215–225. https://doi.org/10.1177/0146167295213003
Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132. https://doi.org/10.1146/annurev.psych.53.100901.135153
Eccles, J. S., & Wigfield, A. (2020). From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation. Contemporary Educational Psychology. https://doi.org/10.1016/j.cedpsych.2020.101859
Eccles, J. S. (2005). Subjective task value and the eccles et al. model of achievement-related choices. In A. J. Elliot & C. S. Dweck (Eds.), Handbook of Competence and Motivation (1st ed.). Guilford Press.
Estrada, M., Eroy-Reveles, A., & Matsui, J. (2018). The influence of affirming kindness and community on broadening participation in STEM career pathways. Social Issues and Policy Review, 12(1), 258–297. https://doi.org/10.1111/sipr.12046
Gasiewski, J. A., Eagan, M. K., Garcia, G. A., Hurtado, S., & Chang, M. J. (2012). From gatekeeping to engagement: A multicontextual, mixed method study of student academic engagement in introductory STEM courses. Research in Higher Education, 53(2), 229–261. https://doi.org/10.1007/s11162-011-9247-y
Gaspard, H., Dicke, A.-L., Flunger, B., Schreier, B., Häfner, I., Trautwein, U., & Nagengast, B. (2015). More value through greater differentiation: Gender differences in value beliefs about math. Journal of Educational Psychology, 107(3), 663–677. https://doi.org/10.1037/edu0000003
Gaspard, H., Häfner, I., Parrisius, C., Trautwein, U., & Nagengast, B. (2017). Assessing task values in five subjects during secondary school: Measurement structure and mean level differences across grade level, gender, and academic subject. Contemporary Educational Psychology, 48, 67–84. https://doi.org/10.1016/j.cedpsych.2016.09.003
Goodman, I., & Cunningham, C. (2002). Final Report of the Women’s Experiences in College Engineering (WECE) Project (No. ED507394). Goodman Research Group.
Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Giffen, C. J., Blair, S. S., Rouse, D. I., & Hyde, J. S. (2014). Closing the Social Class Achievement Gap for First-Generation Students in Undergraduate Biology. Journal of Educational Psychology, 106(2), 375–389. https://doi.org/10.1037/a0034679
Harackiewicz, J. M., Canning, E. A., Tibbetts, Y., Priniski, S. J., & Hyde, J. S. (2016). Closing achievement gaps with a utility-value intervention: Disentangling race and social class. Journal of Personality and Social Psychology, 111(5), 745–765. https://doi.org/10.1037/pspp0000075
Harackiewicz, J. M., & Priniski, S. J. (2018). Improving student outcomes in higher education: The science of targeted intervention. Annual Review of Psychology, 69, 409–435. https://doi.org/10.1146/annurev-psych-122216-011725
Harris, R. B., Mack, M. R., Bryant, J., Theobald, E. J., & Freeman, S. (2020). Reducing achievement gaps in undergraduate general chemistry could lift underrepresented students into a “hyperpersistent zone”. Science Advances, 6(24), eaaz5687. https://doi.org/10.1126/sciadv.aaz5687
Henderson, C., Beach, A., & Finkelstein, N. (2011). Facilitating change in undergraduate STEM instructional practices: An analytic review of the literature. Journal of Research in Science Teaching, 48(8), 952–984. https://doi.org/10.1002/tea.20439
Hester, S. D., Nadler, M., Katcher, J., Elfring, L. K., Dykstra, E., Rezende, L. F., & Bolger, M. S. (2018). Authentic inquiry through modeling in biology (AIM-Bio): An introductory laboratory curriculum that increases undergraduates’ scientific agency and skills. CBE Life Sciences Education, 17(4), ar63. https://doi.org/10.1187/cbe.18-06-0090
Hulleman, C. S., Barron, K. E., Kosovich, J. J., & Lazowski, R. A. (2016). Student Motivation: Current Theories, Constructs, and Interventions Within an Expectancy-Value Framework. In A. A. Lipnevich, F. Preckel, & R. D. Roberts (Eds.), Psychosocial skills and school systems in the 21st century (pp. 241–278). Springer International Publishing. https://doi.org/10.1007/978-3-319-28606-8_10
Hulleman, C. S., Kosovich, J. J., Barron, K. E., & Daniel, D. B. (2017). Making connections: Replicating and extending the utility value intervention in the classroom. Journal of Educational Psychology, 109(3), 387–404. https://doi.org/10.1037/edu0000146
IBM Corp. (2019). IBM SPSS Statistics for Macintosh (28.0) [Computer software]. IBM Corp.
Johri, A., & Olds, B. M. (2011). Situated engineering learning: Bridging engineering education research and the learning sciences. Journal of Engineering Education, 100(1), 151–185.
Jones, B. D., Paretti, M. C., Hein, S. F., & Knott, T. W. (2010). An analysis of motivation constructs with first-year engineering students: relationships among expectancies, values, achievement, and career plans. Journal of Engineering Education, 99(4), 319–336. https://doi.org/10.1002/j.2168-9830.2010.tb01066.x
Kang, J., Hense, J., Scheersoi, A., & Keinonen, T. (2019). Gender study on the relationships between science interest and future career perspectives. International Journal of Science Education, 41(1), 80–101. https://doi.org/10.1080/09500693.2018.1534021
Keller, J. M. (2010). Motivational design for learning and performance. Springer, US. https://doi.org/10.1007/978-1-4419-1250-3
Krapp, A., & Prenzel, M. (2011). Research on interest in science: Theories, methods, and findings. International Journal of Science Education, 33(1), 27–50. https://doi.org/10.1080/09500693.2010.518645
Marra, R. M., Rodgers, K. A., Shen, D., & Bogue, B. (2012). Leaving engineering: A multi-year single institution study. Journal of Engineering Education, 101(1), 6–27. https://doi.org/10.1002/j.2168-9830.2012.tb00039.x
McColl-Kennedy, J., & Schneider, U. (2000). Measuring customer satisfaction: Why, what and how. Total Quality Management, 11(7), 883–896. https://doi.org/10.1080/09544120050135434
Miyake, A., Kost-Smith, L. E., Finkelstein, N. D., Pollock, S. J., Cohen, G. L., & Ito, T. A. (2010). Reducing the gender achievement gap in college science: A classroom study of values affirmation. Science, 330(6008), 1234–1237. https://doi.org/10.1126/science.1195996
National Academy of Engineering [NAE]. (2008). Grand Challenges - 14 Grand Challenges for Engineering. http://www.engineeringchallenges.org
National Research Council [NRC] (Ed.). (2012). Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering. National Academies Press. https://doi.org/10.17226/13362
National Science Foundation [NSF]. (n.d.). U. S. National Science Foundation definition of groups underrepresented . US NSF - MPS - DMR - Broadening Participation. Retrieved March 5, 2021, from https://www.nsf.gov/mps/dmr/diversity.jsp
Osborne, J. W., & Walker, C. (2006). Stereotype threat, identification with academics, and withdrawal from school: Why the most successful students of colour might be most likely to withdraw. Educational Psychology, 26(4), 563–577. https://doi.org/10.1080/01443410500342518
Oyserman, D., Bybee, D., & Terry, K. (2006). Possible selves and academic outcomes: How and when possible selves impel action. Journal of Personality and Social Psychology, 91(1), 188–204. https://doi.org/10.1037/0022-3518.104.22.168
Pajares, F. (1996). Self-efficacy beliefs in academic settings. Review of Educational Research, 66(4), 543–578. https://doi.org/10.3102/00346543066004543
Patrick, B. C., Hisley, J., & Kempler, T. (2000). “What’s Everybody So Excited About?”: The Effects of Teacher Enthusiasm on Student Intrinsic Motivation and Vitality. Informa UK Limited. https://doi.org/10.1080/00220970009600093
Payne, C., Crippen, K. J., Imperial, L., Wu, C.-Y., Brucat, P., & Korolev, M. (2019). Persistence of First-year Engineering Majors with a Design-based Chemistry Laboratory Curriculum In- and Out-of-Sequence. Proceedings of the American Society for Engineering Education. American Society for Engineering Education (ASEE) Annual Conference & Exposition, Tampa, FL. https://doi.org/10.18260/1-2--32326
Payne, C., Crippen, K. J., & Imperial, L. (2021). An Exploration of Perceptions of Justice in a Career-Forward Undergraduate Chemistry Laboratory Course. Journal for STEM Education Research. https://doi.org/10.1007/s41979-021-00063-1
Perez, T., Cromley, J. G., & Kaplan, A. (2014). The role of identity development, values, and costs in college STEM retention. Journal of Educational Psychology, 106(1), 315–329. https://doi.org/10.1037/a0034027
Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A Meta-Analysis. Psychological Bulletin, 130(2), 261–288. https://doi.org/10.1037/0033-2909.130.2.261
Robinson, K. A., Perez, T., Carmel, J. H., & Linnenbrink-Garcia, L. (2019). Science identity development trajectories in a gateway college chemistry course: Predictors and relations to achievement and STEM pursuit. Contemporary Educational Psychology, 56, 180–192. https://doi.org/10.1016/j.cedpsych.2019.01.004
Robinson, K. A., Perez, T., Nuttall, A. K., Roseth, C. J., & Linnenbrink-Garcia, L. (2018). From science student to scientist: Predictors and outcomes of heterogeneous science identity trajectories in college. Developmental Psychology, 54(10), 1977–1992. https://doi.org/10.1037/dev0000567
Rosenzweig, E. Q., Wigfield, A., & Eccles, J. S. (2019). Expectancy-Value Theory and Its Relevance for Student Motivation and Learning. In The cambridge handbook of motivation and learning (pp. 617–644). Cambridge University Press. https://doi.org/10.1017/9781316823279.026
R Core Team. (2020). R: A language and environment for statistical computing (4.0.2) [Computer software]. R Foundation for Statistical Computing. https://www.R-project.org
Sauro, J., & Dumas, J. S. (2009). Comparison of three one-question, post-task usability questionnaires. Proceedings of the 27th International Conference on Human Factors in Computing Systems - CHI 09, 1599. https://doi.org/10.1145/1518701.1518946
Sauro, J. (2018). Benchmarking the User Experience. MeasuringU Press.
Schmidt, M., Tawfik, A. A., Jahnke, I., & Earnshaw, Y. (Eds.). (2020). Learner and user experience research an introduction for the field of learning design & technology. Ed Tech Books. https://edtechbooks.org/ux
Schoffstall, A. M., & Gaddis, B. A. (2007). Incorporating guided-inquiry learning into the organic chemistry laboratory. Journal of Chemical Education, 84(5), 848. https://doi.org/10.1021/ed084p848
Schukajlow, S., Leiss, D., Pekrun, R., Blum, W., Müller, M., & Messner, R. (2012). Teaching methods for modelling problems and students’ task-specific enjoyment, value, interest and self-efficacy expectations. Educational Studies in Mathematics, 79(2), 215–237. https://doi.org/10.1007/s10649-011-9341-2
Smith, J. L., Cech, E., Metz, A., Huntoon, M., & Moyer, C. (2014). Giving back or giving up: Native American student experiences in science and engineering. Cultural Diversity & Ethnic Minority Psychology, 20(3), 413–429. https://doi.org/10.1037/a0036945
Su, R., & Rounds, J. (2015). All STEM fields are not created equal: People and things interests explain gender disparities across STEM fields. Frontiers in Psychology, 6, 189. https://doi.org/10.3389/fpsyg.2015.00189
Tabachnick, B. G., & Fidell, L. S. (2012). Using Multivariate Statistics (p. 1024). Pearson Higher Ed.
Talanquer, V., & Pollard, J. (2010). Let’s teach how we think instead of what we know. Chemical Education Research and Practice, 11(2), 74–83. https://doi.org/10.1039/C005349J
Thoman, D. B., Brown, E. R., Mason, A. Z., Harmsen, A. G., & Smith, J. L. (2015). The role of altruistic values in motivating underrepresented minority students for biomedicine. BioScience, 65(2), 183–188. https://doi.org/10.1093/biosci/biu199
Trede, F., Macklin, R., & Bridges, D. (2012). Professional identity development: A review of the higher education literature. Studies in Higher Education, 37(3), 365–384. https://doi.org/10.1080/03075079.2010.521237
Tullis, T., & Albert, B. (2013). Measuring the user experience: Collecting, analyzing and presenting usability metrics (Second). Elsevier.
Verdin, D., & Godwin, A. (2015). First in the family: A comparison of first-generation and non-first-generation engineering college students. IEEE Frontiers in Education Conference (FIE), 2015, 1–8. https://doi.org/10.1109/FIE.2015.7344359
Walton, G. M., & Cohen, G. L. (2007). A question of belonging: Race, social fit, and achievement. Journal of Personality and Social Psychology, 92(1), 82–96. https://doi.org/10.1037/0022-3522.214.171.124
Walton, G. M., & Cohen, G. L. (2011). A brief social-belonging intervention improves academic and health outcomes of minority students. Science, 331(6023), 1447–1451. https://doi.org/10.1126/science.1198364
Walton, G. M., Logel, C., Peach, J. M., Spencer, S. J., & Zanna, M. P. (2015). Two brief interventions to mitigate a “chilly climate” transform women’s experience, relationships, and achievement in engineering. Journal of Educational Psychology, 107(2), 468–485. https://doi.org/10.1037/a0037461
Weston, T. J. (2019). Patterns of switching and relocation. In E. Seymour & A.-B. Hunter (Eds.), Talking about Leaving Revisited: Persistence, Relocation, and Loss in Undergraduate STEM Education (pp. 55–85). Springer International Publishing. https://doi.org/10.1007/978-3-030-25304-2_2
Wheeler, L. B., Clark, C. P., & Grisham, C. M. (2017). Transforming a traditional laboratory to an inquiry-based course: importance of training TAs when redesigning a curriculum. Journal of Chemical Education, 94(8), 1019–1026. https://doi.org/10.1021/acs.jchemed.6b00831
Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25(1), 68–81. https://doi.org/10.1006/ceps.1999.1015
This study was approved by the University of Florida Institutional Review Board (IRB201600944). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the US National Science Foundation. Thank you to the following colleagues for their time and expertise in helping the authors conceptualize the design challenges: Jennifer Andrew, Department of Materials Science and Engineering, University of Florida; Jean-Claude Bonzongo, Department of Environmental Engineering Sciences, University of Florida; Jennifer Bridge, Department of Civil and Coastal Engineering, University of Florida; Anuj Chauhan, Department of Chemical and Biological Engineering, Colorado School of Mines; David Cornwell, Department of Environmental Engineering Sciences, University of Florida and Cornwell Engineering Group; Katherine Deliz Quifones, Department of Environmental Engineering Sciences, University of Florida; Jonathan Scheffe, Department of Mechanical and Aerospace Engineering, University of Florida; Kyle Riding, Department of Civil and Coastal Engineering, University of Florida; Eric McLamore, Department of Agricultural and & Biological Engineering, University of Florida; Anne Meneffee, Department of Energy and Mineral Engineering, The Pennsylvania State University.
This material is based upon work supported by the US National Science Foundation under Grant No. (DUE-1625378).
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Crippen, K.J., Imperial, L., Payne, C. et al. User experience and motivation with engineering design challenges in general chemistry laboratory. Innov Educ 3, 9 (2021). https://doi.org/10.1186/s42862-021-00016-w