What Factors Impact Psychological Safety in Engineering Student Teams? A Mixed-Method Longitudinal Investigation

Engineering organizations around the globe are increasingly relying on team-based activities [1,2] based on teams’ abilities to solve complex problems more effectively than individuals alone [2,3]. This increase in performance has been attributed to the wide range of knowledge, experience, and expertise present in a team [3]. In other words, the “wisdom of the collective” [4] is often better than the output of the sum of the individual team members [5,6]. In an engineering context, this improvement is essential because companies need to produce new solutions to problems to thrive in the fast-paced market dependent on innovation [7]. Because of this demand from engineering industry, engineering education has also recognized and incorporated team processes into the engineering classroom [8,9] especially through cornerstone (first-year) and capstone (final year) undergraduate design courses [10].

Despite the increase in the integration of teamwork in engineering education, how to cultivate teamwork skills in the classroom remains unclear [11]. While meta-analysis substantiates that various factors that could affect psychological safety in different scenarios [12,13], the relative importance to engineering design education remains unclear. To have a better understanding of team performance, there should be more information on the dynamic assembly of teams, which is also lack current literature [14]. Moreover, there is also little information on what, how, and when to successfully intervene in team activities for better team performance.

To answer these questions, the study of the longitudinal trajectory of psychological safety in an engineering context can be applied to the engineering classroom. Specifically, psychological safety is, “a shared belief that the team is safe for interpersonal risk taking” [15]. When people feel psychologically safe, they are more likely to speak up about their own opinion without fear of failure, which would allow them to learn and contribute more efficiently [12]. Psychological safety as a predictor has shown its consistency and generalizability when measuring outcomes from individuals, teams, and groups [12]. Specifically, meta-analysis outside of this has substantiated the relationship between psychological safety and performance [15]. When tasks are complex, knowledge intensive, and innovative, this relationship is strongest. These are the traits that characterize the tasks in engineering design. However, there is limited research on psychological safety in an engineering context.

Despite the lack of studies of psychological safety in engineering, research in innovation management points to why this relationship may be critical to explore. Particularly, the prior work has shown that psychological safety shares a relationship with creativity through promoting individuals to feel safe to share novel ideas and give constructive feedback [16,17], skills that are critical to foster in engineering education [11]. These skills guide teams throughout the engineering design process, which relies on an iterative process of three phases: generation, evaluation, and communication [18]. Low performance throughout these phases can hurt overall performance of the final design [18], substantiating the importance of investigating how psychological safety develops over time.

In light of this prior work, the goal of the current article was to explore the role of psychological safety in student interactions at each of the conceptual phases of the engineering design process. Specifically, qualitative responses were assessed to understand what positive and negative interactions contributed to teams’ ratings of psychological safety. The results of this study provide empirical evidence for the factors that contribute to the building and/or waning of psychological safety across the trajectory of an engineering design project. Furthermore, these results can be used to develop intervention methods geared toward particular issues (e.g., communication issues) at specific points in the design process.

In this section, we review the literature on psychological safety, focusing on how it is validated as a construct, and expand upon how it may impact processes and interactions that occur throughout the engineering design process. We also review studies of psychological safety to support the need to extend empirical knowledge to the engineering design context.

Psychological safety is a team construct established through deep interactions and conversations that facilitate how team members perceive their treatment from team members [19]. In other words, teams with high psychological safety share a general belief that nobody will be punished or humiliated for sharing ideas, making mistakes, or asking questions, for example. The construct first came to focus in organizational research in the 1990s, and it has since been identified as a pivotal factor of teamwork, team learning, voice, and organizational learning [12]. The concept can be analyzed at the individual, group, and organizational level [12], making it applicable in a variety of industries such as healthcare [20] and manufacturing [14]. Particularly, meta-analysis has identified various constructs that feed into and come from psychological safety at the individual and team levels [12]. Some of these inputs are problem-solving efficacy, task conflict, social interactions, team characteristics (e.g., personality), and team leadership [12]; all of which can contribute to how psychological builds, wanes, or establishes as a team construct. From there, psychological safety can impact information sharing within a team, how teams handle conflict, and how teams learn—ultimately impacting performance [12]. Because building psychologically safe teams can be dependent on these constructs, identifying when teams’ understandings of their psychological safety become established, and how and why psychological safety changes within teams over time remains a fundamental goal for expanding its research in engineering design.

The first step in building a psychologically safe team is understanding when a shared understanding of psychological safety exists across teams. This is critical to establish, as psychological safety is a team level construct [15], where team members possess a shared belief of feeling safe for interpersonal risk-taking, meaning that team members can make decisions that affect the team without fear of repercussions if plans fail. However, this shared understanding may take time to establish. Specifically, psychological safety is a dynamic construct, meaning that it takes time to manifest through team interactions [21], which helps teams develop a shared mental model within the team [22,23]. Therefore, analysts must justify aggregating individual psychological safety ratings to the team level to validate the construct. This requires a two-step process, where the first step requires interrater reliability (IRR) to be calculated as Cronbach’s alpha [24]. This identifies when individuals’ perceptions of the what the scale is measuring matches the intended construct to be measured [25]. In some cases, lower Cronbach’s alpha values below the α = 0.70 threshold [26] can indicate poor interrelatedness, or there are heterogeneous constructs present in the scale [24]. The second step requires interrater agreement indices to be calculated, such as r_wg and intraclass correlation (ICC) values, which justify whether the scale can be aggregated to the team level [25]. Specifically, when all the individual members’ ratings are similar with other individuals on their team (here, the individual psychological safety ratings), then the team construct (team psychological safety) can be considered a shared team level construct [27].

Beyond aggregations to the team level, understanding when a shared perception of psychological safety becomes established within a team can pose a challenge. While meta-analysis showed that psychological safety is consistent, generalizable, and a multilevel predictor of various outcomes in various contexts [12], many of these studies examined psychological safety from a “snap-shot” view; failing to measure psychological safety over time. This is problematic when considering recent work in engineering design [28], as design teams must work together through different tasks at each stage of the design process over duration of time [18]. Although many studies show the importance of psychological safety in fostering team performance, when to address issues due to low psychological safety, as well as types of intervention methods to promote psychological safety, remains unclear. This calls to attention the importance of identifying whether psychological safety is stable across the different time points of a team’s lifespan.

While psychological safety has seen limited treatment in engineering design, there are multiple benefits that support its potential utility in engineering design education. One major theme of psychological safety is that it facilitates the contribution of ideas and thus stimulates the team performance [12]. Specifically, meta-analytic evidence demonstrated that the relationship between psychological safety and learning, as well as performance, is the strongest, especially when having complex, knowledge-intensive tasks that involve creativity and sense-making [13]. Higher psychological safety has the potential to increase creativity by allowing people to express their opinions in a dignified and respectful manner [29]. With creativity influence on engineering design, it is worth noticing the relationship between creativity and psychological safety. Studies also suggest that the development of psychological safety could enable creativity by encouraging participants to speak up with their novel ideas (concept generation), provide feedback to other members (concept selection), and challenge solutions throughout the process (e.g., during prototyping) [16,30]. Particularly, the generation [31] and selection [32] of ideas are important for success of the final design, as prior research suggests that creativity and the likelihood of people speaking up could play an important role during the design process. These could also have an impact on the elucidation of sound decisions during the development stages of the project, which are also related to the success of the final design [33]. However, within these stages, task conflict could occur. Task conflict has been demonstrated to have a relationship with team performance [34], showing that team members that are less likely to agree on tasks easily tend to produce creative outputs, promoting team performance [15]. Particularly, in engineering design, this conflict provides an opportunity for team members to challenge others’ ideas to improve performance [35]. However, psychological safety can mediate whether this conflict either hurts or helps teams [34]. For these reasons, investigating how psychological safety wanes and/or builds throughout the engineering design stages is critical to understanding how to leverage team performance.

In addition to the dynamics of the various stages of the engineering design process, it is also important to consider team behaviors in relation to psychological safety. As the prior work indicates, psychological safety that results from team behaviors can impact the level of safety within an environment where individuals participate [12]. This implies that psychological safety could be subject to several factors. For example, interpersonal relationships, group dynamics (also an output of psychological safety that impacts further workplace practices [13]), leadership, and organizational norms are just four of factors that have been identified as inputs of psychological safety [36]. In addition, specific team behaviors or learning behaviors could positively relate to psychological safety. For example, when team members are being respected by other members, they are more likely to gain confidence and speak up, knowing others will not hold their mistakes against them [12]. In addition, both listening to each other [37] and the frequency of communication [38] are proposed to have an influence on psychological safety. At this individual-to-individual level of psychological safety, other factors could feed into why team members may choose to treat specific members a certain way. Particularly, surface-level diversity, such as gender and ethnic background, has been shown to have differential effects on team performance and relationship conflict in teams [39]. Beyond surface-level diversity, factors grouped under deep-level diversity can also contribute to such differential effects [39]. Specifically, deep-level diversity can come in various forms, such differences are due to cognitive style and level in the form of cognitive gap [40,41], a contributor of conflict in teams [41]. These are important to keep in mind as confounding factors of psychological safety, as diversity can impact how team members treat others and feel treated.

In addition to team behaviors, there are several other factors that can impact psychological safety. In lieu of the limited research on industry-based engineering teams, we turn our focus to student teams, as these groups cannot be compared on the same grounds [42]. Importantly, student teams tend to be shorter in duration, which could weaken the effectiveness of team development [43]. To investigate these teams, we can apply Salas et al.’s Nine Critical Considerations of Teamwork (9 C’s) [44]. Importantly, not all factors apply to our student teams (see Sec. 4.3); therefore, the current discussion focuses on role of team composition, communication, coordination, cooperation, and conflict. For our study, team composition refers to the individual factors that are relevant to team performance such as knowledge, skills, and attitudes, stemming from the team characteristics (personality) construct [12]. For example, the “Big Five” personality traits [45], such as openness and neuroticism, as well as emotional stability, have been posited to be correlated with psychological safety. For example, openness can potentially increase the possibility of an individual feeling safe to take risks [46]. In addition, team communication is used to reflect factors such as giving and receiving feedback [12], being respectful to others [12,47], and listening to each other [37]. This was based on the social interaction and team characteristics constructs, which can impact information sharing [12]. Moreover, conflict, which refers to the incompatibility of team members’ interests, beliefs, or views, may also impact team psychological safety as it has been shown to have a relationship with team performance [34]. This stemmed from the task conflict construct that is mediated by psychological safety and impacts how teams handle conflict [12]. Interestingly, higher psychological safety can help teams leverage the benefits of conflict to produce creative outputs [48]. Differing from conflict, coordination focuses on the participants’ behavior patterns when performing tasks, promoting team effectiveness [44], which comes from the problem-solving efficacy and team characteristic constructs [12]. Specifically, the prior work has shown that coordination can impact the efficiency and success of designs in engineering [49,50]. Finally, stemming from the team leadership construct, cooperation allows teams to develop a problem-solving orientation that promotes leaders and members to discuss and learn from mistakes [12,51].

In addition to these Critical Considerations of Teamwork, the cohesiveness and creativity of the team may also affect their psychological safety. Cohesiveness refers to interpersonal attraction, commitment to task, and/or group pride [52]. While meta-analysis has pointed to the importance of these factors in relation to psychological safety [13], the results are mixed. For example, where one study indicated that team cohesiveness can reduce the likelihood of interpersonal risk-taking in psychological safety [53], another study linked cohesiveness in teams to feeling safe in sharing opinions [54]. These studies bring to question the impact of team cohesiveness in building psychological safety. Finally, creativity, which can emerge as a construct from psychological safety, may also impact team psychological safety [12]. Creativity is heavily emphasized in engineering design [55]; however, its impacts on the team psychological safety remain unclear.

While the empirical work shows support that the aforementioned factors may impact psychological safety, some important questions remain. Specifically, how, what, and when should we intervene in team activities to reach higher psychological safety and better team performance? To respond to the research needs, we measured the reliability and applicability of psychological safety in an engineering design project in this study. In addition, we explored the factors that could contribute to the construct of psychological safety, as it has been identified as a key research need [12,13].

The purpose of this study was threefold. First, it sought to understand the longitudinal reliability of the psychological safety construct in engineering student teams. This is critical, as reliability certifies whether a scale is measuring what it claims to measure under specific conditions, such as whether psychological safety can be measured in engineering design. Second, it sought to explore the factors that impact the building or waning of psychological safety in engineering design student teams. Third, it sought to examine which factors affected psychological safety during each of the five time points. The results of this study provide a better understanding of what kinds of intervention might lead to better team performance, how to successfully intervene, and when is a better time to intervene. This work also provides some of the first evidence of the reliability of psychological safety in an engineering context.

To answer the research questions stated as part of this study, a longitudinal study was conducted over the course of 2 years with 260 participants. See Table 1 for the detailed number of teams and participants in each semester. Data collection started during the Summer 2018 term and concluded in the Spring 2020 term. Over the condensed summer terms, the duration of the study was 4 weeks, whereas over the fall and spring terms, the study was 8 weeks instead, but the activities and timeline remained the same. Importantly, all teams spent the same amount of time together, where the summer sessions met twice as long each week, and see Fig. 1 for the study timeline. The remainder of this section describes the data that were collected as part of the current investigation.

In total, 260 participants between the ages 18–19 years formed into 68 teams and participated in the study (189 males and 71 females), and see Table 1 for summary. The participants were all enrolled as engineering majors at a large northeastern university.

The study was completed within a first-year introduction to engineering design class (EDSGN 100). A total of 11 sections of students were involved in the current investigation. While seven sections took place during a typical semester (8 weeks), four sections happened over condensed summer terms (4 weeks). The study data were collected over five time points, including (1) start of the project, (2) idea generation, (3) concept selection, (4) prototyping, and (5) final design deliverable (see Fig. 2). Specifically, at the end of each time point, a survey including seven questions developed by Edmondson [15] and two open-ended questions were completed by the participants. The first seven questions centered around the level of how comfortable team members felt when they made mistakes without criticism, provided suggestions with good intentions, as well as whether they felt accepted as a member of the group [15], see Supplementary Materials on the ASME Digital Collection for survey items. Psychological safety was measured via calculating the average of the scale items from the Team Psychological Safety survey for each individual, and then team scores were aggregated to the team level by taking an average of the team members’ individual psychological safety scores. These questions were followed by two open-ended questions, which explicitly asked students to state the positive and negative interactions that had an impact on their answers to the psychological safety survey items. All participants consented at the start of the study (time point 1) under the institutional review board guidelines set forth at the university. Importantly, time in between time points could last one class session in between four and five, depending instructors’ needs to cover other assignments besides the main project. Each main design session occurred within one class period.

During time point 1, participants were assigned to a group of thee to four people based on Kirton’s Adaption-Innovation inventory (KAI) [40]. Although not the focus of this paper, half of the teams were constructed to be homogeneous and half of the teams were heterogeneous, as based on the KAI scores. Later, teams engaged in a design challenge, which differed by the instructors of each section, see Table 1. Then, students performed research on the contextual area of the design problem to understand the problem for their design projects. At the end of the class, students completed the first psychological safety survey.

During time point 2, students were given a lecture on how to investigate customer needs and how to develop problem statements. Next, instructors introduced a module to the students on the importance of creativity. Later, students were guided through methods of idea generation and were asked to generate as many ideas as possible in 15 min. For the next 10 min, the teams were encouraged to brainstorm as a group by making modifications to the ideas generated by each team member. At the end of the session, students took the second psychological safety survey.

During time point 3, teams were led through concept selection activities where the participants were asked to assess ideas generated by their respective team members in time point 2. Specifically, participants were asked to assess ideas individually in a random order by categorizing the ideas using the individual concept assessment sheets into “Consider” or “Do Not Consider,” see Fig. 3. Ideas in the “Consider” category were concepts that the participants felt would most likely satisfy the design goals, while the “Do Not Consider” category contains ideas that the participants felt had little to no likelihood of satisfying the design goals. Afterward, the students were asked to do a final round of screening, which they decided on the final ideas as a group. At the end of the class, all of the participants completed the psychological safety survey.

During time point 4, participants were asked to embody the ideas selected in time point 3 by building low-fidelity prototypes for 20 min while using commonly available materials (e.g., foam cores, cardstock, post-it notes). From there, participants were asked to give an “elevator pitch” to their classmates from other teams introducing their prototypes. After their classmates asked some critical questions, such as those related to the technical feasibility and economic impacts, participants gathered the feedback from their classmates and made additional changes to their design. At the end of the class, all students completed the psychological safety survey.

During time point 5, teams presented their final design ideas with a PowerPoint presentation and a final rendered prototype using CAD (see Fig. 3). At the end of class, students completed the psychological safety survey.

To analyze the open-ended prompts in the survey, “Please describe any positive/negative team interactions or activities that impacted the rating,” a qualitative analysis was conducted. Specifically, prior studies showed that factors such as group dynamics [36], interpersonal relationships [36], communication [38], and personal traits [56] are known to have influence on psychological safety. Consequently, a codebook was developed through a directed content analysis approach [57], building upon prior work, see Supplementary Materials on the ASME Digital Collection. Specifically, the responses were coded using five of the Critical Considerations for Teamwork Framework (Composition, Communication, Coordination, Cooperation, Conflict) [44], and two additional Critical Considerations (Creativity and Cohesiveness [52]) that are relevant to this study and engineering design teams in general. We call this the “Seven Critical Considerations (7 C’s) of Psychological Safety in Engineering Design Teamwork” model. Although psychological safety is derived from an aggregate score for all items [15], items that map to the constructs under investigation probe respondents for how they are able to bring up problems (Conflict and Communication), feel safe to take risks, and make mistakes (Conflict and Cooperation), and difficulty in asking others for help (Cooperation, Communication, and Coordination). In addition, the scale gets at feeling rejected for being different (Cohesiveness and Composition), not feeling that efforts are undermined (Cohesiveness and Cooperation), and feeling that unique skills and talents are utilized (Creativity and Coordination). After coding the qualitative data into the themes mentioned earlier, we performed further axial-coding and re-grouped the answers into secondary themes.

This section highlights the results from exploring longitudinal study on psychological safety. During the study, the average psychological safety scores for all 68 teams from time point 1 to time point 5 were 5.59, 6.02, 6.10, 6.12, and 6.07, respectively. The interrater reliability of the qualitative data was assessed via two raters; one PhD Industrial Engineering student and one undergraduate Industrial Engineering student, coded using the same codebook at an overlap of 20% of the open-ended questions using NVivo Pro. An interrater reliability (Cohen’s Kappa) of 0.76 was reached. The remainder of this section presents the results in reference to our research questions. The statistical data were analyzed via the spss v.26.

RQ1: How Applicable and Reliable Is Psychological Safety in Engineering Student Teams Over Time?

Before conducting further analyses with the psychological safety scores, the first step is to ensure that the scale applies to the engineering design team environment. Without this validation, further analyses with survey data and open-ended responses would be subject to usability. To justify the reliability of the psychological safety scale used for RQ1, Cronbach’s alpha (α) was computed based on the data collected from five different time points throughout the course for all teams within each time point. Cronbach’s alpha measures the internal consistency of a scale by identifying how similar two responses are to randomly selected sets of items within the scale [26]. Generally, a Cronbach’s alpha value greater than 0.7 is acceptable, regardless of random error calculated using Ref. [24].

The results computed for each of the five time points demonstrate that the scale was reliable since Cronbach’s alpha values range from 0.70 to 0.82 across time points 1 through 5, as shown in Fig. 4. These values were equal to 0.70, 0.72, 0.75, 0.74, and 0.82 for each time point, respectively. Among these values, time point 1 exhibited the lowest value at α = 0.7. This may be because psychological safety in a team takes time to develop over the course of a project, because it begins at the individual level and manifests at the team level over time [23,27]. Because time point 1 represented the first meeting, this signaled that there may not have been enough time for the perception of psychological safety to manifest. After time point 1, the Cronbach’s alpha increased over time, while the survey length remained the same. Therefore, we can conclude that at time point 1 when the team construct might not have been stabilized, it could have caused difficulties in examining the psychological safety in this longitudinal study.

After assessing the internal consistency of the scale’s reliability, we also examined consistency of scores among the team members. This is because psychological safety is a shared group property that manifests at the team level [15]. It commences with individual feelings but as more interaction takes place, psychological safety turns into a collective team phenomenon [23,27]. Therefore, it is necessary to justify aggregating the scale to the team level because psychological safety describes team perceptions instead of individual [15]. When all the individual members in the team agree with each other on the overall psychological safety level, psychological safety can be considered a shared team level construct [27]. As a result, it is important to justify aggregation to the team level through interrater reliability and agreement within the teams to establish psychometric integrity in assessing team psychological safety [25].

The commonly accepted value for r_WG is 0.70 and above [58]. While the r_WG accounts for the agreement of one question, the r_WG(J) has been computed to represent a multi-item index since a total of seven questions are included in the psychological safety survey [15]. Because an r_WG(J) value has been calculated for each team, an average and median r_WG(J) values have been reported for each of the time points. The average of the r_WG(J) ranged from 0.88 to 0.91, and the median of the r_WG(J) ranged from 0.91 to 0.94. These results demonstrate that the interrater agreement of the psychological safety level within teams is acceptable. See Table 2 for mean and median r_WG(J) values for each time point.

The second commonly used measure to justify aggregation that captures both interrater agreement (consensus) and reliability (consistency) are the ICCs [22]. Particularly, ICC values are typically used to represent the effect size due to group measurement, whereas r_WG tends to be the stronger measure for suggesting aggregation [25]. Two types of intraclass correlations, ICC(1) and ICC(2), were calculated for this study (see Table 2). ICC(1) uses team membership to estimate the total proportion of variability [58], which can be explained by the extent to which one team member could represent every member who took the survey in the group. A larger ICC(1) value indicates a larger degree of interchangeability between the team members—or the extent to which one team member’s rating may represent all ratings within the group [58]. In the study, ICC(1)s ranged from 0.08 to 0.26 throughout five different time points. Results from time point 1 showed that the psychological safety variance due to team membership was greatest at 26%. While surprising, as we would expect lower ICC(1) values earlier on due to limited team interactions, the low α may play a role in this distinction from other time points. As time progressed, results from time point 2 and 3 revealed decreases in ICC(1) values at 18% and 13%, respectively. Then, ICC(1) was lowest at time point 4 (8%), suggesting relatively little variability in psychological safety due to team membership, pointing to lack of a shared model of psychological safety. This could be because time point 4 represents a later point in the semester, where team members were likely to be working on different tasks related to the main project, such as building prototypes or working on individual assignments (such as CAD homework) [44]. When team members were not expecting to work together on tasks and engaged in “divide and conquer” tactics, the team developed less of a shared mental model over time, which resulted in a small percentage of variability in psychological safety due to team membership [44]. The ICC(1) then recovered at time point 5 with an 11% variability in psychological safety scores due to team membership. This increased value exhibited that when the team worked together on the same tasks for the final project, the shared mental model [44] promoted the increase in amount of variance explained by team membership.

While ICC(1) captures the interrater agreement, ICC(2) captures the reliability of the mean ratings [25]. In this case, ICC(2) takes into account how many people there are in a group to denote how reliable the team means would be based on the consensus and consistency of member ratings [58]. Estimation of ICC(2) on the reliability of mean ratings are generally higher in magnitude than ICC(1) estimates because it is adjusted based on the group size [22]. Similar to the pattern of ICC(1) results, ICC(2) showed the highest value at time point 1, where ICC(2) = 0.57. After time point 1, the ICC(2) decreased at time point 2–4 with values of 0.43, 0.35, and 0.24, respectively. Similar to ICC(1), the ICC(2) also recovered a little at time point 5, where ICC(2) = 0.31. Overall, the ICC(1) and ICC(2) values resulted in a decreasing pattern from time point 1–4, which dropped the most at time point 4, where students worked on different tasks, weakening the shared mental model. However, the ICC values slightly recovered at time point 5, demonstrating that the students were back to working with each other more so than in time point 4.

RQ2: What Factors Impact the Building or Waning of Psychological Safety in an Engineering Design Student Team?

To understand the factors that impact the building or waning of psychological safety in student engineering teams, qualitative responses from the open-ended survey questions were analyzed in an exploratory study. Specifically, this research question sought to discover what factors were correlated with higher or lower psychological safety across all teams using our Seven Critical Considerations (7 Cs) of Engineering Design Teamwork. The open-ended questions from the survey were analyzed to identify major themes that may have impacted psychological safety. While 2742 responses were received from participants, 650 were discarded due to a lack of detailed response by the participants (e.g., “Not Applicable”, “Nothing to report”), and the 62 “Other” responses that were irrelevant to the questions asked were discarded as well (i.e., “second guessed a lot”) totaling 2030 open-ended responses. Importantly, these responses came from individuals across all teams, and therefore, they represent individual perceptions of psychological safety.

In all, 7 main discussion topics and 18 subtopics were defined after the investigation of the data. As detailed in Fig. 5, “Communication” was referred to the most for positive team interactions, while “Coordination” contained the highest frequency count for negative team interactions. The following section presents detailed descriptions and examples of the discussion topics that may have impacted the formation or decline of psychological safety.

The discussion topic that was the most frequently cited interaction or activity that impacted participants’ ratings of psychological safety during the five time points was the communication (f = 730) within the team. Three subtopics in this area were also identified, including being respectful of other’s ideas (f = 136), listening (f = 63), and general communication (f = 531). Specifically, under the general communication subtopic, where members provided and received information which led to the attitudinal, behavioral, and cognitional change of a team, teams indicated whether they were able to communicate well throughout the process. There were usually no specific details to explain the communication as presented in the survey responses, but rather it was stated in a general way under this subtopic. For example, a participant in Team 32 expressed that, “We communicated very well without hurting each other’s feelings when [we] chose not to consider an idea.” Along with general communication, being respectful to other’s ideas is a topic that participants frequently mentioned. Particularly during concept screening, this respectfulness was mentioned frequently. For example, a participant from Team 38 commented that, “Everything we do is respectful and considerate of the feelings and ideas of others. Supportive language is used almost always.” Participants also mentioned listening as another subtopic. Under listening, team members paid attention and did not interrupt others. A negative example of this came from Team 40 in the quote, “Sometimes I had to repeat myself a few times to be listened to.” Communication was the most-often occurring positive interaction in our sample (f = 544).

The second most frequently discussed topic was coordination or team behavioral mechanism that can cause team recourses to result [44] (f = 617). Discussion about coordination has been categorized into six subtopics, including efficiency (f = 158), work contribution and participation (f = 185), task completion (f = 155), time management (f = 57), absence and lateness (f = 32), and not explicitly stated (f = 30). Specifically, under coordination, one of the most frequent topics was when participants discussed they had accomplished tasks in an efficient manner. For example, a participant in Team 30 indicated that “We did work very efficiently today…” Another topic that participants brought up many times was whether group members actively participated in the activities or whether everybody contributed the same effort without having an idle team member not doing anything. For example, a participant in Team 26 commented, “Not evenly split up the jobs.” Meanwhile, a participant in Team 31 indicated that “Everyone contributes and everyone puts in their work.” In addition, students commented about how their teams accomplished their tasks in a timely manner. For example, a participant from Team 13 stated that, “We waited last minute for everything.” These results indicate that task performance was positively related to psychological safety [13] in an engineering context. Also, our qualitative results agree with the notion that problem-solving efficacy also correlates with psychological safety in a group-level research [12]. More specifically, the results suggest that coordination including team efficiency, members’ contribution and participation, task accomplishment, time management, and the absence and punctuality of team members were likely to be linked to the formation or decline of psychological safety.

The third topic participants discussed the most involved cooperation. During team activities, cooperation is a motivation driver for a team, which the team builds upon with attitude and beliefs [44]. Cooperation had been discussed in a high frequency (f = 350). More specifically, there was a higher frequency that a participant mentions cooperation when asked about positive interaction (f = 328) than negative interaction (f = 22). For example, a participant in Team 27 mentioned that, “We [are] all able to collaborate our ideas to make a great product.” Our results show that collaborating well, working together, and helping each other could impact the formation of psychological safety. These results also demonstrate the existence of a relationship between cooperation and psychological safety [51].

The fourth most frequent topic involved team composition (f = 136). Discussions about the composition included participants commenting on the characteristics of their team members and the individual factors that are related to the outcome of the team performance [44]. These were broken down into two categories: personal traits (f = 94) and not explicitly stated (f = 42). Usually, participants discussed two of the personality traits out of the “Big Five” [45], which are openness (f = 59) and extraversion (f = 35). During the team activities, especially for time points 2 and 3, these activities required participants to share their own and evaluate others’ ideas. For example, a participant from Team 49 discussed that, “Everybody is open-minded to new ideas.” These results suggest that having open and extroverted team members (f = 85) could promote the building of psychological safety. In comparison to the frequency of positive composition interactions (f = 110), negative interactions were relatively small (f = 26).

The fifth most frequent topic that participants discussed was conflict. Conflict describes the incompatibility of ideas and perspectives regarding the tasks and assignments [44]. Throughout the data, teams mentioned reaching consensus in making decisions when providing answers for positive interactions (f = 55). They also mentioned that conflict occurred when team members possessed different views (f = 33) could also impact their ratings that decreased their psychological safety scores. A positive response from participant in Team 37 was stated, “We could all agree on what ideas were good and what ideas were bad.” In contrast with other topics that had relatively huge differences between the frequency of positive and negative interactions, frequency counts for “conflict” for both cases were relatively close. The results indicate that there may be a connection between having team conflict and the climate of psychological safety, and conflict could act like a double-edged sword when it comes to psychological safety. This is similar with the relevant research, which stated that conflict is an important factor when it comes to improving team performance [34], as psychological safety is also positively related to team performance [15].

The sixth most frequent topic involved discussions on the creativity of the ideas. Starting at time point 2 (Idea Generation), students were told by the instructors not to hold back from generating any ideas that came to mind to inspire more creativity during the process. Some participants mentioned team members generating creative ideas as positive interactions (f = 62) for building psychological safety. For example, a participant from Team 28 commented, “We came up with many unique and good ideas.” While infrequent, participants complained about lack of creative ideas as a negative interaction (f = 10).

The final, least frequently discussed topic was the cohesiveness in the group (f = 38); interpersonal attraction, commitment to task, and/or group pride [52] when team members worked toward a common goal. Specifically, there were some participants who talked about having good or poor connections with their teammates when discussing positive and negative interactions. For instance, a participant from Team 26 commented, “We are able to laugh and get closer to each other.” Prior research demonstrates that group cohesiveness is distinct from psychological safety, as team cohesiveness could reduce the likelihood of interpersonal risk-taking [53], as team psychological safety focuses on whether individuals consider themselves safe for risk-taking [15].

RQ3: What Factors Impact the Building or Waning of Psychological Safety in an Engineering Design Student Team at Each of the Five Time Points?

In addition to reviewing the prevalence of these factors throughout the project, we also wanted to explore how these factors impacted psychological safety at each time point. To achieve this, 14 linear regressions were applied to understand the effect of the number of positive and negative comments for each of the 7 Cs at each of the five time points, see Table 3 for summary. This allowed us to investigate what factors contribute to lower or higher psychological safety during an engineering design project. Before conducting this analysis, assumptions were checked. Specifically, scatterplots were used to verify the linearity of the variables. In addition, homoscedasticity and normality of the residuals were verified, and no outliers were found. To account for the potential inflation of type I error, a Bonferroni correction was applied [59]. Specifically, the typical p = 0.05 value was divided by 14 (number of linear regressions), giving an adjusted p-value of p = 0.00357 to define statistical significance. Importantly, team level aggregations of psychological safety were used, with the degrees-of-freedom reported in the following paragraph. Importantly, Fig. 6 shows the breakdown of these topics across all time points for all individuals; however, the frequencies were normalized within each team and related back to team psychological safety scores for the analysis.

The results of the regression analyses at time point 1 (start of the project) revealed one significant prediction variables. Specifically, the normalized negative conflict comments during the start of the project statistically significantly predicted team psychological safety at time point 1, F(1,53) = 15.020, p < 0.0005, accounting for 22.1% of the variation in team psychological safety with adjusted $R2=20.6%$⁠, a medium effect size. The results of the regression analyses at time point 2 (concept generation) revealed two significant prediction variables. Specifically, the normalized positive communication comments a team documented statistically significantly predicted team psychological safety, F(1,54) = 12.143, p = 0.001, accounting for 18.4% of the variation in team psychological safety with adjusted $R2=16.8%$_, a medium effect size. In addition, the normalized cohesiveness negative comments statistically significantly predicted team psychological safety, F(1,54) 9.924, p = 0.003, accounting for 15.5% of the variation in team psychological safety with adjusted $R2=14.0%$_, a medium effect size. Finally, the results of the regression analyses at time point 3 (concept screening), time point 4 (prototyping), and time point 5 (final design deliverables) failed to reveal any significant prediction variables.

These results are important because they demonstrate what factors could affect team psychological safety during each of the five time points, such as negative communication and conflict comments being linked to lower psychological safety during prototyping. The number of negative communication comments was significant during time point 1 (start of project), with positive communication comments significant during time point 2 (concept generation). In addition, negative cohesiveness comments at time point 2 were also significant. Based on these results, communication had the largest impact on psychological safety within this study, particularly at the earlier stages. This demonstrates the importance of promoting communication early in a team.

The main objective of this study was to understand how applicable and reliable the measure of psychological safety was in engineering student teams over time. In addition, we sought to identify potential factors that impact the building and waning of psychological safety in these teams over time. The main findings from this study were as follows:

• The psychological safety scale was reliable for all time points, where Cronbach’s alpha values ranged from 0.70 to 0.82, with the lowest at time point 1.

• Psychological safety scores among teams were consistent with ICC(1) and ICC(2) being highest at time point 1 at 0.26 and 0.57, respectively.

• In total, communication contained the highest number of negative comments, whereas coordination contained the highest number of positive comments.

• Negative communication was significantly negatively related to psychological safety at time point 1.

• At time point 2, positive communication was significantly positively related to psychological safety, and negative cohesiveness was significantly negatively related to psychological safety.

Cronbach’s alpha values met the threshold for reliability at all time points and the intraclass correlations, ICC(1) and ICC(2), were highest at time point 1 at 0.26 and 0.57, respectively. This indicates that the amount of variance explained by team membership (ICC(1)) [25], and thus the extent to which mean ratings can be distinguished reliably among groups (ICC(2)) [25], was highest at the first time point. Following time point 1, ICC(1) and ICC(2) gradually decreased over time until time point 4 (prototyping) and then recovered at time point 5 (final design deliverables). This demonstrates that the shared team model of psychological safety weakened over time, but recovered slightly at the last time point. This may be related to teams making an effort to “get on the same page” as time point 5 mainly focuses on the processes of turning in the remaining the final project deliverables. While we may not expect such a decrease at time point 4 due to dominant personalities potentially taking over and teams coming together before sharing low-fidelity prototypes, all prototyping activities were condensed as one time point, so we lack the granularity of analysis to make any statement on its effects. Conversely, coordination work had the highest number of positive and negative comments at time point 5, so coordination may contribute increase in the shared team model. This aligns with the prior work, which showed that team members working on different assignments separately can weaken the shared mental model of a team [44], decreasing interrater agreement and reliability. Findings also indicate that sometimes the survey may not be reliable throughout the design process, limiting to what extent psychological safety exists as a team construct.

In addition to the IRR and IRA analyses, the qualitative analyses helped to identify some interactions occurring within the teams. Specifically, positive coordination and negative communication comments were the highest overall, emphasizing the importance of coordination for being successful in an engineering design project, and how poor communication can hurt a team’s chances of success. As problem-solving and evaluation activities rely on good communication, whether that be through words, diagrams, or numbers [18]. Conversely, coordination is critical to producing design outputs in engineering both efficiently and successfully [49,50]. Our findings are further substantiated by the statistical analyses of the normalized comments per team, where normalized positive communication comments were correlated with an increase in psychological safety at time point 2 (concept generation). This aligns with prior work, as communication during the generation stage is necessary for being able to share new ideas or alternatives to tackle a design problem [18]. However, normalized negative communication comments at time point 1 (start of project) were correlated with lower psychological safety, implying that lack of discussion within teams made it difficult for team members to feel psychologically safe with one another. This can be detrimental to both performance and developing a strong, shared model of psychological safety, as good communication is necessary discussing problems effectively [12,15]. Furthermore, it helps to mitigate the negative impacts of diversity that are inconducive to a safe team atmosphere [12]. Therefore, future work should look to developing ways of fostering more communication and discussions within teams at the beginning of a team project. Finally, the third analysis showed that the normalized number of negative cohesiveness comments was correlated with lower psychological safety at time point 2. While contradictory to some prior work [53], our findings suggest that lack of interpersonal attraction, lack of task commitment, and not doing things as a team can lower team psychological safety. Thus, these findings compliment the prior work that demonstrated the importance of cohesiveness in overcoming the fear of risk in sharing individual opinions [54].

While this study was exploratory in nature, the results help us begin to answer some difficult fundamental questions on supporting team performance in engineering education, such as when do teams need interventions to support team performance, what types of interventions might be successful, and how can we successfully intervene in these scenarios? These are complex questions, however, that the current study only begins to shed light on. Other factors contributing to team performance, such as team expertise, personality, and interaction behaviors also need to be considered.

Although the current study presents some of the first evidence on the reliability of psychological safety in engineering teams and is among the first to examine the longitudinal trajectory of psychological safety, there are several limitations that suggest directions for future work. First, while actual psychological safety scores were applied to the analyses for RQ2 and RQ3, RQ1 just focuses on the validation process due to the varying trajectories of teams’ scores, limiting our ability to make any generalizations on psychological safety score trends over time. In fact, preliminary graphs showed lack of any trend in the building/waning of psychological safety among teams. Importantly, while the ICC values seem low on the surface, these values focus on the effect size due to team membership [25], meaning that less team interaction due to dividing tasks is the most probable cause for low ICC values. Furthermore, the study does not include important drivers of the building or waning of psychological safety—such as the composition of the team in the form of cognitive style (deep-level diversity) [40,41], and gender and racial background (surface-level diversity) [39], for example. In addition, the study was conducted in a cornerstone engineering design class. Thus, the results may only be applicable to this population. Finally, qualitative studies tend to contain selection biases based on the sample [60], but we feel that impacts on the data would be minimal, as the survey items did not explicitly contain words from the 7 Cs, such as communication, in the questions. However, we acknowledge that the word “interaction” in the open-ended question may be biasing, eliciting more responses about communication, for example. Further biasing could come from teaching styles, although the focus is on team interactions; therefore, we would expect minimal influence from this. Importantly, these findings are limited to engineering design student teams, as industry teams tend to differ in terms of motives [42], consequently affecting perceptions of positive and negative interactions. Finally, students tend to work outside the classroom on tasks; however, how teams divided tasks was unreported and could impact psychological safety. Future work will be geared toward exploring the trajectory of psychological safety on a larger scale, including a larger range of design problems and working preferences.

Along with expanding the study to a larger scale, it is also important to consider how psychological safety impacts team design outputs to justify investigation. Particularly, while the prior work has identified the importance of psychological safety in enabling creativity and providing critical feedback [16,30], the current study did not study the impact of psychological safety on design process outputs. Therefore, future work will also examine the role of psychological safety on engineering design team performance throughout the course of a design project. Taken together, these new investigations will lead us even closer to understanding how to support students within team design activities through intervention methods that promote performance.

The aim of this study was to understand the longitudinal reliability of the psychological safety construct in engineering student teams. In addition, we tried to seek insights on what factors could impact the building or waning of the psychological safety in the 68 teams. The results of this study leave several implications for engineering classroom instruction and engineering design research. First, our results show that psychological safety can be reliably measured throughout the trajectory of an engineering design project; however, it is more likely to be unreliable toward the beginning. Particularly, the shared team model of psychological safety had a tendency to decrease over time, which implies that individual perceptions of psychological safety tend to differ more within the teams. Based on these findings, we suggest implementing methods to encourage strong communication within teams to develop an understanding of and promote a shared model of psychological safety. Second, our results show that communication and coordination play an important role in the perceived success and failure of teams’ abilities in working. This complements prior work that emphasizes the importance of communication [18] and coordination [49,50] in engineering design. Finally, statistical analyses showed the impact of communication on psychological safety at the start of the project and concept generation, as well as the impact of cohesiveness on psychological safety during concept generation. These results emphasize the need to ensure that good communication can be maintained to solve problems effectively [12] and demonstrate the importance of cohesiveness in feeling psychologically safe to share individual opinions, aligning with prior findings [54]. As such, these findings provide a framework for future intervention methods in promoting psychological safety in engineering design teams over a design project trajectory.

Special thanks are given to Zibing (Janice) Gong in helping with the analysis of the reliability of the psychological safety scale and Katie Heininger in collecting data for Summer 2018. This study is based on the work supported by the National Science Foundation (Grant No. 1825830).

There are no conflicts of interest.

The datasets generated and supporting the findings of this article are obtainable from the corresponding author upon reasonable request.