Assessments of the extent to which the ERC Program is succeeding in preparing graduates who are effective in industry were obtained from two independent sources: (1) the industrial representatives to ERCs whose firms had hired ERC students or graduates; and (2) the supervisors of engineers who had had ERC experience in graduate school. Both groups compared specific skills, capabilities, and knowledge of ERC graduates with those of peers. The views of industry representatives whose firms had ERC students as temporary employees (21.8% of the firms) and graduates as full-time employees (27.3% of the firms) were obtained as part of a second study of ERC-industry interactions. Supervisors' assessments were obtained in a separate survey process under this Effectiveness study that included surveying of the graduates as well. Four hundred thirty-three ERC graduates working in industry, academia, the Federal government, and other sectors responded to a written survey. Table 4-1 provides demographic data on graduate survey respondents.

At the time that these surveys were conducted, ERC graduates had been working for the same organization for an average of about four years and had had the same supervisor for an average of two to three years. While the primary interest was in the effectiveness of ERC graduates in industry, individuals employed in other sectors and their supervisors were surveyed as well. Where findings relating to individuals working outside industry differ from those for individuals employed in industry, those findings are noted.

Table 4-1: Demographics of Graduate Survey Respondents

	Male	Female	African Amer.	Asian Amer.	Hispan.	Native Amer.	Pacific Islander	White
Masters Graduates	88%	12%	1%	23%	1%	0%	1%	72%
Doctoral Graduates	90%	10%	0%	30%	2%	0%	0%	66%

Overall Assessment of ERC Graduates

As Table 4-2 shows, supervisors of ERC graduates and corporate representatives of firms that had hired ERC graduates or students judged their ERC-trained employees to be better than peers on six key performance dimensions; between a third and a half of the same respondents characterized their ERC-trained employees as "much better" than their peers.

Table 4-2: Supervisors' and Representatives' Rating of ERC Graduates As Superior to Peers

Dimension	Supervisors	Representatives
Overall preparedness	89.4	80.2
Contribution to technical work	84.8	77.3
Depth of technical understanding	85.0	80.2
Ability to work in interdisciplinary teams	80.3	64.3
Breadth of technical understanding	80.7	74.3
Ability to apply knowledge and use technology	69.9	72.3

SCALE: 1 = "much worse"; 2 = "somewhat worse"; 3 = "about the same"; 4 = "somewhat better"; 5 = "much better"

NOTE: Values shown in table are percent responding at level 4 or 5.

On all other performance dimensions--familiarity with engineering systems approach; ability to use knowledge to develop technology; the amount of training the employees needed when hired; and ability to solve problems within time, money, and human restraints--half to two-thirds of supervisors and corporate representatives rated their employees with ERC experience as superior to their peers (4 or higher). There is little overlap between supervisors and industrial representatives, as only 17.5% of the latter had supervised any ERC students or graduates.

Corporate representatives whose firms received significant benefit from hiring ERC students and graduates described the results of such hires in a variety of ways, including:

"We have had a significant technology transfer through a recent graduate from the ERC. This has given us new and better approaches to R&D (product development) and a new focus on claim substantiations."

"Since the formation of the ERC, we have hired four of their graduates. They are top contributors to our R&D and were well prepared to work and contribute in industry. This is the primary benefit of the ERC."

"The most significant benefit has been access to students for cooperative education programs and full-time employees. ERC students have excellent applied skills in our company's competencies as a result of the academic and research environment of the ERC. The ERC is the best source of technical talent we have experienced."

"As a small company, we have had relationships with graduate students that have been powerful for them and us. Students' projects at our company have educated both the students and our employees. We have been able to develop practice-oriented engineers."

Graduates' Employment Characteristics

Career Paths

A number of differences exist across employment sector with regard to graduates' career paths up to the time of the survey. Nearly all of those working in industry and most of those in academia at the time of the survey--92% and 86%, respectively--began work following their final degree in that sector. In contrast, less that three-quarters of those employed in federal laboratories or other sectors began work there. Of those moving into these non-industry, non-academic sectors, nearly two-thirds had previously been in industry and the rest came from academia.

Overall, ERC graduates had developed stable but varied career paths with the same organization. Graduates employed in industry and academia in 1995 had worked on average around four years with their present employer, while those in federal laboratories and other sectors had been with the same employer for an average of five years. The extent to which they identified the field in which they were working as the same as that of their final degree varied. Not surprisingly, 86% of academics were working in their field of training. However, non-academic graduates were more likely to have shifted to another field, as 75% of those in industry and 70% of those in federal laboratories or other work settings no longer considered the field of their work to be the same as that of their highest degree.

Job Activities

ERC graduates working in industry¹ reported responsibilities in a wide range of activities, with the emphases being process and product engineering, manufacturing, fundamental or basic research, and technical management. Marketing/sales and general management were not prominent among their responsibilities. In their current jobs, they were most likely to use knowledge or techniques learned through their ERC work and to make use of specific ERC research results. They are unlikely, however, to continue working on things that they had been doing at the center. This is not surprising, given the variety of their current responsibilities and the likelihood that their jobs fit in with a broader corporate strategy or agenda that has little relation to ERC activity. In addition, since the graduates had worked an average of four years with their existing industry employer at the time they were surveyed, most had moved on in their careers and were not new degree recipients.

Graduates working in industry engaged in a variety of activities with and without academic involvement. Approximately half of these graduates reported having both applied for patents and had articles accepted for publication, and over 70% also made presentations at professional meetings that did not include academic participation. On the other hand, more than two-fifths also indicated that they were involved in collaborative work with university researchers and over half have obtained technical advice from academic faculty or research staff; approximately one-third of those involved in such activities were collaborating with ERCs. Finally, approximately a quarter of the ERC graduates had made presentations based on collaborative work with university researchers or were coauthors of publications with academic researchers. The majority of these activities were performed with individuals and groups not associated with an ERC.

In certain respects, graduates working in federal laboratories and those in industry were similar. For example, those in federal laboratories were even more likely than their colleagues in industry to collaborate with university researchers and to make presentations at professional meetings about work not related to collaborations with universities; but those in industry generally had a higher level of involvement with ERCs than did those in federal laboratories. Additionally, two-thirds of federal laboratory employees established or participated in cross-disciplinary research and nearly half carried out engineering systems-based research.

Those employed in academia identified their primary job activities as conducting their own research and supervising research performed by students and, where applicable, postdoctoral fellows. Less important duties included administration not related to research, industrial liaison, and outside consultation. Faculty had, on average, a medium level of responsibility for teaching; but individuals tended to have either a high or relatively low level of teaching responsibilities.

As with those working in federal laboratories, those in academia participated in activities involving industry and the general approach to research espoused by the ERC Program. Over half pursued collaborative research with industry, served as a consultant to industry, and received research support from industry. Most indicated that these activities were encouraged at their institution. Additionally, nearly 80% had established or participated in cross-disciplinary research groups and about 60% had worked on engineering systems-based projects. A majority also indicated that their research was focused to a substantial extent or entirely on problems directly relevant to industry.

Graduates' Most Valued Job Skills and Capabilities

As Figure 4-1 shows, the sector of a graduate's employment had a considerable influence on the skills and capabilities that were deemed most useful in his or her present job.

In the case of each skill or capability, however, there were individuals for whom it was very important. These results reflect variation in both the numbers of those for whom each was valuable and the extent of the value to each. (Similar patterns emerged in Section 3, "ERC-Industry Interaction," in discussion of the range and extent of benefit each firm received from interaction with an ERC.)

Graduates' Self-Assessment of Job Performance

Using the list of 16 job capabilities and skills with which supervisors rated their employee(s) and that graduates used to identify the items that are valuable in their present job (see below), the ERC alumni also assessed themselves relative to peers in their own organization.

Using a scale in which 1 equalled "much worse than average," 3 represented "about the same," and 5 was "much better than average," graduates ranked themselves highest (mean values of 4.20-4.34) in general capabilities. These include:

• ability to grasp quickly the key features of new problems;
• contribution to the firm's technical work;
• breadth of technical understanding; and
• ability to define the steps needed to solve new problems.

The group of capabilities immediately below this (mean values of 4.02-4.18) included:

• ability to communicate ideas in writing;
• creativity and innovativeness;
• depth of technical understanding;
• integrating and synthesizing information from different fields;
• ability to work in interdisciplinary teams; and
• ability to communicate ideas verbally.

The items in the remaining group were less positively rated (mean values of 3.54-3.90):

• understanding the relationship between work and customer needs;
• meeting business goals while satisfying technical requirements;
• leadership ability;
• solving problems within the constraints of time, money, and human resources;
• ability to transfer outside technology into the firm; and
• networking within the company.

Graduates working in academia rated themselves slightly lower on these skills and capabilities than did those employed in either industry or federal laboratories and other sectors. The general pattern of strengths and weaknesses across all groups, however, was essentially the same.

Overall, ERC alumni gave themselves a mean ranking of 4.32 in job performance compared to their peers. Supervisors assessed the overall performance of their ERC-trained employees slightly higher, at 4.35. In 12 of the individual skills and capabilities as well, the graduates ranked themselves slightly lower than their supervisors did. The biggest differentials occurred with three items on which the graduates rated themselves lowest: networking, transferring outside technology into the firm, and understanding the relationship between work and customer needs. Similarly, ability to work in interdisciplinary teams was rated significantly higher by supervisors than by graduates: 4.27 compared with 4.11.

Average ratings by ERC graduates working in industry were closest to those of their supervisors. In contrast, ratings by supervisors of those graduates working in federal laboratories and especially in academia were more favorable than those of the graduates themselves.

Graduates in industry and their supervisors had slightly differing views of the amount of post-employment training the graduates needed to do their job. Using a five-point scale in which 1 meant "much less training needed than average" to become a net contributor to the company's work and 5 meant "much more training needed" than average, the mean rating by the graduates in industry was 1.96. Their supervisors provided a slightly higher mean ranking of 2.09; although they still placed their ERC-trained employees in the category of needing less training than their peers, supervisors did not see the difference as being quite as large as did the graduates themselves.

ERC Experience and Job Performance

Impact of ERC and non-ERC Graduate Experience on Job Performance

One of the underlying assumptions of the ERC Program has been that significant involvement in ERC activities during graduate study builds upon, or is complementary to, traditional graduate education. To examine the extent to which this has been the case, graduates rated separately the impact of their ERC experiences and non-ERC graduate experiences on their performance in each of the 16 capability or skill areas listed in the preceding section.

Using a 1-to-5 scale in which 1 was defined as "very negative" impact and 5 meant "very positive," graduates described the impact of their ERC experiences as ranging from neutral to somewhat positive, while their ratings for other graduate training experiences were slightly negative to positive. The impact of non-ERC experiences on those working in industry was most noticeable in six capabilities and skills: contribution to the company's technical work; breadth of technical understanding; depth of technical understanding; ability to grasp quickly key features of new problems; ability to define the steps needed to solve new problems; and ability to apply knowledge from different disciplines in an integrated fashion to solve problems. On average, the impact of non-ERC experiences on the graduates' job performance in these six areas was rated by those in industry approximately 0.25 higher than was the impact of ERC experiences. Performance in two other skills--creativity and innovation plus being able to communicate ideas in writing--also was slightly more affected by non-ERC graduate training.

Conversely, ERC experiences had a greater impact on performance of the remaining half (8) of the skills and capabilities, all of which were consistent with ERC Program intentions. The four with the greatest difference between ERC and non-ERC experiences were: ability to work in interdisciplinary teams; ability to develop solutions that meet business goals as well as satisfying technical requirements; understanding the relationship between their work and the needs of the company's customers; and skill at networking in the company. The differences between ERC and non-ERC impact were smaller in the remaining four items: ability to communicate ideas verbally; ability to identify and transfer useful technology from outside sources such as universities or national labs; leadership ability; and ability to solve problems within constraints of time, money and human resources. Looking across graduates in all employment sectors, the impact of ERC experiences on these items was strongest among those employed in industry.

Graduates also rated the overall impact of both types of experiences on their job performance. Non-ERC experiences were rated to have had a higher overall impact on performance, compared with the impact of ERC experiences. Finally, looking at the ERC and non-ERC experiences across each of the 16 skills and abilities plus overall rating, the grand mean for non-ERC training was fractionally above that for ERC experiences. Given the greater amount of time students spent in non-ERC experiences while in graduate school, these difference are not surprising. Looking at the relative impact of ERC and non-ERC experiences on graduates across employment sectors, the ERC impact was greatest among those working in academia.

Graduate School Activities

Examination of what ERC graduates did while in graduate school sheds additional light on how ERC experiences add value to the rest of their graduate education. Masters and doctoral degree recipients in this study did substantially different things while in graduate school, and with notably different frequencies (see Appendix B). Of the 23 activities in which they might have engaged, only three were identified by more than half of the masters degree holders as things they had done: worked on research projects with ERC faculty, attended one or more professional meetings, and took non-ERC cross-disciplinary or interdepartmental course. Note that there is a roughly equal balance between ERC-related and non-ERC activities in which 25%-49% of the masters graduates engaged. Doctoral graduates were split relatively evenly between ERC and non-ERC activities throughout the full range of activities.

Note that the second most frequent activity for both masters and doctoral graduates was attending professional society meetings, and that in many cases respondents from both groups also presented papers at such meetings. While the graduates were not asked how the trips were financed, it is safe to assume that at least some of the graduates received funds from their ERC to attend such meetings. This is particularly likely if the individuals were making presentations about their ERC research projects.

Looking across all of their graduate activities, ERC graduates engaged in a surprising amount of ERC-like activities outside of ERCs as well. Table 4-3 provides some comparisons for both masters and doctoral graduates in terms of rates of participation in ERC and non-ERC activities.

Table 4-3: ERC and non-ERC Graduate School Activities of ERC Graduates

	Masters Graduates		Doctoral Graduates
Activity	ERC	non-ERC	ERC	non-ERC
Collaborated with corporate researchers	24%	15%	37%	30%
Made presentations to industrial audiences	23	24	50	55
Published papers with industrial coauthors	5	5	21	15
Industrial internships--20% of masters	25	75	--	--
Industrial internships--18% of doctoral	--	--	48	52
Cross-disciplinary/cross-dept. courses	28	50	44	63
Presented talks at seminar series	18	25	45	60

The following section will examine the extent to which there are in fact differences in the impact of these seemingly similar ERC and non-ERC activities on graduates' subsequent job performance.

Statistical Relationships Between Graduate School Experiences and Job Performance

None of the results presented thus far dealing with the impact of ERC and non-ERC activities on graduates' job performance has been statistically significant. Thus, the possibility that they might be due to chance cannot be ruled out. However, more complex analyses (stepwise regressions) have identified statistically significant relationships between graduate school experiences and job performance.

Looking first at ERC-trained masters degree recipients who went to work in industry immediately upon receipt of the degree, those who took at least one ERC-developed or sponsored course while pursuing that degree rated their communications and leadership abilities somewhat higher than did those who did not take such courses. Among those who received a PhD and took a position in industry, there were several relationships between activities engaged in while pursuing that degree and subsequent job performance. Looking at ERC-linked activities, the following relationships emerged:

• Those involved with ERC prototyping projects assessed their performance in a number of technical areas to be superior to that of peers. Specifically, 60% of those previously involved in such projects identified their job performance as "much better than average" in their contribution to their company's technical work. They also rated themselves comparatively highly on their overall ability to carry out their job responsibilities and their breadth of technical understanding.

• Doctoral recipients who took ERC courses rated their ability to work in interdisciplinary teams significantly higher than did those who took no such courses. Over 40% of all doctoral recipients who took courses developed or sponsored by an ERC rated themselves "much better than average" in interdisciplinary team work. Conversely, only 28% of PhD graduates who did not take any ERC courses rated their performance as highly on this dimension.

• PhD recipients who had coauthored papers with corporate sponsors of the ERC while in their doctoral program rated themselves relatively higher than did those who did not engage in such coauthorship in their ability to apply knowledge from different disciplines and their ability to transfer useful technology from outside sources.

Several relationships between non-ERC activities and job performance also emerged. Specifically:

• Those who gave talks in non-ERC seminar series and published papers with non-ERC corporate authors while in doctoral programs rated themselves significantly higher in two areas--their ability to communicate ideas both verbally and in writing and their leadership abilities--than did doctoral graduates who did not engage in these activities.

• PhD recipients who presented talks at non-ERC seminar series also rated themselves relatively higher than did those who gave no such talks on (1) a variety of technical capabilities and skills, (2) their ability to apply knowledge from different disciplines, and (3) their ability to identify and transfer useful technology from outside sources.

ERC Aspects with Greatest Career Impact

First, the ERC graduates who participated in this study looked back over their various experiences with an ERC and listed the aspects of their ERC involvement that had the greatest impact on their careers. Table 4-4 contains these aspects in order of frequency of identification:

Table 4-4: ERC Aspects with Most Impact on ERC Graduates' Careers

Internship in industry, "real-world" experience	31%
Specific courses, faculty, advisors	23
Multidisciplinarity; exposure to different disciplines	21
Good facilities and equipment	19
Teamwork	16
Research focus	14
Contacts, networking	12
Presentation, oral, or writing skills	7
Mention of a specific substantive area	6

¹The analysis of employment characteristics focuses on graduates working in industry, unless otherwise noted.

If you have specific comments and questions on the contents of this site and/or its use, please contact webmaster@erc-assoc.org. To report technical problems, contact techhelp@erc-assoc.org.