Schools of management, as elsewhere in graduate and undergraduate education, have been called upon to account for the effect of their MBA programs. Students, parents, and prospective employers, are asking management schools to show results on their unique "bottom line"-what are the retained learnings? The lack of agreement as to the goals of management education (Duncan, 1983) often leads the discussion of accountability to faculty effectiveness.

Faculty in graduate schools are the first gatekeepers of entrance into the occupations and professions they represent. Accrediting/licensing bodies and professional associations can be viewed as the second and third gatekeepers. Knowledge, appropriate norms of conduct, and values are transmitted to graduate students as they are socialized into a field or profession. Faculty are not the only sources of impact on students in graduate school. Exposure to other students, internships, projects, clubs, professional associations and community activities also have an impact, but none command more attention, time, or engagement than faculty. Therefore, examining the effectiveness of professional education on its two main objectives of adding value to students and evaluating their competence to enter a profession can be turned into an examination of the effectiveness of faculty.

Before faculty effectiveness can be assessed, the overall impact of MBA programs on students must be documented. Although a number of studies (Porter and McKibbin, 1988; Boyatzis and Renio, 1989; Boyatzis, Renio, and Thompson, 1990) have found that MBA programs have a positive impact on students, there is still confusion concerning the nature and degree of this impact. Outcome studies conducted to assess the added value of programs to students' abilities (i.e., comparing graduating students to entering students) have shown statistically significant positive increases on various abilities from five MBA programs (Boyatzis and Sokol, 1982; Development Dimensions International, DDI, 1985; Boyatzis and Renio, 1989; Boyatzis, Renio, and Thompson, 1990). These studies focused on abilities because prior discussion as to the effectiveness of professional education have typically focused on knowledge. While effective transmission of knowledge by faculty in schools of management has been assumed (Keys and Wolfe, 1988), the discussions have often resulted in debates as to the relevance of various bodies of knowledge. The traditionally exclusive focus of the discussion of faculty effectiveness on knowledge has often resulted in ignoring the impact of faculty and graduate programs on students' abilities. The latter have been shown to have greater impact on performance than only possession of knowledge in various professions (McClelland, 1973) and management (Crooks, Campbell, and Rock, 1979).

Even with regard to abilities, the relevance and appropriateness of what is taught must be examined prior to assessing the effectiveness of the teaching. Employers and faculty should be studied to determine what abilities graduates will need in management or other related jobs to perform effectively. Surveys of employers and studies of effective and less effective managers, effective and less effective financial analysts, salespeople, and so forth regarding each of the occupations MBA graduates enter will answer part of the question. Various studies have been conducted in this manner (Boyatzis, 1982; Howard and Bray, 1988; Kotter, 1988; Campbell, Dunnette, Lawler, and Weick, 1970).

Similarly, faculty can be surveyed as to what they believe to be the relevant and appropriate abilities. Porter and McKibbin (1988) collected information from various stakeholders regarding their views of the impact of MBA programs. The study included interviews and questionnaires of 2,055 faculty and 1,835 MBA students from the 620 schools in the American Assembly of Collegiate Schools of Business (AACSB) regarding skills and personal characteristics. They found that faculty and students agreed as to where the current emphasis was placed in MBA programs. Analytical and decision making skills were seen as "emphasized very much" by more than half of the students and more than one third of the faculty. With the exception of computer skills, the order of the skills on the basis of the percentage of responses saying it was "emphasized very much" is identical. Both students and faculty felt that the order of emphasis was: analytical (62% and 38%, respectively); decision making (51% and 32%); planning and organizing (51% and 18%); written communication (42% and 15%); oral communication (28% and 14%); leadership and interpersonal (26% and 14%); initiative (21% and 11%); and risk taking skills (8% and 5%). Computer skills ranked eighth in the view of students and fourth in the view of faculty (14% and 15%). In summary, the faculty and students view of MBA programs' emphasis on particular abilities appears consistent with employer observations that students with MBA degrees enter the work force with a great deal of analytic and quantitative ability and less interpersonal, communications, and entrepreneurial ability than desired (Porter and McKibbin, 1988; Byrne, Norman, and Miles, 1988).

While providing a great deal of useful information, Porter and McKibbin's (1988) survey asked faculty and students for their overall view of program emphasis. This type of inquiry may have suffered from distortion because faculty and students' generalized views as to what occurs may have been different than what does occur (Argyris, 1985). A study focused on the classroom behavior and outcome would provide further information relevant to the assessment of faculty effectiveness.

The effectiveness of faculty, therefore, can be assessed through a comparison of the faculty intent and student outcome. The intentions of faculty include their learning objectives, determination of relevant topics, and organization of material in the design and delivery of their courses. It reflects how faculty assess and guide themselves during the delivery of a course. Student outcome is the progress, or change, shown in students between entering and graduating from a program. The present study attempted to compare faculty intent and student outcome on various abilities related to effectiveness in management and occupations of graduating MBAs.

In the spring and summer of 1988, faculty at the Weatherhead School of Management were interviewed by a special senior faculty committee about specific courses each had recently taught in the MBA program. Forty-two of the forty-seven full time faculty who taught in the MBA program during the prior year were interviewed. Twenty-nine faculty were interviewed regarding the required courses. At least two faculty were interviewed regarding each of the thirteen required courses. Of the thirty-eight elective courses taught during the same year, twenty-one faculty were interviewed regarding twenty-nine (76%) of the elective courses. Some faculty were interviewed about required and elective courses, but in such cases the interview protocol was completed concerning each course separately.

The interview involved three questions and administration of one instrument. Questions were asked about each specific course: (1) In general, what are you trying to teach in this course? What are your overall objectives?; (2) What topics do you see as important to cover in the course?; and (3) What other objectives, besides course topics, are you trying to cover in this course? Following the second question, the syllabus was reviewed and discussed.

Following the answers to these questions, the faculty member was asked to repeat the answer to the third question in a way that could assist analysis across courses by means of an adaptation of the Executive Skills Profile (ESP). The ESP is a card sort instrument in which a person is asked to describe his/her skills and the demands of his/her job by sorting seventy-two cards with statements of specific skills or activities. The cards are placed into stacks, up to a maximum of seven, reflecting various levels of the skill. The scales have been shown to be reliable and valid measures of skills in a number of studies (Boyatzis and Kolb, in press). In this case, the instructions were modified and the faculty member was asked to describe the specific course being discussed by placing each of the seventy-two cards into one of the following five categories: (1) Skills that are of primary importance to me in designing and conducting my class; (2) Skills that are important but not primary in designing and conducting my class; (3) Skills that are less clearly addressed or "implicit" in designing and conducting my class; (4) Skills that are mentioned or addressed only once or twice in designing and conducting my class; and (5) Skills that are not addressed and/or not applicable in designing and conducting my class. The faculty member was then asked to review each stack to insure appropriate placement of the cards.

The numeric value assigned to each statement (i.e., each card) corresponded to the number of the stack into which the faculty member placed the card. Scale scores were computed as a total of the items in each scale.

The full response of each faculty member to all questions for each course was written and reviewed with the faculty members, including a list of all of the ESP items he/she listed as a level "5" for the course. Any changes desired by the faculty members were made, and the resulting written documents and the complete ESP scores were used in the analysis. Only the analysis regarding the abilities assessed with the ESP are reported in this study. For each course in which several faculty provided information, an average score per item in the ESP was calculated. The resulting scores for the forty-two courses (i.e., thirteen required and twenty-nine elective courses) were used for the analysis.

Student outcome data was used from the first year of a multi-year study of the impact of the MBA program at the Weatherhead School of Management (Boyatzis and Renio, 1989). The study assessed a random sample of twenty-six entering part time students (10% of the population), seventy-two of the entering full time students during the orientation program (72%), a random sample of twenty-seven graduating full time students (44%), and a random sample of twenty-three graduating part time students (51%). The ESP was one of the instruments used in the study.

Analysis of ESP scores for all courses indicated that faculty intent was to emphasize Adapting Skills above all, then Planning Skills, followed by Information Analysis Skills, Entrepreneurship Skills, Setting and Managing to Goals Skills, Quantitative Analysis Skills, and Information Gathering Skills, as shown in Table 1. Other skills were relatively less important in terms of faculty intent. They were, in order of decreasing importance: Technology Management Skills, Leadership Skills, Taking Action Skills, Relationship Skills, and Helping and Delegating Skills.

-------------------------

Insert Table 1 about here

-------------------------

Comparison of the graduating and entering students' scores on the ESP are summarized in Table 1. The mean difference is reported with the associated significance level from analyses of variance (Boyatzis and Renio, 1989). Examination of the skills on which significant change occurred reveals that they tended to be the skills emphasized by the faculty in their courses. A visual comparison of the entering students' scores, the graduating students' scores, and the faculty's intent scores are shown in Figure 1. Graphically, it appears that the students increase on a skill during their MBA program when their level at point of entry into the program is lower than the level of faculty intent, or desired impact on the skill.

--------------------------

Insert Figure 1 about here

--------------------------

To help understand the faculty intent, the six items with the highest mean score and the six items with the lowest mean score are shown in Table 2. The items with the highest overall mean score, showing that they were considered "primary" and "important but not primary" in the design and conducting of courses, concerned conceptual, analytic, and planning skills. Meanwhile, the items with the lowest mean score, showing that they were "not addressed at all," "implicit," or "mentioned only once or twice" concerned managing people, interpersonal relationships, and self-awareness.

-------------------------

Insert Table 2 about here

-------------------------

A transformation of the ESP data was required to compare faculty intent and student outcome in more detail. The percentage of each scale score to the total score of all scales was computed to adjust for the difference between a seven point and a five point scale. The adjusted score indicates relative emphasis of a skill within each set of data.

A comparison of the entering MBA students' skills versus the faculty emphasis using the adjusted scores is shown in Table 3. The entering students had significantly greater relative strength on Leadership Skills, Relationship Skills, Helping and Delegating Skills, Taking Action Skills, and Entrepreneurship Skills than the faculty relative emphasis in their courses. The faculty had significantly more relative emphasis on Adapting Skills, Information Analysis Skills, Planning Skills, Quantitative Analysis Skills, and near significant Technology Management Skills than the relative strength of the entering students' on these skills.

-------------------------

Insert Table 3 about here

-------------------------

A summary of the comparisons are shown in Figure 2. The graduates improved, as compared to entering students, on all four skills for which the relative emphasis of faculty was significantly greater than entering students' relative abilities. These were : Adapting Skills; Information Analysis Skills; Planning Skills; and Quantitative Analysis Skills. On the two skills where faculty emphasis was greater than entering students' relative abilities at a near significant level, there was significant change on Technology Management Skills, but no change on Setting and Managing to Goals Skills. On four of the six skills on which there was no difference between faculty emphasis and entering students' relative abilities or the entering students had relatively more ability than the faculty emphasis, there was no improvement in students between entering and graduating from the MBA program. They were: Information Gathering Skills; Leadership Skills; Relationship Skills; and Helping and Delegating Skills. On Taking Action Skills and Entrepreneurial Skills, entering students had relatively more ability than faculty emphasis, and yet graduates improved as compared to entering students.

--------------------------

Insert Figure 2 about here

--------------------------

The way the faculty design and conduct their classes has a significant impact on how students change during an MBA program. Students increase on seven skills and remain about the same on five skills, and it appears that faculty drive the change in the direction they desire. In those areas where faculty intention is high, improvement of students is seen, specifically on Information Analysis Skills, Planning Skills, Quantitative Analysis Skills, Entrepreneurial Skills, and Adapting Skills. In those areas where faculty intention is low, little change occurs in students, specifically on Information Gathering Skills, Leadership Skills, Relationship Skills, and Helping and Delegating Skills. One exception is regarding Setting and Managing to Goals Skills. A relatively high faculty intention does not appear to affect student change on Setting and Managing to Goals Skills.

At the same time, it can be noted that although faculty had relatively low intentions regarding certain skills, students do significantly improve on Technology Management Skills and Taking Action Skills. The positive impact could be attributed to effects of the school, program, faculty contact outside of courses, or community activities. Using a median to split the scales into High Faculty Intent and Low Faculty Intent, it appears that students improve, significantly or near significantly, on five of the High Faculty Intent scales (83%), and do not improve or change on four of the Low Faculty Intent scales (67%).

The pattern of student change showed further support a year later in another study of the impact of this program. A sample of 106 MBA students were selected in the same manner (Boyatzis, Renio, and Thompson, 1990). This sample included a very small Intent and Outcome longitudinal sample of 9 students from the first year study. Both the sample from the second study and the small longitudinal sample showed significant improvement in students' Information Analysis, Planning, and Quantitative Analysis skills. The small longitudinal sample also showed a significant increase on the Technology Management skill.

The number of items in each scale for which students showed significant or near significant improvement further illustrated the impact of faculty intent. The number of items on which students showed improvement for the five scales on which faculty intent was greater than entering students skills was 4.0 per scale (see Table 3). For the two scales on which faculty intent and entering students showed no or near significant differences, the number of items that showed improvement was 1.0 per scale. For the five scales on which entering students were significantly greater than faculty intent, the number of items that showed improvement was 1.4 per scale.

An alternative explanation would be that students compensate for their skill levels at point of entry into the MBA program. They might seek to improve on their relatively weak skills and not pay attention to their relatively strong skills. This effect should work regardless of faculty intent. The concept of compensating development worked on seven of the twelve skill areas. A median split on entering students skill levels revealed that students improved on four of the six skills with which they entered the program relatively low (67%), and showed little or no change on three of the skills with which they entered the program relatively high (50%). Four of the five skills on which the student change data did not confirm the compensating effect are involved in implementation and are the most action oriented skills assessed in the ESP. It appears that students change the most on those abilities on which they are relatively weak and do not change much on those abilities on which they are relatively strong.

Despite the impact of the compensating development effect on students' learning, it appears that faculty intentions have a profound effect on students' learning. The good news is- when faculty intend a change, the students improve. The bad news is- when faculty do not intend a change, the students do not improve.

Although this study provides a more rigorous assessment of faculty intentions than Porter and McKibbin (1988), both studies show strong faculty emphasis on similar skills, with the exception of this program's significant impact on Taking Action Skills, Entrepreneurial Skills, and Technology Management Skills. Does this suggest that much of the relationship between faculty intentions and student outcome found in this study would also be characteristic of other management programs? Until MBA programs assess this relationship it will be difficult to determine how similar or dissimilar these findings are to other programs' faculty intentions and student changes.

It is also possible that the relationship found in this program is similar to the impact of graduate professional education in other fields. A number of studies of alumni of graduate engineering and social work programs showed increased Information Analysis, Information Gathering, Planning, and Quantitative Analysis Skills attributed to their graduate education. In a similar vein, Friedman (1989) reported that female, middle level managers with Masters Degrees had significantly higher levels of the Information Analysis, Planning, and Quantitative Analysis Skills than their less educated colleagues. The intentions of the faculty in those graduate programs was not known so it is difficult to conclude that student change was a function of an emphasis on similar abilities from the faculty of the various graduate programs.

These findings suggest that regarding impact on students' abilities, graduate professional education increases abilities regarding information collection and analysis, planning, and quantitative analysis. Likewise, it appears that interpersonal and self-awareness abilities are not affected. It seems likely, therefore, that the positive impact of this particular program on Technology Management, Taking Action, and Entrepreneurial Skills is related to either its specific program or characteristic of the field of management. The faculty views of relative emphasis of MBA education reported in Porter and McKibbin (1988) suggest that there is relatively low emphasis on these skills in most programs.

It could be argued that the faculty intent shown here and in other graduate professional education programs is, in part, a result of the student impact. That is, over the years, faculty come to realize that students only improve, or add value, on these information collection and analysis abilities. Therefore, the faculty focus their intent only on those abilities they believe can be changed during graduate education.

While the influence of past impact on students cannot be eliminated as a contributor to socialization of faculty, and thereby, affecting their expectations and intent, there seems to be literature suggesting that faculty seek to have impact on students in broader arenas. Some faculty have expressed doubt in faculty meetings about the efficacy of influencing a person's interpersonal ability, ability to work with others in groups, or action and implementation abilities. At the same time, there seem to be far more faculty who feel that they have a positive impact on a student's maturation and improvement on a broad range of skills. Literature in the field contends that faculty in schools of management should have even greater impact than we have had in many of these other areas (Porter and McKibbin, 1988).

The desirability of influencing many of a student's abilities is neither a new phenomenon nor an archaic one. The apprenticeship and tutorial aspects of advanced, professional education first popularized by Socrates, Plato, and Aristotle, continue to have advocates in John Dewey, Malcolm Knowles, David Kolb, and many others for engaging the whole student. The "affective education" movement of the 1960's, the "experiential learning" movement of the 1970's, the "competency-based education" movement of the 1980's, and the efforts of professional schools to become closely connected to the settings and expectations in which their graduates work share the intent to develop the "whole" student. They want to help the student develop any and all knowledge and abilities related to the practice of their profession and improvement of that practice. Within this context, it is difficult to embrace the notion that faculty have limited their intent to only those information collection and analytic abilities on which students have been shown to change in the past.

CONCLUSION

Faculty intent appears to affect the degree of graduate student learning in terms of abilities. Other aspects of the program assessed may also have a positive impact on students in facilitating change, but on the whole, if faculty do not view a skill as a key item on which to focus, the students will not change during their graduate program. Student outcomes indicate that faculty are effective in their teaching of abilities.

Unfortunately, the abilities on which faculty focus may be too limited for the intended and desired occupations of their graduates. If the criticisms reported in Porter and McKibbin (1988) and gaps in substantive progress in terms of innovation in management education (Keys and Wolfe, 1988) are taken seriously, then faculty must choose to either change their focus within courses, and/or to change the focus of the graduate program through other aspects of the program. Faculty could change their focus on skills without changing the content of their courses by changing the way in which they are taught, the type of assignments, and the type of interaction expected from students. Other program elements can also be introduced into a curriculum, such as learning teams (Caie, 1987; Prideaux and Ford, 1988), to bolster attention on those skill areas not thoroughly addressed in courses.

Whether the desire for curriculum change and change of the impact on students comes from employers, students, parents, or administrators, the faculty must be directly involved. The faculty already have intentions, in terms of learning objectives, selection of topics, the design, and delivery of their courses. These intentions address content topics and abilities. To stimulate a change effort, the faculty must "buy into" the effort.

Once a deficiency in a curriculum is noted, the standard academic response is to add a course on the topic. The findings of this study suggest that merely adding another course in the curriculum would not have the desired effect. For example, adding a course on leadership or a course on ethics will not significantly alter the overall impact of the courses and program on a student as to leadership abilities or ethical behavior. The time and experiences to which he/she is exposed in one course are miniscule compared to the combined effect of all courses, faculty, and the entire program. The results of this study suggest that expansion of faculty focus on abilities will, most likely, result in more impact on students, more value-added, and greater retained learnings than is currently experienced. While a sense of purpose and focus shared by the faculty maybe difficult to achieve, it is an essential first step in the direction of increased positive impact on students' abilities.

Endnotes

1. The author wishes to thank John Aram, David Bowers, David Kolb, Eric Neilsen, Jack Ruhl, Tojo Thachankery, and Xiaoping Tian for assisting in the collection and analysis of the information reported in this study, and to Anne Renio and Lorraine Thompson for assistance in data analysis.

Faculty Student Change:

Intent, Graduates (N=50) as

N=42 Compared to Entering

Skill Courses Students (N=98)(1)

Adapting 23.40 1.6!

Planning 22.55 4.1***

Information Analysis 22.00 4.2***

Entrepreneurial 19.52 2.1*

Setting/Managing

to Goals 19.19 1.1

Quantitative

Analysis 19.10 6.1***

Information

Gathering 18.62 1.2

Technology Management 15.98 3.3**

Leadership 15.67 0.9

Taking Action 15.42 2.2**

Relationship 15.33 0.7

Helping & Delegating 12.83 0.6

(!p<.10; *p<.05; **p<.01; ***p<.001)

(1) From Boyatzis and Renio, 1989

Item Mean Score

Identifying and defining problems 4.62

Building conceptual models/conceptual thinking 4.48

Seeing how things fit in the big picture 4.38

Making decisions under conditions of risk

and uncertainty 4.17

Innovating/developing new solutions to problems 4.10

Adapting to changing circumstances 4.07

Establishing trusting, dependable

relationships with co-workers 2.14

Establishing relationships with co-workers in

which honest feedback is given and received 2.07

Helping others gain opportunities to

develop their abilities 2.07

Directing and supervising the work of others 2.05

Understanding and being influenced by the

feelings of others--empathy 2.05

Being aware of and understanding yourself,

assessing yourself accurately 2.02

Percentage of Total

Skills Score____

Entering Faculty

Students Intent

Skill N=98 N=42 t

Leadership 8.30% 7.04% 4.00***

Relationship 9.79% 6.87% 7.76***

Helping and Delegating 8.80% 5.70% 7.69***

Adapting 9.03% 10.80% -5.46***

Information Gathering 8.63% 8.40% 1.04

Information Analysis 8.21% 10.16% -5.43***

Planning 7.04% 10.44% -10.03***

Quantitative

Analysis 6.98% 8.86% -3.32**

Technology Management 6.63% 7.35% -1.76!

Setting/Managing

to Goals 8.18% 8.68% -1.63!

Taking Action 8.91% 6.86% 7.75***

Entrepreneurship 9.51% 8.86% 2.30*

(!p<.10; *p<.05; **p<.01; ***p<.001)

(this figure is not computerized)

Faculty Intent Student

Skill vs. Entering MBA Outcome

Adapting Faculty>>Students Grads>Entering

Information

Analysis Faculty>>Students Grads>>Entering

Planning Faculty>>Students Grads>>Entering

Quantitative

Analysis Faculty>>Students Grads>>Entering

Technology

Management Faculty>Students Grads>>Entering

Setting/Managing

to Goals Faculty>Students ns

Information

Gathering ns ns

Leadership Students>>Faculty ns

Relationship Students>>Faculty ns

Helping and

Delegating Students>>Faculty ns

Taking Action Students>>Faculty Grads>>Entering

Entrepreneurship Students>>Faculty Grads>>Entering

(">" means near significant, p<.10; ">>" means significant, p<.05)