Continuous Improvement System - ABET Accredited B.S. Program

Metallurgical Engineering SDSM&T Rapid City, SD

 

Program Educational Objective Evaluation Process

 

 

Establishing Program Educational Objectives

The department has a long tradition of external evaluation dating to 1970. Periodic surveys of both alumni and their employers were routinely performed and acted on. The department was the source of the current campus student opinion surveys starting in 1971. The department was also the point of initiation for Industrial Advisory Boards (now more commonly called the Advisory Boards) beginning in the mid 1970s.

 

The design of the continuous improvement system began in 2000 and was followed by a staged collection of materials beginning in the 2001-2 academic year. During the subsequent two years, the system was continually refined and brought to full implementation. Although informal reviews and system refinements were occurring on a weekly basis throughout 2001-2003, the first comprehensive objective review involved all data collected up to the end of 2003. This initial “closing of the loops” occurred during the Spring Semester of 2004. With the substantial faculty retirements (Stone, Han, and Marquis) from 2005-2007, subsequent biannual Advisory Board reviews were renewed in 2007 with the newly contracted faculty (Medlin, West, and Cross).

 

During the period from 2001 to 2004 the entire department faculty has met once or twice a week during the academic year to create the continuous improvement system now in place. Dr. Howard attended several conferences on ABET methodology during the 2001-3 period. Dr. Howard trained as an ABET evaluator in the period 1999-2000 period. Dr. Kellar has also attended ABET training sessions for chairs. Program faculty members have attended numerous campus sessions on continuous improvement methodologies.

 

Since the 2004 ABET review, three of the five full-time tenured track faculty members have retired and been replaced: Drs. Medlin and West were unfamiliar with the department’s continuous improvement system while Dr. Cross being an internal candidate from the department research program had some familiarity with the system. Extensive training was provided these new faculty members. As of 2010, all program faculty members are well versed and directly involved in supporting and managing the continuous improvement system. All teaching faculty members in the metallurgical engineering program are actively engaged in periodic reviews of the program educational objectives.

 

The program faculty members proposed initial ABET-conforming program objectives in 2001. During the subsequent year, program constituents were asked to review the objectives. The first review was conducted by the 2002 Advisory Board, followed by reviews in 2004 by comprehensive and often overlapping assemblages of program constituents grouped as follows:

 

Primary

          Constituent focus groups

          Alumni surveys

          Employer surveys

          Graduate student surveys

          Recent outstanding graduate awardees surveys

          Advisory Board reviews

Secondary

          SDSM&T constituent departments surveys

          SDSM&T undergraduate student opinion surveys

          SDSM&T Student Satisfaction-Importance (SSI) Survey

 

After this pre-2005 exhaustive review of program educational objectives for use in our ABET-conforming continuous improvement process, a more streamlined review has been employed that relies on the Advisory Board, alumni surveys, and constituent focus groups. These groups overlap but have unique functions and input methodologies both of which preclude consolidating the program educational objective review under the purview of any one group. For example, the Advisory Board may engage in the review of departmental personnel matters, which would be inappropriate to consider in a body including students. There are many such matters requiring the use of several groups to complete a review of program educational objectives. The secondary sources above are not specifically employed since the departments are represented in the current constituent focus groups, opinion surveys are confidential, and the Satisfaction-Importance Survey is more germane to assessment functions.

 

Since 2004 the program objectives have been reviewed by the

         Advisory Board

         Constituent focus group

         Alumni survey.

 

The composition of the 2007-2012 Advisory Board is as follows:

  • Dr. Ray Peterson, Aleris International, Advisory Board Chairman
  • Dr. Everett Bloom, Oak Ridge National Laboratory – Retired
  • Mr. Mark Benson, US Bank
  • Ms. Wendy Craig, Mac Steel
  • Mr. Christopher Misterek, John Deere
  • Mr. Shawn Veurink , RPM and Associates
  • Mr. Shane Vernon, Nucor Steel
  • Mr. John Walenta, Caterpillar Inc.
  • Mr. Richard Wensel, Micron Technology

 

The department holds biannual meetings with its Advisory Board to conduct a review of Program Objectives and the department’s success in achieving them. The review also includes a re-examination of the objectives to assure they are current and significant. Materials presented to the board include the results of alumni and employer surveys, which are designed to gauge the extent to which program graduates are achieving the program objectives. The board members are selected to represent as many of the program’s constituents as possible.

 

The Advisory Board is provided the most recent survey information from alumni, current students, constituents, etc. making their review the most comprehensive whereas the alumni surveys and the constituent focus groups are generally asked to offer input on specific topics such as the currency of the program education objectives. However, every group is encouraged to offer any constructive comments they wish.

 

Alumni surveys query all alumni who graduated since the previous alumni survey. This means all alumni are asked for their input. Owing to the variation in numbers of program graduates, alumni surveys are grouped in four-year groups to preserve anonymity and ensure large enough numbers for statistical meaningfulness. A total of 54 alumni were surveyed using Survey Monkey in 2008 with 51 responses. This unusually high response is in itself an indication of the strong, favorable relationship between faculty and program graduates. The three responses not received were caused by incorrect email addresses.

 

As shown in Table 2-1 the Advisory Board reviews are scheduled every two years; alumni surveys every four years; and constituent focus group meetings every four years. The two surveys are staggered by two years since some of those surveyed are the same individuals. This staggering precludes pestering some alumni for the same information in the same year yet, assures important constituent input for objective input. The schedule was revised in 2006 to restart the cycle in 2007 because of the 60 percent replacement in program faculty that occurred between 2005 and early 2007. Restarting the cycle in fall of 2007 provided for all the new faculty members to participate in a common evaluation review cycle early in their tenure.

 

The program faculty members review all program evaluation input every two years after the results of the Advisory Board are available. The review culminates with action statements.

 

Figure 2-1 shows a schematic of the continuous improvement process used by the metallurgical engineering program to determine progress towards program objectives. Figure 2-2 shows this process interfaced with the process to determine progress in meeting program outcomes.

 

 

Table 2-1. Program Educational Objectives Assessment and Evaluation Schedule

 

04

05

06

07

08

09

10

11

12

13

14

15

16

Alumni Surveys

 

 

 

 

 

 

 

 

 

 

 

 

 

Advisory Board Review

 

 

 

 

 

 

 

 

 

 

 

 

 

Constituent Focus Group

 

 

 

 

 

 

 

 

 

 

 

 

 

Department Review

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

Achievement of Program Educational Objectives

The following process is used to determine the extent to which program educational objectives are being achieved:

         All program alumni are specifically surveyed on the achievement of the program objectives. The survey asks for their self-assessment. It also asks for input related to the achievement of the objectives, such as the extent of community and professional service. The survey results are used by the program faculty to evaluate the level of attainment of the objectives.

         Program constituent focus groups are convened to review the program’s direction and level of achievement. The focus group includes employers, alumni, and other departments who engage our graduates (most often as graduate students). The focus group is led by a non-program campus professional to assure free exchange and anonymous input who writes a summary report to be used by the program faculty in their evaluation.

         The Advisory Board has all pertinent surveys and reports available to them when they conduct their review. They are specifically charged with reviewing the program educational objectives and progress in achieving them. They submit a report of their findings and recommendations.

         The program faculty members meet several times a month on program and departmental matters. Many of these meetings are specifically to review ongoing continuous improvement matters. Over a period of several meetings consisting of an initial review, discussions, and a final review with written documents is generated. Each biannual review begins with reviewing the previous objective actions followed by a review of the alumni survey, focus group report, and the Advisory Board reports. A summary is written as to the how the action items were addressed and the result of the actions taken. Needed new action items for the coming period are then formulated and documented.

 

As described above, the program faculty members review all program evaluation input every two years after the results of the Advisory Board are available. The review process always consists of two action categories:

1.      Improvement in the evaluation process and

2.      Improvement in the achievement of the program educational objectives.

 

The action statements summarize actions taken within the evaluation process and the curriculum. The departmental faculty meets several times each month to discuss program operations and progress in the implementation of needed program objective improvements. The CIS Officer is monitors, tracks, and documents assessment information and evaluation work.