Montana State University: Guidelines for Program Reviews

Updated January 2014

 

Program reviews serve a variety of purposes including meeting the requirements of the Board of Regents of the Montana University System (BOR) and providing valuable input into the planning and analysis processes at the University. The Board of Regents requires a review of every degree (including options and minors) every seven years (Policy 303.3), but the Montana State University administration can require a Special Program Review at any time. Program accreditation reviews may be used to satisfy the provisions of BOR policy. However, accreditation reviews may or may not provide the information needed for institutional planning and analysis. Therefore additional information may be required from programs with external accreditation.

Programs Requiring Intensive Review – Productivity Benchmarks


In the past, the BOR asked for increased scrutiny of programs that fall into any of the following categories:

  • Undergraduate Programs: An average graduation rate of fewer than 10 students per year
  • Masters Programs: An average graduation rate of fewer than 3 students per year
  • Doctoral Programs: An average graduation rate of fewer than 2 students per year
  • Loss of Majors: The number of enrolled majors has decreased by 20% or more since the last review.
  • Loss of SCH: The number of student credit hours generated has decreased by 20% or more since the last review.1

These benchmarks are still used with program reviews at Montana State University (MSU).

Scope of Program Reviews

Reviews will be conducted at the departmental level, since departments are the primary organizational structure for academic programming at MSU. A departmental review will cover all degrees, programs, options, certificates, and minors housed in the department. The scope of departmental reviews will not be limited to degree and certificate programs, and will include the full range of departmental activities.

Types of Review

Program reviews may be external or internal.

  • External reviews are conducted by a team of two or three disciplinary experts from other institutions, possibly assisted by an MSU faculty member from a closely related field. External reviews may or may not involve a site visit. The reviewers are selected by the Provost, but it is anticipated that the College Dean will provide a list of prospective reviewers after consulting with the Department Head and department faculty.
  • Internal reviews are conducted by a team of three MSU faculty. Typically, one or two reviewers will be selected by the Dean, and one by the Provost. Faculty reviewers should have closely related disciplinary expertise and/or come from departments with students who regularly take courses in the department under review.

The decision on whether to use an internal and/or external review committee is determined by the Provost with extensive input from the college Dean and the Department Head and faculty. When a site visit is required, the Office of the Provost may cover a portion of the costs, however costs associated with professional accreditation visits are borne by the colleges. A request of the Provost to cover a portion of the review costs should be made before the reviewers are identified.

External reviews are normally conducted during a one or two-day visit; final reports are to be submitted within three weeks of the visit. Internal reviews should be completed within two months, with final reports due within three weeks of the conclusion of the final scheduled meeting. Departments have the primary responsibility for scheduling all events associated with a site visit.

Checklist for Program Reviews

 Associate Provost notifies Dean and Department Head of Program Review

 Provost decides whether reviewers will be internal or external (or mixed)

 Department Head completes self-study

 Dean and Department Head establish date for site visit, notify Associate Provost

 Dean and Department Head establish schedule for site visit

 Dean forwards reviewer nominations (from faculty, Department Head, and Dean) to Associate Provost

 Provost selects reviewers

 Dean arranges for reviewer travel and accommodations

 Associate Provost prepares draft charge letter for Provost

 Reviewers meet with Provost, Dean and Department Head (Charge meeting)

 Reviewers interview identified groups according to schedule provided

 Reviewers meet with Associate Provost, Dean, and Department Head (Exit meeting) 

 Reviewers complete draft report, submit to Department Head for fact check

 Department Head responds to reviewers within five days

 Reviewers complete final report, submit to Dean

 Dean sends copies of final report to Associate Provost and Provost

 Dean submits recommendations to Provost

 Provost develops response actions

 Provost meets with Dean and Department Head re actions (Closure meeting)

 Department Head meets with faculty

 Associate Provost writes Board of Regents summary report

Preparing for a Program Review – Self-Study Report

In preparation for the review (both internal and external), the department prepares a self-study report to assist the reviewers in better understanding all aspects of departmental activities. This self-study report can rely heavily on existing documents and institutional data so as to reduce as much as possible the burden on departments. However, a key aspect of the self-study is the department’s strategic plan, which if not regularly reviewed, should be reviewed in preparation for this process. Specifically, the self-study must contain the following items:

  1. Institution-provided data (provided by the Office of Planning and Analysis)

                a. Institutional overview (provided for external reviewers)

                b. Key performance indicators (KPIs)

                c. Delaware data

                d. Productivity benchmark data

       2. Department response to institution-provided data (2 pages)2

         3. Department strategic plan3

       4. Self-assessment of the department’s progress on strategic goals (2 to 10 pages)

       5. Relevant material from MSU catalog (including curriculum listings)

       6. Narrative describing any significant curriculum changes in any of the department’s programs since the last review (1 to 5 pages)
       7. Departmental assessment plan, including program learning outcomes, and recent annual assessment reports.4

       8. Self-assessment of the department’s programs, including alignment with MSU’s Core Themes

           (4 to 20 pages, depending upon number of programs)

       9. Current CVs for faculty (suggested length: 2-3 pages each, include all publications for past 5 years)

The department may include other material aimed toward helping the reviewers better understand the department, its faculty, and its students.

Although likely to be compiled by a lead individual or team, departments are expected to engage all faculty in the process of generating the self-study document, and a majority of the faculty must approve the self-study report before it is submitted to the reviewers.5 The date of faculty approval should be included on the self-study report cover page.

Program Review Site Visit

Most program reviews include a site visit, unless the scope of the review is very narrowly focused. During a program review site visit, reviewers typically meet with the following individuals or groups:

  1. Provost
  2. College Dean
  3. Vice President for Research
  4. Graduate Dean (if graduate programs are reviewed)
  5. Department Head or Chair
  6. Tenured and Tenure-Track Faculty
  7. Adjunct Faculty, Post-Doctoral Fellows, and other Non-Tenured Faculty
  8. Staff
  9. Graduate Students
  10. Undergraduate Students
  11. External Consituencies (if appropriate)
  12. Others at the request of the reviewers

The reviewers conclude their visit with an exit interview with the Provost, Dean, and Department Head. During this interview, and in the subsequent written report, the reviewers will be asked to address the following issues:

  1. The strengths of the department. Please list any specific commendations.
  2. Overall observations and determinations regarding the quality and the rigor of the academic programs.
  3. The effectiveness of the department’s Assessment Plan and assessment activities, including program learning outcomes (in the Assessment Plan) and course learning outcomes (in course syllabi).
  4. Status of the each program curriculum in terms of breadth and currency with the discipline. That is, is each curriculum still relevant and has the curriculum kept pace with changes in the discipline?
  5. Overall level of faculty productivity as it relates to the stated missions of the department and university.
  6. Alignment of each of the department’s academic programs with the Core Themes and strategic priorities of the institution.
  7. Diversity of the department’s faculty and student body.
  8. Overall assessment of the quality of graduates produced by the programs in the department.
  9. Any weaknesses or unrealized opportunities, with specific recommendations for action.

The Reviewers’ Report

We request that the final written report be organized using the following general headings, although the committee is welcome to adjust this to fit its unique needs:

  1. Members of Review Committee (including name, title, institutional affiliation)
  2. Process
    Please provide a brief overview of the structure of the review with special attention paid to any components not described in this document.
  3. Scope of Review
    Please provide a short narrative describing the specific areas that were reviewed. Also, if any areas of departmental activity were not reviewed, please list these specifically.
  4. Assessments
    a.  Department Strategic Plan

    b.  Academic Programs (quality, rigor, relevance)

    c.  Department Productivity

    d.  Alignment with Core Themes and Institutional Priorities

    e.  Diversity of the department’s faculty and student body.

    f.   Department Learning Outcomes Assessment Program6

  5. Strengths

    Areas of notable success, or where the department excels relative to its peers should be documented in the Reviewer’s Report.

  6. Challenges
    The challenges should relate specifically to the department’s ability to meet the goals of its own strategic plan, and with special attention to any academic program failing to meet any of the productivity benchmark values listed under Programs Requiring Intensive Review.
  7. Opportunities
    These are the specific recommendations that the review team makes designed to assist the department in achieving its goals.

Disposition of the Reviewer’s Report

The following steps occur after the Review Team has prepared a draft report:

  • The Review Team Chair sends the Draft Reviewer’s Report to the Department Head who has an opportunity (5 business days) to correct any factual errors in the report.
  • The Review Team Chair then sends the Final Reviewer’s Report to the Dean for review. This completes the work of the Review Team.
  • The Dean forwards the Final Reviewer’s Report to the Provost with his/her recommendations for the reviewed programs.
  • The Provost meets with the Department Head and Dean to communicate the Provost’s recommendations for the reviewed programs.
  • The Reviewer’s Report is made available to the department faculty.
  • A summary of the program review and recommendations is reported to the Board of Regents. This summary is a public document.

 

A dowloadable PDF of this information can be found here.


 

NOTES

1 Data for student credit hour generation is typically available at the department level rather than the program level.

2 Page lengths listed here are typical, but can vary for individual departments.

3 A strategic plan describes the department’s mission, its strength (including evidence for those strengths) and describes specific goals with clearly articulated priorities. It should address all areas of departmental activity including, but not limited to, undergraduate education, graduate education, research, outreach, and college/university service. In short, the plan answers the questions, What do we care about? What are we good at? How do we know we are good at these things? What do we want to become? and How do we align with the institution’s priorities? In addition to providing ongoing direction for departmental decision-making, the value of the strategic plan is that it allows reviewers to assess the department against its plan.

4 Departmental assessment plans, available at www.montana.edu/wwwprov/assessment/assessmentplans.htm, describe the expected learning outcomes for all degree programs within the department and describe the process by which the department regularly monitors its progress in helping students meet those goals. The updates provide evidence that the plan is being implemented and the resulting data acted upon as appropriate. Additional detailed information not included in the public updates may be included in the self-study.

5 Department Heads should provide a copy of the self-study to the dean as early in the process as possible.

6 Specifically, we are asking reviewers to assess the following: (1) Are the stated program learning outcomes appropriate and sufficient? (2) Are the department’s assessment processes appropriate and being used? (3) Is there evidence that the students are achieving the program learning outcomes? (4) If weaknesses in assessment are identified, has the program developed a plan for addressing the weaknesses and coming into compliance with institutional assessment requirements?