Materials Science Program Assessment

Standard Graduate Assessment Plan

MSU’s Mission

Montana State University, the state’s land grant institution, educates students, creates knowledge and art, and serves communities by integrating learning, discovery and engagement.

Materials Science PhD Program Mission

Students who graduate with a Ph.D. in Materials Science will go on to assume leadership roles in their respective fields at nationally and internationally recognized institutions.  

Ph.D. Learning Outcomes

All students in the program will:

1. Understand how classes of materials derive their properties from the atomic and molecular properties of their constituents and be familiar with the growing set of materials fabrication, assembly, processing and characterization tools and techniques.
2. Demonstrate effective oral and written communication skills.
3. Show awareness of the economic, societal, and broader impacts of materials research and application.
4. Demonstrate the ability to conceive, plan, design, conduct, analyze, defend, publish, and communicate original and creative research that advances understanding in an area important to materials science.
5. Demonstrate knowledge of basic laboratory safety and the requirements to assist in establishing a safe laboratory environment.  
6. Understand ethical issues and responsibilities especially in matters related to professionalism, data collection, the laboratory setting and in writing and publishing theses, dissertations and scientific papers.
7. Professionalization into the field of study: publications, presentations, attended conferences, received funded fellowships, and professional association activities.

Doctoral Program

¹ The Program will keep track of qualifying/comprehensive/ thesis/dissertation defenses. The rubrics for outcomes 2-4 are to be completed at the exam/defense.  These rubrics will not be used to assess or evaluate individual students and will not inform the decision regarding whether a student passes a defense or course.  The data will be aggregated for all students in the program over a two-year period in order to assess the success of the program in meeting its program learning outcomes (see sample rubric attached).² The rubric for outcomes 5 and 6 is the completion of lab safety and ethics training.³ Thesis Points of students will be collected at the time of their defense.  The thesis points document will be used to provide evidence of appropriate professional activities to the particular field of study (these may be published papers, research training, fellowships received and presentations at conferences).

Response Threshold

• 100% of students will be ranked at a level 2 (acceptable) or 3 (exceptional) in subject content knowledge, written communication, and oral communication skills.
• 100% of our students will pass their Ph.D defense on their first attempt.
• 100% of students will successfully complete the ethics training and lab safety training.
• 100% of students will demonstrate more than one form of professionalization in their field.

Schedule of Assessment

• Assessment reports for Doctoral programs will be submitted in September of odd-numbered years.

Process for Assessing the Data

Data are collected as students advance through the degree program.  The graduate assessment coordinator will tabulate the scores from the rubrics after defense of the comprehensive exam and PhD defense.  The program coordinator will also prepare lists of students taking qualifying/comprehensive exams and the results (pass or fail) of such exams.  The coordinator will also identify, based on an analysis of thesis points documents, the percent of students demonstrating acceptable professionalization into the field of study. The coordinator will write an assessment report and share it with MTSI stakeholders in order to improve programming and learning.

• If an acceptable performance threshold (as outlined in the plan) has NOT been met, a response from the Director of the MTSI program is required.  It should include some strategy for addressing improving areas where the threshold has not been met.
  • Gather additional data next year to verify or refute the result.
  • Change something in the curriculum or program to try to improve performance.
  • Change the acceptable performance threshold (must provide reasoning behind such a strategy).
  • If MTSI faculty members identify new strategies for meeting the learning outcomes, they may respond to assessment results even if the acceptable performance threshold has been met.
  • Acceptable outcomes include deciding that changes are not needed when students are demonstrating proficiency with each learning outcomes.  In this case participating MTSI will note (in writing) that they are satisfied with student outcomes and, based on assessment data, that the program is achieving its goals.

A summary of the year’s assessment activities and faculty decisions are reported to the Dean of the Graduate School.  

Materials Science Ph.D. Program Assessment- September 2017

The University of Montana (UM), Montana State University (MSU), and Montana Tech (MTech) participate in the Materials Science (MTSI) Ph.D. program. The program is a collaborative effort between the three campuses, and involves multiple departments, faculty, courses, and research infrastructure.  As stated in the Materials Science Handbook, “the curriculum integrates a broad range of physical science and engineering disciplines with an even broader range of applications: from health and medicine to nanotechnology to energy, environment, and natural resources. Courses are coordinated and teaching responsibilities are shared by the three campuses, taking advantage of on-line instructional technologies where appropriate. To earn a Ph.D. degree in Materials Science, each student must complete original, independent research culminating in a dissertation.”

Program requirements include a qualifying exam (administered after the first year in the program), the comprehensive exam (proposal and defense scheduled no later than the 6^th semester in the program) and the writing and defense of the dissertation (intended to take place in a student’s 10 semester).  Additional requirements to be completed by each student to obtain a Ph.D. in MTSI are (1) participation in the program’s annual summer symposium, (2) presenting a public seminar, and (3) holding annual committee meetings. All program requirements provide a framework for assessment.  This report will only include data from the Materials Science cohorts from Montana State University. 

This is the first assessment document submitted on behalf of the Ph.D. Materials Science Program. The first cohort entered the program in Fall of 2014. 

Cohort 1 - Fall of 2014

Four students successfully passed the mandatory MTSI coursework (GPAs ≥ 3.0) in AY 2014-2015. 

Qualifying Exam

All students in the Fall 2014 cohort sat for the qualifying exam in August of 2015.   Students had to achieve a composite score of 70% and not less than 50% on any given question in order to pass the exam. 

Three of the four students passed the exam on the first attempt.  One student did not demonstrate mastery of the material in 2 of the 6 areas tested.  The student’s  aggregate score was 66.6%.  The student successfully passed the qualifying exam on the second attempt.  All four students are in good standing in the program.   

Cohort 2 - Fall of 2015

Both students successfully passed (3.0 or better) the 20 credits of mandatory MTSI coursework in AY 2015-2016.

After the first summer in the program, one student decided he wanted to leave the PhD program.  Because the Materials Science program does not offer an MS track, he applied to the MS coursework program in the Department of Chemistry and Biochemistry and finished his coursework MS degree in December of 2016.

Qualifying Exam

One student sat for the qualifying exam in late August of 2016 and did not demonstrate mastery in 3 of the 6 areas tested.  His aggregate score was less than 60%.  (An aggregate score of 70% or higher is required to pass the exam.) He was offered the opportunity to sit for a second exam in November but declined the offer.  The student decided to exit the PhD program in December of 2016.  He applied to the Chemistry Department for a MS degree and met the requirements for a coursework MS.  He graduated in May of 2017.

Cohort 3 - Fall of 2016

Four students entered the PhD program in Materials Science in the Fall of 2016.

All students have passed the required MTSI courses in AY ‘16-’17. Each student has an overall GPA above 3.0 to remain in good standing with The Graduate School requirements.

Qualifying Exam

Three students sat for the qualifying exam in August of 2017.  One student passed the exam and two other students received conditional passes. The students receiving conditional passes, their composite scores were above the 70% requirement, however, both did not demonstrate mastery on one of the questions.  Each student will sit for their respective exam question in the fall of 2017.

Summary of Program Outcomes

PhD Learning Outcomes

1. Understand how classes of materials derive their properties from the atomic to the macroscopic level and be familiar with the growing set of materials fabrication, assembly, processing and characterization tools and techniques.
2. Demonstrate effective oral and written communication skills.
3. Become aware of the economic, societal, and broader impacts of materials and materials research.
4. Demonstrate that they can conceive, plan, design, conduct, analyze, defend, publish, and communicate original and creative research that advances understanding in an area important to materials science.
5. Demonstrate knowledge of basic lab safety and the requirements to assist in establishing a safe lab environment.  
6. Understand ethical issues and responsibilities especially in matters related to professionalism, data collection, the laboratory setting and in writing and publishing theses, dissertations and scientific papers.
7. Professionalization into the field of study: publications, presentations, attended conferences, received funded fellowships, and professional association activities.

Outcome 1- 100% of our students (Cohorts 1-3) have met the coursework requirement for materials science and are in good standing academically.  Cohort 3 will be completing their 12 credits of electives AY “17-18 academic year.

Outcome 2.  Students in Cohort 1 have completed their written and oral comprehensive exam.  All students received passing marks.

Outcomes 2,3,4 will be assessed in more depth at dissertation defenses.

Outcome 5.  We are working to create a mandatory lab safety workshop for all incoming MTSI students. However, four students affiliated with the Materials Science program through the Chemistry Department has received lab safety training. 

Outcome 6.  100% of our students in all cohorts have received ethical training at the Graduate School fall orientation.

Outcome 7.  Several Students presented posters at the 2016 and 2017  “Under the Big Sky” Materials Science Symposium.  Four students in Cohort 1 have publications.

Summary and Conclusion

The MTSI Ph.D program, although small, is meeting program outcomes. Funding for a minimum cohort size of 5 students is assured by an MOU with the Office of The Provost and administered by the Dean of the Graduate School. The MOU is re-visited every 3 years.  As the program becomes more established, we hope to secure additional support to increase the number of students per cohort/year. We fully understand program outcomes may be susceptible to “impactful” changes due to the small number of students in each cohort. 

Significant benchmarks of the program are addressed below:

• All students in the program have a 3.0 or better overall G.P.A.
• All students (Cohorts 1 and 3) have secured a “research home” with an advisor.
• All students in the program were research assistants in the summer of 2016. 
• All students in Cohort 1 have passed their qualifying and comprehensive exam requirement. 
• Two students in Cohort 1 have publications- one is a first author.
• All students in Cohort 1 have demonstrated at least one form of professionalization in their field of study.
• All students in Cohort 3 have passed or have conditional passes on their qualifying exam requirement.
• Cohort 4 -composed of three individuals started in the Fall of 2017.

Small setback- It was unfortunate that both individuals in Cohort 2 left the Ph.D program and obtained their coursework MS degrees in Chemistry.  Two discussion points that emerged from their departures need to be addressed at the next annual meeting. First, finding an alternative path for those students who opt out of the PhD program. Second, update the MTSI student handbook with additional text for clarification to assist students with meeting and fully understanding program requirements.  One student in Cohort 3 recently left the program to pursue employment at a local software firm.  

Assessment will be continuous.  The MTSI program has an external advisory board (EAB) that annually reviews the progress and status of the Materials Science Ph.D. program.  At the last MTSI Symposium, (Fall of 2016) 4 EAB members met with MTSI campus constituents from University of Montana, Montana Tech and Montana State University to assess learning outcomes, discuss program growth and to make recommendations to become a self-sustaining competitive PhD program for MUS.  The EAB had no significant programmatic or policy concerns.  Based on EAB’s recommendation, no changes will be made to the PhD program for 2017-2018 AY.