2021-22 Academic Year CAEP Standard 4.2 Impact and Outcome Measures

These measures relate to our programs accredited by the Council for the Accreditation of Educator Preparation.

 

There were a total of 268 responses to our 2022 Completer Survey.  Of those 268 responses only, complete responses were analyzed (n=195).  There was a 4-point Likert scale (1-not effective, 2-somewhat effective, 3- effective, and 4-very effective).

Of the 20 questions, only 3 averaged above a 3.0 indicating completers felt that MSU was effective in their preparation of those areas.  Completers felt that their teacher education program was most effective in preparing them for implementing instruction aligned with Montana State Content Standards (M=3.09, SD=.85), demonstrating accurate content knowledge (M=3.06, SD=.77), and engaging with colleagues in a professional manner (M=3.06, SD=.79).  In contrast, their teacher education program was less effective in preparing them for differentiate instruction effectively for English Learners (EL) (M=1.88, SD=.86), responding productively to negative behavior (M=2.10, SD=.91), and initiating professional conversations with supervisors (M=2.54, SD=.99).

MSU TEP Completer Survey 2022 - Chart of data in table below

Click chart to enlarge.

Completer Survey 2022

 

Mean

SD

N

% Not Effective

% Somewhat Effective

% Effective

% Very Effective

Design instruction based on learners' development.

2.71

0.73

195

6%

29%

55%

11%

Differentiate for English Learners (EL).

1.88

0.86

195

39%

38%

18%

5%

Collaborate to meet needs of all students.

2.79

0.81

195

5%

31%

44%

20%

Actively engage students in learning.

2.98

0.79

195

3%

23%

47%

27%

Respond productively to negative behavior.

2.10

0.91

195

29%

39%

24%

8%

Purposefully use instructional technology.

2.91

0.84

195

4%

28%

41%

27%

Demonstrate accurate content knowledge.

3.06

0.77

195

2%

22%

45%

31%

Encourage critical thinking for problem solving.

2.91

0.79

195

3%

28%

45%

25%

Analyze assessment data to improve instruction.

2.74

0.84

195

7%

31%

43%

19%

Plan instruction based on knowledge of community context.

2.61

0.80

195

8%

34%

46%

12%

Implement instruction aligned with Montana State Content Standards.

3.09

0.85

195

4%

21%

38%

37%

Use a variety of instructional strategies.

2.95

0.82

195

4%

25%

44%

27%

Participate in ongoing professional development opportunities.

2.79

0.85

195

7%

29%

44%

21%

Reflect on how instructional choices affect students.

2.89

0.85

195

5%

28%

41%

26%

Respect beliefs, norms, and expectations of families.

2.97

0.87

195

7%

19%

44%

30%

Apply knowledge of the legal responsibilities of teachers.

2.65

0.90

195

10%

34%

37%

19%

Initiate professional conversations with supervisors.

2.54

0.99

195

19%

26%

38%

17%

Engage with colleagues in a professional manner.

3.06

0.79

195

3%

19%

47%

31%

Accurately incorporate the 7 Essential Understandings of Indian Education for All.

2.63

0.96

195

13%

33%

32%

22%

Relevance of preparation for job

2.66

0.81

195

6%

37%

42%

15%

We value our stakeholders and utilize their feedback through a variety of ways in our Educator Preparation Program.

Field Placement

For in-area placements, administrator approval is collected via emails sent from the MSU Field Placement Office with lists of all pre-service/in-service teacher matches for practicum and student teaching. In this way, placement partnerships are verified at both district and building levels. Currently, MSU FPLO houses MOUs with districts in 20 states, and in 2021-22, began a systematic process of updating all local, in-state, and out-of-state MOUs to reflect stakeholder feedback. Feedback from CTs is also highly valued. Each spring, CTs are included in the Partner Survey. This survey openly invites a variety of feedback from all partner stakeholders.

Employer Survey

We participate in the state-wide Continuous Improvement Cooperative in collecting data from employers of our program completers.

Stakeholders’ Breakfast

Each spring, local administrators and district leaders are invited to campus for the annual Stakeholders' Breakfast.  Although curtailed during the pandemic, this meeting was renewed in Spring 2021. The event is a valued opportunity to meet, discuss policy, elicit feedback, and build relationships.

Montana State University - Bozeman

Employer Survey

Survey sent to 133 employers of MSU-Bozeman teacher education program graduates.

48 started the survey (36.1%)

36 completed the survey (27.1%)

Map with twenty-one markers across Montana and one in Salt Lake City

Locations where surveys were completed according to reported longitude and latitude data.

Please indicate the type of school which you currently lead.
Please select the option(s) which is most representative of your school.

N

N
Reply*

[no response]

1 0

Elementary P/K-5

11 6

Elementary P/K-5, Middle School 6-8, High School 9-12

1 1

Elementary P/K-8

6 5

Elementary P/K-8, Middle School 6-8

1 1

Middle School 6-8

6 4

Middle School 6-8, High School 9-12

3 2

High School 9-12

9 9

Full District P-12

10 8

Totals

48 36

 

Please identify grade band in which your teacher(s) is assigned.
Choose all that apply.

N

N
Reply*

[no response]

1 0

Preschool P, Elementary K-5

3 2

Preschool P, Elementary K-5, Middle School 6-8

1 1

Elementary K-5

17 10

Elementary K-5, Middle School 6-8

5 5

Elementary K-5, Middle School 6-8, High School 9-12, K-12 Subject-specific

1 1

Elementary K-5, High School 9-12

1 1

Middle School 6-8

4 2

Middle School 6-8, High School 9-12

5 4

High School 9-12

9 9

K-12 Subject-specific

1 1

Totals

48 36

*Surveys links were emailed to 133 school adminstrators known to have hired MSU graduates. Of those, 48 began the survey, while 36 completed the evaluative questions.

Nineteen items were evaluated on a scale from Very Satisfied (1) to Very Unsatisified (4). Note that lower scores indicate more satisfaction.

Seventeen respondents evaluated our graduates with a mean Very Satisfied (1.00 to 1.47) including five respondents evaluating our graduates with Very Satisfied on all nineteen items.  Eighteen respondents evaluated our graduates with a mean Somewhat Satisfied (1.53 to 2.39). One respondent evaluated our graduates with a mean Somewhat Unsatisfied (2.53).

The most common response from all respondents on all items was Very Satisfied.

Response Score N
Very Satisfied 1 340
Somewhat Satisfied 2 265
Somewhat Unsatisified 3 70
Very Unsatisfied 4 4

 

The overall mean from all respondents on all items was Somewhat Satisfied.

N Mean SD
679 1.61 0.69

 

The mean scores on each of the nineteen items were as follows:

Question N Mean SD

Design instruction based on learners’ development.

36 1.58 0.69

Differentiate instruction effectively for English Learners (EL).

34 1.91 0.75

Collaborate to meet the learning needs of all students.

36 1.47 0.61

Actively engage students in learning.

35 1.57 0.70

Respond productively to negative behavior.

36 1.78 0.76

Purposefully use instructional technology.

35 1.29 0.46

Demonstrate accurate content knowledge.

36 1.39 0.60

Encourage critical thinking for problem solving.

36 1.64 0.59

Analyze assessment data to improve the effectiveness of instruction.

36 2.03 0.81

Plan instruction based on knowledge of students in their community context.

36 1.83 0.74

Implement instruction aligned with Montana State Content Standards.

36 1.36 0.49

Use a variety of instructional strategies.

36 1.64 0.64

Participate in ongoing professional development opportunities.

36 1.53 0.61

Reflect on how instructional choices affect students.

36 1.75 0.73

Respect beliefs, norms, and expectations of families.

35 1.54 0.74

Apply knowledge of the legal responsibilities of teachers.

36 1.56 0.77

Initiate professional conversation with supervisors.

36 1.61 0.60

Engage with colleagues in a professional manner.

36 1.44 0.61

Accurately incorporate the 7 Essential Understandings of Indian Education for All.

36 1.75 0.81

 

     

Stakeholders Map with indicators at Willow Creek, Three Forks, Manhattan, Belgrade, Bozeman, and Livingston

Each year, the Department of Education, partnered with the Field Placement and Licensure Office, hosts Stakeholder Meetings. We invite K-12 administrators and teachers from our local area (all public and private schools from Livingston, Bozeman, Belgrade, Manhattan, Three Forks, and Willow Creek), our current students, Department of Education staff and faculty, the EHHD Dean’s office, and our University Teacher Education Committee partner faculty to meet and discuss our Teacher Education Program.  We provide updates to the program, talk about student trends, receive input and perceptions of our students in the field, and talk about ways our program can be bolstered via work the districts are undertaking.

 

mailboxCall for Comments - The Department of Education at Montana State University—Bozeman is hosting an accreditation visit by the Council for the Accreditation of Educator Preparation (CAEP) on September 23, 2023. Programs under review are the undergraduate Teacher Education Program, the graduate Master of Arts in Teaching, the graduate Master of Education in Educational Leadership/Principal Preparation and the graduate Superintendent Certificate. Interested parties are invited to submit third-party comments to the CAEP evaluation team on or before August 12, 2023. Laudatory, critical, and merely observational comments are all welcome. Details at this link.

Each graduating undergraduate pre-service teacher (PST) seeking Montana licensure was required to complete the Praxis Test, an external exam that assesses PST content knowledge, administered and assessed through the Montana Office of Public Instruction. PSTs seeking a K-8 or P-3 licensure have content questions related to Math, Language Arts, Social Studies and Science, while those seeking K-12/5-12 licensure complete the exam designated for their specific content area. For example, an undergraduate PST seeking Secondary Science Broadfield licensure will complete an exam containing questions in the fields of Biology, Chemistry, Physics and Earth Science. PSTs in the Master of Arts in Teaching are initially evaluated for subject matter knowledge through completion of the appropriate content area PRAXIS exam.

The US Median score here is the median of all scores on that test taken between August of 2019 and July of 2022. This gives us a national benchmark to compare with MSU's average score for tests taken by completers who finished in the four academic years between 2019-20 and 2022-23. If too few completers in an area take the test, those low-N scores cannot be reported for reasons of student privacy as well as statistical validity. Some completers elect to delay the taking of the Praxis test, so scores may change over time as more come in.

 

Praxis Test of Content Knowledge Scores
Undergraduate Teacher Education Program
Praxis
Test
Code

 

Praxis Test

 

N

 

MSU Mean Score

 

US Median

5025 Early Childhood Education 13 179 170
5018 Elementary Education 193 176 167
5134 Art 19 166 164
5857 Health and Physical Education 19 169 165
5174 French: World Language N too small to report
5183 German: World Language N too small to report
5195 Spanish: World Language 11 162 176
5113 Music 17 171 166
5435 General Science 25 176 161
5081 Social Studies 35 171 167
5701 Agriculture 29 169 167
5235 Biology N too small to report
5245 Chemistry N too small to report
5038 English Language Arts 40 180 178
5122 Family and Consumer Sciences 8 166 163
5941 World and U.S. History 10 166 161
5161 Mathematics (old test) 20 163 157
5165 Mathematics (new test) N too small to report
5265 Physics N too small to report
5051 Technology Education 7 180 179
5204 Teaching Reading 11 175 170
5571 Earth Science 5-12 (minor only) N too small to report
5911 Economics N too small to report
5931 Government/Political Science N too small to report
5652 Computer Science N too small to report

This data was last updated 7/26/2023.

Unlike undergraduates whose program includes their subject area content knowledge, graduate students in the Master of Arts in Teaching program must already possess that knowledge. They take their Praxis content knowlege tests as part of their admission process to the program. Elementary level candidates take the four-part Elementary Education: Multiple Subjects test. Secondary level candidate take the content knowledge test appropriate to their subject area.

 

Praxis Test of Content Knowledge Scores
Master of Arts in Teaching
Praxis
Test
Code

 

Praxis Test

 

N

 

MSU Mean Score

 

US Median

5002 Elementary Education: Reading and Language Arts Subtest 28 179 170
5003 Elementary Education: Mathematics Subtest 28 182 172
5004 Elementary Education: Social Studies Subtest 28 184 166
5005 Elementary Education: Science Subtest 28 183 169
5038 English Language Arts 6 187 178
5081 Social Studies 16 172 167
5161 Mathematics N too small to report
5435 General Science 13 175 161

This data was last updated 7/26/2023.

Teacher candidates who wish to graduate and attain licensure must reach competency standards during student teaching for the Danielson Performance Assessment through evaluation by both their field supervisor and their cooperating classroom teacher. Content knowledge is measured through a Content Area Assessment of proficiency (evaluated by their cooperating teacher), GPA, and performance on the Praxis Exam (an external evaluation administered and assessed by the Office of Public Instruction) collectively labeled the MACK (Montana Assessment of Content Knowledge).

The Danielson Tool

Both field supervisors and cooperating teachers evaluate pre-service teachers using this performance assessment at the halfway point and at the end of student teaching. Any unsatisfactory score (a score of less than 2 on the rubric) assigned by either the field supervisor or the cooperating teacher at the end of student teaching may result in the pre-service teacher needing to re-take student teaching. 

Praxis Exam

During the review cycle, each graduating UG PST was required to complete the Praxis Exam, an external exam that assesses PST content knowledge, administered and assessed through the Montana Office of Public Instruction. PSTs seeking a K-8 or P-3 licensure have content questions related to Math, Language Arts, Social Studies and Science, while those seeking K-12/5-12 licensure complete the exam designated for their specific content area. For example, a UG PST seeking Secondary Science Broadfield licensure will complete an exam containing questions in the fields of Biology, Chemistry, Physics and Earth Science. PSTs in the MAT are initially evaluated for subject matter knowledge through completion of the appropriate content area PRAXIS exam. (See scores under Measure 3 on this page.)

The MACK

The Montana Assessment of Content Knowledge (MACK) is a single score based on content area GPA, a content area assessment during Student Teaching/clinical practice, and the Praxis content knowledge score. Points are assigned to each area.

Completer Endorsements

In the 2021-22 academic year, 142 individuals completed and graduated from the MSU-Bozeman undergraduate educator preparation program. Of these, 101 were prepared to teach in a single subject area, 22 prepared to teach in two areas, and 19 prepared to teach in three areas. It is important to note that completers often choose not to go directly into teaching following graduation (see table below) and so may not seek licensure until a later date, and some requests for licensure are in process. As of July 18, 2023, the MSU Field Placement Office has recommended 81 of these individuals to teach in one or more of the areas for which they were prepared. Of those, 75 were recommended for licensure in Montana, and 11 were recommended for licensure in other states. Similar situations exist for the graduate Master of Arts in Teaching program as well as the advanced Principal and Superintendent programs.

 

Educator Preparation Program Undergraduate Completers
Data as of July 18, 2023
Academic Year Number of Completers Number Recommended
by Field Placement 
Recommended
in Montana
Recommended
Out of State
2022-23 133    74    73 4
2021-22 142    81    75 11
2020-21 172    140    138 11

 

Master of Arts in Teaching Program Graduate Completers - Elementary
Data as of July 18, 2023
Academic Year Number of Completers Number Recommended
by Field Placement 
Recommended
in Montana
Recommended
Out of State
2022-23 11 3 2 1
2021-22 9 6 6 0
2020-21 11 9 8 1

 

Master of Arts in Teaching Program Graduate Completers - Secondary
Data as of July 18, 2023
Academic Year Number of Completers Number Recommended
by Field Placement 
Recommended
in Montana
Recommended
Out of State
2022-23 10 2 2 0
2021-22 10 6 6 0
2020-21 9 9 8 1

These are the CAEP Accountability Measures for the undergraduate and graduate initial teacher licensure programs. We have also posted our CAEP Accountability Measures for the Educational Leadership program.