Liberal Studies Student Learning Outcome Assessment Report

 

Course Title:                                                     LS301 – Integrative Seminar

Author of Report:                                           Bridget Kevane

Outcome Being Assessed:                           Written Reflective Communication

Semester and Year                                        Spring 22

Course Enrollment:                                       22

Number of Course Sections:                       2

Number of Assignments Assessed:          6 (27%)

 

Assessment Team:

Kevane, Bridget; Schweppe, Nina; Storment, Jennifer; Denny, Jaclyn (forth reader)

 

Method of Selecting Student Work:

Instructors used a random number generator online and generated 3 numbers at random, per class.  The numbers generated told them which artifact to select, by order submitted in Brightspace D2L.

 

Method of Ensuring Inter-rater Reliability:

Student work was sent to Program Manager, Jennifer Storment, who removed identifying information and assigned an artifact number to each.  Three assessors, identified as 1 – 3 by Jennifer, assessed each individually.  Jaclyn Denny was identified as a fourth reader, to have on standby should any of the scores differ by more than a point; in such a case, Jaclyn would have read the artifact, scored it, and help remove the outlier.  Having three assessors plus a fourth reader helped prevent scoring discrepancies and allowed for an identification of scoring outliers, if present. 

 

Notes about Scoring:

In no case was there more than a 1 point difference in the score of the three assessors.  If any score differed by more than 1 point, however, the forth reader could have read and scored the artifact so that the outlier could have been removed.  This allowed for a more accurate measure of overall student performance. Students in LS 301 are doing well and 100% of papers met at least the criteria for “Meets Expectations” while 33.3% met the criteria for “Exceeds expectations.”

 

Artifacts on expectations scale

Below Expectations

0

0.0%

Meets Expectations

4

66.7%

Exceeds Expectations

2

33.3%

 

 

Assessments (rubric attached):

 

Exceeds Expectations= 4 - 5

Meets Expectations =   3 - 4

Below Expectations = 1 - 2

 

 

Assessor Scores, by artifact

        

Artifact #

Assessor

Content & Development

Organization

Language

Conventions

Average

1

Assessor 1

4

4

3

4

3.75
 

Assessor 2

4

4

3

4

3.75
 

Assessor 3

4

4

3.5

4

3.88
 

Avg:

4

4

3.17

4

3.79

2

Assessor 1

4

4

3

3

3.50
 

Assessor 2

3

3

3

3

3.00
 

Assessor 3

4

4

3

4

3.75
 

Avg:

3.67

3.67

3

3.33

3.42

3

Assessor 1

5

5

4

5

4.75
 

Assessor 2

4.5

4.5

4.5

4.5

4.50
 

Assessor 3

5

4

5

5

4.75
 

Avg:

4.83

4.5

4.5

4.83

4.67

4

Assessor 1

5

4

3

3

3.75
 

Assessor 2

4

5

4

3

4.00
 

Assessor 3

4.5

4

4

4

4.13
 

Avg:

4.5

4.33

3.67

3.33

3.96

5

Assessor 1

5

5

5

5

5.00
 

Assessor 2

5

5

5

5

5.00
 

Assessor 3

5

4.5

4.5

4.5

4.63
 

Avg:

5

4.83

4.83

4.83

4.88

6

Assessor 1

5

4

3

3

3.75
 

Assessor 2

4

3.5

3.5

3.5

3.63
 

Assessor 3

5

4

4

4

4.25
 

Avg:

4.67

3.83

3.5

3.5

3.88
 

Avg:

4.44

4.19

3.78

3.97

4.10

 

 

Average Scores, by Artifact

Artifact Evaluated

Content & Dev.

Organization

Language

Conventions

Artifact Average

1

4.00

4.00

3.17

4.00

3.79

2

3.67

3.67

3.00

3.33

3.42

3

4.83

4.50

4.50

4.83

4.67

4

4.50

4.33

3.67

3.33

3.96

5

5.00

4.83

4.83

4.83

4.88

6

4.67

3.83

3.50

3.50

3.88

Overall Avg.

4.44

4.19

3.78

3.97

4.10

Score by indicator-5.0 scale

88.89%

83.89%

75.56%

79.44%

81.94%

 

 

Percentage of Indicators on Expectations Scale

  

 

Content & Dev.

Organization

Language

Conventions

Below Expectations

0.0%

0.0%

0.0%

0.0%

Meets Expectations

16.7%

33.3%

66.7%

50.0%

Exceeds Expectations

83.3%

66.7%

33.3%

50.0%

 

 

Results:

Criteria

Below Expectations

Meets Expectations

Exceeds Expectations

Content & Dev.

0.0%

16.7%

83.3%

Organization

0.0%

33.3%

66.7%

Language

0.0%

66.7%

33.3%

Conventions

0.0%

50.0%

50.0%

 

100% of artifacts were in the “Meets Expectations” or above for each indicating criteria.  The areas ranked the highest are in “Content and Development”, where 83.3% of artifacts exceeded expectations, and in “Organization”, where 66.7% of artifacts exceeded expectations.  “Language” and “Conventions” were also still fairly high, at 33.3% and 50% exceeding expectations. 

 

Recommendations for LS 301 and the Liberal Studies Program:

The overall scores were excellent for LS 301 artifacts.  More than a quarter of the LS 301 student artifacts were assessed, allowing for a strong sampling.  Because all of the scores met or exceeded expectations, there are not obvious areas of improvement that should be focused on moving forward.  Results will be shared with LS 301 instructors along with the Liberal Studies Core Team, to start a conversation on what is going well, and to also determine as a team if revisions should be considered, or if a revised rubric should be in place.

 

Reviewer (circle one):   1     2     3     4

Artifact #: ____________

 

Level of Achievement

Score

Indicators of Effective Writing

1

2*

3

4**

5

 

Content & Development: ideas, reflections, examples, reasons & evidence, & point of view

Topic/reflection is poorly developed, support is only vague or general; ideas are trite; wording is unclear, simplistic; reflects lack of understanding of topic and audience; minimally accomplishes goals of the assignment

 

Topic/reflection is evident; some supporting detail; wording is generally clear; reflects understanding of topic and audience; generally accomplishes goals of the assignment

 

Thesis topic/reflection idea is clearly stated and well developed; details/wording is accurate, specific, appropriate for the topic & audience with no digressions; evidence of effective, clear thinking; completely accomplishes the goals of the assignment

 

Organization: focus, coherence, progression of ideas, & developed thesis

Disorganized and unfocused; serious problems with coherence and progression of ideas; weak or non- existent thesis

 

Generally organized & focused, demonstrating coherence & progression of ideas; presents a thesis and suggests a plan of development that is mostly carried out

 

Clearly focused and organized around a central theme; thesis presented or implied with noticeable coherence; provides specific & accurate support

 

Language: word choice & sentence variety

Displays frequent & fundamental errors in vocabulary; repetitive words and sentence types; sentences may be simplistic and disjointed

 

Competent use of language and sometimes varies sentence structure; generally focused

 

Choice of language & sentence structure is precise & purposeful, demonstrating a command of language and variety of sentence structures

 

Conventions: grammar, punctuation, spelling, paragraphing, format; (as applicable) documentation

Errors interfere with writer’s ability to consistently communicate purpose; pervasive mechanical errors obscure meaning; inappropriate format; in text and ending documentation are generally inconsistent and incomplete; cited information is not incorporated into the document

 

Occasional errors do not interfere with writer’s ability to communicate purpose; generally appropriate format; in text and ending documentation are generally clear, consistent, and complete; cited information is somewhat incorporated into the document

 

Control of conventions contribute to the writer’s ability to communicate purpose; free of most mechanical errors; appropriate format; In text and ending documentation are clear, consistent, and complete; cited information is incorporated effectively into the document

 

 

 

 

 

 

Total

 

Rubric for the Assessment of Written Reflective Communication

*Exhibits most characteristics of ‘1’ and some of ‘3’; **Exhibits most characteristics of ‘3’ and some of ‘5’ 

 

*********************************************

 

Research (R) Core Student Learning Outcome Assessment Report

 

Course Title:                                                  LS 490R: Senior Capstone

 

Author of Report:                                         Bridget Kevane

 

Outcome Being Assessed:                         Research

 

Semester and Year                                      Spring 2021

           

Course Enrollment:                                      28

 

Number of Course Sections:                      2 Spring Sections

 

Number of Assignments Assessed:           8 (28.6%)

 

Assessment Team:                          

Bridget Kevane, Director; Jennifer Storment, Program Manager; Sarah Coletta-Flynn, Online Program Manager/LS Instructor; John Townsend-Mehler, Online Program Advisor/LS Instructor

 

Method of Selecting Student Work:

The Liberal Studies director requested that each of the two instructors submit four randomly chosen papers, totaling eight in all. Randomization for these artifacts was done by looking at the order in which papers were submitted, assigning a number.  I.e. The first paper submitted was “1”, the tenth paper was “10”, and so on and so forth.  From there each instructor used a random number generator website to come up with 4 randomly chosen numbers for each section to determine which papers to submit, which were then renumbered as artifacts 1 – 8.  The assignment assessed was their final capstone paper. 

 

Method of Ensuring Inter-rater Reliability:

The assessment team met and discussed the rubric and expectations for different rankings of students’ final capstone papers. Because the two instructors also serve on our assessment committee, papers were distributed so that no one assessed their own students’ work, to maintain objectivity.  No one on the assessment team was able to see other assessors’ rankings, and rankings were not recorded until all had submitted their completed rubrics, to prevent scoring bias.    

 

There were only three instances out of 40 where the score varied by more than 1 point on the same paper in the same scoring category.  In these instances, those scores were thrown out as outliers and not calculated into any averages or overall scores.  This allowed for a more consistent measure of overall student performance.

 

Notes about Scoring:

For each category, rubric scores were coded as “Exemplary” (4), “Good” (3), “Acceptable” (2), and “Unacceptable” (1). Assigning values to the rankings with 4 being the highest score, and 1 being the lowest, allowed us to monitor averages and more easily compare and contrast final results.  See the results tables below for reference.    

  

Results:

 

Spring 2021 Research Core Scoring Results\

R-Core Assessment-LS 490R Capstone

        

Exemplary = 4

 

          

Good = 3

 

          

Acceptable = 2

 

          

Unacceptable = 1

 

          
             

 
               

Paper #

Assessor

Well-Defined Research Q/ Purpose

Context

Use of References

Quality of References

Consideration of Alt. Perspectives

Paper Average, 4.0 scale

1

Assessor 1 (BK)

4

4

4

4

2

3.33
 

Assessor 3 (JTM)

2

2

3

3

2

2.67
 

Avg:

 

 

3.5

3.5

2

3

2

Assessor 1 (BK)

3

3

3

4

2

3.00
 

Assessor 3 (JTM)

2

3

2

3

1

2.20
 

Avg:

2.5

3

2.5

3.5

1.5

2.6

3

Assessor 1 (BK)

4

4

4

4

3

3.75
 

Assessor 3 (JTM)

2

3

4

3

3

3.25
 

Avg:

 

3.5

4

3.5

3

3.5

4

Assessor 1 (BK)

4

3

4

4

2

3.40
 

Assessor 3 (JTM)

3

4

4

4

3

3.60
 

Avg:

3.5

3.5

4

4

2.5

3.5
 

1-4 Avg:

3

2.5

3.5

3.625

2.25

3.15

5

Assessor 2 (JS)

3

3

3

3

2

2.80
 

Assessor 4 (SCF)

3

2

3

3

2

2.60
 

Avg:

3

2.5

3

3

2

2.7

6

Assessor 2 (JS)

3

3

3

3

2

2.80
 

Assessor 4 (SCF)

2

2

3

2

2

2.20
 

Avg:

2.5

2.5

3

2.5

2

2.5

7

Assessor 2 (JS)

3

4

4

3

2

3.20
 

Assessor 4 (SCF)

3

3

3

3

2

2.80
 

Avg:

3

3.5

3.5

3

2

3

8

Assessor 2 (JS)

2

2

2

3

2

2.20
 

Assessor 4 (SCF)

2

2

2

3

2

2.20
 

Avg:

2

2

2

3

2

2.2
 

5-8 Avg:

2.625

2.625

2.875

2.875

2

2.6
 

Overall Average, Papers 1 - 8

2.8125

2.5625

3.1875

3.25

2.125

2.875
                         

 

   

Research Q/ Purpose

%

Context

%

Use of References

%

Quality of References

%

Consideration of Alt. Perspectives

%

Total Unacceptable (1)

0

0%

0

0%

0

0%

0

0%

1

6%

Total Acceptable (2)

4

33%

5

31%

3

19%

1

6%

12

75%

Total Good (3)

7

58%

7

44%

7

44%

10

63%

3

19%

Total Exemplary (4's)

1

8%

4

25%

6

38%

5

31%

0

0%
   

*12

100%

16

100%

16

100%

16

100%

16

100%
   

*4 outlier scores thrown out)

                  

 

What was learned for LS490R Capstone:

We were pleased that all artifacts were scored at or above an acceptable score (2.0+ on a 4.0 scale), most scores averaging around a good score, or 3.0.  We will discuss the results as a Liberal Studies team, to include our capstone instructors, to determine if any areas require additional focus and/or revision.