Fiscal Year 1997-1998
General Education
Definition: While the general education curriculum is preparation for study in advanced baccalaureate courses, the content of a general education curriculum also "stands alone" as it gives essential skills and knowledge to citizens involving themselves responsibly in a sophisticated, democratic society. Therefore, general education as defined here is the set of transferable skills related to effective communication and analytical and computational ability that, along with the basic philosophical, historical, aesthetic and scientific knowledge base, enable the educated person to understand and judge human experiences and to live as a responsible citizen.
Indicators: Two years ago (1995-96), USCB reported a comprehensive assessment of General Education and Student Development on campus which included the following instrumentation:
As discussed in that report, the exercise was as much an analysis of the instrumentation and the process as an evaluation of USCB's effectiveness at engendering General Education skills in its students.
Since 1996, we have continued to assess the validity of different instruments as we come to terms with what it means to be educated and what feedback is precise enough to direct curricular change.
Indicators:
As it turned out, while the survey results could be compared to earlier assessment results
along general groupings of general education skills (e.g. "oral communication"), the
specific skills within these were not always comparable to measures in 1996 and the survey.
The survey was self-report. Even knowing the problems inherent in that, data on many of
the questions was ultimately dismissed when faculty struggled to make meaningful
interpretations. Self-report systems simply will not be accepted by many on this campus
now. The survey served a heuristic purpose of focusing students on the objectives of college.
All in all, the results and reception of the survey proved disappointing and suggested a
major mid-course correction in measurement technique. However, the silver lining is that
the sudden requirement for General Education reporting this spring caused us to look
summatively at this method and the data and make a change in instruments. Also, the in
depth analysis of these components of general education continues to help less-initiated
faculty come to see the need for assessment of institutional aims greater than the sum of its
parts.
Assessment Methodology:
In 1996, USCB like other USC campuses reported on General Education assessment findings. Since this key component of our program is so difficult to define, let alone measure, the assessment of this was (and is) a metaanalysis of the general education assessment process. Thus, much of the report of results of findings related to the instruments as well as what the instruments tell us about our program and our students. However, where consistencies in the results of the two administrations seem apparent, these are addressed here.
Written Communication
Previously, in 1996 and in concert with other campuses of USC, USCB administered a writing assessment using a prompt designed at USC for rising juniors. Judging were one USCB faculty member and 2 external faculty. Papers were judged simply yes-or-no on a series of competencies -percentages show those meeting each competency:
Criteria USCB Professor: USC Professor: SchoolTeacher: Composite
Readerl Reader2 Reader3
1. Addresses topic/
shows knowledge 77% 100% 94% 90%
Demonstrates
Awareness of reader 54% 65% 100% 73%
3. Reflects
Appropriate format 77% 76% 53% 69%
4. Reflects
Appropriate format 38% 88% 71% 66%
5. Relates ideas
23% 94% 53% 57%
6. Style Personal
Voice Coherence 92% 94% 82% 89%
Comprehension,
Analysis, Synthesis 85% 71% 88% 81%
131
There appeared to be discrepancy between judges ("inter-rater reliability problems") in criteria 4 and 5. These two similar criteria were deemed -problematic even before comparison of the readers' reports. (e.g., the USC external reader reported being consciously'loo lenient" on students on these items because the writing prompt specified a 3-paragraph structure. Dr. Tombe also believes the problem lies not with the grading rubrics but instead with the writing prompt, which constricts student writing by imposing the structure.)
Since 1996: The survey given over the last three semesters revealed little discrimination between accomplishment of various writing skills. The following shows about students perception ... My college experience at the University of South Carolina Beaufort has strengthened my ability to:
# Variable N N* Av.
(1 =Strongly Agree
2=Agree,
3=Disagree,
4=Strongly Disagree)
40 Write Sentences 249 36 1.747 Write using good sentence
structure.
31 Write Well 252 33 1.75 Write using correct grammar and
punctuation.
18 Demonstrate 251 34 1.7968 Demonstrate my creativity
through writing.
7 Write Logically 268 17 1.8172 Write using logical organization.
N*=number of students who skipped or marked "not applicable" to this item
The survey does little toward discrimination among various components of writing skill. While we might make something of the fact that 'logical organization" is what students self-reported as a relative weakness in the new survey and the judges deemed this a relatively weaker skill in 1996, we stand on shaky ground in making any conclusions about strengths or weaknesses. We certainly cannot pinpoint a specific area needing to be addressed by a change in the curriculum.
Oral Communication
Previously, USCB assessed oral communication with the help of an extensive general education survey developed by the USC General Education Assessment Advisory Committee. This reports responses from Students Completing Associate Degree At USC Beaufort to the question
My college experience at USC Beaufort has developed or improved my ability to speak with:
Criteria Strongly Agree Disagree Strongly
Agree Disagree
8. Knowledge of the subject/topic. 39% 56% 6% 0%
9. Awareness of the audience/group. 0%
10. Organization appropriate to the purpose and to the 44% 39% 11% 6%
interaction between speaker and audience.
11. Vocal delivery which encourages listening. 50% 39% 11% 0%
12. Physical presentation (use of body) appropriate to 50% 28% 22% 0%
the speaking situation.
13. Appropriate sentence structure and word choice. 39% 56% 6% 0%
14. Improved my ability to listen and extract information 33% 67% 0% 0%
and meaning from oral communication.
Since 1996: The results this year showed the following responses regarding oral communication aspects of instruction at USCB: My college experience at the University of South Carolina Beaufort has strengthened my ability to:
# Variable N N* Average
19 LISTEN 267 18 1.7116 Listen more effectively.
24 SPEAK WELL 259 26 1.7259 Speak more confidently.
41 SPEAK WELL 255 30 1.7569 Speak in an organized and
understandable manner.
32 SHA EIDEAS 262 23 1.7786 Share ideas appropriately.
14 SPEAK CORRECTLY 252 33 1.8294 Speak using correct grammar.
N*=number of students who skipped or marked "not applicable" to this item
Relative to areas such as computer literacy and some specific content areas, students are especially positive about USC Beaufort's effectiveness in this area.
Use of Assessment Findings: Students appear to think USCB does a very good job at developing oral communication skills! Analyzing more specifically, while it appears again that there is some lack of comparability between the 2 reporting cycles, (e.g., students rated "grammar" as a relative weakness in one instrument but no parallel item is found in the other) we might, however, consider possible consistencies over the two report periods which appear somewhat plausible in "listening" and "organization" items when we receive future feedback in this area. However, again, these results will not lead to immediate change in the curriculum.
Student Development
Previously, the surprising finding was the degree to which students attributed developmental growth to USCB. Perhaps not-so-surprising but just as striking was the degree to which they felt this growth resulted from in-class activities on this non-residential campus.
It appears that "community involvement" is perhaps a weakness that might require further investigation. With the exception of "raising a family," which is arguably an uneasy inclusion in a list of developmental items shaped by college, the following were areas where no development took place at USCB:
"Community Involvement" (48.5% said "change attributed to college") "in touch wl Feelings" (50.8%); "Autonomy" (50.8%); "Managing Emotions" (52.3%); and "Developing Integrity" (60.9% -- 44% of students felt USCB contribute some to this area and an additional 17% said USCB contributed greatly.)
It is not surprising that the topmost items dealt more with cognitive aspects of development: Intellectual Competence; Critical Thinking Skills; Knowledge of Your Field; Understanding Abilities; Positive Selfconcept (which ranged from 97% to 82% of gain attributed to college.)
Since 1996: The survey administered over the last three semesters, again, showed little:
# Variable N N* Average
3 441ndependent 261 24 1.6973 Work in an independent and self-directed manner.
5 6 Consider 269 16 1.7175 Consider ideas different from mine.
Ideas
6 17 Set Goals 261 24 1.7395 Set goals for myself and pursue them.
5 21 Open to 270 15 1.7407 Be more open to new ideas.
Ideas
1 38 Self-Worth 248 37 1.746 Appreciate my self-worth, talents, and potential for
success.
5 30 Appreciation 258 27 1.7674 Appreciate the values and beliefs of others.
5 35 Be Fair 254 3' 1.7953 Be Fair with a variety of other people.
1 22 Interact 264 21 1.8144 Interact in small groups.
3 13 Accountable 241 44 1.8216 Be accountable for my actions
7 5 Keep Choices 257 28 1.856 Keep my choices/behavior consistent with sound moral
principles.
7 23 Moral 243 42 1.8683 Recognize moral conflicts and identify means of
resolving them.
7 9 Define Values 254 31 1.8858 Define my own values and beliefs.
1 26 Environment 221 64 2.0136 Enhanced my concern for the environment.
N*=number of students who skipped or marked "not applicable" to this item
It may be noteworthy that public awareness/activism items were the "outliers" in both assessments - community participation in the first and environmental concern in the latter. But again, it is easy to make too much of this. Public conscience issues, though, should be a primary focus of a liberal arts education and these findings indicate that this may be an area for future concentrated assessment.
Critical Thinking
In 1996 -- and again in 1997-- USCB faculty observed roundtable sessions where students answered (as a group) questions requiring critical thinking. (Answering as a group required students thus to "think out loud." ) As with other aspects of general education assessment, the simulation was arranged to involve faculty as much as possible in our continued emphasis on "closing the loop." Faculty, of course, are the ones Who will take any meaningful action on assessment findings.
Assessment Methodology: A provocative situation was presented to set up the critical thinking exercise. This exercise was part of a Pre-graduation workshop to which all Associate Degree candidates were invited. The 18 graduating associate-degree students participating in this exercise were asked to become policy analysts and choose beneficial initiatives from a list of 12 laws currently under consideration. 10 USCB faculty presented background information on each public policy proposed. First, a faculty member offered a rule-of-thumb on determining a proposed law's constitutionally. Then, other faculty members shed light from their academic specialties on the 12 public policy issues. After this, students prepared their 1 -page reports as groups, observed by faculty. The report included prompts asking for demonstration of all items of Bloom's Taxonomy.
Since 1996: The second year, instead of a simulation, students attended a speech by a popular actor and author, Greg Alan Williams. Faculty observed as students responded, as groups, to a series of prompts on a critical evaluation sheet.
Neither year resulted in operational feedback, though faculty got an "eye full" of general impressions about our students' critical thinking abilities. While the faculty were generally gratified at the students' level of thinking, seeing a student's whole cognitive background brought to bear on analyzing problems encouraged faculty to focus on the total institutional output in new ways.
Replacing the simulation this year is data from the General Education (self-report) survey administered over the last three semesters. Survey results:
# Variable N N* Average
8 Apply New 272 13 1.6949 Apply new knowledge.
29 Solve Problems 256 29 1,7734 Intelligently solve problems.
4 Reach 271 14 1.7749 Reach my own conclusions after evaluating
Conclusions information.
15 Reasoning/Thinkin 268 17 1.8022 Reason and think logically.
q
10 Think New 267 18 1.8052 Think in new ways about familiar subjects.
39 New Ways 258 27 1.8217 Create new ways to solve problems.
27 Original 260 25 1.8269 Originate new ideas on my own.
1 Find/Evaluate 269 16 1,8439 Find, evaluate, and organize information logical
N*=number of students who skipped or marked "not applicable" to this item
Findings: ROUNDTABLES: Since the student groups in the round table exercises demonstrated different skill levels, generalizing is difficult. However, faculty commented the first year that while students were competent at handling activities requiring analysis, it appeared to faculty that these students seemed to have the most difficulty with Synthesis and Evaluation. Last year's evaluation (roundtable) exercise also yielded expression of faculty concerns about synthesis skills.
This year's survey data suggested that students believe USCB helps them think better, but the data did not go far in discriminating specific strengths or weaknesses among forms of critical thinking. Not only was variability low, the survey questions did not seem to follow a taxonomy and the 1 st question, 'FINDEVAL", did not address a discrete competency, but rather three orthogonal issues, one of which was also addressed in #4 REACHCON.
Knowledge, Traditional Disciplines, and Interdisciplinary Perspectives:
Definition: This category includes math and science, social science, and humanities/cultural studies. With the possible exception of computer literacy, content knowledge associated with these disciplinary areas have wide-ranging application, which are different from those skills in the category of "transferable" skills (writing, critical thinking, speaking, etc.).
Previous Assessment Findings: With the exception of the "Humanities/Cultural Studies" components, USC Beaufort students overwhelmingly stated that their coursework built up their abilities along general education objectives. Overwhelmingly high marks for success in Math, Science, and Social Science objectives were surprising even for those of us most positive about our program. Update Status: Again, conclusions cannot be made given the lack of variability between the items. Discipline Specific Content (Average not meaningful since this is not a coherent factor.)
135
# Variable N N* Average
20 Professional 262 23 1.7328 Develop the knowledge and skills required for
professional growth.
36 Humanities 237 48 1.7553 Recognize the role literature, history, philosophy and
the arts have played in shaping cultures.
11 Relate Culturally 253 32 1.7905 Relate to people from different cultural backgrounds.
43 Community 238 47 1.8571 Utilize my abilities, resources & talents in community
Activities activities.
42 Math/Statistical 229 56 1.8996 Use mathematical and statistical tools to analyze
information.
34 Understand 225 60 1.92 Better understanding of political issues.
Politics
12 International 237 48 1.9283 Better understanding of political issues.
3 Understanding 231 54 1.9307 Understand the contribution of science & technology
to society.
2 Intellectually 237 48 2.0042 Intellectually solve mathematical problems.
N*=number of students who skipped or marked "not applicable" to this item
The lack of definitive findings is especially frustrating here. These are skills and abilities at the core of liberal arts. It would be very useful to know, for example, if students would like to see greater emphasis on political issues or more grounding in mathematics. The criticism of the survey by the faculty representatives is just. However, the process has helped us see that a method such as this survey which can compare strengths of disparate content areas - but with credibility -may be very valuable. (eg, it may suggest interdisciplinary reinforcement of certain competencies.)
Computer Literacy
Previously, the conclusion of the general education assessment was that students felt less secure about their computer training than any other area (with the exception of several humanities items related to specific skills in foreign language and art.) However, at the time this survey was taken, USCB was a different place than it is today. Increasingly, faculty are requiring students to use programs such as Mini Tabs and DISCUS on-line data searches. And the expectation for presentation of papers is rising. In any case, previously, the USC survey results showed students in disagreement with the statement
My college experience at the USC Beaufort has developed or improved my ability to use computers to:
Strongly Agree Disagree Strongly Criteria
Aaree Disaqree
25% 42% 25% 9% 15. Create, edit, and revise written texts.
14% 42% 32% 12% 16. Analyze quantitative data.
20% 42% 24% 14% 17. Access information and databases.
16% 38% 33% 14% 18. Integrate graphical, visual and statistical information in
to written presentations.
14% 27% 41% 17% 19. Send and receive electronic communication.
Update Status: This year, the three "computers' items were again among the last five out of all survey questions. While there was not specific comparability between the 1996 measure and the current one, it is clear that in both sets of evaluations, students felt least well about their training in computers at USCB. But, as stated earlier, faculty are increasingly incorporating computer use into their syllabi.
# Variable N N* Average
37 Computer 211 74 1.9005 Recognize and accept new computer technology
33 Computer 204 81 1,9951 Employ computer technology to more efficiently
accomplish tasks.
25 Computer 206 79 2.0388 Develop the computer knowledge and skills required
for professional growth.
N*=number of students who skipped or marked "not applicable" to this item
If the campus is indeed weak in this area, several issues must be considered. As one USCB faculty member expressed, students may not know what they expect to learn about new technologies and may have hopes unrealistic for 2-year colleges. However, USCB and higher education in general is responding to such expectations. Certainly, the PC revolution is still in its infancy (especially if compared other revolutions in disciplinary areas for which the University has assumed responsibility.) And while perhaps not precisely current with aspirations, higher education is investing heavily. USCB has routinely invested the equivalent of four faculty salaries each year to support and upgrade its computers and infrastructure.
In this time of transition, USCB like many other liberal arts colleges, is trying to meet technological needs of all constituencies and it changes its spectrum of responsibilities and assumes new roles. USCB will continue to solicit feedback from all constituencies as it responds to changing expectations for itself in this burgeoning skill area.
Overall Result of this Assessment:
The status of assessment at USCB is in dynamic transition. We believe we are entering the phase of acceptance (and we can hope even embrace) of the assessment process. The reaction itself shows perhaps some minor paradigm shifts taking place on campus. What are these shifts? We HOPE faculty are coming to see that:
1 . The end result of a liberal arls education is greater than the sum of its parts. (General education
goals require different approaches than do the discreet objectives taught in individual classes.)
2. The contribution of the individual classroom to the whole must be carefully considered, even to the
point of breaking down the broad goals to attribute contribution of each core course.
3. Responsibility for one's contribution to the Institutional goals must be shouldered.
4. Faculty must find and adopt meaningful assessment techniques.
Faculty resistance to using current results to change curricula shows they are conceding to all elements (e.g. "closing the loop") of the assessment cycle. This faculty "investment" into the process and the findings of assessment is essential if general education is going to be adequately operationalized, observed, analyzed, evaluated and its results synthesized into our improvement efforts. We believe this process has brought about additional faculty installments on that investment, as indicated by increased expression and interest and commitment to meaningful institutional assessment.
The faculty have been and are engaged in assessing USCB's ability to provide students with positive, meaningful growth in general education and developmental objectives. While it seems this campus is engaged in reacting to recent methods, key faculty are feeling a greater sense of responsibility for helping make assessment meaningful. And while we cannot tie all changes to a documented assessment loop, changes have occurred: Certainly improvements in the computer program have been the most visible, but faculty are continuing to incorporate new teaching techniques and materials to improve general education competencies, particularly critical thinking and synthesis of information and ideas.