|
|
|
The views expressed on this page are
solely
those of the author and do not
necessarily represent the views of County
News Online
|
|
GETTY IMAGES PLUS
Inside Higher Ed
Creating Rich Transcripts for Career Activation
Institutions should be embarrassed by the standard transcripts they
have been issuing, unchanged for a century, and students should demand
better, argues By Fred Cutler
January 20, 2021
Around the world, many people have questioned or criticized the
impoverished traditional transcript. A 2017 report from the Higher
Education Quality Council of Ontario reported, for example, that the
“Current credential and accreditation system does not serve students
well.” The fact is that institutions should be embarrassed by the
standard transcripts they have been issuing, unchanged for a century,
and students should demand better.
Some colleges have launched programs to revise transcripts so they
represent students’ co- and extracurricular experiences, attaching to
the standard transcript some electronically provisioned add-on that
shows students’ activities outside their coursework. Unlike the
traditional transcript, where most courses are titled in terms of their
content, such supplements often emphasize skills. Other tweaks on the
standard transcript have supplemented the abbreviated course titles
with some other information or, in the case of electronic transcripts,
links to student production or a course website.
In alignment with those efforts and other broader initiatives to assist
students in the transition to careers, the political science department
at the University of British Columbia, where I am a professor, has
worked to develop for its graduates a supplementary “rich transcript”
that includes:
The student’s courses’ full titles;
A word cloud built from the instructors’ detailed course descriptions for their courses (not the generic calendar descriptions);
Aggregated statistics for each student on the number of writing
assignments, pages written, peer reviews, oral presentations, hours of
group work, research designs, primary research, internships and service
learning; and
A list of 23 skills showing in how many of the student’s courses each skill was a key learning outcome.
Every student deserves this kind of report on their learning and skill
development. In this article, I describe the five-year process that
produced the rich transcripts for a pilot project for graduates of one
major at my university in hopes it can inform others to consider how
they might improve their students’ transcripts, as well.
In our case, it all began when I started thinking about university
transcripts and wondered if students could be given a better summary of
their learning experiences. Those transcripts had the official stamp of
the institution, some overabbreviated course names and, of course,
grades -- and nothing more. Not a great parting gift for students who
had worked so hard, had a variety of learning experiences and developed
a wide range of skills.
Fortunately, it was around this same time that the faculty of arts at
the university was defining program learning outcomes and building a
data mart including enrollment and course information. In addition, the
career office was looking for more tools to help students articulate
the skills they develop pursuing their degrees.
So the time seemed right to propose a pilot project: the Course
Characteristics Census. The idea was to inventory our courses to find
out how the instructor had designed the learning, what students were
being asked to do in the course and what skills those instructors
intended students to develop. Among the many benefits, we imagined that
the course data could be aggregated by student to summarize each
person’s learning experiences, academic output and skill development.
Gathering the Data: A Course Inventory
We started by examining how we might best collect the relevant
information about each course. The natural place to start was to gather
syllabi and code them. The dean’s office and other department heads
suggested that approach because they thought the syllabi would contain
a clear indication of the learning activities and skills to be
developed in the class. Moreover, faculty members wouldn’t have to be
asked to do anything.
But it soon became clear that the syllabi were extremely varied, with
only a few providing the information required, especially on learning
objectives in terms of skill development. Some were even inaccessible
because departments had not been required or encouraged to archive
their syllabi, and some former instructors were unreachable.
We decided to try a different approach to gather a broader set of, and
more reliable, data. In consultation with experts on pedagogy and
assessment and, of course, the department itself, we built a
questionnaire to gather detailed information on characteristics of
courses: learning outcomes defined as skills, teaching modalities,
learning activity structure, assessments, work time expectations,
technology use and special features like community service learning or
primary research.
But unlike an ordinary internal university survey that might have value
with, say, a 20 percent response rate, we realized we would need a 100
percent response rate if we were going to be able to aggregate the
course information for each student covering all of their learning in
the political science major, without any gaps. In other words, we would
need to conduct a census, not a survey.
The final questionnaire typically took 20 minutes per instructor per
course. Obviously, we didn’t ask instructors to repeat it if they had
not significantly changed their course design from year to year. Still,
it was a tremendous challenge getting information on every course
taught in the department by full-time and part-time faculty over the
course of four years. We only succeeded because we had the strong
participation of the department chair and I was able to twist my
colleagues’ arms in a way that only a colleague can do. Even then, we
were missing a few courses, particularly those that sessional lecturers
and postdocs taught.
An academic unit trying to gather this information should consider the
time lag from beginning to collect the data to the point when the rich
transcript can be issued for the first time. It is theoretically
possible to start by trying to gather the data back five years to cover
the course history of the students about to graduate. In our case, we
gathered it over four years and then did the data processing and issued
these rich transcripts. Even then, at the end of those four years, we
had to go back to fill in some missing responses and try to collect
data on courses as far back as seven years.
Lesson 1: To provide rich transcripts to students based on
instructor-provided data, you must obtain full buy-in from the
department chair to use all methods to ensure that every instructor
fills out the questionnaire for all of their courses over a few years.
Lesson 2: Different strategies are required to ensure the participation
of different types of instructors. We had to write a census response
requirement into the contract of part-time instructors. And we had to
tell full-time faculty repeatedly that this was mandatory because,
without full participation, students would not get a complete rich
transcript and any reputational or student-satisfaction benefit to the
department would not be realized.
Manipulating the Data: Creating the Transcripts
We needed course information for 237 distinct course-instructor pairs.
That is, if both Professor Apple and Professor Plum taught POLI367, we
needed data from each of them. But if Professor Apple taught the course
four times, we only needed that one data point from her unless she
significantly changed the course design. Our goal was to match that
course-instructor data to the enrollment histories of two cohorts (550
graduates) from our political science major.
We first defined what indicators we wished to include in the rich
transcript -- what features of the courses could be aggregated and be
most useful for students. Broadly, those were output, experiences and
skills.
Lesson 3: Define what indicators you want to surface for students and
for the program before composing the questionnaire. Consider how the
data will be aggregated, and formulate questions to facilitate easy
data processing.
We settled on these items as the content of the rich transcript:
A list of the student’s courses with the real substantive titles, not the official calendar titles;
A word cloud from the course descriptions written by the instructors, not the calendar description;
For each student, a tally of their output: the number of writing
assignments; written output in pages; number of peer reviews and oral
presentations; hours of group work; and the number of some enriched
learning activities like a research design, primary field research,
community and global service learning, and so on;
A tally of the number of courses for that student in which 23 different
skills were a key learning objective, including common ones like “Write
clearly and effectively” as well as others like “Develop or clarify a
personal code of ethics” or “Perform mathematical or formal/logical
analysis.”
After we settled on the content, we had to design the final product.
The approach was to create a two-page infographic-style document. A
designer was brought in to work with our data scientist who would be
batch-producing the rich transcripts in Tableau, a data science and
data visualization tool.
The result is a document that looks official as well as engaging and is
suitable for both print and screen. We designed it to be a document
that a reader can digest in just a minute or two but with enough detail
that a student can point to it in a job interview and provide examples
of their output, learning activities and skill development.
The team’s data scientists merged the course and enrollment data and
then produced PDF files for each student. Finally, the PDF transcripts
were attached to individual emails to students through a mail-merge
operation.
Surveying Student Reactions
We sent a follow-up survey to the students to gauge their reactions as
well as to prod them to look at their transcript again and use it in
their career-development activities and job searches. We asked students
if they saw value in their rich transcript and how they imagined using
and sharing it.
All but a handful of those who responded had opened their transcript,
and, in fact, two-thirds had opened and reviewed it more than once. The
mean overall rating of the value of the transcript was four on a zero
to five scale, while the rating for “usefulness for career” was
slightly lower, at 3.4. When asked on a zero to 10 scale if they would
recommend that other colleges offer this to their students,
three-quarters gave a response at seven or above, with 30 percent
giving a 10.
About two-thirds of the students had already shown it to friends or
family when we followed up with them a few weeks after they first
received it. Sixty percent said they thought they were likely to show
it to a prospective employer at some point.
Many of the student reactions spontaneously referred to the benefits of
their rich transcript for career activation. One person said, “I was
originally hired as a panel administer for a broadcast measurement
company and was promoted in March to a position that requires a lot of
writing and teamwork. For my job application and interview, I was able
to use the specifics in the transcript to identify what I did during
undergrad. Thank you and the department for putting it together!”
We received suggestions for further development of the rich
transcripts. One of the most common was to allow students to access the
report online as they progress through their degree, so they can see
what skills they are accumulating. Some even suggested that the
information about learning activities and skills development be
available while students are selecting courses so that they can “fill
gaps” in their skill set by choosing courses with particular learning
activities and outcomes. We intend to pursue those suggestions. We will
also reach out to employers for their views about the value of the
current rich transcript document and any suggestions for improvement.
Future Directions
Going forward, we will now systematize and streamline the
data-gathering and production processes for scaling up to multiple
departments. We’ll also work to integrate the rich transcripts with
efforts to support students and alumni in their career preparation and
job search by building a set of accompanying materials to help students
understand how to use their rich transcript as they start their career
journey.
During this consolidation phase, we will elaborate a process framework,
providing a kit that any department can use to conduct a course
characteristics census and produce rich transcripts. The process phases
will be, roughly:
Definition of desired rich transcript content;
Development of the instructor questionnaire through consultation;
Questionnaire administration, including timing;
Data manipulation;
Joining course data to student enrollment data;
Rich transcript visual/graphic design;
Production of rich transcripts;
Distribution;
Integration with student and alumni career resources; and
Evaluation
Over the long term, we should also be able to first, issue interim rich
transcripts at the end of each semester so students can track their
learning experiences and skills development, and second, make the
course census information available to students as they choose their
courses. Students could then more consciously and accurately build
their own program to acquire and develop a wide range of skills -- and
perhaps deep competency in one or two.
|
|
|
|
