CASE STUDY
Conducting Moderated Usability Testing for Pratt Institute’s Center for Career and Professional Development Website
The Challenge
Through informal user feedback, CCPD representatives uncovered a need for a more effective web presence to improve communications and to drive engagement with CCPD services. The CCPD team contacted the Pratt Center for Digital Experiences to partner with usability experts to explore options for improvements.
The Client
Pratt Institute’s Center for Career and Professional Development (CCPD) has the overall goal of assisting Pratt students and alumni with personalized career development services which include assistance with portfolio and resume creation, facilitation of workshops surrounding interviewing for jobs and internships, help with authoring grants and proposals, and the scheduling of professional networking events.
Project Goals
Enhance
the overall usability of the CCPD website following sound usability principles and practices.
Identify
elements of the website that students find challenging to navigate, gathering insights to guide targeted improvements.
Recommend
adjustments to the site that will provide maximum value in a short time and within the technical constraints provided by the client.
Our Team…
John Veon
My Responsibilities:
N. Gowda
L. MacGuire
Interviewing the client with the team, capturing notes and sharing in a Google Doc
Generating a timeline with milestones to help the team stay on track
Creating a dynamic screener survey in Google Forms, used for recruiting
Creating a consent form in Google Forms for selected participants
Writing the participant selection email sent to our 8 participants
Collaborating with the team to generate the 5 tasks used in testing
Conducting remote usability testing with 2 participants
Performing data analysis via affinity mapping with the team
Generating the specific “Deep Links” recommendation for the client
With the team, preparing the formal presentation deck and report for the client
Zoom meeting organizer and team cheerleader 😎
B. Yang
The Process
Client Interview
What is the need?
Pre-Test
How to proceed?
Recruiting
Who will participate?
Usability Testing
Data collection!
Data Analysis
What’s important?
Findings
What improvements?
Client Interview
“We received an external review which made mention that we could communicate more clearly.”
“We’d like to focus on students for now, specifically undergrad students.”
“We’d like to drive engagement with the website and provide better content.”
“Some of the sections in the navigation aren’t related to CCPD offerings.”
Pre-Testing Activities
Team planning using a digital whiteboard was invaluable for asynchronous collaboration. Our ability to work independently on this whiteboard and then meet to collaborate on ideas and findings remotely and in person drove both rapid ideation and efficiency in decision making for next steps.
The team moved quickly to generate a pre-recruiting screener questionnaire and participant consent form to identify target participants and to move rapidly to the testing phase, keeping us ahead of schedule.
Recruiting Methodology
Flyers posted at campus locations
Email blast via Handshake
Discord posting on undergrad server
The team used a spreadsheet to track respondents and to collectively analyze our pool of recruits for participant selection…
Flyers posted around campus
Participant Demographics
Our recruiting efforts allowed the team to select a diverse set of participants for our study quickly and efficiently…
Images created by team member N. Gowda 😺
Remote Moderated Usability Testing
Qualitative and quantitative feedback was collected through remote testing sessions by introducing participants to 5 different scenarios aligned with the project’s goals. Usability testing was conducted using Zoom, with recordings of the sessions allowing for deeper analysis after testing had been completed.
The team asked additional questions both before and after the specific scenarios to get an overall indication of the CCPD site’s usability and navigability. For example, following scenario testing we asked participants to provide us with three words to describe their overall experience navigating the site.
Data Analysis
Following testing, the team engaged in qualitative and quantitative data synthesis and analysis. Using a collaborative spreadsheet for capture during interviews and later dropping our data into a digital affinity map for grouping allowed us to visualize conclusions about common trends that participants were experiencing.
Averaged responses following each task. Lower numbers indicate better results.
Participant Insights…
“I’d rather just email the CCPD for information instead of searching through every tab...”
“I usually attend the CCPD workshops - they are quite helpful…”
“ I’ve already forgotten how I got to this page…I can’t even find the link any more…”
Recommendations
Findings:
Imagery selected for each section doesn’t feel relevant for participants.
Text is very dense for each page; users are getting lost in the wall of text presented.
Links are not immediately evident and sometimes buried in the text.
Recommendations:
Distill and focus content to the specific service for easier scan / search for users. Reduce and clarify!
Introduce more contextual, relevant imagery to help guide students to their intended area.
Findings:
The current CCPD website features an overly extensive array of content, much of which fell outside the purview of the CCPD.
The typical scope of career services is sandwiched between nonrelevant links, hindering the effectiveness of user navigation.
Recommendation:
Eliminate content that falls outside the scope of CCPD to streamline the user experience. We saw this performed effectively in other sections of the Pratt website and felt that it was within technical bounds.
Findings:
Some participants feel overwhelmed with the CCPD website experience. Information is difficult to locate, and some users felt that it would be easier to contact the CCPD directly - especially students that would be graduating soon.
Recommendation:
Introduce a floating Help button on every CCPD page that user can click at any time while browsing the CCPD website.
This button should link to the current FAQ, which is a concise page of information including contact details for the CCPD office, providing immediate help regardless of the topic at hand.
Findings:
There are no specific indicators in the left nav that critical services are available; portfolio and resume help, interview assistance, etc.
The appointments link in the current left hand navigation direct students to log into Handshake, but users need to wade through a wall of text to discover it.
Once at Handshake, students became frustrated with attempting to address their specific need…Handshake is not helpful in guiding users to the desired end result.
Recommendations:
When a participant clicks link to schedule an appointment for a specific need (resume help, portfolio assembly)…
…that link points to the appointment group that relates to that topic in Handshake. Guide users to their destination using deep links!
Conclusion
Pratt Institute’s Center for Career and Professional Development provides a wide array of useful, valuable services to students and alumni. Though remote moderated usability testing we found that while participants were positive and appreciative of CCPD services, they were disgruntled with the experience of trying to find helpful resources through the current Pratt CCPD website.
We feel that by moving the needle a bit at a time, the CCPD can improve communications with students around critical services such as portfolio assembly, resume writing, and interview preparation. We consider that a) reorganizing content to be less dense and eliminating some imagery, b) introducing deep links to commonly requested services on Handshake, c) refining the left nav to be CCPD-focused, and d) injecting buttons for immediate help are all opportunities to effect immediate usability improvements via low cost, high impact results within the technical constraints of the current website. The CCPD staff was highly encouraged by our findings, especially since they remained within the limitations that were expressed during earlier conversations. As a team, we feel that the CCPD site could further benefit from additional usability testing and implementation of improvements based on those findings.
Further detailed documentation surrounding this project may be viewed via the client presentation deck and formal client report.
We are highly appreciative of the CCPD staff for partnering with us on this project.