Talley Student Union Sustainability
Talley Student Union Sustainability
Embedded Files
Usability Evaluation / Matching behavior to the intention
North Carolina State University
Fall, 2016
Project advisor: Dr. Anne Collins McLaughlin
Team members: Xing Huang, Jenna Darrah, David Feltner
My role: Study design, task analysis, heuristic analysis, survey design, new intervention design and test.
Methods: Task analysis, heuristic evaluation, user survey, cognitive walkthrough, pre-post test.
Background
Background
Composting is a simple process that reuses physical waste to eventually produce clean and environmentally friendly products. The newly renovated Talley Student Union at NC State is a hub for students, visitors, and NCSU employees. With this many people, Talley is a place where a composting intervention can make a substantial difference. However, Talley patrons only compost a tiny portion of potential materials (according to NCSU's Sustainability Report, Talley patrons only composted 8.2% of overall compostable waste in 2015).
©Robert Benson Photography
Mission Statement
Mission Statement
People may understand the benefits of composting yet their behaviors do not follow their beliefs. We sought to analytically and systematically evaluate and solve the lack of composting options in public restrooms of the Talley Student Union by evaluating the task of composting, current user needs, and using this information to create and test an efficient system.
Study Design
Study Design
Figure 1 - The study design: a multi-method approach
This project consists of two parts: Evaluating the current system and design/test a new system. In the first part, we evaluated the current composting system by knowing the task and knowing the users. Task analysis, heuristic evaluation, and user survey were used to:
Know the task and know the users.
Determine baseline knowledge and behavior.
Figure 2 - The Integrated Behavioral Model (IBM) serves as the foundation for how we view the entirety of the composting system in the restrooms of the Talley Student Union.
In the second part, through the lens of the Integrated Behavioral Model (see Figure 2), the new interventions are created based on the findings from the first part. We then conducted cognitive walkthrough, pre-post testing, and a follow-up survey to analyze the efficacy of the intervention.
— Part 1 - Knowing the task, knowing the users —
Task analysis
Task analysis
The Task analysis takes a macro look at the physical processes of hand-washing and composting and delves into the user’s cognitive processes, environmental constraints, potential errors associated with each step of the task. There are 11 basic tasks that a user must execute, going from washing their hands, to locating the receptacle, to finally throwing away the rubbish and walking out the door. Table 1 shows an example of putting this process in a fully described context and informs what actions the user will take.
Table 1: A step-by-step task analysis
Table 1 also shows:
"Environmental constraints": physical impediments that a user must overcome to accomplish the goal of disposing of a paper towel. E.g., the composting bin is close to a dispenser, distinct and visible.
"Interventions": design that is directly tied to a specific module in the Integrated Behavioral Model, that way each action is taken is intentional and purposeful. E.g., the design of stickers on dispensers, signs on bins, and placement of bins.
"Target": refers to which aspect to the Integrated Behavioral Model that the intervention is using to change the user's behavior. E.g., environmental constraints, knowledge, habit...
"Errors": errors that can happen throughout the execution of the task. E.g., fail to locate compost bin; users do not understand composting.
"Error Consequence": impact the errors will take on the system. E.g., the user may choose a composting bin for trash, which will contaminate the entire bin.
The Task Analysis helps us to prioritize our focus.
Some errors can happen throughout the task, but we need to prioritize our interventions to prevent the catastrophic error: contamination of the composting bag (if any contaminant is in a composting bin, then the entire compost bag is deemed unfit and redirected to the landfill).
Physical barriers, visual reminders, and the physical location of the composting bins play a significant role in achieving user adherence.
Heuristic Evaluation
Heuristic Evaluation
We conducted a Heuristic Evaluation to identify usability issues with the composting system.
A few insights came to the forefront during the evaluation in the form of "Visibility of System Status" and "Error Prevention". Users will need to break their automatic process of throwing away paper towels and be aware of the new system. In this way, interventions must simultaneously physically prevent potential errors and educate users about composting and the impact of their actions. For example, signs need to be clearly visible to users; error prevention is necessary (if any contaminant is in a composting bin, then the entire compost bag has to be redirected to the landfill).
User Survey
User Survey
To better understand users' intentions, knowledge, motivations, and attitudes towards composting, we conducted an initial user survey of 28 randomly sampled Talley patrons and asked them questions, for instance:
"Do you know what composting is?"
"Have you composted in the last 24 hours?"
"Which of the following do you think can be composted?"
"On a scale of 1 (not important) to 5 (extremely important), how important is composting to you?"
"Where are the compost bins located in Talley Student Union?"
"Are there composting bins for paper towels in the Talley restroom?"
"On a scale of 1 (not likely at all) to 5 (extremely likely), how likely are you to compost paper towels in restrooms?"
The result shows: some people have good insights about composting, but many could improve their knowledge of composting.
Part 1 Key Takeaways
Part 1 Key Takeaways
Users
Users
Have positive intentions but lack of knowledge about how-to and benefits of composting.
Need a simple system that effectively communicates to all users.
The physical design of the system needs to facilitate error-free usage.
System requirement
System requirement
Error prevention (contamination) is the highest design priority.
Physically separate composting bins to prevent confusion.
Signs need to be simple, visible and approved by NCSU marketing.
— Part 2 - Design + Evaluate interventions —
During the second phase, we developed design alternatives for the composting system and tested our design decision by using Cognitive Walkthrough. We then implemented our design to evaluate their efficiency using a pre-post test and a follow-up survey.
The design considerations
The design considerations
The consideration of environmental constraint
Easy physical body access;
Easy eyesight reach;
Interruptions - after drying hands to drop paper towels into a compost bin;
Avoid throwing trash into the compost bins;
Diminishing the mistake of throwing paper towels into a trash bin.
To design the sign, we determined from the previous survey that we needed to design an intervention that would allow the user's decision to compost easily and simply. We found that many people were not knowledgeable about composting, so it may be beneficial to avoid the word "compost" in our design.
Three Options & A Brief Cognitive Walkthrough
Three Options & A Brief Cognitive Walkthrough
We generated three design alternatives as shown below. Justifications for decisions are made based on the relative strengths and weaknesses of each system. As a result, we chose to use the first option for further tests. We used a brief cognitive walkthrough to test the new design before the actual implementation. Specifically, we followed Nielsen's Usability Inspection Methods and ask the following questions for every sub-task:
Will the user try to achieve the effect that the subtask has?
Will the user notice the correct action is available?
Does the user get appropriate feedback?
Pre-post Testing (Evaluation)
Pre-post Testing (Evaluation)
To test the effectiveness of the new system, we conducted a pre-post test.
Evaluation period: 2 academic weeks (Monday - Friday); Opening - 2:30 pm → to encompass the entire duration of peak usage hours
Location: I men's + 1 women's restroom on the main floor
Measurement:
1st week: The baseline assessment → examine a typical week of paper towel use → totaling for an average trash weight.
2nd week: Collection including the implementation of one compost bin in the men's restroom and another in the women's restroom.
Follow-up measurement focused on:
Determine the weight of the compost bags compared to the regular garbage bags.
Determine how much diversion of paper towel material that correlates to the implementation.
Contamination
Results & Findings
A summary shows the baseline (week 1) versus intervention (week 2) spread of waste and compost items.
From the data we collected during the one-week period, we were able to determine that 41.7% of the total waste collected from the restrooms was placed in the compost bin.
Extremely high success rate of composting without contamination in restrooms. Only 1 contaminant was found throughout 5 days of data collection during the second week.
A paired sample t-test reveals significantly less waste was produced when a composting option was introduced in the restrooms, t(4)= 4.89, p=.008.
Follow-Up Survey
Follow-Up Survey
To gauge the effectiveness of our design, we surveyed 21 users in Talley (15 students, 3 visitors, and 3 employees) who had recently exited the restroom for user knowledge and sign memorability. Based on our usability survey, several of the users were not able to recall much about the sign, meaning that it was not memorable enough, so we proposed a few improvements.
Having more prevalent signage throughout the restroom and using of "throw" instead of "compost" on extra signs
Change the shape of the sign to an arrow/triangle shape to point toward the opening of the compost bin.
Users are slowly gaining knowledge about composting but need prompts to assist with successful composting
Discussions & Design Suggestions
Discussions & Design Suggestions
In summary, in Part 1, we began by understanding the user and the task. The Integrated Behavior Model gave us a foundation in which to devise interventions that would most effectively change users' behavior. We collected baseline data and implemented our new system to determine any effect. A follow-up survey helped us to better understand how users perceived our interventions.
We learned that composting in restrooms is very possible and that through gram-by-gram adding up, Talley has the possibility of composting over 400 pounds in one year just in paper towels.
The physical layout worked well; although having the composting bin away from 2 towel dispensers limited the amount of waste diverted, it stood as an effective physical barrier that prevented contamination.
One of the unexpected outcomes is that for this level of action, people do not really need to understand the thought behind an action in order to carry it out. People will inherently believe and follow the instructions that a sign gives them, such as "compost paper towels", even if they don't know that paper towels can be composted.
Proposed future design: We need to improve the visual design of the signs to make it more salient and more memorable; it also needs to be vetted through NCSU marketing.
Page updated
Google Sites
Report abuse