research-article Open Access
- Authors:
- Hana Habib Carnegie Mellon University, Pittsburgh, PA, USA
Carnegie Mellon University, Pittsburgh, PA, USA
View Profile
- Sarah Pearman Carnegie Mellon University, Pittsburgh, PA, USA
Carnegie Mellon University, Pittsburgh, PA, USA
View Profile
- Jiamin Wang Carnegie Mellon University, Pittsburgh, PA, USA
Carnegie Mellon University, Pittsburgh, PA, USA
View Profile
- Yixin Zou University of Michigan, Ann Arbor, MI, USA
University of Michigan, Ann Arbor, MI, USA
View Profile
- Alessandro Acquisti Carnegie Mellon University, Pittsburgh, PA, USA
Carnegie Mellon University, Pittsburgh, PA, USA
View Profile
- Lorrie Faith Cranor Carnegie Mellon University, Pittsburgh, PA, USA
Carnegie Mellon University, Pittsburgh, PA, USA
View Profile
- Norman Sadeh Carnegie Mellon University, Pittsburgh, PA, USA
Carnegie Mellon University, Pittsburgh, PA, USA
View Profile
- Florian Schaub University of Michigan, Ann Arbor, MI, USA
University of Michigan, Ann Arbor, MI, USA
View Profile
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing SystemsApril 2020Pages 1–12https://doi.org/10.1145/3313831.3376511
- 50citation
- 2,664
- Downloads
Metrics
Total Citations50Total Downloads2,664Last 12 Months714
Last 6 weeks116
- Get Citation Alerts
New Citation Alert added!
This alert has been successfully added and will be sent to:
You will be notified whenever a record that you have chosen has been cited.
To manage your alert preferences, click on the button below.
Manage my Alerts
New Citation Alert!
Please log in to your account
- Publisher Site
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
"It's a scavenger hunt": Usability of Websites' Opt-Out and Data Deletion Choices
Pages 1–12
PreviousChapterNextChapter
ABSTRACT
We conducted an in-lab user study with 24 participants to explore the usefulness and usability of privacy choices offered by websites. Participants were asked to find and use choices related to email marketing, targeted advertising, or data deletion on a set of nine websites that differed in terms of where and how these choices were presented. They struggled with several aspects of the interaction, such as selecting the correct page from a site's navigation menu and understanding what information to include in written opt-out requests. Participants found mechanisms located in account settings pages easier to use than options contained in privacy policies, but many still consulted help pages or sent email to request assistance. Our findings indicate that, despite their prevalence, privacy choices like those examined in this study are difficult for consumers to exercise in practice. We provide design and policy recommendations for making these website opt-out and deletion choices more useful and usable for consumers.
Skip Supplemental Material Section
Supplemental Material
a384-habib-presentation.mp4
mp4
46.8 MB
Play streamDownload
Available for Download
zip
pn5776aux.zip (5.5 MB)
Session_Script.pdf: Script used to conduct each participant session, including questions asked prior to and after completion of the study tasks
References
- Alessandro Acquisti, Idris Adjerid, Rebecca Balebako, Laura Brandimarte, Lorrie Faith Cranor, Saranga Komanduri, Pedro Giovanni Leon, Norman Sadeh, Florian Schaub, Manya Sleeper, and others. 2017. Nudges for Privacy and Security: Understanding and Assisting Users' Choices Online. ACM Computing Surveys (CSUR) 50, 3 (2017), 44.Google ScholarDigital Library
- Terence S Andre, H Rex Hartson, Steven M Belz, and Faith A McCreary. 2001. The User Action Framework: A Reliable Foundation for Usability Engineering Support Tools. International Journal of Human-Computer Studies 54, 1 (2001), 107--136.Google ScholarDigital Library
- BackgroundChecks.org. 2019. JustDelete.me. (2019). https://justdeleteme.xyz.Google Scholar
- California State Legislature Website. 2018. SB-1121 California Consumer Privacy Act of 2018. (2018). https://leginfo.legislature.ca.gov/faces/ billTextClient.xhtml?bill_id=201720180SB1121.Google Scholar
- Martin Degeling, Christine Utz, Christopher Lentzsch, Henry Hosseini, Florian Schaub, and Thorsten Holz. 2019. We Value Your Privacy... Now Take Some Cookies: Measuring the GDPR's Impact on Web Privacy. In Proceedings of Network and Distributed System Security Symposium (NDSS).Google ScholarCross Ref
- Digital Advertising Alliance. 2009. Self-Regulatory Principles for Online Behavioral Advertising. (July 2009). http://digitaladvertisingalliance.org/principles.Google Scholar
- Pardis Emami-Naeini, Henry Dixon, Yuvraj Agarwal, and Lorrie Faith Cranor. 2019. Exploring How Privacy and Security Factor Into IoT Device Purchase Behavior. In Proceedings of the Conference on Human Factors in Computing Systems (CHI).Google ScholarDigital Library
- European Commission. 2018a. Article 29 Data Protection Working Party. Guidelines on Transparency under regulation 2016/679. (2018). http://europa.eu/rapid/press-release_SPEECH-11--461_en.htm.Google Scholar
- European Commission. 2018b. EU Data Protection Rules. (2018). https://ec.europa.eu/commission/priorities/ justice-and-fundamental-rights/data-protection/ 2018-reform-eu-data-protection-rules_en.Google Scholar
- Benjamin Fabian, Tatiana Ermakova, and Tino Lentz. 2017. Large-Scale Readability Analysis of Privacy Policies. In Proceedings of the International Conference on Web Intelligence (WI). 18--25.Google ScholarDigital Library
- Federal Trade Commission. 2009. CAN-SPAM Act: A Compliance Guide for Business. (2009). https://www.ftc.gov/tips-advice/business-center/guidance/can-spam-act-compliance-guide-business.Google Scholar
- Federal Trade Commission. 2017. Children's Online Privacy Protection Rule: A Six-Step Compliance Plan for Your Business. (2017). https://www.ftc.gov/tips-advice/business-center/guidance/childrens-online-privacy-protection-rule-six-step-compliance.Google Scholar
- Stacia Garlach and Daniel Suthers. 2018. 'I'm supposed to see that?' AdChoices Usability in the Mobile Environment. In Proceedings of the Hawaii International Conference on System Sciences (HICSS).Google ScholarCross Ref
- Global Privacy Enforcement Network. 2017. GPEN Sweep 2017: User Controls over Personal information. (2017). https://www.privacyenforcement.net/system/files/2017% 20GPEN%20Sweep%20-%20International%20Report_0.pdf.Google Scholar
- Colin M Gray, Yubo Kou, Bryan Battles, Joseph Hoggatt, and Austin L Toombs. 2018. The Dark (Patterns) Side of UX Design. In Proceedings of the Conference on Human Factors in Computing Systems (CHI).Google ScholarDigital Library
- Hana Habib, Yixin Zou, Aditi Jannu, Neha Sridhar, Chelse Swoopes, Alessandro Acquisti, Lorrie Faith Cranor, Norman Sadeh, and Florian Schaub. 2019. An Empirical Analysis of Data Deletion and Opt-Out Choices on 150 Websites. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS).Google Scholar
- Jovanni Hernandez, Akshay Jagadeesh, and Jonathan Mayer. 2011. Tracking the Trackers: The AdChoices Icon. (2011). http://cyberlaw.stanford.edu/blog/2011/08/tracking-trackers-adchoices-icon.Google Scholar
- IAB Europe. 2011. EU Framework for Online Behavioural Advertising. (2011). https://www.edaa.eu/wp-content/uploads/2012/10/ 2013--11--11-IAB-Europe-OBA-Framework_.pdf.Google Scholar
- IAB Europe. 2019. GDPR Transparency and Consent Framework. (2019). https://iabtechlab.com/standards/gdpr-transparency-and-consent-framework/.Google Scholar
- Saranga Komanduri, Richard Shay, Greg Norcie, and Blase Ur. 2011. AdChoices? Compliance with Online Behavioral Advertising Notice and Choice Requirements. A Journal of Law and Policy for the Information Society 7 (2011).Google Scholar
- Pedro Giovanni Leon, Justin Cranshaw, Lorrie Faith Cranor, Jim Graves, Manoj Hastak, Blase Ur, and Guzi Xu. 2012. What Do Online Behavioral Advertising Privacy Disclosures Communicate to Users?. In Proceedings of the Workshop on Privacy in the Electronic Society (WPES).Google ScholarDigital Library
- Thomas Linden, Hamza Harkous, and Kassem Fawaz. 2018. The Privacy Policy Landscape After the GDPR. arXiv:1809.08396 (2018).Google Scholar
- Mary Madden and Lee Rainie. 2015. Americans' Attitudes About Privacy, Security and Surveillance. (2015).Google Scholar
- Arunesh Mathur, Jessica Vitak, Arvind Narayanan, and Marshini Chetty. 2018. Characterizing the use of browser-based blocking extensions to prevent online tracking. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS).Google Scholar
- Jonathan R Mayer and John C Mitchell. 2012. Third-Party Web Tracking: Policy and Technology. In Proceedings of the IEEE Symposium on Security and Privacy (S&P).Google ScholarDigital Library
- Aleecia M McDonald and Lorrie Faith Cranor. 2010. Americans' Attitudes About Internet Behavioral Advertising Practices. In Proceedings of the Workshop on Privacy in the Electronic Society (WPES).Google ScholarDigital Library
- William Melicher, Mahmood Sharif, Joshua Tan, Lujo Bauer, Mihai Christodorescu, and Pedro Giovanni Leon. 2016. (Do Not) Track Me Sometimes: Users' Contextual Preferences for Web Tracking. Proceedings on Privacy Enhancing Technologies 2016, 2 (2016), 135--154.Google ScholarCross Ref
- Michael Morgan, Daniel Gottlieb, Matthew Cin, Jonathan Ende, Amy Pimentel, and Li Wang. 2018. California Enacts a Groundbreaking New Privacy Law. (2018). https://www.mwe.com/en/thought-leadership/publications/2018/06/california-enacts-groundbreaking-new-privacy-law.Google Scholar
- Ambar Murillo, Andreas Kramm, Sebastian Schnorf, and Alexander De Luca. 2018. "If I press delete, it's gone" - User Understanding of Online Data Deletion and Expiration. Proceedings of the Symposium on Usable Privacy and Security (SOUPS) (2018).Google Scholar
- Network Advertising Initiative. 2018. NAI Code of Conduct. (2018). https://www.networkadvertising.org/sites/default/files/nai_code2018.pdf.Google Scholar
- Nielsen Norman Group. 2018. Top 10 Design Mistakes in the Unsubscribe Experience. (2018). https://www.nngroup.com/articles/unsubscribe-mistakes/.Google Scholar
- Donald A. Norman. 1986. Cognitive Engineering. In User Centered System Design: New Perspectives on Human-Computer Interaction. Lawrence Erlbaum Associates, 31--61.Google Scholar
- Donald A. Norman. 1990. The Design of Everyday Things. Doubleday.Google ScholarDigital Library
- Norwegian Consumer Council. 2018. Deceived by Design: How Tech Companies Use Dark Patterns to Discourage Us from Exercising Our Rights to Privacy. (2018). https://fil.forbrukerradet.no/wp-content/uploads/2018/06/2018-06--27-deceived-by-design-final.pdf.Google Scholar
- Online Trust Alliance. 2018. Email Marketing & Unsubscribe Audit. (2018). https://www.internetsociety.org/resources/ota/2018/2018-email-marketing-unsubscribe-audit/.Google Scholar
- Enric Pujol, Oliver Hohlfeld, and Anja Feldmann. 2015. Annoyed Users: Ads and Ad-Block Usage in the Wild. In Proceedings of the Internet Measurement Conference.Google ScholarDigital Library
- Iskander Sanchez-Rola, Matteo Dell'Amico, Platon Kotzias, Davide Balzarotti, Leyla Bilge, Pierre-Antoine Vervier, and Igor Santos. 2019. Can I Opt Out Yet?: GDPR and the Global Illusion of Cookie Control. In Proceedings of the ACM Asia Conference on Computer and Communications Security.Google ScholarDigital Library
- Florian Schaub, Aditya Marella, Pranshu Kalvani, Blase Ur, Chao Pan, Emily Forney, and Lorrie Faith Cranor. 2016. Watching Them Watching Me: Browser Extensions' Impact on User Privacy Awareness and Concern. In Proceedings of NDSS Workshop on Usable Security (USEC).Google ScholarCross Ref
- Fatemeh Shirazi and Melanie Volkamer. 2014. What Deters Jane from Preventing Identification and Tracking on the Web?. In Proceedings of the Workshop on Privacy in the Electronic Society (WPES).Google ScholarDigital Library
- United States Congress. 1999. S.900 Gramm-Leach-Bliley Act. (1999). https://www.congress.gov/bill/106th-congress/senate-bill/00900.Google Scholar
- Blase Ur, Pedro Giovanni Leon, Lorrie Faith Cranor, Richard Shay, and Yang Wang. 2012. Smart, Useful, Scary, Creepy: Perceptions of Online Behavioral Advertising. In Proceedings of the Symposium on Usable Privacy and Security (SOUPS).Google ScholarDigital Library
- Christine Utz, Martin Degeling, Sascha Fahl, Florian Schaub, and Thorsten Holz. 2019. (Un)informed Consent: Studying GDPR Consent Notices in the Field. In Proceedings of Conference on Computer and Communications Security (CCS).Google ScholarDigital Library
- Ari Ezra Waldman. 2019. There is No Privacy Paradox: How Cognitive Biases and Design Dark Patterns Affect Online Disclosure. Current Opinion in Psychology (2019).Google Scholar
Cited By
View all
Index Terms
"It's a scavenger hunt": Usability of Websites' Opt-Out and Data Deletion Choices
Human-centered computing
Human computer interaction (HCI)
Empirical studies in HCI
Security and privacy
Human and societal aspects of security and privacy
Usability in security and privacy
Social and professional topics
Computing / technology policy
Privacy policies
Recommendations
- Why Johnny can't opt out: a usability evaluation of tools to limit online behavioral advertising
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
We present results of a 45-participant laboratory study investigating the usability of nine tools to limit online behavioral advertising (OBA). We interviewed participants about OBA and recorded their behavior and attitudes as they configured and used a ...
Read More
- What do online behavioral advertising privacy disclosures communicate to users?
WPES '12: Proceedings of the 2012 ACM workshop on Privacy in the electronic society
Online Behavioral Advertising (OBA), the practice of tailoring ads based on an individual's online activities, has led to privacy concerns. In an attempt to mitigate these privacy concerns, the online advertising industry has proposed the use of OBA ...
Read More
- An empirical analysis of data deletion and opt-out choices on 150 websites
SOUPS'19: Proceedings of the Fifteenth USENIX Conference on Usable Privacy and Security
Many websites offer visitors privacy controls and opt-out choices, either to comply with legal requirements or to address consumer privacy concerns. The way these control mechanisms are implemented can significantly affect individuals' choices and their ...
Read More
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in
Full Access
Get this Publication
- Information
- Contributors
Published in
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
April 2020
10688 pages
ISBN:9781450367080
DOI:10.1145/3313831
- General Chairs:
- Regina Bernhaupt
Eindhoven University of Technology, Netherlands
, - Florian 'Floyd' Mueller
Monash University, Australia
, - David Verweij
Newcastle University, UK
, - Josh Andres
RMIT, Australia
, - Program Chairs:
- Joanna McGrenere
University of British Columbia, Canada
, - Andy co*ckburn
University of Canterbury, New Zealand
, - Ignacio Avellino
University of Maryland Baltimore County, USA
, - Alix Goguey
Grenoble Alpes University, France
, - Pernille Bjørn
University of Copenhagen, Denmark
, - Shengdong (Shen) Zhao
National University of Singapore, Singapore
, - Briane Paul Samson
Future University Hakodate, Japan & De La Salle University, Philippines
, - Rafal Kocielnik
University of Washington, USA
Copyright © 2020 Owner/Author
This work is licensed under a Creative Commons Attribution International 4.0 License.
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 23 April 2020
Author Tags
- data deletion
- email marketing
- privacy
- privacy controls
- targeted advertising
- usability
Qualifiers
- research-article
Conference
Acceptance Rates
Overall Acceptance Rate6,199of26,314submissions,24%
Funding Sources
Other Metrics
View Article Metrics
- Bibliometrics
- Citations50
Article Metrics
- View Citations
50
Total Citations
2,664
Total Downloads
- Downloads (Last 12 months)714
- Downloads (Last 6 weeks)116
Other Metrics
View Author Metrics
Cited By
View all
PDF Format
View or Download as a PDF file.
eReader
View online with eReader.
eReader
Digital Edition
View this article in digital edition.
View Digital Edition
HTML Format
View this article in HTML Format .
View HTML Format
- Figures
- Other
Close Figure Viewer
Browse AllReturn
Caption
View Table of Contents
Export Citations
Your Search Results Download Request
We are preparing your search results for download ...
We will inform you here when the file is ready.
Download now!
Your Search Results Download Request
Your file of search results citations is now ready.
Download now!
Your Search Results Download Request
Your search export query has expired. Please try again.