Date of Award
2025
Document Type
Honors Thesis (Colby Access Only)
Department
Colby College. Computer Science Dept.
Advisor(s)
Naser Al Madi
Abstract
Background: Previous research into generative AI systems reveals widespread optimism in usefulness, despite high levels of distrust in the outputted con- tent. These findings are consistent across disciplines and motivate the search for ways to improve known points of distrust, such as explainability.
Objective: We aim to assess the impact of stylistic factors in the perceived trustworthiness of AI-generated responses. By specifically instructing the model to include or exclude a combination of references and writing style, we isolate the effects of these components in participant feedback.
Method: We conduct an eye tracking experiment with a series of controlled AI responses. Feedback is gathered via a trustworthiness, accuracy, and bias index after each stimulus is presented. This combination of data will be used to ascertain the individual weight each component of an answer has through eye tracking and data analysis.
Results: We found clear differences in perceived values of trustworthiness and bias between the reference and style groups. Participants found gener- ated responses with citations to be significantly more trustworthy and less biased. Professional style resulted in higher bias and higher trustworthiness, but not significantly so. Neither accuracy nor eye tracking metrics showed clear differences across groups.
Keywords
eye-tracking, generative AI, reading, eye-movement, interfaces, trust
Recommended Citation
Raymond, Owen, "Assessing Factors that Impact AI Response Trustworthiness" (2025). Honors Theses. Paper 1491.https://digitalcommons.colby.edu/honorstheses/1491