Trust Building in Recommender Agents
Li Chen
1
, Pearl Pu
2
Human Computer Interaction Group, School of Computer and Communication Sciences,
Swiss Federal Institute of Technology at Lausanne (EPFL), Switzerland
1
EPFL IC IIF GR-PU, BC-144, Station 14, CH-1015 Lausanne, Switzerland
2
EPFL IC IIF GR-PU, BC-107, Station 14, CH-1015 Lausanne, Switzerland
Abstract. Trust has long been regarded as an important factor influencing us-
ers’ decision to buy a product in an online shop or to return to the shop for more
product information. However, most notions of trust focus on the aspects of be-
nevolence and integrity, and less on competence. Although benefits clearly ex-
ist for websites to employ competent recommender agents, the exact nature of
these benefits to users’ trusting intentions remains unclear. This paper presents
some preliminary results of these issues based on a trust model that we have
developed for recommender agents. We describe a carefully constructed survey
in an attempt to reveal the relationship between users’ perception of the agent’s
trustworthiness based on its competence and consumer trusting intentions, and
more importantly, the role of explanation-based recommendation interfaces and
their media format on trust promotion.
Keywords: recommender agents, trustworthiness, explanation-based interfaces,
competence, trusting intentions, e-commerce
1 Introduction
In online commerce (or e-commerce), the traditional salesperson is often replaced by
a product recommender agent (or a virtual salesperson). Given the lack of face-to-
face interaction consumer, trust is difficult to build and easy to lose in a virtual store,
which has impeded customers from participating in e-commerce environments. Thus,
trust has been established to be a key factor to the success of e-commerce [4, 8]. It is
widely accepted that trust in a technological item (like the recommender agent) is
based on competence, benevolence, and integrity, just like trust in a person [10].
Although trust-related issues have been explored broadly in the fields of e-
commerce and human computer interaction, many limitations still exist. As a matter
of fact, most notions of trust have concentrated on how to improve the online shop’s
security, privacy policy and reputation, i.e. the benevolence and integrity of trust
constructs, and less on its competence. As recommender agents have been increas-
Chen L. and Pu P. (2005).
Trust Building in Recommender Agents.
In Proceedings of the 1st International Workshop on Web Personalisation, Recommender Systems and Intelligent User Interfaces, pages 135-145
DOI: 10.5220/0001422901350145
Copyright
c
SciTePress
ingly employed in websites to assist users in choosing products and making decisions,
it is necessary to pay attention to how an agent’s competence influences and builds
consumer’s trust. That is, it is meaningful to investigate how the system design fea-
tures, such as its interface display techniques, recommender algorithms, and user-
system interaction models interact within the trust building process.
Another main limitation of the field is the lack of empirical studies detailing the
exact nature of trust-induced benefits. In the electronic environment, trust is widely
defined as a kind of behavioral intention [5], referred as “trusting intentions” by
McKnight et al. [14]. It has been established that customer trust is positively associ-
ated with customer’s intention to transact, purchase a product, or return to the website
[4, 8, 11]. However, there is no further exploration of which construct of trust most
contributes to one specific intention. Moreover, it is unclear whether users, rather
than e-stores, can actually benefit from the trust. For example, can users actually
improve their task performance due to their increased trust in the recommender
agent?
The contribution of this paper is the development of a trust model, which identifies
a set of system features that contribute towards building competence-inspired trust in
recommender agents. It considers different aspects of the system design, in particular
the role of explanation-based recommendation interfaces and their media format on
building trust. To understand the effect of these design issues, we have conducted a
survey among 53 users to understand the benefits of trusting a recommender agent
based on its competence and the effective means to develop trust using explanation-
based interfaces. The results showed that a positive perception of a recommender
agent’s competence increases users’ intention to return to the agent, but does not
necessarily affect their intention to purchase. The explanation facility integrated in the
recommendation interface was positively correlated with users’ trust-building in the
agent. The survey further demonstrated that an alternative organization-based expla-
nation technique was more effective than the simple “why” construct used in most e-
commerce websites.
This paper is organized as follows: section 2 presents our trust model for recom-
mender agents and its different constituents; section 3 introduces different explana-
tion techniques applicable in recommendation interfaces; section 4 reports our hy-
potheses and survey results; and section 5 concludes the paper’s work followed by
several directions for future research.
2 Trust Model
We have conceptualized a general trust model for recommender agents. It consists of
three components: system features, trustworthiness of the agents, and trusting inten-
tions (see Fig. 1). The system features mainly deal with those design aspects of a
recommender agent that can contribute to the promotion of its trustworthiness. We
classified them into three groups: the interface display techniques, the recommender
algorithms (e.g. collaborative filtering, case-based reasoning, and preference-based
search techniques), and user-system interaction models such as how an agent elicits
users’ preferences. In this paper, we focus our treatment of systems features on inter-
136
face display techniques, especially explanation-based interfaces, and we detail how to
select content, media format and richness for such interfaces.
The agent trustworthiness is a trust formation process based on users’ perception
of the agent’s competence, reputation, integrity, and benevolence, which has been
regarded as the main positive influence on the trusting intentions [6, 13]. In this paper,
we primarily consider the competence perception and its essential contribution to
trust-induced benefits.
The trust intentions are the benefits expected from users once trust has been estab-
lished by the recommender agents. Trusting intentions include the intention to pur-
chase a recommended item, to return to the store for more information on products or
purchase more recommended products, and to save effort. The intention to save effort
is of particular interest to us because it examines whether upon establishing a certain
trust level with the agent, users will exert less effort to process all information them-
selves by selecting the recommended items much earlier in the recommendation cy-
cles.
In addition to the agent trustworthiness, another influence on trusting intentions
would be the individual propensity to trust. Studies of trust as a purely psychological
attribute revealed that each person possesses a stable personality characteristic, which
influences one’s willingness to extend trust in specific situations [3]. We are inter-
ested to know whether this factor will make an impact on users' behavior intentions in
recommender agents.
Fig. 1. Trust model for recommender agents
3 Building Trust Using Explanation-based Interfaces
Explanation has long been employed as one main approach to improve system’s
transparency in the domains of expert system [9], recommender systems [7], and
interactive data exploration systems [1]. Practical application of explanation-based
interfaces can be found in decision support systems (Logical Decisions:
www.logicaldecisions.com), and commercial web sites (Active Decisions:
www.activedecisions.com).
System Features
Trustin
g
Intentions A
g
en
t
Trustworthiness
Recommender
al
go
rithm
s
Intention to bu
y
Intention to
return
Intention to save
effort
Benevolence
Re
p
utation
Com
p
etence
Inte
rit
Pro
p
ensit
y
to trust
Interface display
Ex
p
lanation
User-system
in
te
r
act
i
o
n
137
However, the benefits of explanation for trust formation have not been well estab-
lished. Herlocker et al. [7] have shown that in automated collaborative filtering (ACF)
based recommender systems, providing explanation facility of recommendations can
improve the acceptance of the system and filtering performance of users, but no fur-
ther work on the relationship between explanation and trust building. More specifi-
cally, it is still unclear whether explaining how recommendations are computed can
increase user’s trust in the recommender agent. In this section, we therefore primarily
consider trust building by the different design dimensions of explanation-based inter-
faces. In particular, we investigate the modality of explanation, e.g., the use of graph-
ics vs. text, the amount of information used to explain, e.g., whether long or short text
is more trust inspiring, and most importantly whether alternative explanation tech-
niques exists that are more effective in trust building than the simple “why” construct
currently used in most e-commerce websites.
Fig. 2. The “why” explanation facility used in most e-commerce websites
The explanation generation comprises the steps of content selection and organiza-
tion, media allocation, and media realization and coordination [1]. Content selection
determines what information should be included in the explanations. For instance, the
neighbors’ ratings can be included to explain the recommended items in the collabo-
rative filtering based recommender systems [7]. Once the content is selected, we must
know how to organize and display it. The simplest strategy is to display the content in
a rank ordered list with a “why” tool tip for each recommendation (see Fig. 2).
Fig. 3. Organization-based explanation interface, where the category title replaces the “why”
component
138
As an alternative and potentially more effective technique, we have designed an
organization-based explanation interface where recommendations that provide trade-
off alternatives are grouped in one category (see Fig. 3). This idea was inspired by
McCarthy et al.’s work [12] which showed that suggesting products in groups of
compound critiques enabled users to reach their decisions much faster.
The ranch house seems better than Japa-
nese house according to your preferences,
since it has advantages on garage size,
condition, needed repairs, purchase price,
systems, kitchen and other features. How-
ever, the Japanese house still has some
benefits on surroundings quality, operating
costs, exterior appearance and upstairs
size.
Fig. 4. Explanation realized in text vs. graphics. The right figure (adapted from logical deci-
sions software) is using graphics to explain the difference between two houses in terms of their
attribute values. The left text gives the same content in the style of conversational sentences
Media allocation and realization considers the concrete mapping between the dif-
ferent portions of the selected content and the corresponding media. Currently, there
are mainly two media used to implement explanation. One medium is text (see Fig. 4),
which is used in many commercial web sites (see Fig. 2) and expert systems [9]. The
research direction has been to make the explanation more conversational and argu-
mentative to make people feel at ease.
Another medium uses graphics for realizing explanation content (see Fig. 4). The
advantage is that visual information can allow people to develop a deep understand-
ing of the data. Herlocker et al. have proven that the histogram with grouping of
neighbor ratings was the best performing explanation component for collaborative
filtering recommendations [7]. However, their experiment didn’t compare the histo-
gram with text for the same explanation content. In general, few existing works indi-
cate which media is more preferred by users in general or in a specific circumstance.
House 18 is an interesting house. In fact, it
has a convenient location in the Ecublens
neighborhood. House 18 is close to work (1.7
miles).
House 18 is an interesting house. In fact, it
has a convenient location in the Ecublens
neighborhood. Even though house 18 is
somewhat far from the park (3 miles), it is
close to work (1.7 miles) and a rapid trans-
portation stop (1 mile). House 18 offers a
beautiful view, and it has a wonderful
exterior.
Fig. 5. Short and concise explanation sentences vs. Long and detailed ones
The issue of media richness of explanation is also not well understood. For exam-
ple, is a short and concise explanation text preferred to a long and concise one (see
139
Fig. 5)? Carenini and Moore [2] have developed one method to generate argumenta-
tive text tailored to the user's multi-criteria preference model. They also indicated that
the effective arguments should be concise, presenting only pertinent and cogent in-
formation. However, their evaluation was specific in the domain of searching for a
house, and did not measure the effectiveness from the aspect of trust building.
4 Survey
We have conducted a survey among 53 users in order to understand the interaction
among the three components of our trust model: the effect of an agent’s competence
in building users’ trust, the influence of trust on users’ problem solving efficiency and
other trust intentions, and the effective means to build trust using explanation-based
interfaces. The result of this survey will hopefully help us focus our future direction
in conducting empirical studies to understand the quantitative relationships among
these components of trust.
4.1 Hypotheses and Survey Questions
We have developed 9 hypotheses and divided them into three categories: agent trust-
worthiness in terms of its competence, trusting intentions assessment, and explanation
techniques on trust formation. For each hypothesis, we have designed a question for
participants to respond on a 5-point Likert scale (see the hypotheses and correspond-
ing questions in the following tables). To illustrate the hypothesized scenarios, a set
of pre-designed interfaces was used as references. For instance, the interface inte-
grated with the “why” construct (adapted from a website powered by Active Deci-
sions, see Fig. 2) was shown along with another similar display without such explana-
tion facility (see Fig. 6) to our participants when they were asked whether they would
trust more in the recommender agent which could explain how it works than the agent
without any explanation.
Fig. 6. The recommendation interface without “why” explanation facility
140
The first hypothesis tested in our survey was about the direct relationship between
the recommender agent’s competence and users’ trust in the agent.
Hypothesis Question
Hypothesis 1: A positive perception of the
recommender agent’s competence will induce
the user’s tendency to trust that agent.
The recommender agent gave me some
really good suggestions. Therefore, the
agent can be trusted.
We further predicated that a positive perception of the agent's competence could
increase a user's intention to return and save effort, but not his/her intention to pur-
chase because purchase intention would depend on other variables as well. Therefore,
we have developed the following three hypotheses related to the effect of agent's
competence on trusting intentions.
Hypothesis Question
Hypothesis 2: A positive perception of the
recommender agent’s competence may not be
the only element contributing to users’ dispo-
sition to buy a product from the website.
Even though I got some really good sug-
gestions from the agent, I am not yet in-
clined to buy the product from the website
where I found the recommender agent.
Hypothesis 3: A positive perception of the
recommender agent’s competence may neces-
sarily lead to users’ intention to return to the
agent for other product recommendation.
The recommender agent gave me some
really good suggestions. Therefore, I will
return to this website for other product
recommendations.
Hypothesis 4: A high level of trust in the
recommender agent may necessarily lead to
users’ intention to completely rely on the
agent to make a decision.
If I trust the recommender agent, I will rely
on it more to help me make a decision,
rather than processing all of the informa-
tion myself.
Then we measured the effectiveness of explanation-based display techniques on
trust building in recommender agents. The hypotheses 5 and 6 were about the benefits
of explanation on trust promotions, and the remaining hypotheses aimed at determin-
ing the effect of media format and the richness of explanation, and more importantly
whether an alternative organization-based explanation technique (see Fig. 3) would
perform better than the simple “why” construct (see Fig. 7).
Hypothesis Question
Hypothesis 5: Explanation is positively
correlated with user’s trust in the recom-
mender agent.
If there are two recommender agents, one
with an explanation of how it works (see
Fig. 2), and another one without (see Fig.
6), I will definitely trust the first one more.
Hypothesis 6: Explaining how suggestions
are computed increases users’ trust in the
agent.
If I know how the suggestions are com-
puted and ranked, I will be less likely to
want to see the alternatives the agent does
not suggest.
Hypothesis 7: The explanation of suggestions
in text form is more effective than in graph-
ics.
I prefer to see an explanation in familiar
language rather than in diagrams such as a
histogram or a table (see Fig. 4).
Hypothesis 8: Explanation in short and con-
cise sentences is preferred to long and de-
tailed ones.
I prefer short and concise explanation
sentences to long and detailed ones (see
Fig. 5).
Hypothesis 9: Well-organized recommenda-
tions are more effective than a simple list of
If the suggestions are well organized into
different groups according to their differ-
141
suggestions with explanations. ences (see Fig. 3), it will be easier for me
to compare them and make a quicker
choice, compared to a rank-ordered sug-
gestions with detailed explanations (see
Fig. 7).
Fig. 7. The recommendations with simple “why” explanation component
4.2 Survey Participants and Procedure
A total of 53 (7 females) undergraduate students taking the Human Computer Interac-
tion course participated in the survey for partial course credit. To make sure that all of
them have had some experience in online shopping environments before the survey,
we have instructed these student participants to search for a Tablet PC at PriceGrab-
ber (www.pricegrabber.com).
This survey was conducted in the form of a carefully constructed questionnaire,
based on a series of hypothesis and corresponding applicable questions. In the begin-
ning, participants were required to attend a pre-test of their familiarity with e-
commerce environment and propensity to trust, with the aim to check whether these
factors would influence their survey answers. Afterwards, the survey started by ask-
ing users to respond to each of the nine questions on a 5-point Likert scale from
“Strongly disagree” to “Strongly agree” respectively. Since most of the students’
native language is French, each question was accompanied with a corresponding
translation so as to avoid any language misunderstanding.
4.3 Results and Discussion
The survey results give us some useful implications on trust building in recommender
agents (see Table 1). It indicates that the competence of recommender agents would
not be the only contribution to users’ trust formation process (hypothesis 1: median
=3 “not sure” and mode=3), but it is positively correlated with the trusting intention
to return. In fact, most of participants agreed with the items measured for hypothesis
2 and 3 (mean>3, median=4 “agree” and mode=4). This indicates that if users pos-
sessed a high perception of the recommender agent’s competence, they would be
more inclined to return to the agent for other products’ information and recommenda-
142
tion, but they would not necessarily intend to buy the product from the website where
the agent was found. Post-survey discussion indicated that they would visit more
websites to compare the product’s prices before making a purchase. The website’s
security, reputation, delivery service and privacy policy were also their important
considerations in buying a product. As for the trust benefit to users’ problem solving
efficiency (hypothesis 4), the most frequently recurring answer was “disagree”
(mode=2), implying that many participants would still want to take time to process
information themselves, rather than entirely relying on the agent to choose an item.
Table 1. The analysis results of survey on hypotheses (possible range from 1: strongly disagree
to 5: strongly agree)
Mean (St.d.)
N=53
Median
N=53
Mode
N=53
Hypothesis 1 3.15 (0.73) 3 3
Hypothesis 2 3.55 (0.94) 4 4
Hypothesis 3 4.23 (0.63) 4 4
Hypothesis 4 2.89 (1.09) 3 2
Hypothesis 5 3.64 (0.99) 4 4
Hypothesis 6 3.06 (0.98) 3 4
Hypothesis 7 2.38 (0.94) 2 2
Hypothesis 8 2.85 (1.12) 3 2
Hypothesis 9 3.91 (1.03) 4 4
The positive responses to hypothesis 5, 6 and 9 (mean>3 and mode=4) indicate
that explanation can be an effective means to achieve user’s trust, and the organiza-
tion would be a more effective explanation technique than the simple “why” con-
struct. However, the other two aspects of explanation, i.e. the modality (hypothesis 7)
and richness (hypothesis 8), were not conclusive in this study (mean<3 and mode=2).
That is, it is unclear whether graphics or text is more effective to realize explanation,
and whether long or short explanation text would be more trust inspiring. From the
participants’ viewpoints, these two aspects were mostly dependent on the concrete
product domain. Users would prefer a short and concise conversational sentence for
the so-called low-risk products such as movies and books, but if they were looking
for products carrying high financial and emotional risks such as cars and houses, a
more detailed and reasonable explanation would be favored. In addition, people from
different educational background seemed to have different preferences on the media
richness. For example, students majoring in mathematics were more likely to prefer
the explanation using graphics than explanation in text form.
The correlations between pre-tested variables with measured hypotheses indicated
that participants’ propensity to trust and familiarity levels with the e-commerce didn’t
have significant correlations with their resulting ratings on the hypothesized items,
except the frequency of using online shopping tools was significantly positively cor-
related with the hypothesis 9 (p<0.05), which suggests that if participants had more
online shopping experience, they would more likely prefer a well-organized recom-
mendations to a simple list with the “why” explanation construct.
143
5 Conclusion and Future Work
This article describes the early stage of our investigation of trust issues in recom-
mender agents and the qualitative relationship between consumers’ perception of
trustworthiness based on an agent’s competence and their trusting intentions.
Through a carefully designed survey, we have shown that a recommender agent’s
competence is positively correlated with users’ intention to return, but not necessarily
with their intention to purchase. Further, explanation-based interfaces provide a
promising approach to build a competence-inspired trust relationship. More impor-
tantly, an organization-based explanation technique is likely to be more effective than
the simple “why” construct.
These initial results provide useful insights to several directions for future work.
For example, we have recently started a large-scale empirical study to quantitatively
measure the difference of user’s speed in identifying their most preferred items be-
tween the interface using the “why” construct and the interface using the information
organization strategies. We are also planning a controlled experiment to measure if
users can improve their decision accuracy using the explanation-based interface. In
addition, we also believe that users will likely save their decision-making effort with
agents which are trustworthy.
References
1. Carenini, G. and Moore, J. Multimedia Explanations in IDEA Decision Support System.
Working Notes of the AAAI Spring Symposium on Interactive and Mixed-Initiative Decision
Theoretic Systems (1998)
2. Carenini, G. and Moore, J. An Empirical Study of the Influence of Argument Conciseness
on Argument Effectiveness. Proceedings of the 38th Annual Meeting of the Association for
Computational Linguistics (ACL-00) (2000)
3. Chopra, K. and Wallace, W. A. Trust in Electronic Environments. In Proceedings of the 36
th
Hawaii International Conference on System Sciences (HICSS'03) (2003)
4. Gefen, D. E-Commerce: The Role of Familiarity and Trust. Omega: The International Jour-
nal of Management Science, 28, 725-737 (2000)
5. Gefen, D., Rao, V.S. and Tractinsky, N. The Conceptualization of Trust, Risk, and Their
Relationship in Electronic Commerce: The Need for Clarifications, Working paper (2003)
6. Gefen, D. and Straub, D.W. Building Consumer Trust in Electronic Commerce, Working
paper (1999)
7. Herlocker, J. L., Konstan, J. A. and Riedl, J. Explaining Collaborative Filtering Recommen-
dations. In ACM Conference on Computer Supported Cooperative Work (2000)
8. Jarvenpaa, S. L., Tractinsky, N. and Vitale, M. Consumer trust in An Internet store. Informa-
tion Technology and Management, 1(1-2), 45-71 (2000)
9. Klein, D. A. and Shortliffe, E. H. A Framework for Explaining Decision-theoretic Advice.
Artificial Intelligence 67: 201-243 (1994)
10. Komiak, S., Wang, W. and Benbasat, I. Comparing Customer Trust in Virtual Salespersons
with Customer Trust in Human Salespersons. Proceedings of 38
th
Hawaii international confer-
ence on system sciences (2005)
11. Koufaris, M., Hampton-Sosa, W. Customer Trust Online: Examining the Role of the Ex-
perience with the Web Site, CIS Working paper series (2002)
144
12. McCarthy, K., Reilly, J., McGinty, L. and Smyth, B. Experiments in Dynamic Critiquing,
Proceedings of the International Conference on Intelligent User Interfaces (2004) 175-182
13. McKnight, D.H., and Chervany, N.L. What Trust Means in e-Commerce Customer Rela-
tionships: Conceptual Typology. International Journal of Electronic Commerce, 6(2), 35-59
(2002)
14. McKnight, D. H., Cummings, L. and Chervany, N. L. Trust Formation in New Organiza-
tional Relationships. Working paper (1995)
145