31
4.1.3 SYSTEM FEATURE
Designing and manipulating system features often turns out to be a critical step in many IIR user
studies, especially in user-oriented IR evaluation research. As it is discussed in Chapter 2, some
researchers choose to manipulate existing system or interface features (e.g., SERP size, query term
suggestion) for system evaluation, whereas other researchers design new features, aordances, and
search assistant tools (e.g., search trail tool, chatbot) in order to create and observe new interaction
patterns. Yamamoto and Yamamoto (2018) sought to promote critical thinking in the search pro-
cess via query priming and their results demonstrated that using query suggestions can facilitate
searchers’ critical thinking and analysis and thereby improve their search performances in certain
types of search tasks. In this case, the researchers manipulated a widely-adopted existing system
feature (i.e., query term suggestions) and evaluated various versions of the feature or aordance
with users based on their interactive behaviors and performances. One of the prominent advan-
tages of using existing, widely applied system and interface feature(s) is that participants usually
do not need to take too much time and eorts to practice and get familiar with the manipulated
version of the system (because the manipulated feature here is not entirely new). erefore, com-
pared to introducing brand new features and conducting IR system evaluation, it might be much
easier for researchers to elicit natural search behavior and responses from participants with this
type of study settings.
Diering from Yamamoto and Yamamoto (2018), some user studies designed and evaluated
new system or interface features in search interactions. For instance, Ruotsalo et al. (2018) designed
an innovative assistant tool (i.e., interactive intent modeling) and visualized user intents using
keywords for supporting search interactions. en, the researchers evaluated the new system with
a between-subject user study design (i.e., experimental system group versus baseline group) and
demonstrated that mining, predicting, and visualizing search intents can eectively facilitate users’
query formulations and thereby improve their search performance and overall experience. Similarly,
Bateman, Teevan, and White (2012) created and assessed a new tool named Search Dashboard that
provides a useful interface for users to reect on multiple aspects of their own search strategies
(e.g., query formulation, click and result examination). eir research demonstrated that reecting
on personal search behavior can be benecial to users as it changes their views and understanding
of their own search tactics and what can be accomplished with the current search system. Nichol-
son et al. (2014) reported a user-centered evaluation of Panopticon, a video surrogate system as a
support search tool for nding information within video lectures. Using the standard video player
(YouTube) as the baseline to compare with in evaluation, the study indicated that their new search
support tool can signicantly expedite information nding within the lecture videos. Regarding the
system features “behind the scenes,” Syed and Collins-ompson (2017) proposed novel retrieval
algorithms that can generate personalized search results tailored to a variety of human learning
4.1 STRUCTURE OF THE FACETED FRAMEWORK
..................Content has been hidden....................
You can't read the all page of ebook, please click here login for view all page.