Monday 19 December 2016

A New Search Results Evaluation Model from Google

Search Results (SERPS) are no longer about showing pages that are ordered by rankings for a query term. A Google paper shows us a different way of thinking about them in our age of structured Snippets and featured snippets mixed with URL search results. The paper is:

Incorporating Clicks, Attention and Satisfaction into a
Search Engine Result Page Evaluation Model
by Aleksandr Chuklin, Google Research Europe & University of Amsterdam and Maarten de Rijke, University of Amsterdam

Search engine results have gone through some significant changes over the past couple of years. A paper from the CIKM’16 conference in October 24-28, 2016, recently published on the Research at Google pages describes some of the user behavior that may take place around search results. The benefit that the paper brings us is that it describes:

In this paper we propose a model of user behavior on a SERP that jointly captures click behavior, user attention and satisfaction, the CAS model, and demonstrate that it gives more accurate predictions of user actions and self-reported satisfaction than existing models based on clicks alone.

Good Abandonment

Sometimes people search and expect to find answers to their questions in search results. In these days of features snippets, that may happen more frequently than when answers might just happen to appear in snippets for pages featured in results. So, sometimes a set of SERPS can answer questions without any clicks.

This paper doesn’t describe the idea of entity metrics, but it reminds me of how the Google patent about them presented SERPS in a way that was different that just a presentation of URLs that were ordered based upon IR and PageRank scores. It’s recommended reading:

Structured Data & The SERPs: What Google’s Patents Tell Us About Ranking In Universal Search

There are different reasons why some pages or entities show up in search results than just a rank, and they can have value.

Back seven years ago, I wrote about a paper from Google and Yahoo researchers that talked about satisfaction with search results titled, Evaluating the Relevancy of Search Results Based upon Position. That one focused more on the importance of where results were ranking in search results. There are possibly other things to consider these days.

Like the idea that a searcher might find an answer to their question and not have to click through to a page to find their answer, which can save them time and effort in finding pages pointed out by search results. That would be a case of “good abandonment.”

Click Models

This new paper also talks about user clicks being a sign of possible satisfaction. Very different from the kind of biometric satisfaction that I wrote about in, Satisfaction a Future Ranking Signal in Google Search Results?.

This paper instead talks about a Clicks, Attention, and Satisfaction (CAS) model. It provides some information about each of those things, in the context of human raters that might interact with search results.

The paper tells us about some of the crowdsourcing approaches used with human evaluators, including a glimpse at some questions asked to determine the relevance of results:

Crowdsourcing Direct Relevance

Takeaways

The model described to evaluate search results doesn’t tell you about how to show up in search results or rank higher, but it does tell us about how search results are changing and evolving. Search engines are trying to better understand when results might answer the questions posed by searchers in search results. They also tell us that they are using human evaluators to better understand how relevant search results might be and how satisfied searchers might be with the search engine.


Copyright © 2016 SEO by the Sea. This Feed is for personal non-commercial use only. If you are not reading this material in your news aggregator, the site you are looking at may be guilty of copyright infringement. Please contact SEO by the Sea, so we can take appropriate action immediately.
Plugin by Taragana

The post A New Search Results Evaluation Model from Google appeared first on SEO by the Sea.



source http://feedproxy.google.com/~r/seobythesea/Tesr/~3/u8cnGWwfhm4/

No comments:

Post a Comment