This A/B test have one control group and two test groups that show learning-to-rank results.

This test ran from 20 September 2017 to 05 October 2017 on arwiki, dewiki, fawiki, fiwiki, frwiki, hewiki, idwiki, itwiki, jawiki, kowiki, nlwiki, nowiki, plwiki, ptwiki, ruwiki, svwiki, viwiki, zhwiki. There were 3 test groups: control, ltr-1024, ltr-i-1024. This report includes fulltext searches. Refer to Phabricator ticket T175771 for more details.

Data Clean-up

Fulltext search events: Deleted 60061 duplicated events. Deleted 1383174 unnecessary check-in events and only keep the last one. Deleted 91 events with negative load time. Removed 8265 orphan (SERP-less) events. Removed 0 sessions falling into multiple test groups. Removed 199 sessions with more than 50 searches.

Data Summary

Select one of these three tabs:

Test Summary

Days Events Sessions Page IDs SERPs Unique search queries Searches Same-wiki clicks Other clicks
16 1,624,956 455,646 1,183,582 950,181 949,282 763,822 243,252 9,672

Select one of these sub-tabs:

Events

Event type identifies the context in which the event was created. Every time a new search is performed a searchResultPage event is created. When the user clicks a link in the results a visitPage event is created. When the user has dwelled for N seconds a checkin event occurs. If the user clicks an interwiki result provided by TextCat language detection, there is a iwclick event. If the user clicks on a sister search result from the sidebar, that’s an ssclick. If the user interacts with a result to explore similar (pages, categories, translations), there are hover-on, hover-off, and esclick events.

Searches