Comment
Author: Admin | 2025-04-28
At the Amazon Store, we strive to deliver the product recommendations most relevant to customers’ queries. Often, that can require commonsense reasoning. If a customer, for instance, submits a query for “shoes for pregnant women”, the recommendation engine should be able to deduce that pregnant women might want slip-resistant shoes. Mining implicit commonsense knowledge from customer behavior. To help Amazon’s recommendation engine make these types of commonsense inferences, we’re building a knowledge graph that encodes relationships between products in the Amazon Store and the human contexts in which they play a role — their functions, their audiences, the locations in which they’re used, and the like. For instance, the knowledge graph might use the used_for_audience relationship to link slip-resistant shoes and pregnant women.In a paper we’re presenting at the Association for Computing Machinery’s annual Conference on Management of Data (SIGMOD) in June 2024, we describe COSMO, a framework that uses large language models (LLMs) to discern the commonsense relationships implicit in customer interaction data from the Amazon Store.COSMO involves a recursive procedure in which an LLM generates hypotheses about the commonsense implications of query-purchase and co-purchase data; a combination of human annotation and machine learning models filters out the low-quality hypotheses; human reviewers extract guiding principles from the high-quality hypotheses; and instructions based on those principles are used to prompt the LLM.To evaluate COSMO, we used the Shopping Queries Data Set we created for KDD Cup 2022, a competition held at the 2022 Conference on Knowledge Discovery and Data Mining (KDD). The dataset consists of queries and product listings, with the products rated according to their relevance to each query.In our experiments, three models — a bi-encoder, or two-tower model; a cross-encoder, or unified model; and a cross-encoder enhanced with relationship information from the COSMO knowledge graph — were tasked with finding the products most relevant to each query. We measured performance using two different F1 scores: macro F1 is an average of F1 scores in different categories, and micro F1 is the overall F1 score, regardless of categories.When the models’ encoders were fixed — so the only difference between
Add Comment