Auto Content Moderation for Trust & Safety

Summary

Mercari uses its machine learning-based Content Moderation System to monitor the listings and transactions conducted by more than 20 million monthly active users, contributing to a safer and more secure marketplace.

Purpose

Mercari uses its machine learning-based Content Moderation System to monitor the listings and transactions conducted by more than 20 million monthly active users, contributing to a safer and more secure marketplace.

Description

To realize a safe and secure marketplace, Mercari detects and removes offending items. In addition to detection rules manually created by customer support operators, we also use machine learning models that utilize data from past violations.

Our models for detecting violations are based on violation categories, such as models that detect counterfeit brand-name items and models that detect guns and other weapons that are prohibited by law from being bought and sold. These models are independently trained and operated.

An item determined to be in violation by the model is then visually checked by a customer operator, and a final decision on whether to remove the item is made by a human.

The Content Moderation System development team is currently working on the following technical issues.

1. Because marketplace listing trends change over time, the models need to be regularly re-trained with the latest data. We are now working on creating a system to achieve this efficiently.
   Labels on the training data contain noise, and only some of the data is labeled, resulting in unlabeled data. We are working on building a system that can train models even using data such as this.
   We are also working on developing multimodal models. Items listed in the marketplace include a variety of data, such as text, images, and metadata about the product, and we are searching for new ways to use such data effectively.