Interview
AI Listings, Image Search… CEO Shintaro Yamada and Director Shunya Kimura Look Back at Mercari’s AI Journey
During 2017-2018, Mercari released a variety of features powered by AI technology, including AI listings, personalized timelines, and image search. These are still core features of Mercari. What was the process leading up to that? Shintaro Yamada, Representative Director & CEO, and Shunya Kimura, Director of AI & Search, look back at the history of Mercari's active use of AI, the challenges they faced along the way, and the organizational transformation that resulted.
Mercari has always had a data-driven decision-making culture
Shunya Kimura: At Mercari, we’ve been focusing on the use of AI for some time now. When did you decide to make AI the focus?
Shintaro Yamada: Mercari was originally a company with many employees who had worked in game development. I was also at a company called Zynga, and when I founded Mercari, we had many employees who joined us from DeNA and mixi in particular. From the days when we were creating game services for feature phones through the transition to smartphone culture, we all felt the importance of continuing to be data-driven. That's why we've been doing A/B testing and data-driven decision-making and product development.
Kimura: Was that inspired by your own first-hand experience?
Yamada: When I was at Zynga, we actively used A/B testing to select button colors and try out new game systems in the course of system development. By implementing the PDCA (Plan-Do-Check-Act) cycle, we were able to see the impact on our business, such as improved customer retention and increased sales. When we released Mercari in 2013, we didn't even have BigQuery yet, but we were doing data enhancement from the very beginning. Utilizing those systems, we saw ways to utilize more value-added data, including AI and machine learning.
Kimura: When did that happen?
Yamada: I remember it was around 2015-2016 that the world began to take notice of the active use of deep learning in AI. We asked ourselves if we might be able to use AI to develop a variety of features. We started development, and in 2017, we started developing AI listings and other products that incorporate machine learning.
The catalyst for the use of AI was witnessing the use of data by Chinese companies
Kimura: What inspired you to actively promote the use of AI? Was it your own sense the power of AI?
Yamada: Looking back at the developments within the company, the flow of events was as I just described, but personally, I was really struck by how the various services in China were using AI. At the time, Merpay was in its start-up phase, and we began getting many reports within the company from our employees who went to research FinTech services not only in China but also other countries. In the summer of 2017, I visited China with Aoyagi (Naoki Aoyagi, CEO of Merpay and Mercoin) before he joined the company.
ByteDance, which would later become famous in Japan for TikTok, developed a personalized news service using deep learning called TouTiao. It's a service similar to SmartNews in Japan. Meituan developed Dianping, a gourmet food site similar to Japan's Tabelog site, and then expanded into food delivery and ride sharing. In each case, they progressively applied AI. Baidu's search engine was also amazing.
What they had in common was they collected data from users, placed that data at the center of their business strategy, and developed their services by thinking about how they could use it and what they could do with it. In the case of Meituan, they had just entered the ridesharing market using their data logged from food delivery bikes and other data.
At the time, seeing this company that was gaining momentum in China that was a data company making good use of data, it made me realize that it was possible to develop a range of businesses in this way. And looking at WeChatPay, Alipay, and so on, I could see that there was a new form of finance ahead.
Kimura: Indeed, the use of data by Chinese companies back in 2017 was quite a shock.
Start of implementation of core Mercari features “personalized timeline” and “AI listings”
Kimura: The first step in the use of AI at Mercari was the development of the "AI listing" feature. This is a feature that, when you take a picture of a product, suggests information about it and even estimates the price. It made an impact on the world, and some people still find this feature so useful that they say it’s one of the best things about Mercari. How did you feel about it at the time?
Yamada: The first thing that struck me was simply how far AI had come. It was a way to take a picture of a product with your phone, identify the products and brand, and even estimate the price. When I heard an overview of the feature, I honestly thought that we would not be able to achieve that level of accuracy. However, we took suggestions from the product side and proceeded with the implementation. When I tried out the actual feature, I was really surprised because the accuracy was higher than I expected. The image recognition technology is implemented correctly and incorporated into the feature in a way that is consistent with the Mercari experience. In fact, the completion rate from the time the listing screen is displayed to the time the listing is completed has also clearly improved. After this series of experiences, I felt the need to invest more in technology in the future.
After that, we implemented a “personalized timeline” feature that personalizes the order of products displayed on your timeline. At the time, Mercari just displayed all the product categories in order of recency, but we released a feature that estimates categories of interest from the customer's browsing history and puts them on the timeline. This feature was very effective and had a significant impact in terms of total transaction value.
Kimura: The great thing about AI is that it can make suggestions that are one step ahead of, or at an angle to, what you might want to buy.
Yamada: I think what's nice about AI is that you don't need to manually construct a rule-based display order. The system is able to handle and process a wide range of data and automatically derive the most relevant output. How do you change the numbers in a particular product category for a particular customer segment? How do you adjust when the response is not good? The beauty of AI is that it can do that kind of complicated automation. I'm also inspired by the personalized services that TouTiao and Meituan have come up with.
In a rapidly expanding organization, don’t try to march in lockstep, just create synergies where you need them
Yamada: When you first joined the company, what was the organizational situation like?
Kimura: In July 2017, when I joined the company, there weren’t as many engineers as there are now. We didn't even have an AI team yet. AI-driven development is now done by ML engineers, but back then it was done by back-end engineers. I remember that the AI team was formed in early 2018. We assigned AI engineers to various projects such as violation detection, AI listing, photo search, price suggestion, and for the US version of Mercari, weight estimation and shipping method recommendation.
At the time, we had created Merpay and were growing as an organization, and we kept the projects separate. Our ML engineers were split into different organizations such as Merpay and US version of Mercari. Under this arrangement, we didn’t try to align our AI development efforts, but proceeded with development in each individual organization. Rather than working too closely together, which slows down the development process, each organization works independently, loosely connected to the others. We emphasize speed while creating synergies where we need them. In this context, where Mercari's services are diversifying and globalizing, each organization is doing its own development using AI. I find that really interesting.
The data management project that became a major task for advancing the use of AI
Yamada: The headcount has increased considerably since then. We now have several hundred engineers. There was a time when the A/B testing framework was full and we started to have conflicts. From that time on, I felt that automation of A/B tests was essential. I also felt it was extremely important to have the data acquisition system organized before reaching that phase.
At the time in 2019, Mercari had already grown into a service used by many customers, but we had challenges such as a lack of consistency in how data is obtained and where data is stored. After that, we started a project to revamp how we obtain data.
Kimura: We first tackled the development of a proper data infrastructure around 2019. As background, amid the acceleration of data-driven decision-making and rising expectations for AI development, the need to improve the accuracy and timeliness of data was growing across the company. In order to strengthen AI utilization and data-driven decision-making, it became necessary to reorganize the structure of the data Mercari was obtaining and the ways we were obtaining it.
At the time, the system was designed to make data storage quite free and easy, because we were focusing on flexibility in analysis. As a result, we ended up with large amounts of data stored in databases, making data maintenance and management increasingly difficult. There is also a debate going on about data management, with each stakeholder having their own opinions from a legal, engineering, and analytics perspective. Meanwhile, the development of Merpay started, and the situation became quite complicated.
This is still a problem that needs to be tackled company-wide, and there are a large number of stakeholders. Jeff, the incoming CEO of Mercari Japan, organized the situation, moved forward with the implementation, and gained company-wide consensus. Since then, we’ve unified how data is obtained and where data is stored, and we now have a more robust data platform that can store data in real time. Furthermore, a project to improve data governance is currently underway company-wide.
In this article, the two discuss Mercari's vision of AI ethics and a concrete path forward to achieving a recycling-oriented society.
Photo : Shun Nakayama (Web/Twitter)
Text : Ayumi Iga (Twitter)