Work at JD.com

Several people have asked me about my work at JD.com. This page describes some of my work and the research that came out of this work. During 2017-2019, I was on leave from Stanford and worked in JD’s Silicon Valley office as Chief Scientist for Business Strategy and afterward was affiliated with JD’s Intelligent Ads Group (2019-2021).

JD.com Overview

JD.com is China’s second largest e-commerce company. JD’s 2020 revenues of USD 114.3B makes it China’s largest internet company by revenue: see here for more facts about the company.

Broad Focus

My role was focused on using marketing science and applied econometrics to drive growth for JD and its partner brands in China, leveraging JD’s data assets and AI-driven technology platform. This article describes at a high level some of the goals of the broad effort: https://finance.sina.cn/2018-10-18/detail-ifxeuwws5653433.d.html (translation required). See here and here for some broad overviews of the JD Marketing 360 platform.

Together with Paul Yan, President of JD Business Growth, engineering leaders Jack Lin and Lei Wu, and our incredible data science and economics team,  I developed products related to e-commerce marketing, pricing and advertising for improved brand-building, monetization and ad-strategy for JD and participating brands.

Ad-Experimentation Platform

One effort involved developing an external facing ad-experimentation platform for the JD ad-business. The platform comprises a set of products that together facilitate scalable, self-serve experimentation by advertisers to evaluate and enhance their advertising campaigns on JD. Together, the idea behind these products is for a publisher to provide experimentation “as a service” to its participating firms.

Specific Products

A few specific products developed for the platform are described below. These products are currently deployed or in production on JZT (https://jzt.jd.com/), JD’s campaign management platform.

  • JD Multi Touch Attribution (MTA) uses a Recurrent Neural Network trained on granular user data along with Shapley Values to develop a data-driven attribution system to assess campaign performance for advertisers on JD.  A related product,  “Path to Purchase,” visualizes consumer click-steams on JD so as to provide more data-driven context to the reported attribution. This article provides an overview: http://papers.adkdd.org/2019/papers/adkdd19-du-causally.pdf.
  • Touchpoint Mix Modeling (TMM) develops an advertiser-facing product for campaign-level automated bidding and budget optimization tailored for ads bought via Real Time Bidding (RTB). The system integrates with modern MTA systems to provide bid and budget optimization taking attribution from MTA models as an input. A premium is placed on transparency and interpretability of the model to preserve advertiser trust in the automation. Paper available here, which reports on algorithm, deployment and randomized controlled evaluation: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3913039
  • JD Conversion Lift for RTB develops a system and experimental design for measuring ad-lifts in an RTB environment from a publisher’s perspective. The publisher controls the ad-auction mechanism and the system exploits knowledge of the publisher’s real-time auction-queue. A new aspect of the framework is it facilitates simultaneous parallel experimentation across multiple competitors in such an environment, defining the relevant treatment effects. This article provides an overview: https://arxiv.org/abs/1903.11198.
  • JD Conversion Lift for AdX develops a system and experimental design for measuring ad-lift in an real-time bidding (RTB) environment from a Demand Side Platform (DSP’s) perspective. The DSP, of which JD is an example, submits bids on behalf of advertisers into an external ad-exchange (AdX). It cannot control the AdX’s auction or observe its auction-queue. The framework facilitates experimentation in such an environment by randomizing the bids submitted by the DSP into the AdX auction. The randomization is implemented adaptively, via a contextual bandit, so that the optimal bidding policy is also learned alongside inference of the ad-lift, so that the algorithm simultaneously delivers on the advertiser’s payoff-maximizing goal and her causal-inference goals. This article provides an overview: https://arxiv.org/abs/1908.08600.
  • JD Comparison Lift develops a system to help an advertiser designing a campaign to discover via online experimentation, a creative-target audience combination that provides her the highest expected payoff. The target audiences can be complex, potentially overlapping with each other, and the creatives can be any type of media (picture, video, text etc.). The algorithm is set up as a contextual bandit that adaptively allocates traffic during the test so as to minimize the cost to the advertiser from experimentation.  Broad overviews here and here.
  • Blending ads and organic content is particularly important in e-commerce as ads are embedded into content throughout the platform, implying that the impact of both ads and organic content depend on each other. We develop a practical method to blend ads and content on e-commerce platforms that respects content interactions and externalities and balances multiple platform objectives. The method involves a deep-learning system for click-through prediction combined with a “virtual-bid” formulation for balancing objectives that is auto-tuned from historical data. The formulation is a building block for an auction payment system such as the VCG mechanism that respects externalities. This system has been deployed on JD.com’s mobile app and is currently serving millions of users. See paper at https://arxiv.org/abs/2105.13556.
  • Helping users discover new products is important for e-commerce platforms to serve as a healthy marketplace for brands, and also for the success of marketplace sellers on the platform. But e-commerce platforms have substantial difficulty exposing new products to their users on account of an information problem: in the early stages of launch, new products have not accumulated enough sales, orders, or other user-engagement information in order to reliably assess them of high-enough quality so as to rank them high in search listings. We redesign the search engine of JD to boost the rankings of new products that have been recently advertised. Our premise is that sellers’ advertising decisions reflect private information that sellers possess about product quality, and if sellers tend to advertise more the products they believe have higher quality, boosting advertised products can help surface better new products to users. A large-scale field experiment implemented on JD shows that incorporating ad propensity information into the search ranking algorithm benefits both the platform and consumers, in the short run. Our findings showcase a new channel by which advertising can improve outcomes for consumers and platforms in e-commerce, through its ability to reveal information that can be used by platforms to improve search algorithms. Also, it highlights the usefulness of economic theory-driven feature engineering and calls for blanket separations between ad and product markets to be re-thought. See paper at https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3895782.
  • Digital Couponing is particularly powerful in e-commerce. Digital coupons are complex pricing contracts that enable differential pricing to be implemented at scale in a way that is targeted, personalized, with individualized deadlines, product scope and contactual features, while reducing psychological reactance from consumers to price variation. JD’s couponing platform delivers a wide variety of coupons and vouchers to users.  A paper that leverages field experiments developed to evaluate the efficacy of JD’s consumption vouchers as a policy tool to stimulate household spending and help mitigate the economic costs of COVID-19 is https://papers.ssrn.com/sol3/papers.cfm?abstract_id=3679362.

Overall, by leveraging causal inference, experimentation, and machine learning with micro-data in a scalable way, these products help advertisers better measure and implement their digital marketing campaigns when working with publishers and to better optimize their pricing and advertising strategies, while at the same time, promoting healthy growth of the platform and its marketplace.