Sunday 10:15–11:00 in Tower Suite 1

Combining Computer Vision and NLP for Multi-Task Fashion Attribute Modeling at Shoprunner

Michael Sugimura

Audience level:
Intermediate

Description

Generating fashion attributes of products is key for allowing search and filtering in online retail. To automatically generate attribute tags for millions of products the Shoprunner Data Science team built an ensemble of custom multi-task CNNs and fine-tuned Google’s Bert in Pytorch.

Abstract

Shoprunner aggregates millions of products from 140 retailers which represent thousands of brands. In order to make these products findable and searchable by users it is important for Shoprunner to be able to standardize the attributes (style, color, pattern etc) of these millions of products.

Even after defining what attributes to model, choosing the best way to predict attributes is difficult because every product can be represented in a variety of forms such as images, product description, title,and brand name. These different data representations each have their strengths and weaknesses. Images encode information such as color and pattern well while other attributes related to length and cut may be well captured in text descriptions. This session will go through the multi-task learning ensemble that the Data Science team at Shoprunner has built using both custom multi-task CNNs for images and fine-tuned Bert model for text classification in Pytorch for attribute modeling.

Subscribe to Receive PyData Updates

Subscribe