Teaching AI to Identify Clothes

Agustinus Nalwan
12 min readJan 20, 2020

This blog is a sequel to my first blog on building faAi, an AI system which supposed to assist my wife to choose an outfit to wear. In the previous blog, I shared how I built the Clothes Diary, the first module of faAi which automatically record a snapshot of what my wife wears everyday, tagged with weather condition.

In this blog, I will share how I built the second module, an automatic Clothes Cataloguing system which identify clothes captured in the Clothes Diary. Why do we need to identify clothes, you probably ask? Because, being able to identify a clothing is a very crucial step towards being able to make recommendations.

So, the goal here is quite clear, to build a cataloging system which automatically assigns an ID to each unique outfit seen by the Clothes Diary with additional tagging such as skirt, pants, shorts, vest, etc. As you definitely have more than one of each kind, the final id will be more like skirt-1, skirt-2, pants-1, shorts-1, shorts-2, etc.

Desired labelled outfits. Notice that Long sleeve top-1, trousers-1 and long sleeve top-2 were identified twice on separate photos.

In order to uniquely identify clothes, we need to know what makes an outfit different from others. First, the structural shape, e.g. pants is longer than shorts, skirts is short but has a wider bottom, etc. Second, the texture (including colour), e.g. wool, cotton, red, yellow, etc. So, I need a system which can locate all outfits in an image and identify each one by their…