A personal tailor application using computer vision for online clothing retailers

In modern life it is almost impossible to imagine a day without a mobile phone. We use them all the time, carry them around everywhere we go, and store very personal information on them, to the point where they become more than just a phone and more than just a gadget. With the technological advancements in mobile phone production, our phones are now faster, smarter and more affordable, making them more attractive and available to a wider range of customers. According to some statistics, more than 7.26 Billion people owned a smartphone in 2021, which makes up more than 90% of the world’s population! With such a huge market, the number of mobile apps and associated gadgets rises immensely. Smartphones have evolved beyond just communication technology – they are used to take photos and videos, make payments, track fitness and health, and so much more. In light of this, it’s easy to see how the market related to mobile phone apps has unlimited potential to grow and create new solutions to make people’s lives easier.

At Grid Dynamics we are always eager to develop new solutions to make lives easier using Machine Learning and advanced AI technologies. Think of voice assistants at home, chatbots on store websites, and autonomous cars on city streets. But what about the retail industry? Retail, as one of the oldest and most traditional sectors, still has a lot of potential to evolve and improve its business outcomes by using advanced AI-based solutions. Considering the numerous fitting rooms in clothing stores, there are still areas where we can apply our expertise to help every customer and business. For some, even before COVID-19, it wasn’t easy to find the most suitable clothes without a personal stylist or fashion designer in a store, boutique or tailoring studio. It can be a tedious and time-consuming task to find such a studio, find the right master and take the right measurements.

Imagine if you could take all the necessary measurements a tailor would take in the comfort of your own home? What if your customers could order clothes online in exactly the right size without having to order adjacent sizes, which would result in your warehouses eliminating the incurred costs of pre-stocking unnecessary sizes, as well as the delivery and courier costs associated with the inevitable returns? This idea drove us to develop a completely new approach to the customer experience and customer journey map. We have utilized Image Segmentation and Key Points Detection technology to extract precise body measurements from images of a person. A customer only needs to point the camera, take a photo and our app will calculate the body dimensions from the image.

If this story has sparked your imagination, keep on reading, because in the following section we will tell you all about the subtleties of our solution, how it works and how it can help your business to evolve and thrive in the digital economy.

Our solution is based on a mobile application that customers can use to take a picture of themselves in full body length to determine their measurements. This is how it works:

1. The first step is for a customer to take a picture in an adjusted pose.
2. The second step involves determining exactly where the person is in the photo. We achieve this by creating a segmentation mask that can point to a person’s presence, pixel by pixel. A common solution for this is to use neural networks (NN) for the determination of the location of a person on screen. Most sophisticated models allow getting segmentation masks for separate body parts. The main downside of such an approach is that NN requires a lot of computations. Fortunately, many modern mobile phones have embedded NN acceleration and even embedded solutions for popular ML tasks. That allowed us to implement all the necessary computations on the device.
3. The third step, after segmenting the human body in a photo, involved detecting the exact position of certain body parts.
4. The fourth step is to calculate the body measurements.

In Figure 1 you can see the example of the segmented image. The next step is detecting the exact position of specific body part points.

Detecting the location of the key body points (hips, shoulders, etc.) was a crucial task because the error in point detection would highly affect the final calculations. For this task we tried multiple approaches. Our baseline approach was using statistical knowledge of the human body to infer relevant parts; for example, hips are usually located in the middle half of the body. This approach provided good results, but we decided to improve them further by using advanced machine learning techniques, such as keypoint detection. Using keypoint detection we were able to achieve better accuracy when detecting body part locations in the photo.

Calculating body measurements from images proved to be a difficult task since errors in previous steps of the pipeline, such as camera deformations, camera position, baggy clothes, etc., vastly influenced the accuracy of our measurements. In our initial approach we tried different pure mathematical methods to calculate chest, waist and hip circumference, with different formulas achieving better results for particular body parts. After refining our baseline approach we moved on to complex methods. We decided to gather human measurements data and conducted extensive exploratory data analysis. Our findings indicated that certain body measurements were highly correlated with other body measurements. For example, In Figure 2 you can see how certain body measurements correlate with each other.

This proved to be the key to developing better algorithms for human body measurement calculations, since it was easier to calculate some measurements more than others. Armed with this knowledge, we tried different machine learning approaches, from simple regression to more advanced boosting methods. This gave us significant reduction in measurement errors.

We didn’t want to stop on just tuning the algorithm to enhance the precision of the predictions, we wanted to go one step further and make a user oriented solution. We developed a mobile phone app that uses our analytical solution in the background to deliver body measurements from the image. See the app in action here!

During the COVID-19 pandemic, the retail industry suffered a loss in clothing sales due to customers not being able to come to the shops and try clothes on. One can also take into account that even during the best of times, some people simply don’t enjoy the experience of manually trying clothes on in a tiny in-store cubicle. Online shopping can help to overcome this, but it still doesn’t solve the problem of choosing the correct size clothing.

With our app, the customer can get accurate measurements without having to leave home or look for a tailor to take measurements. There is, however, still the problem of different sizing scales for different brands. Depending on the style of clothing, the same size may fit differently. Some clothing brands come up with their own sizing scale, completely different from traditional scales. Some clothing brands also intentionally make their clothes larger than others, so customers should wear a smaller size than usual.

Guess what? Our solution helps solve this problem too! The measurements obtained using our application can be applied by the customer to the product catalog in the mobile application and, depending on the chosen brand, receive a recommendation of the most appropriate size within the size scale of this brand.

If you’re interested in a body measurement app for your retail store, get in touch with us to start a conversation.