Introducing augmented reality for e-commerce

Augmented Reality (AR) is a type of computer vision technology that allows real-time views to be overlaid with virtual, computer generated objects. As opposed to virtual reality, which replaces the real world with a completely simulated one, augmented reality is designed to enhance people’s perception of reality. Powerful modern computer processing has also opened the doors of the mobile world to this technology and hundreds of companies are now actively producing thousands of different AR applications.

While mobile AR started out mainly as entertainment, the broader potential of the technology is now beginning to be realized in many different areas including medicine, construction, and architecture. But it is in the e-commerce sector that AR is already starting to dramatically change what is possible for people to experience. 

Modern mobile devices are so powerful that they have the capability to fully render virtual worlds around us. The expanded use of AR is also being helped by the standardization of the core algorithms that applications rely on to utilize the technology. This was largely driven by both Apple and Google recognizing that this would help lead to the expanded use of augmented reality for mobile devices. Both companies released corresponding software development kits (SDKs) and made the technology an integral part of their iOS and Android platforms. Now the threshold for developers to start using AR technology is much lower and millions of developers can embrace the possibilities offered by AR.

At one time we did all our shopping in stores, then became familiar with buying online, but it is not until now that we are able to shop online and know exactly how a new sofa will look in an empty room. While online shopping has become a regular part of our lives, it does not perfectly replicate the in-store experience of being able to actually see and feel a product. You can buy a new poster online but not until it is delivered will you be able to tell if it really goes well with your furniture or the color of the wall. Even if you do buy it in a store, it is still hard to know exactly how it will look until you bring it home. 

Augmented reality is working to fill in the missing pieces of the online experience by allowing you to actually see a product exactly how it would look hanging on your wall or sitting in your living room. By simply launching an application, customers can preview products and see exactly how they will look. This helps make online purchasing decisions easier as it takes some of the risk out of big purchases and removes the likelihood of needing to return a product. This capability makes customers happier and more decisive, which helps retailer’s online sale conversions.

AR technology means you don’t need to close your eyes to imagine how a product would look in your home. Instead, all you need to do is take your smartphone and launch any AR application. Then choose a product, look at your smartphone screen and you will immediately see the item as if it was right in front of you. You can move closer or further away, turn lights on or off, and change your viewing angle and it will still be there.

Consider a piece of art, which you would like to be in harmony with the rest of your room. Making a perfect choice is not that straightforward unless you have the picture in your hand. Now it’s simple. You can also see this via a demo of our AR application below.

Just choose an image and view it on the wall you want to put it on. 

You can compare different artworks and change their location with your fingers to wherever you want them. 

Want to change the light? Just switch the light on or off in the physical world. The image on your device will immediately reflect any changes in ambient illumination. 

Now it’s time to choose a frame. 

And that’s it! Before pressing the Buy Now button, all you need to do is also choose a new sofa to match the new artwork you’re going to place above it. 

IKEA, following the release of iOS 11 in September, 2017 immediately rolled out its ARKit application. Now, nearly their whole collection of sofas, armchairs, coffee tables, and storage units are available as 3D models. Items can be rotated and moved around in any room so you can get an idea of the size of a piece of furniture relative to what’s already in the room. However, at the time of writing, despite creating a huge range of 3D models, which can be moved or rotated freely to suit the customer, the user experience is still far from ideal.

Less than two months later, Amazon added the AR Views featureto its mobile application. It allows the user to swipe through a menu of various categories including “living room,” “home décor,” “bedroom,” “kitchen,” “electronics,” and even “toys and games.” It is also possible to rotate any item, move it with a finger touch, or walk around it. Target too has taken a step toward augmented reality e-commerce and updated its mobile site with AR features.

Wayfair, which pioneered a mobile AR e-commerce application in June 2016, has gradually scaled the number of 3D SKUs available today to more than 40,000, making it one of the most impressive applications currently on the market. Another promising utilization of AR technology can be seen in Sephora’s updated AR beauty application. The Sephora Virtual Artist feature was developed in partnership with AR specialists, ModiFace, and allows users to scan in a 3D model of their face. The application then automatically detects the location of the user’s lips and eyes, allowing them to be shown a huge variety of different looks exactly as they would actually appear.

Apple recently released ARKit, which requires iOS 11 and an A9, A10, or A11 processor, which are found in its more recent mobile devices. Google rolled out ARCore, which works only on the more recent flagship Android phones such as Google’s Pixel or the Samsung Galaxy S8. Both SDKs offer a similar range of features, however the accuracy of ARKit depth calculations is a little more advanced than ARCore since Apple’s version employs dual cameras and infra-red sensors for better distance recognition. There is also another AR SDK from Google called Tango, which is essentially a more advanced version of ARCore designed for devices with dual-cameras and IR-sensors. 

Each of the Apple and Google SDKs incorporates the following key features:

• Environmental mapping
• Virtual object rendering
• Motion tracking
• Light level detection
• User interaction
• Object anchoring

Leaving aside the more complex math, here is a more detailed explanation of how the technologies behind augmented reality really work. More recent Android smartphones come pre-installed with Google’s Augmented Reality kit, ARCore. ARCore relies on visual odometry to accurately track the world around it. This computer vision technology is enhanced by a motion detection data sensor, also known as an inertial measurement unit, which senses how the camera moves within a room with a high degree of accuracy. ARCore delivers quite good results on single lens camera devices, with Google’s Tango technology offering improved stereo vision and better depth recognition for Android devices with dual cameras. 

The ARCore or Tango software uses the visual odometry technology to create a virtual world that remains intact irrespective of the camera’s motion. To do this, the software can recognize horizontal surfaces and their boundaries. Once at least one plane has been recognized, a global coordinate system is established and is continuously updated to improve the virtual world model that is being created. The position and orientation of this coordinate system relative to the physical world is determined by Pose. Pose is comprised of both translation and rotation. Translation is the vector between the origins of the physical world coordinate system and the local one associated with the camera. Rotation is a quaternion, or a set of four values, which represent a vector and axis rotation by an angle around the vector. 

These technologies allow virtual objects to be anchored in the physical coordinate system. In many aspects, ARCore’s geometry is similar to the one used in OpenGL as the ARCore Pose object uses the same types of transformation matrices. Essentially, they share the same linear algebra and any corresponding tutorial can be used to understand ARCore. Having sequentially applied all the transformation matrices, ARCore calculates the projection of any virtual 3D object onto the screen.

At the moment, ARCore is not able to recognize vertical planes. However, when a horizontal plane (such as the floor) intersects with a vertical plane (such as a wall), it is possible to detect that boundary. If the user is prompted to tap somewhere on the screen that shows the location of a wall, then ARCore can determine the orientation of the wall within the phone’s coordinate system. Having combined this vector with the plane beneath it, it is then possible to place a virtual object right at the boundary between the horizontal and vertical planes (e.g. to hang a piece of art on the wall), since the elevation of the camera above the horizontal surface has been determined.

The ARCore software doesn’t render any virtual objects itself. Instead, its role is to track the exact position and direction of the camera screen respective to objects recognized in the physical world. It is OpenGL that actually renders 3D objects.

AR offers the possibility to bridge the gap between offline and online shopping experiences. Following earlier attempts to create dedicated hardware such as Google Glass, advances in hardware and software have turned smartphones into a useful, familiar, and ubiquitous AR platform. Mobile developers can utilize their existing skills and established distribution mediums to seamlessly integrate AR into e-commerce user stories. Some industry-specific applications may require creative tweaks of vendor-provided libraries and we’ll be seeing more of these as more retailers discover AR as a way to differentiate and enhance their customer experiences.

Having made AR a part of their platforms, Google and Apple are very serious about further promotion of the technology:

“There will be hundreds of millions of phones that will be AR ready. And the current stage is … we are helping developers … build the experiences … so that it becomes a daily habit. And once it does, I suspect the natural evolution of monetization comes through.” (Amit Singh, vice president of business and operations for virtual reality (VR) at Google, Web Summit technology conference in Lisbon, Portugal.)

The obstacle for e-commerce is that customers so often face the uncertainty of whether a certain product is right for them. This “imagination gap” is already widely discussed, especially in respect to the home décor market. AR offers additional confidence for the customer that they are making the right product choice and therefore helps to motivate purchasing decisions.

Research suggests that 61% of customers like the idea of being able to try different options before committing to a decision. 30% would also use augmented reality to get opinions on purchases, such as making posts via social media. 2018 will definitely be a year that sees major retailers enhance their mobile applications by adding augmented reality features.