The DIGITOUR system is an end-to-end pipeline for creating digital tours of real-estate properties. It involves capturing 360-degree images in each area of a property, tagging each of these areas with bi-colored paper tags, and using machine learning algorithms to stitch together a coherent tour.
To create a tour, an operator places paper tags at various points throughout the property. The tags are 6″x6″, numbered, and come in two specific colors. These properties help with the identification and sequencing of the tags. A 360-degree camera then captures images of each tag area. The camera staff used was a Ricoh-Theta model.
In a complex process, these images are converted from their native equirectangular format, which is stretched and distorted, into a cube map projection consisting of six images that together represent a much less distorted view. This is done using the Python library vrProjector.
Next, the system uses the state-of-the-art YOLOv5 model to detect and locate the tags in each of the cube maps. It then recognizes the digits on the tags using a custom version of the MobileNet model. Once the tags are located and read, they can be mapped back onto the original equirectangular images. From there, all the images corresponding to a single property are stitched together to create the digital tour.
The final digital tour represents a property as a graph, where each node is a location with a tag, and each edge is a transition from one location to the next based on the sequencing of the tags. The system collected data from properties in Gurugram, India, and they experimented with different tag colors (green and bi-colored). Over time, the system was able to create 100% accurate tours for just under 95% of the properties it processed.
DIGITOUR shows promise in enhancing the real estate industry by providing efficient, automatic creation of high-quality, 360-degree virtual tours. The approach standardizes the process of creating such tours, reducing the need for professional photographers and enabling more properties to be showcased. The pipeline’s end-to-end performance demonstrated mean Average Precision (mAP) of 88.12 and an f1-score at 0.5 IoU threshold of 95.81, averaged (weighted) over all classes.
