Content Based Analysis and Retrieval of Architectural Floor Plans

Abstract

Architects often refer to existing layouts, while designing new projects. This process aids in providing insight into how such similar architectural situations were solved in the past. By studying one or several previous reference projects the architect tries to derive a solution for the current problem. The manual look-up process for such similar projects through the layouts can be rather cumbersome. With several architectural projects archived in digital form, the research and development of fast automatic retrieval techniques in floor plans is the need of the hour. Floor plan analysis is a special case of document image understanding. It aims at extracting semantic and structural details of an architectural layout by analysis of the 2D image of the floor plan. Symbol spotting and retrieval in architectural layouts have been solved as individual problems in the past. Moreover, both image and sketch has been used as modalities for the symbol spotting task. Thus, upon performing a keen analysis related to the existing work in the area of architectural floor plans, it was concluded that retrieval in this particular domain is a challenging yet less researched area and has a variety of applications in today’s digital scenario. Specific requirements by buyers during property rent/sale can be met using a composite, automated framework that takes into account semantics, as well as the content inside a floor plan for retrieval. Existing floor plan datasets like Systems Evaluation SYnthetic Documents (SESYD) and Computer Vision Center- Floor Plan (CVC-FP) dataset are not fit for a retrieval task as they don’t suffice in number and variation in the floor plan samples for the task of floor plan retrieval. Keeping in mind the current status of floor plan analysis research, two publicly available benchmark datasets Repository Of BuildIng plaNs (ROBIN) and Sketched-Repository Of BuildIng plaNs (S-ROBIN) are proposed in this thesis that will aid research for the community in the area of floor plan analysis and retrieval. In this thesis, techniques to analyse architectural floor plans and extraction of different features to aid in content based retrieval in floor plans is proposed. In one such attempt, retrieval in floor plans under the query by example paradigm, based on similar overall layout designs as well as the arrangement of the room decor present in the layouts, is proposed. Here query is taken as a floor plan image. A room layout segmentation and adjacent room detection algorithm is presented to represent layouts as an undirected graph. The vertices of the graph represent the rooms, while the edges represent the connectivity between them. Also a novel graph spectral embedding feature is proposed to uniquely represent the layout of the architectural floor plan. This helps in effective and efficient matching of the room layouts. To match the semantic similarity between a pair of floor plans, a two stage matching technique is proposed and high retrieval accuracy is obtained. An interactive graphical user interface to aid users to select, analyse and retrieve similar floor plans is also proposed in this thesis. In another attempt a Convolutional Neural Network (CNN) framework to extract both low and high level semantic features is proposed for floor plan retrieval. Experiments were conducted on publicly available datasets as well as ROBIN. The key contributions in the proposed approach are, a novel deep learning framework to retrieve similar floor plan layouts from repository. Also, in this part of the work the effect of the individual deep convolutional neural network layers for floor plan retrieval task is analysed. Experiments have shown that the deep learning frameworks work very well when the target image itself has a lot of features. Examples include natural images, textual documents, etc. On the other hand for images, which are not that feature rich, the off-the-shelf CNNs are not that effective. Moreover, deep features mostly capture the global similarity in an image. It was envisaged that effective combination of domain specific features may give superior results. The effect of combining various extracted features in a weighted manner to aid in giving preference to a certain feature while retrieval is also proposed. A novel end-to-end framework for extracting high level semantic features like area and room-wise decor arrangement for the task of fine grained retrieval is proposed. Further, a technique to perform feature fusion to aggregate high-level semantic features extracted is also proposed. Weighted feature fusion helps in setting preferences to particular characteristics of the floor plan while retrieval and satisfying specific user demands. In an attempt to explore other query modes than query by example in the form of floor plan image, sketch based retrieval is proposed. Sketch based retrieval comes with its own set of challenges in terms of both representation and recognition. However, this mode of query can aid in better correspondence while capturing the user’s intent in a query. A composite network comprising of Cyclic Generative Adversarial Networks (Cycle-GAN) along with CNN is proposed to bridge the gap between sketch and image domains while retrieval. An improved approach using autencoders in conjunction with Cyclic Generative Adversarial Networks is proposed, which outperforms all other state-of-the-art techniques for sketch based floor plan retrieval by using an efficient domain mapping approach.