Our general research area is computer vision and computer graphics, and our research problem is 3D reconstruction of window for architectural modeling from window images. The 3D reconstruction of window from images required the learning of 3D properties and viewpoint interpolation of window images. 3D representation will includes PBR mapping, depth mapping and viewpoint interpolation of image. This research is related to computer graphics and computer vision because research involves the artificially rendering of window images for Deep Learning and the Deep Learnning from generated window images. We conduct the research firstly by using auto CAD software to generate huge mount of gray images and their normal mapping, depth mapping and ortho mapping. After that, we use holistically nested edge(HED) detector and Pix2pixHD neural networks to process window images generated from auto CAD software and to generate realistic window images. With realistic window images, we will explore different kinds of neural networks to train model which can predict 3D properties such as PBR mapping. We already finish the generation of gray window images and their normal mapping, depth mapping and ortho mapping, and we are currently working on the using HED detector and Pix2pixHD neural network to generate realistic window images from gray window images. After getting realistic window images, the research and use of different neural networks, such as UNet, DeepLab and CapsNet, will give us way to learn the 3D representation of window images and the 3D reconstruction of window will be possible. The outcome of this resaerch also can be extended to other part of an architecture. This research will brings a great experience on researching in computer vision and computer graphics.
Author: Kunting Qi
Advisor: John Femiani, Department of Computer Science and Software Engineering
You must be logged in to post a comment.