3D Mapping of Unstructured Outdoor Environments with High Uncertainty Using Monocular Vision Sensors and IMUs
Overview:
This project entails inferences from the data collected using a monocular sensor mounted on an unmanned robot. This work focuses on outdoor environments, i.e., mixed-built environments. For the collected data, a computer vision and data science workflow is required for generating a static or dynamic 3D map that is local, depth-aware, and navigational. This map is determined for situational awareness and localization without the GPS information. The implementation of such workflows leads to inferences from multiple scans of the environment by the robot. The outputs of the workflow are intended to be used for path planning for a robot or content for building a virtual world for graphics applications, including VR.
Given the heterogeneity of the environments owing to mixed elements (natural and synthetic) and multiple scans, the monocular data is expected to contain high uncertainty concerning its propensity for inference. Hence, this project also includes collecting LiDAR and stereo-depth data for generating the ground truth for the data science workflows. LiDAR is not the primary data source for this work owing to its operational cost for data collection.
This project falls in the scope of cognitive computing as it needs such computing paradigms for awareness of the environment of the robot [1]. This work combines our previous work on 3D virtual world reconstruction [2], and also road geometry reconstruction [3,4], and 3D object detection [5] from 3D automotive LiDAR scans.
Reference:
[1] Schuetz, S., & Venkatesh, V. (2020). The rise of human machines: How cognitive computing systems challenge assumptions of user-system interaction. Journal of the Association for Information Systems, 21(2), 460-482.
[2] Victor, A. C., & Sreevalsan-Nair, J. (2021). Building 3D Virtual Worlds from Monocular Images of Urban Road Traffic Scenes. In Advances in Visual Computing: 16th International Symposium, ISVC 2021, Virtual Event, October 4-6, 2021, Proceedings, Part II (pp. 461-474). Springer International Publishing.
[3] Katkoria, D., & Sreevalsan-Nair, J. (2022). Rosels: Road surface extraction for 3d automotive lidar point cloud sequence. In Proceedings of the 3rd International Conference on Deep Learning Theory and Applications-DeLTA (pp. 55-67).
[4] Katkoria, D., & Sreevalsan-Nair, J. (2022, July). Evaluating and Improving RoSELS for Road Surface Extraction from 3D Automotive LiDAR Point Cloud Sequences. In International Conference on Deep Learning Theory and Applications (pp. 98-120). Cham: Springer Nature Switzerland.
[5] Katkoria Dhvani Shaileshbhai, Spatiotemporal Analytics for Environmental Perception to Assist Autonomous Driving, Master of Science by Research Thesis, IIIT Bangalore, October 2022.
TEAM MEMBERS:
- Prof. Viswanath G., IIIT Bangalore, for semantic segmentation.
- Prof. Bharat Lohani, IIT Kanpur, for LiDAR sensing.