The task of digitalizing meandering complex spaces in 3D is a challenging one even with the most advanced instrumentation like lightweight terrestrial laser scanner or portable/wearable Mobile ...Mapping Systems (MMSs). The complexity and extension of architectonic spaces such as staircases, corridors and passages are such that the acquisition time using static devices becomes prohibitive and the accuracy using mobile devices gets affected by drift error leading to warped models or requiring abundant control measurements. This paper presents a photogrammetric portable fisheye multicamera solution for the 3D survey of complex areas that aims at being both handy and fast in the acquisition as well as more reliable ad accurate than common MMSs. The paper showcases a stress test conducted on five complex reconstruction trajectories selected from the meandering connection passages of Milan’s Cathedral. The tests are constructed as worst-case scenario to evaluate the accuracy and drift error amount of the proposed system in open-ended unconstrained paths. The results, though still suffering from moderate drift error, highlights the potential of the solution, especially in retaining the overall shape and orthogonality of the architectonic elements acquired.
HoloLens 2 is a state-of-the-art of Mixed Reality device that can augment the natural environment with virtual content in the form of holograms. Spatial mapping is the capability of the device to ...scan the environment and provide the user with a detailed 3D mesh model representation of its surroundings. The device can place holograms about real-world objects, perform occlusions and simulate physical interactions only having the geometric knowledge of its operating environment. HoloLens 2 mapping capabilities have been extensively tested. Nevertheless, no test was performed on monumental heritage buildings, where dimensions far exceed the typical sensor working range of the device. The authors want to evaluate the capability of the device to map monumental spaces. In these environments, dimensions, repetition of elements and heterogeneity of areas provide a real challenge for out-of-the-box Microsoft localization and mapping algorithms. The test case will be Milan Cathedral, inside the research project to build an MR application to help and enhance the systematic and planned maintenance intervention and inspections. Mapping capabilities have been tested in very different spaces: i) the south nave; ii) the empty volumes between the internal vaults and the external roof ("Sordine"); iii) the narrow spiral staircase connecting the south nave with the roofs lower level.
The research illustrated in this article aimed at identifying a good standard methodology to survey very narrow spaces during 3D investigation of Cultural Heritage. It is an important topic in ...today’s era of BIM modelling applied to Cultural Heritage. Spaces like staircases, corridors and passages are very common in the architectural or archaeological fields, and obtaining a 3D-oriented survey of those areas can be a very complex task when completeness of the model and high precision are requested. Photogrammetry appears to be the most promising solution in terms of versatility and manoeuvrability also considering the quality of the required data. Fisheye lenses were studied and tested in depth because of their significant advantage in the field of view if compared with rectilinear lenses. This advantage alone can be crucial to reduce the total amount of photos and, as a consequence, to obtain manageable data, to simplify the survey phase and to significantly reduce the elaboration time. In order to overcome the main issue that arise when using fisheye lenses, which is the lack of rules that can be employed to design the survey, a general mathematical formulation to precisely estimate the GSD (Ground Sampling Distance) for every optical projection is presented here. A complete survey of a real complex case study was performed in order to test and stress the proposed methodology, and to handle a fisheye-based survey from beginning to end: the photogrammetric survey of the Minguzzi Staircase. It is a complex service spiral-staircase located in the Duomo di Milano with a total height of 25 meters and characterized by a narrow walkable space about 70 centimetres wide.
Digital technology provides methods to record and preserve cultural heritage, support conservation and restoration efforts, and share our collective past with a worldwide audience. Between 2011 and ...2017, the 3D Survey Group from Politecnico di Milano operated an annual workshop in the medieval village of Ghesc in which photogrammetry and laser-scanner surveys were carried out. The point cloud data acquired in these activities has become “time slices” documenting different stages of the preservation interventions in Ghesc and the evolution of advanced survey techniques. The main objective of this research is to streamline the workflow of delivering immersive and interactive experiences for complex heritage by directly utilising the 3D survey point cloud data, whether derived from a photogrammetric survey, static laser scanner, or mobile mapping.A point cloud-based multiplatform application is designed and delivered with versatile functions. It runs on PC and VR devices to provide virtual access to the village and narrate its revitalisation story. Additionally, it operates on mobile devices with an AR feature that brings vibrancy to the on-site experience. This application integrates high-fidelity point cloud models, detailed information on vernacular architecture in the Ossola Valley, and information on the preservation project with gamified learning experiences. The unconventional approach of using points as rendering primitives in virtual applications offers a practical solution for visualising complex heritage, enabling an efficient transition from the data collection stage to the data sharing stage without the need for 3D reconstruction and intricate BIM modelling.
In recent years, Artificial Intelligence (AI) methods for 3D point cloud classification has assumed an essential role in the heritage field. The association of semantic information to 3D ...representations became a valuable instrument for measurement, analysis, education or maintenance, in particular in the Cultural Heritage (CH) sector. Moreover, the recent availability and reliability of head-mounted displays and glasses are allowing extraordinary immersive virtual experiences. This paper presents an end-to-end framework to handle large and complex 3D point clouds, from acquisition to semantic segmentation and final access in a Mixed Reality (MR) environment. Three completely different heritage scenarios are considered: the Temple of Neptune in Paestum, the Milan Cathedral and a large portion of Bologna’s porticoes. Mixed Reality experiences are described and shown based on the Microsoft HoloLens 2 device. A video of the results is available at https://youtu.be/Kd_3s0tIX04.
MIXED REALITY CONTENT ALIGNMENT IN MONUMENTAL ENVIRONMENTS Teruggi, S.; Fassi, F.
International archives of the photogrammetry, remote sensing and spatial information sciences.,
01/2022, Letnik:
XLIII-B2-2022
Journal Article, Conference Proceeding
Recenzirano
Odprti dostop
Mixed reality provides on-the-spot and real-time data access capabilities by making virtual models and information more intuitive and accessible. Furthermore, allowing the operator to perceive 3D as ...holograms would allow for a more natural and straightforward manipulation of the perceived 3D content by permitting the augmentation of real objects with various levels of data. This can be accomplished by appropriately registering and superimposing the presented 3D models with the surrounding environment. This work aims to provide a quantitative evaluation of HoloLens 2 capabilities in registering virtual content inside monumental spaces. Two different methodologies are evaluated: Vuforia image targets and Microsoft World Locking Tools (WLTs). Tests have been performed inside Milan Cathedral's monumental spaces. Here, ambience dimensions, single architectural element repetition and non-uniform lighting conditions push out-of-the-box methods to their limits. Results show that WLTs with their space pins API can correctly reference virtual content keeping deviations in the order of 15 cm coping with the scale error produced from sensors’ drifts.
The advent of mobile mapping systems (MMSs) and computer vision algorithms has enriched a wide range of navigation and mapping tasks such as localisation, 3D motion estimation and 3D mapping. This ...study focuses on Visual Simultaneous Localisation and Mapping (V-SLAM) in the context of two in-houses MMSs: Ant3D, a patented five-fisheye multi-camera rig and GeoRizon, a high-resolution stereo fisheye rig. The aim is to leverage V-SLAM to enhance the systems performance in near-real-time and non-real-time 3D reconstruction applications. The research investigates both Monocular and Stereo V-SLAM applied to both MMSs and tackles the challenge of combining the V-SLAM estimated trajectory of one or a pair of cameras with known multi-camera relative orientation. We propose a state-of-the-art code that serves as a flexible and extensible platform for MMSs image acquisition and processing, along with an adapted version of the well-established ORB-SLAM3.0. Evaluation is performed in a cultural heritage challenging setup: the Minguzzi spiral staircase in the Duomo di Milano Cathedral. Performed tests highlight that introducing V-SLAM trajectories as well as pre-calibrated interior orientation and multi-camera constraints improve speed, applicability and accuracy of 3D surveys.
Image classification and object detection techniques have been widely discussed and developed in recent years; they are the basis of various prosperous applications, for example, real-time mapping. ...Promising as it is, the practical test in the cultural heritage field encountered multiple problems. In this paper, the authors attempt to share the research experimentations and the empirical knowledge focusing on the classification and detection of architectural pathology. The tests are built on elaborated training sets annotated with analysed and in-advance defined categories. The trained models were examined from the perspective of evaluation sets, model explanation and unseen datasets. The outcomes indicated the mistakes and confusions behind things and stuff in the object detection efforts, to which cultural heritage and architectural field are closely related. The model also reveals specific visual patterns for recognition from thousands of instances in the training set. By digging into different aspects of model performance, the potential and limitations of these techniques in practical applications can be better understood.
Photogrammetric applications nowadays envisage the use of more and more low-cost cameras such as those equipped on commercial UAV platforms. Typically, these low-grade cameras suffer from extreme ...radial distortion and strong vignetting among other defects. This, initiated a trend among the low-cost cameras’ manufacturers to try to hide the camera defects by applying software pre-corrections to the images. These Built-In Correction Profiles gets applied to both the JPG files, directly in-camera, and usually to the raw files as well, through the opcode functions of the DNG standard. In this paper we rise this issue that is still under-reported in the literature and further assess the accuracy implication of applying or discarding the Built-In Correction Profile in the scenario of UAV mapping. We tested the commercial UAV DJI Phantom 4 Pro v2 in a calibration environment and a field test to compare the performance of pre-corrected versus uncorrected images. In our tests, processing the original uncorrected images led to improved IO calibration and reduced bowing effect in the field test.
Multi-camera devices are increasingly popular in various metrological applications, including cultural heritage digitalisation, where these devices are adopted as low-cost alternatives to more ...traditional methods or mobile mapping systems. They can be of two types: panoramic and non-panoramic configurations, with the former usually more compact and ready-made off-the-shelves and the latter usually custom-developed for metrological applications. In the paper, we compare the accuracy and reliability performance of two types of multi-camera: the spherical camera INSTA 360 Pro2 and the custom multi-camera rig Ant3D. The case study is a challenging spiral staircase environment, typical in many cultural heritage survey projects. The processed image datasets were evaluated in the most common constrain scenario (GCPs at both ends of the staircase) and the worst-case scenario (open-ended path, GCPs at the start). The datasets were processed with precalibrated IO and various degrees of multi-camera constraints up to precalibrated relative orientations. The results highlight that the nominal scale 1:50 can be achieved, e.g. an accuracy of <2 cm plus complete and precise point clouds and mesh results.
PDF
