GISdevelopment.net --> Proceedings --> Map Asia 2002 --> Keynote Sessions
Generation and Visualization of 3D-city and facility models using CyberCity Modeler
Dr. Franz Steidler, Prof. Dr. Armin Gruen and Dr. Xinhua Wang
CyberCity AG Zurich, Switzerland
Dr. Franz Steidler, Prof. Dr. Armin Gruen and Dr. Xinhua Wang
CyberCity AG Zurich, Switzerland
The efficient generation of data for 3-D city models and their handling in Spatial Information Systems has become feasible. The derivation of this data from aerial and terrestrial images with semi-automated techniques constitutes the most powerful tool currently available to fulfill this task. Semi-automated object extraction has become a viable concept for the generation of 3-D city models. CyberCity-Modeler (CCM) has been developed with the aim of creating not only buildings, but also other objects pertaining to a city model efficiently and with a high degree of flexibility concerning the level of detail. In its commercial implementation, CCM has been confronted with a number of user requirements which needed to be observed. This led to some extensions in functionality, which are addressed in this speech: Geometrical regularization of buildings, editing functions for topology adjustment, integration of facades and other vertical walls and modeling of overhanging roofs. These extensions of the original concept make CyberCity-Modeler an even more powerful tool for 3-D city modeling.
(Visualization left with CyberWalk, right with TerrainView)
Introduction
CyberCity-Modeler (CCM) represents a methodology for semi-automated object extraction and modeling of built-up environments from images of satellite, aerial and terrestrial platforms. It is generic in the sense that it allows to model not only buildings, but all objects of interest which can be represented as polyhedral model, which includes DTM, roads, waterways, parking lots, bridges, trees, ships and so forth. As such it produces 3-D city models efficiently, with a high degree of flexibility with respect to metric accuracy, modeling resolution (level of detail), type of objects and processing speed.
CCM is a commercial software product, marketed by the ETHZ spin-off company CyberCity AG (www.cybercity.tv). As a matter of fact, there is a steadily increasing interest for 3-D city models, with the current major customers being city planning and surveying offices, industrial facilities (chemical and car industry) and telecom companies. With the different types of customers comes a great variability in project specifications. Here it turned out to be of advantage that CCM was set up from the very beginning as a technique with high degree of flexibility. In spite of that, some additions had to be developed to extend of the original functionality in order to fulfill specific requests.
This paper reports about these extensions. After a brief review of the original CCM concept we will address the issues of regularization of buildings, editing functions and the integration of building facades, automated modeling of roof overhanging roofs as well as mapping wall textures.
CCM – the concept
CyberCity-Modeler, was designed as a tool for data acquisition and structuring for 3-D city model generation. From the very beginning, CCM has been devised as a semi-automated procedure. This was done in view of the need to observe the following constraints:
Figure 1 describes the work- and dataflow of CCM. The operator measures on an Analytical Plotter or on a Digital Station in the stereomodel individual points that fully describe the visible part of an object, i.e. the roof of a building.
Figure 1. Work- and dataflow of CCM
CCM presents a new method for fitting planar faces to the resulting 3-D point cloud. This face fitting is defined as a consistent labeling problem, which is solved by a special version of probabilistic relaxation. In theory there are various labeling methods available, but only one solution is desired, which meets the inherent topological constraints of the object. From a geometrical point of view, the inherent topological constraints can be summarized as:
The DTM, if not given a priori, can also be measured and integrated.
Texture from aerial images is mapped automatically on the terrain and on the roofs, since the geometrical relationship between object faces and image planes has been established. Facade texture is produced semi-automatically via projective transformation from terrestrial images, usually taken by camcorders or still video cameras. If it is necessary to map more than one image for one facade, the images are mapped together by CCM.
The system produces its own internal 3-D data structure, including texture. Interfaces to major public data formats are available.
The system and software are fully operational. In the order of 100’000 buildings at high resolution have been generated already with this approach. Figure 2 shows one of the models, the Congress Center RAI, Amsterdam, location of the XIX th ISPRS Congress 2000.
Figure 2. 3-D model of the Congress Center RAI, Amsterdam, produced with CCM (vector data, overlaid with natural texture).
Extensions of CCM
What looks like a complete approach and system from a scientific point of view may not necessarily fulfill some specific practical requirements efficiently. This is the case with the original approach of CCM. Whereas it is acceptable in most projects very well there are always some specifications, which require modifications and additions. One of those is geometric regularization. While CCM was built to model the objects as close to their existing size and shape as possible, there arises sometimes the need to regularize the geometry. Under these constraints do fall the requests to make straight lines parallel and perpendicular where they are actually not, or to have all points of a group (e.g. eaves or ridge points) at a unique height. Another problem grew from the fact that CCM was designed to handle individual buildings sequentially and independent of each other. Building neighborhood conditions were not considered. The geometrical inconsistencies originating from that fact, like small gaps or overlaps between adjacent buildings (in the cm/dm range), are not dramatic and tolerable in many applications, especially those
which are purely related to visualization. However, the topological errors constitute a serious problem in projects where the 3-D model is subject to legal considerations or some other kind of analysis which requires topologically correct data.
Another significant extension refers to the precise modeling of building facades. Facades are usually not visible in aerial images, but available in cadastral maps. We combine this facade information with the roof landscape modeled with CCM in order to be able to represent the roof overhangs.
In the following, we will describe some extensions in more detail. Figure 3 shows an example of improving geometric quality.
Figure 3. Workflow of CCM extensions
Geometrical regularization and neighborhood topology
Geometrical regularization refers to the task of modifying the geometry in such a way that regular structures are obtained. Measurements from images are always erroneous, although the errors may be very small. In addition, in particular with older buildings, the geometry deviates from regular patterns sometimes significantly. Edges are not parallel, intersections not perpendicular, roof faces not planar. We therefore have developed two strategies for regularization: A fully automatic adjustment based on least squares and a semi-automated approach of CAD editing. Both approaches are integrated in the software package CC-Edit.
The requirements for geometrical regularization are as follows:
We solve these requirements by formulating the constraints as stochastic constraints, i.e. as weighted observation equations in a least squares context. Details may be found in Gruen Wang 2001.
Figure 4. Correction of roof group by automated geometry regularization.
Regularization by CAD editing
This is a semi-automated supervised procedure, which operates only in planimetry. Therefore, it requires that the equal height condition is already observed during the point measurement phase. Then a grid of parallel construction lines is generated and overlaid to the measured lines. The measured lines are automatically adjusted to the direction of the grid. The grid's direction itself is derived from the average direction of the measured lines concerned. The selection of the concerned lines can be done automatically or manually. The overlay display is used for checking and manual editing if something went wrong.
The right angle, collinearity and the planar face constraints are automatically observed by that procedure. Since we use hard constraints here, the results are strict. An example is shown in Figure 5.
Figure 5. Line rectification (dotted line: before, solid line: after).
Topology adjustment
Inconsistencies in topology between adjacent buildings may arise because of measurement errors and because of mutually overlapping roofs.
Figure 6 shows a typical topology problem, which may exist even after the previous geometry regularization. For its solution, we provide both an automated and a semi-automated procedure.
Figure 6. CC-Edit: CCM user interface for editing with an example of topological inconsistency between adjacent buildings.
In the automated mode, the system selects a reference borderline which is kept fixed and onto which the points of the other lines are projected perpendicularly. As reference line, we select the longest line (which is supposed to be the most stable). In the semi-automated mode, this reference line is selected manually.
The functioning of the automated and semi-automated procedures described above can be monitored by an operator within an editing window as shown in Figure 6. This has of course a certain similarity with a CAD interface. It actually contains many typical CAD functions, but also others which are unique to our system and application-related. An example of automatic topology correction is shown in Figure 7.
Figure 7. CC-Edit: Result of editing with example of false (left) and automatically corrected topology (right).
Building facade integration
The aim is a higher level of detail in building modeling. Since facades are in general not visible in aerial images we use digital cadastral maps, which show the outer walls of buildings as part of the legal definition of real estate property. By integrating this infomation into the rooflandscape we are able to model the roof overhangs. What sounds like a simple problem at first sight turns out to be a formidable task to automate. In terms of structural detail, the rooflandscape looks very different from the facade landscape. Sometimes the maps are outdated and the roofs do not match the map content at all. Maps may also be inaccurate to an extent that the facade appears shifted and rotated with respect to the roof by a substantial amount. Facades can show a lot of additional, peripheral details, as for instance stairs and other add-ons (Fig. 8).
Figure 8. Plan view of roof landscape (dark) and the related facade representation from a map (light).
Figure 9 shows a result of automated facade integration. The problem is not yet solved in general terms and still needs some manual interference in complex situations. We will report about technical details of our approach in another publication.
Figure 9. Automated facade integration. Left: Plan view of roofs and facades from map. Right: Integration result.
Beyond facade integration, also other vertical wall sections, as they may appear on parts of a building and not be available from maps, need to be explicitly modeled as faces. This holds for all vertical building sections that do not constitute the legal building boundary. We also have developed a solution for this problem based on the intersection of gutter point projections onto other building parts like roofs, balkonies and terraces.
Visualization
Visualization is performed by realtime tools like TerrainView or Cyber Walk, level of details is included to reach sufficient performance. TerrainView and Cyber Walk are two competitive virtual reality based products with an interactive user interface. In particular they are able to display terrain and buildings as separate objects. Other visualization tools just present a common Digital Surface Model, where buildings are included in the landscape. They allow also free navigation within a selected geographic area. 3D enquiries can be performed by links to a 3D database, e.g. oracle. Both programs work with levels of detail (LOD) , what means foreground and background are displayed with different data. LOD is used to optimize performance and storage capacity. Additionally data is compressed and is decompressed during visualization. There are many more sophisticated features like flight paths, weather simulation. You may get additional information on TerrainView may at www.viewtec.ch and on Cyber Walk on www.muellersystemtechnik.de.
Conclusions
We have presented some extensions to the standard approach of CyberCity-Modeler, which were dictated by the requirements of users. We have developed solutions for geometry regularization, topology adjustment and vertical wall integration. In all cases we have provided automatic and semi-automatic approaches, with process and result monitoring possibilities for the operator.
As the requirements for high-resolution, precise, reliable and complete city modeling are increasing continuously, these additional functions are becoming more and more important.
The original concept of CCM, to use a semi-automated approach, has proven a valuable and successful concept and has also been the underlying philosophy of these new developments.
References
(Visualization left with CyberWalk, right with TerrainView)
Introduction
CyberCity-Modeler (CCM) represents a methodology for semi-automated object extraction and modeling of built-up environments from images of satellite, aerial and terrestrial platforms. It is generic in the sense that it allows to model not only buildings, but all objects of interest which can be represented as polyhedral model, which includes DTM, roads, waterways, parking lots, bridges, trees, ships and so forth. As such it produces 3-D city models efficiently, with a high degree of flexibility with respect to metric accuracy, modeling resolution (level of detail), type of objects and processing speed.
CCM is a commercial software product, marketed by the ETHZ spin-off company CyberCity AG (www.cybercity.tv). As a matter of fact, there is a steadily increasing interest for 3-D city models, with the current major customers being city planning and surveying offices, industrial facilities (chemical and car industry) and telecom companies. With the different types of customers comes a great variability in project specifications. Here it turned out to be of advantage that CCM was set up from the very beginning as a technique with high degree of flexibility. In spite of that, some additions had to be developed to extend of the original functionality in order to fulfill specific requests.
This paper reports about these extensions. After a brief review of the original CCM concept we will address the issues of regularization of buildings, editing functions and the integration of building facades, automated modeling of roof overhanging roofs as well as mapping wall textures.
CCM – the concept
CyberCity-Modeler, was designed as a tool for data acquisition and structuring for 3-D city model generation. From the very beginning, CCM has been devised as a semi-automated procedure. This was done in view of the need to observe the following constraints:
- Extract buildings, and other objects like traffic network, water, terrain, vegetation.
- Generate truly 3-D geometry and topology.
- Integrate natural (real) image textures.
- Allow for object attributation.
- Keep level of detail flexible. Accept virtually any image scale.
- Allow for a variety of accuracy levels (10 cm to 2 m).
- Produce structured data, compatible with major CAD and visualization software.
- Measurement.
- Structuring of data.
- Visualization, simulation, animation.
- Analysis.
Figure 1 describes the work- and dataflow of CCM. The operator measures on an Analytical Plotter or on a Digital Station in the stereomodel individual points that fully describe the visible part of an object, i.e. the roof of a building.
Figure 1. Work- and dataflow of CCM
CCM presents a new method for fitting planar faces to the resulting 3-D point cloud. This face fitting is defined as a consistent labeling problem, which is solved by a special version of probabilistic relaxation. In theory there are various labeling methods available, but only one solution is desired, which meets the inherent topological constraints of the object. From a geometrical point of view, the inherent topological constraints can be summarized as:
- a 3-D object is a closed multiple-plane object;
- planes are not supposed to pierce each other;
- every two adjacent boundary points are always part of a face.
The DTM, if not given a priori, can also be measured and integrated.
Texture from aerial images is mapped automatically on the terrain and on the roofs, since the geometrical relationship between object faces and image planes has been established. Facade texture is produced semi-automatically via projective transformation from terrestrial images, usually taken by camcorders or still video cameras. If it is necessary to map more than one image for one facade, the images are mapped together by CCM.
The system produces its own internal 3-D data structure, including texture. Interfaces to major public data formats are available.
The system and software are fully operational. In the order of 100’000 buildings at high resolution have been generated already with this approach. Figure 2 shows one of the models, the Congress Center RAI, Amsterdam, location of the XIX th ISPRS Congress 2000.
Figure 2. 3-D model of the Congress Center RAI, Amsterdam, produced with CCM (vector data, overlaid with natural texture).
Extensions of CCM
What looks like a complete approach and system from a scientific point of view may not necessarily fulfill some specific practical requirements efficiently. This is the case with the original approach of CCM. Whereas it is acceptable in most projects very well there are always some specifications, which require modifications and additions. One of those is geometric regularization. While CCM was built to model the objects as close to their existing size and shape as possible, there arises sometimes the need to regularize the geometry. Under these constraints do fall the requests to make straight lines parallel and perpendicular where they are actually not, or to have all points of a group (e.g. eaves or ridge points) at a unique height. Another problem grew from the fact that CCM was designed to handle individual buildings sequentially and independent of each other. Building neighborhood conditions were not considered. The geometrical inconsistencies originating from that fact, like small gaps or overlaps between adjacent buildings (in the cm/dm range), are not dramatic and tolerable in many applications, especially those
which are purely related to visualization. However, the topological errors constitute a serious problem in projects where the 3-D model is subject to legal considerations or some other kind of analysis which requires topologically correct data.
Another significant extension refers to the precise modeling of building facades. Facades are usually not visible in aerial images, but available in cadastral maps. We combine this facade information with the roof landscape modeled with CCM in order to be able to represent the roof overhangs.
In the following, we will describe some extensions in more detail. Figure 3 shows an example of improving geometric quality.
Figure 3. Workflow of CCM extensions
Geometrical regularization and neighborhood topology
Geometrical regularization refers to the task of modifying the geometry in such a way that regular structures are obtained. Measurements from images are always erroneous, although the errors may be very small. In addition, in particular with older buildings, the geometry deviates from regular patterns sometimes significantly. Edges are not parallel, intersections not perpendicular, roof faces not planar. We therefore have developed two strategies for regularization: A fully automatic adjustment based on least squares and a semi-automated approach of CAD editing. Both approaches are integrated in the software package CC-Edit.
The requirements for geometrical regularization are as follows:
- Same height for groups of eaves points, ridge points and other structure points
- Roof patches containing more than 3 points should form planar faces
- Parallelism of straight edges
- Right angles of intersecting roof edges
- Collinearity of edge points
We solve these requirements by formulating the constraints as stochastic constraints, i.e. as weighted observation equations in a least squares context. Details may be found in Gruen Wang 2001.
Figure 4. Correction of roof group by automated geometry regularization.
Regularization by CAD editing
This is a semi-automated supervised procedure, which operates only in planimetry. Therefore, it requires that the equal height condition is already observed during the point measurement phase. Then a grid of parallel construction lines is generated and overlaid to the measured lines. The measured lines are automatically adjusted to the direction of the grid. The grid's direction itself is derived from the average direction of the measured lines concerned. The selection of the concerned lines can be done automatically or manually. The overlay display is used for checking and manual editing if something went wrong.
The right angle, collinearity and the planar face constraints are automatically observed by that procedure. Since we use hard constraints here, the results are strict. An example is shown in Figure 5.
Figure 5. Line rectification (dotted line: before, solid line: after).
Topology adjustment
Inconsistencies in topology between adjacent buildings may arise because of measurement errors and because of mutually overlapping roofs.
Figure 6 shows a typical topology problem, which may exist even after the previous geometry regularization. For its solution, we provide both an automated and a semi-automated procedure.
Figure 6. CC-Edit: CCM user interface for editing with an example of topological inconsistency between adjacent buildings.
In the automated mode, the system selects a reference borderline which is kept fixed and onto which the points of the other lines are projected perpendicularly. As reference line, we select the longest line (which is supposed to be the most stable). In the semi-automated mode, this reference line is selected manually.
The functioning of the automated and semi-automated procedures described above can be monitored by an operator within an editing window as shown in Figure 6. This has of course a certain similarity with a CAD interface. It actually contains many typical CAD functions, but also others which are unique to our system and application-related. An example of automatic topology correction is shown in Figure 7.
Figure 7. CC-Edit: Result of editing with example of false (left) and automatically corrected topology (right).
Building facade integration
The aim is a higher level of detail in building modeling. Since facades are in general not visible in aerial images we use digital cadastral maps, which show the outer walls of buildings as part of the legal definition of real estate property. By integrating this infomation into the rooflandscape we are able to model the roof overhangs. What sounds like a simple problem at first sight turns out to be a formidable task to automate. In terms of structural detail, the rooflandscape looks very different from the facade landscape. Sometimes the maps are outdated and the roofs do not match the map content at all. Maps may also be inaccurate to an extent that the facade appears shifted and rotated with respect to the roof by a substantial amount. Facades can show a lot of additional, peripheral details, as for instance stairs and other add-ons (Fig. 8).
Figure 8. Plan view of roof landscape (dark) and the related facade representation from a map (light).
Figure 9 shows a result of automated facade integration. The problem is not yet solved in general terms and still needs some manual interference in complex situations. We will report about technical details of our approach in another publication.
Figure 9. Automated facade integration. Left: Plan view of roofs and facades from map. Right: Integration result.
Beyond facade integration, also other vertical wall sections, as they may appear on parts of a building and not be available from maps, need to be explicitly modeled as faces. This holds for all vertical building sections that do not constitute the legal building boundary. We also have developed a solution for this problem based on the intersection of gutter point projections onto other building parts like roofs, balkonies and terraces.
Visualization
Visualization is performed by realtime tools like TerrainView or Cyber Walk, level of details is included to reach sufficient performance. TerrainView and Cyber Walk are two competitive virtual reality based products with an interactive user interface. In particular they are able to display terrain and buildings as separate objects. Other visualization tools just present a common Digital Surface Model, where buildings are included in the landscape. They allow also free navigation within a selected geographic area. 3D enquiries can be performed by links to a 3D database, e.g. oracle. Both programs work with levels of detail (LOD) , what means foreground and background are displayed with different data. LOD is used to optimize performance and storage capacity. Additionally data is compressed and is decompressed during visualization. There are many more sophisticated features like flight paths, weather simulation. You may get additional information on TerrainView may at www.viewtec.ch and on Cyber Walk on www.muellersystemtechnik.de.
 1.) LOD 3: Bounding Boxes |
 2.) LOD2: roof structures are loaded |
 3.) LOD1: textures are loaded |
Conclusions
We have presented some extensions to the standard approach of CyberCity-Modeler, which were dictated by the requirements of users. We have developed solutions for geometry regularization, topology adjustment and vertical wall integration. In all cases we have provided automatic and semi-automatic approaches, with process and result monitoring possibilities for the operator.
As the requirements for high-resolution, precise, reliable and complete city modeling are increasing continuously, these additional functions are becoming more and more important.
The original concept of CCM, to use a semi-automated approach, has proven a valuable and successful concept and has also been the underlying philosophy of these new developments.
References
- Grün A., Steidler F., Wang X., 2000, CyberCity Modeler - ein System zur halbautomatischen Generierung von 3-D Stadtmodellen, Vermessungsingenieur 8-00
- Grün A., Wang X., 1998: CC-Modeler: A topology generator for 3-D city models. Presented Paper ISPRS Commission IV Symposium on “GIS - between Vision and Application“, September 7-10, Stuttgart
- Grün A., Wang X., 1999: CyberCity Spatial Information System (SIS): A new concept for the management of 3D city models in a hybrid GIS. Proc. 20th Asian Conference on Remote Sensing, November, Hongkong, pp. 121-128
- Grün A., Wang X, 2001 News from CyberCity-Modeler Monte Verita, Ascona, Switzerland
- Wang, X. & Gruen, A. 2000. A hybrid GIS for 3-D city models. International Archives of Photogrammetry and Remote Sensing, vol. 33, part 4/3, 1165-1172.