Knowledge-Based Image Analysis for 3D Road Reconstruction
When a road segment is selected, the system focuses on the image regions around it and activates a set of image processing tools. Edge pixels are detected with the Canny operator, line extraction and 3D straight line generation are conducted using the methods described in Zhang and Baltsavias (2000).
An unsupervised classification method, ISODATA, is applied in image patches to separate road regions from other objects. For this purpose, 3 bands from different color spaces (derived from original color image) are used: a* from the Lab color space, one band computed with the R and G bands in RGB space as (G-R)/(G+R), and the S band from the HSI color space.
Road marks are a good indication of the existence of roads. In addition, in many cases the correct road centerlines can be derived directly from presented road marks. This is especially useful when the roadsides are occluded or not well defined, such as in cities or city centers. Road marks in high resolution images such as the one used in our work are thin lines with a certain width and distinct color (usually white or yellow), thus the road mark pixels can be roughly detected using color information. The road marks are then extracted by finding thin lines in the detected pixels. The structural matching method developed in Zhang and Baltsavias (2000) is applied to generate 3D road marks. We also detect cars on roads as an additional cue about the road existence, this is still under development.
With the information from existing geographic data and image processing, the knowledge base is established according to the general strategy. Note that one of the important characteristics of the built knowledge base is that all information in it is spatially related, and relations between 2D edges and their corresponding 3D straight lines are kept. The system then extract roads by finding 3D parallel lines that belong to a road and link them in sequence. In case of shadows, occlusions caused by trees and buildings, spatial reasoning is applied using the knowledge base. The main procedures are shown in Fig. 2. The key is the use of knowledge and image context as much as possible, working in 2D images and 3D object spaces, use of 2D and 3D interaction when needed, and reasoning the problematic area. The details of implementation can be found in Zhang (2000).
The system checks extracted lines to find 3D parallel lines. Only lines located in the buffer defined by VEC25, having a similar orientation to VEC25 segments and a certain slope are further processed. Since roads are on the ground, lines above ground are removed by checking with the DHM25. By checking with the image classification results, a relation with the road region (in, outside, at the border) is attached to each line. Two lines are considered as parallel if they have similar orientation in 3D space. The lines of a pair must overlap in direction perpendicular to the lines, and the distance between them must be within a certain range. The minimum and maximum distances depend on the road class defined in VEC25. The found 3D parallel lines are projected into the images and evaluated using multiple knowledge. The region between the projected lines must belong to the class road as determined by the image classification. Image processing tools such as those for road mark extraction are activated to extract additional cues about the road existence in the region.
The parallel lines passing the above check are considered as Possible Road Sides that are Parallel (PRSP). They compose a graph. The nodes of the graph are PRSPs, the arcs of the graph are the relations between PRSPs. Note that in occlusion areas, the arcs also represent the missing parts of a road between a pair of PRSPs. The width of two PRSPs should be similar. If there is no gap between two PRSPs with similar width, i.e. one PRSP shares points with another, and the linking angles between them in 3D space comply with VEC25, they are connected directly. In case of gap existing, the gap area is checked. This is called spatial reasoning in our
