So far, the lines selected by the operator have formed the roof patch in single image and the intersections of the adjacent lines also could represent the corners of roof patch basically. Note here, one complete roof might consist of only one roof patch or several roof patches. For example, two slope roof patches compose one complete roof. It completely depends on the building construction and the generalization of the operator. All selected lines, besides the first one verified by the operator is a 3-D linear segment, might be pure 2-D linear segments without height information or 2-D lines with height datum, here called 3D lines. The subsequent procedure is to use the first 3-D line with correct height and the corners of roof patch to automatically find the equivalent in the other image. We develop three models for the roof reconstruction by simultaneously taking into account some constraints, e.g. height constraint and epipolar constraint. The operator could choose the optimal result from those three models. Hence, the recognition ability of the operator again is integrated into this system and this handling ensures the result correct. If the final result could not be found by these three models, the last choice is to reconstruct the roof patch by pure manual operation, e.g. point-wise measurements. This reconstruction isn’t stopped until all the roof patches are completed.

Reconstruction Models
After identifying the first 3-D linear segment and completing the selection of other lines of roof patch in one single image, the sequential job is to find the counterpart in the other image based on the roof corner in this image and some constraints. Fist, the system calculates the approximate x-parallax from the midpoint coordinates of the first 3-D linear segment. Because the used image pair are normalized images [Cho and Shenk, 1992], the approximate position of roof patch in the other image can be known along the epipolar line. The system searches the junctions, which are extracted by the method [Föstner, 1994], with the spiral way based on the approximate position. The first reconstruction model is to regard the first junction point as the corresponding roof corner in the other image when proceeding the spiral search way. The second model is to find all candidate junctions within 11*11 window size. Roof corners in single image and their candidate corresponding points could calculate the height by the space intersection. The minimum height difference among these candidate corresponding points and the first 3-D linear segment are the optimal choice in our system.

The first model takes the extracted junctions into consideration. Except for the epipolar and parallax constraints, no space information of roof patch is considered. Regarding of the second model, the heights of corresponding roof corners are limited based on the first 3-D height. Speaking about the third reconstruction model, it searches the corresponding lines on the other image according to the height information of the first 3-D linear segment and the azimuth constraint of this 3-D line on the image pairs during the reconstruction procedure. Of course, the wrong matching and linking 3D linear segments might be verified and corrected in terms of the first 3-D linear segment in this processing. Therefore, the first 3D linear segment will play an important role in this approach. The diagram in Fig. 2 will show more detailed about this model.


Fig.2 Digram of the third reconstruction model
Experiments and Results
Fig. 3 illustrates the test stereo image pairs and the results of the tests. The used images are reduced twice after the original photos were scanned with the resolution of 15 mm. The original photos were taken by the normal angle camera with 60% overlap in Tainan City. After reducing, their image scale is about 1: 8000. Meanwhile, there are two complete roofs in this test area. The topmost consists of the flat roof and other two slope roof patches compose the other gable roof. The two complete roofs are all partly occluded by the frame of flowers on the higher flat roof. Moreover, the higher flat roof also occludes the lower gable roof. Some annexes to the higher flat roof, looking like the roof garden, make the extraction of 2-D linear features broken and uncompleted. Of course, it also makes the linking and matching of 3-D linear segments get the wrong results. In Fig 3, those 3-D lines are displayed with yellow color and 2-D lines with cyan color. The red line prompted by the system means the initial 3-D line for roof reconstruction and the green points represent the junctions extracted by the Förstner extraction algorithm. Blue quadrangles mean the borders of roof patches selected by the operator or found by the reconstruction model. Small pink marks show the roof corners. For the sake of viewing easily, these pink marks have been enlarged.