| GISdevelopment.net ---> Technology---> Mobile Mapping |
A Real-time System for Road Management
H. Gontran, J. Skaloud, P.-Y. Gilliéron, F. A. Bayoud
Ecole Polytechnique Fédéral de Lausanne (EPFL),
Geodetic Engineering Laboratory,
1015 Lausanne, Switzerland
Email: herve.gontran@epfl.ch, jan.skaloud@epfl.ch, pierre-yves.gillieron@epfl.ch, fadiatef.bayoud@epfl.ch
ABSTRACT
Precise time registration, georeferencing and instantaneous feature extraction are important challenges for real-time mapping architectures. The Geodetic Engineering Laboratory at the EPFL has designed a real-time mobile mapping system to determine the geometry of the road via a single camera: the Photobus. This paper describes a method and experiences of distributed data preprocessing that enables Photobus to fulfill its mission in the shortest delays. In this context, image grabbing and processing is autonomously achieved by a vertically-oriented camera with an embedded PC, the Ethercam. Triggered by a GPS pulse to guarantee accurate synchronization, the vision system captures frames via CMOS pixels that are less affected by varying illuminations than their CCD counterparts. The resulting histogram-equalized images undergo feature extraction algorithms based upon binarization with an adaptive threshold and blob analysis. Implementing these image processing techniques on the camera-embedded OS helps the computer hosting the mobile mapping software to better focus upon other time-critical tasks. The second distributed system applies the concept of moving base station to provide a real-time determination of the position and heading of the Ethercam for the whole area covered by the Photobus. The proposed implementation relies on a dual-antenna GPS receiver whose primary chip permanently broadcasts RTK corrections to the secondary one, in order to obtain heading with degree-level accuracy. We designed an Internet-based regional DGPS server that contributes to the centimetre-level location of the primary GPS receiver chip on the Photobus through a GPRS connection. While gathering corrections at 5 Hz from an array of base stations, the server broadcasts RTK messages from the nearest reference. The paper describes experiences of GPRS data transfer in a production context, the effects of a changing base station and the advantages of a customized DGPS server compared with surveyor-oriented services.
1. INTRODUCTION
Road databases commonly use a linear referencing system (LRS) for a spatial description of elements of interest. A LRS is directly implemented on the road, with an origin and a set of marks painted on the pavement at each kilometre. Its use does not require an absolute localization of such marks, except for cartographic purposes. Most of the GIS applications now include a procedure for the dynamic segmentation of data that are referenced either in a LRS or in a national reference system. However, the road objects tend to be added using GPS-based positioning, which requires the description of the painted marks and of the centreline geometry in both systems to find the necessary transformation.
The acquisition of the needed transformation parameters initiated the design of a mobile mapping system by the Geodetic Engineering Laboratory of the Swiss Federal Institute of Technology in Lausanne. Our system can be distinguished from its predecessors by its ability to georeference the road centreline through a vertically-oriented CCD camera (Figure 1). This monoscopic technique is simple and economically appealing for rendering the road layout with sub-decimetre accuracy (Gilliéron et al., 2000).
Figure 1. The Photobus system
Nevertheless, the processing of the navigation data derived from a loosely-coupled GPS/INS remains semi-automatic and still requires considerable input from a skilled operator. On the other hand, the variation of light conditions conflicts with the automation of the centreline extraction from video by deceiving most algorithms of contour detection when a part or the whole image is over-saturated. Internet-based GPS-RTK positioning and implementation of a CMOS (Complementary Metal Oxide Semiconductor) image sensor help to deal with these issues. Both solutions are discussed in the paper and contribute to the goal of achieving a near real-time mobile mapping system. This will limit the operator’s intervention to the data collection in the field while ensuring immediate quality control.
2. ENHANCING THE POSITIONING COMPONENT
2.1 Positioning using GPS-RTK
A crucial element of any mobile mapping system is registering the frame pixels in a global coordinate system. This process, known as georeferencing, is partially limited by the accuracy of the positioning and orientation components of the image (Zhang et al., 2003). The decade-old RTK technology has gained a wide acceptance in geodetic engineering and in public works. This technology is an extension of relative positioning based on the interferometric principle of exploiting precise carrier-phase measurements in real-time. The attainable accuracy is at the centimetre level provided that the integer ambiguities can be resolved and fixed (Liu, 2003). In our context of a vehicle that is mapping the road centreline via a vertically-oriented camera, the positions of two GPS-RTK receivers and the derived azimuth provide efficient solutions under small banking angles (<4°). The underlying requirement for a successful RTK operation is the ability to transmit timely and reliably the reference station measurements to both rovers, where the integer ambiguity resolution is performed.
2.2 Format of the reference station measurements
The differential GPS message format plays a significant role in the reliability of any RTK operation since the induced data flow can overload the communication link that conveys the base station measurements to the rovers. The Radio Technical Commission for Maritime services (RTCM) was the first organisation to implement a standard structure for GPS corrections. Each RTCM message contains a variable number of 30-bit words, of which the first two serve as the header. In the frame of real-time positioning, messages 18 and 19 are of primary interest, and the minimal amount of broadcasted data is quantified by:
Flow [bps] = f × 2 × freq × (3 + 2 × N) × 5 (1)
where f = measurement rate
freq = 1 or 2, according to the mono or bi-frequency characteristic of the receiver
N = number of satellites.
However, it is necessary to note that the coordinates of reference station are also transmitted, yet at a slower rate than that of the corrections. Nine 30-bit words are necessary to describe the position of the reference station, which generates the peak output of:
Flow [bps] = f × (2 × freq × (3 + 2 × N) + 9) × 5 (2)
Recently approved for public use, the CMR (Compact Measurement Record) message was developed by Trimble to deliver the corrections over communication lines of reduced bandwidth. In its most recent implementation, the CMR+, the position of the reference station is transmitted in separate segments instead of a single block, as is done with the RTCM message. The formula describing the peak output is:
Flow [bps] = f × (6 + N × (8 + (freq – 1) × 7) + 16) (3)
A numerical example helps to illustrate these concepts. At the time of the signal reception of 7 satellites, at 5 Hz a dual-frequency receiver broadcasts at peak output:
• 5 × (2 × 2 × (3 + 2 × 7) + 9) × 5 = 1925 bytes/s = 15400 bps of RTCM corrections, (4)
• 5 × (6 + 7 × (8 + 1 × 7) + 16) = 635 bytes/s = 5080 bps of CMR+ corrections. (5)
In order to limit bandwidth and thereby avoid the saturation of most of the communication lines, we will base our experiments on the RTK-CMR+ corrections.
2.3 Transmission of GPS corrections via Internet
The choice of a suitable format of corrections is only one aspect of the deployment of the GPS-RTK technique. Extreme attention should be paid to the means of broadcasting data, which must handle the high flow of GPS corrections required for the accurate determination of the vehicle trajectory in real time. Because of the increased capacity of the Internet, on-line radios, which output continuous streams of Internet Protocol (IP) packets, have become well-established services. Real-time GPS data transfer requires relatively little bandwidth compared to these applications. Consequently, the dissemination of RTK corrections over the Internet constitutes an interesting alternative to the use of point-to-point links inherent in the radio and cellular networks.
2.4 NTRIP
In the context of the European reference frame, the Federal Agency of Cartography and Geodesy of Frankfurt (Bundesamt für Kartographie und Geodäsie) has developed a real-time technique for the exchange of GPS data over the Internet (Weber et al., 2003). The method, named NTRIP (Networked Transport of RTCM via Internet Protocol), calls upon a substantial array of servers that allows the simultaneous connection of thousands of users (Figure 2). This feature, as well as the difficulty to implement our own services on a NTRIP server, has driven us to investigate the possibility of using a single workstation as a server of CMR+ corrections.
Figure 2. The NTRIP architecture
As the http port (TCP * 80) is generally not filtered by a firewall, such a server can be built with the assistance of software that converts a serial flow of GPS data into TCP/IP blocks. From the client side, the rovers can gain access to this source of CMR+ messages, provided that they are connected to the Internet. The mobile Photobus platform requires the use of GPRS, a radio data transmission service that uses packet switching on a cellular network. Data are structured in the form of TCP/IP patterns and are able to reach a maximum flow of 171.2 kbps, which satisfies the bandwidth requirement expressed in formula 5. Nevertheless, a GPRS-compatible cell phone cannot transmit GPS corrections to a rover. In the case of a manual introduction of the server IP address, the cell phone web browser attempts to interpret the GPS corrections, which in turn causes the session to time out, thereby losing the GPRS attachment.
The solution lies in the integration of a GPRS module that embeds several Internet communication protocols and converts the TCP/IP data stream back to a serial link. Thanks to such a peripheral, the rovers behave as if they were directly connected to the server by a serial cable (Figure 3).
2.5 Towards a mobile NTRIP
The implementation of the broadcasted CMR+ messages via Internet is based on the condition that the server belongs to a local area network. To carry out an entirely mobile solution implies two simultaneous connections of GPRS modules that are equipping the base and rover stations with Internet access. Unfortunately, this is difficult to achieve as the cellular phone operators dynamically distribute private IP addresses to SIM cards that consequently do not accept any entering connections. To overcome this drawback, two solutions are foreseeable:
2.6 Field tests
The cell phone operators allocate a portion of their infrastructure to GPRS, which decreases the voice capacity and thus challenges the service quality. As opposed to voice, data transmission is particularly sensitive to the network design since each of its bytes is equally meaningful.
An interesting indicator of the availability of the cellular network is the Signal Quality Measure, as defined in GSM07.07 recommendation. Ranging from 0 (no signal) to 32 (excellent reception), such measures can be obtained by a periodic invocation of the Hayes command AT+CSQ. The results reflected in Figure 4 illustrate the constant coverage of Lausanne and its neighbourhood. Consequently, the GSM network in Switzerland is highly suitable for the broadcast of RTK solutions, since there is no significant degradation of performance compared to use of other transportation media (Figure 5). Nevertheless, particular attention should be paid to the conversion of serial GPS messages to TCP datagrams, as its misconfiguration enables only DGPS-code positioning accuracy.
Figure 3. Collecting GPS corrections via GPRS
Figure 4. GSM quality in poorly served areas in Lausanne
Figure 5. NTRIP results at EPFL (baseline < 1 km)
3. IMU/GPS INTEGRATION
Although the GPS provides very accurate position information (and attitude - when three GPS antennas are used) it would not be safe to fully rely on it. In cities, forests and tunnels the GPS signal is not available; so, another positioning system is needed. An IMU consists of a triad of accelerometers and a triad of gyroscopes. When the outputs of these six sensors are integrated with respect to time, the displacement and attitude are determined; however, these sensors suffer from biases and scale factor errors that cause the solution to degrade fast in time.
To tackle these problems, the GPS and the IMU are integrated in a Kalman Filter; since there are two GPS antennas, the external measurements of the Kalman Filter are GPS derived position and azimuth. This IMU/GPS integration, thus, provides a direct georeferencing for the camera (three position components and three attitude angles) in all circumstances. For details in the IMU/GPS integration, the reader can refer to Schwarz and Wei (2000), Titterton and Weston (1997). The development of real-time IMU/GPS Kalman Filter is not trivial. However, the GPS RTK procedure presented above facilitates one aspect of this integration. The other aspect is the real-time monitoring of reliability and integrity of the solution. This is provided by applying advanced statistical techniques and reliability testing.
The accuracy of the IMU/GPS integration depends on two factors:
To test our system we conducted many surveys in Switzerland, where forests and tunnels are common. The GPS outrages in these environments lasted for less than a minute with a very few cases exceeded the one minute. Simulated GPS outages were carried out to study the error in the IMU position. Figure 6 shows the error in the position derived from the stand-alone IMU for one minute.
Figure 6. Error in IMU-derived position
The true multi-base station processing has become a reality in many receivers. In this work, we highly consider the multi-base station processing because of the nature of our mission, where the photobus moves constantly and needs to get the DGPS correction from the nearest reference station. This is done by a training procedure and algorithmic additions that take into account all the reference stations in the vicinity. Before the survey, the reference stations in the vicinity are defined so that the algorithm can switch from one reference station to the other depending on the optimal distance that provides the best DGPS corrections.
4. ENHANCING THE IMAGING COMPONENT BY CMOS TECHNOLOGY
4.1 Introduction of CMOS technology
Over the past five years, there has been a growing interest in CMOS image sensors. Such imagers can be made with standard silicon processes in high-volume foundries. Peripheral electronics, i.e. digital logic, clock drivers, or analog-to-digital converters, are readily integrated with the same fabrication process. To achieve these benefits, the CMOS sensor’s architecture is arranged more like a memory cell or flat-panel display (Figure 7). Each photosite contains a photodiode that converts light to electrons, a charge-to- voltage conversion section, a reset and select transistor and an amplifier section. Overlaying the entire sensor is a grid of metal interconnects to apply timing and readout signals, and an array of column output signal interconnects. The column lines connect to a set of decode and readout (multiplexing) electronics that are arranged by column outside of the pixel array (Mendis et al., 1994). This architecture allows the signals from the entire array, from subsections, or even from a single pixel to be read by a simple X-Y addressing technique.
Figure 7. CMOS structure
4.1 Power consumption
Whereas CCD cameras require numerous chips for the sensor, drivers and signal conditioning, CMOS technology allows the manufacture of imaging devices that can be monolithically integrated as mentioned earlier. The reduced number of parts required has a positive impact on the power consumption while decreasing system size and complexity (Cho et al., 2001).
4.2 Noise and dark current
Fixed Pattern Noise (FPN) and random temporal noise eventually limit the performance of image sensors. FPN is time-dependent and arises from component mismatch due to process variations. Calibration or appropriate electronics can cancel FPN as shown in Figures 8 and 9.
Figure 8. CMOS frame without FPN correction
Figure 9. CMOS frame with FPN correction
The temporal noise includes:
4.4 Bandwidth and saturation
CCDs rely on a process that can leak charge to adjacent pixels when the CCD registers overflows. Thus, bright light blooms cause unwanted streaks on the image. CMOS architecture is inherently less sensitive to this effect. Moreover, smear that is caused by charge transfer in the CCD under illumination is non-existent with CMOS.
4.5 Introduction of a CMOS camera to Photobus
The Ethercam CMOS camera is a complete vision system that combines the functions of image acquisition and digital processing in a compact form (Figure 10). Interpreted results or raw images can be transmitted remotely to host computers through a 10-Mbit Ethernet connection. A serial RS232 interface and three optoisolated I/O lines allow synchronized image acquisition. Consequently, the connection to the computer hosting the mobile mapping software is simple and without a need a frame-grabber.
Figure 10. The Ethercam
The Ethercam embeds a Linux operating system, which supports high level programming languages and thus a wide range of vision libraries. Keeping the computation tasks within the camera helps its host computer to better focus upon time-critical tasks such as the synchronisation of the GPS-RTK position data with the captured frames.
The imaging sensor mounted on the Ethercam is a monochromatic matrix of VGA size, i.e. 640 ×480 pixels. It presents a dynamic range of six decades (120 dB) as a consequence of the logarithmic response of pixels to light intensity (Fossum, 1997). Such a response implies that relative variations of light intensity (.I/I) are perceived with constant sensitivity over the entire range. This property is particularly useful for the analysis of outdoor scenes where light intensity varies substantially from high sunny conditions (100 000 Lux) to dark shadows (10 Lux).
The previous surveys of the road with the CCD sensors showed that most automatic algorithms of centreline detection are deceived by varying light conditions (Gilliéron et al., 2001). In fact, using fast low-level filtration techniques, such as binarization, reject under-exposed pixels of shadowed areas or promote over-saturated pixels under the direct sunlight. Due to its logarithmic response to illumination and unlike a CCD camera, the Ethercam CMOS sensor allows the reproduction of the outdoor scenes without any imperfections such as blooming, smearing, or time lag (Figure 11). Hence, CMOS reproduction of the reality simplifies the methodology for extracting the pixel coordinates of the road centreline.
Figure 11. Light and shadow on the same frame, as seen by CCD (left) and CMOS (right) cameras.
5. APPLICATIONS OF THE SYSTEM AND CONCLUSIONS
Road management is an essential part of transportation engineering and it requires a fast and accurate road mapping. The system proposed here is an integrated system from which the user can study the road geometry and map the road features, which are considered as two critical characteristics for managing safety requirements and warning procedures (Gontran et al., 2005). We explored two promising solutions to map the road centreline in real-time: Internet-based RTK and a logarithmic CMOS camera. The presented enhancements due to the CMOS-based Ethercam relieved the workstation hosting the mobile mapping software from image grabbing and processing. Consequently, most of the computer time can be dedicated to more critical tasks and the processor can be given further help by a real-time operating system that allows a definition of a hierarchy of tasks. Data storage and visualisa-tion of the trajectory and the heading of Photobus are given a lesser priority than pixel georeferencing while the data synchronisation with GPS time is completed first (Figure 13).
To implement our future real-time mobile mapping system, we chose the RTLinux kernel that is well-established in the academic community. This kernel accomplishes real-time performances by monitoring device drivers, the use of interrupt disabling and virtual memory operations that are sources of unpredictability. In fact, the RTLinux kernel lies between the standard Linux kernel and the hardware, whereas the standard Linux kernel sees the real-time layer as the actual hardware. Theoretically, the user can introduce and set priorities to every task. Consequently, we can achieve correct timing for the processes by deciding on the scheduling algo-rithms, priorities and frequency of execution (Yodaiken, 1999).
Figure 13. Real-time approach for Photobus
Further development will involve a tighter integration of the sensors within the scheduled automatic algorithms of RTLinux. This should lead to the implementation of a quality check of mapped data directly in the field.
REFERENCES
H. Gontran, J. Skaloud, P.-Y. Gilliéron, F. A. Bayoud
Ecole Polytechnique Fédéral de Lausanne (EPFL),
Geodetic Engineering Laboratory,
1015 Lausanne, Switzerland
Email: herve.gontran@epfl.ch, jan.skaloud@epfl.ch, pierre-yves.gillieron@epfl.ch, fadiatef.bayoud@epfl.ch
ABSTRACT
Precise time registration, georeferencing and instantaneous feature extraction are important challenges for real-time mapping architectures. The Geodetic Engineering Laboratory at the EPFL has designed a real-time mobile mapping system to determine the geometry of the road via a single camera: the Photobus. This paper describes a method and experiences of distributed data preprocessing that enables Photobus to fulfill its mission in the shortest delays. In this context, image grabbing and processing is autonomously achieved by a vertically-oriented camera with an embedded PC, the Ethercam. Triggered by a GPS pulse to guarantee accurate synchronization, the vision system captures frames via CMOS pixels that are less affected by varying illuminations than their CCD counterparts. The resulting histogram-equalized images undergo feature extraction algorithms based upon binarization with an adaptive threshold and blob analysis. Implementing these image processing techniques on the camera-embedded OS helps the computer hosting the mobile mapping software to better focus upon other time-critical tasks. The second distributed system applies the concept of moving base station to provide a real-time determination of the position and heading of the Ethercam for the whole area covered by the Photobus. The proposed implementation relies on a dual-antenna GPS receiver whose primary chip permanently broadcasts RTK corrections to the secondary one, in order to obtain heading with degree-level accuracy. We designed an Internet-based regional DGPS server that contributes to the centimetre-level location of the primary GPS receiver chip on the Photobus through a GPRS connection. While gathering corrections at 5 Hz from an array of base stations, the server broadcasts RTK messages from the nearest reference. The paper describes experiences of GPRS data transfer in a production context, the effects of a changing base station and the advantages of a customized DGPS server compared with surveyor-oriented services.
1. INTRODUCTION
Road databases commonly use a linear referencing system (LRS) for a spatial description of elements of interest. A LRS is directly implemented on the road, with an origin and a set of marks painted on the pavement at each kilometre. Its use does not require an absolute localization of such marks, except for cartographic purposes. Most of the GIS applications now include a procedure for the dynamic segmentation of data that are referenced either in a LRS or in a national reference system. However, the road objects tend to be added using GPS-based positioning, which requires the description of the painted marks and of the centreline geometry in both systems to find the necessary transformation.
The acquisition of the needed transformation parameters initiated the design of a mobile mapping system by the Geodetic Engineering Laboratory of the Swiss Federal Institute of Technology in Lausanne. Our system can be distinguished from its predecessors by its ability to georeference the road centreline through a vertically-oriented CCD camera (Figure 1). This monoscopic technique is simple and economically appealing for rendering the road layout with sub-decimetre accuracy (Gilliéron et al., 2000).
Figure 1. The Photobus system
Nevertheless, the processing of the navigation data derived from a loosely-coupled GPS/INS remains semi-automatic and still requires considerable input from a skilled operator. On the other hand, the variation of light conditions conflicts with the automation of the centreline extraction from video by deceiving most algorithms of contour detection when a part or the whole image is over-saturated. Internet-based GPS-RTK positioning and implementation of a CMOS (Complementary Metal Oxide Semiconductor) image sensor help to deal with these issues. Both solutions are discussed in the paper and contribute to the goal of achieving a near real-time mobile mapping system. This will limit the operator’s intervention to the data collection in the field while ensuring immediate quality control.
2. ENHANCING THE POSITIONING COMPONENT
2.1 Positioning using GPS-RTK
A crucial element of any mobile mapping system is registering the frame pixels in a global coordinate system. This process, known as georeferencing, is partially limited by the accuracy of the positioning and orientation components of the image (Zhang et al., 2003). The decade-old RTK technology has gained a wide acceptance in geodetic engineering and in public works. This technology is an extension of relative positioning based on the interferometric principle of exploiting precise carrier-phase measurements in real-time. The attainable accuracy is at the centimetre level provided that the integer ambiguities can be resolved and fixed (Liu, 2003). In our context of a vehicle that is mapping the road centreline via a vertically-oriented camera, the positions of two GPS-RTK receivers and the derived azimuth provide efficient solutions under small banking angles (<4°). The underlying requirement for a successful RTK operation is the ability to transmit timely and reliably the reference station measurements to both rovers, where the integer ambiguity resolution is performed.
2.2 Format of the reference station measurements
The differential GPS message format plays a significant role in the reliability of any RTK operation since the induced data flow can overload the communication link that conveys the base station measurements to the rovers. The Radio Technical Commission for Maritime services (RTCM) was the first organisation to implement a standard structure for GPS corrections. Each RTCM message contains a variable number of 30-bit words, of which the first two serve as the header. In the frame of real-time positioning, messages 18 and 19 are of primary interest, and the minimal amount of broadcasted data is quantified by:
Flow [bps] = f × 2 × freq × (3 + 2 × N) × 5 (1)
where f = measurement rate
freq = 1 or 2, according to the mono or bi-frequency characteristic of the receiver
N = number of satellites.
However, it is necessary to note that the coordinates of reference station are also transmitted, yet at a slower rate than that of the corrections. Nine 30-bit words are necessary to describe the position of the reference station, which generates the peak output of:
Flow [bps] = f × (2 × freq × (3 + 2 × N) + 9) × 5 (2)
Recently approved for public use, the CMR (Compact Measurement Record) message was developed by Trimble to deliver the corrections over communication lines of reduced bandwidth. In its most recent implementation, the CMR+, the position of the reference station is transmitted in separate segments instead of a single block, as is done with the RTCM message. The formula describing the peak output is:
Flow [bps] = f × (6 + N × (8 + (freq – 1) × 7) + 16) (3)
A numerical example helps to illustrate these concepts. At the time of the signal reception of 7 satellites, at 5 Hz a dual-frequency receiver broadcasts at peak output:
• 5 × (2 × 2 × (3 + 2 × 7) + 9) × 5 = 1925 bytes/s = 15400 bps of RTCM corrections, (4)
• 5 × (6 + 7 × (8 + 1 × 7) + 16) = 635 bytes/s = 5080 bps of CMR+ corrections. (5)
In order to limit bandwidth and thereby avoid the saturation of most of the communication lines, we will base our experiments on the RTK-CMR+ corrections.
2.3 Transmission of GPS corrections via Internet
The choice of a suitable format of corrections is only one aspect of the deployment of the GPS-RTK technique. Extreme attention should be paid to the means of broadcasting data, which must handle the high flow of GPS corrections required for the accurate determination of the vehicle trajectory in real time. Because of the increased capacity of the Internet, on-line radios, which output continuous streams of Internet Protocol (IP) packets, have become well-established services. Real-time GPS data transfer requires relatively little bandwidth compared to these applications. Consequently, the dissemination of RTK corrections over the Internet constitutes an interesting alternative to the use of point-to-point links inherent in the radio and cellular networks.
2.4 NTRIP
In the context of the European reference frame, the Federal Agency of Cartography and Geodesy of Frankfurt (Bundesamt für Kartographie und Geodäsie) has developed a real-time technique for the exchange of GPS data over the Internet (Weber et al., 2003). The method, named NTRIP (Networked Transport of RTCM via Internet Protocol), calls upon a substantial array of servers that allows the simultaneous connection of thousands of users (Figure 2). This feature, as well as the difficulty to implement our own services on a NTRIP server, has driven us to investigate the possibility of using a single workstation as a server of CMR+ corrections.
Figure 2. The NTRIP architecture
As the http port (TCP * 80) is generally not filtered by a firewall, such a server can be built with the assistance of software that converts a serial flow of GPS data into TCP/IP blocks. From the client side, the rovers can gain access to this source of CMR+ messages, provided that they are connected to the Internet. The mobile Photobus platform requires the use of GPRS, a radio data transmission service that uses packet switching on a cellular network. Data are structured in the form of TCP/IP patterns and are able to reach a maximum flow of 171.2 kbps, which satisfies the bandwidth requirement expressed in formula 5. Nevertheless, a GPRS-compatible cell phone cannot transmit GPS corrections to a rover. In the case of a manual introduction of the server IP address, the cell phone web browser attempts to interpret the GPS corrections, which in turn causes the session to time out, thereby losing the GPRS attachment.
The solution lies in the integration of a GPRS module that embeds several Internet communication protocols and converts the TCP/IP data stream back to a serial link. Thanks to such a peripheral, the rovers behave as if they were directly connected to the server by a serial cable (Figure 3).
2.5 Towards a mobile NTRIP
The implementation of the broadcasted CMR+ messages via Internet is based on the condition that the server belongs to a local area network. To carry out an entirely mobile solution implies two simultaneous connections of GPRS modules that are equipping the base and rover stations with Internet access. Unfortunately, this is difficult to achieve as the cellular phone operators dynamically distribute private IP addresses to SIM cards that consequently do not accept any entering connections. To overcome this drawback, two solutions are foreseeable:
- Come to an agreement with a cell phone service provider in order to obtain routable IP addresses for SIM cards that require GPRS communications.
- Carry out a GPRS connection of the base and rovers to the fixed IP address of an Internet server, which will authorize the exchange of GPS data.
2.6 Field tests
The cell phone operators allocate a portion of their infrastructure to GPRS, which decreases the voice capacity and thus challenges the service quality. As opposed to voice, data transmission is particularly sensitive to the network design since each of its bytes is equally meaningful.
An interesting indicator of the availability of the cellular network is the Signal Quality Measure, as defined in GSM07.07 recommendation. Ranging from 0 (no signal) to 32 (excellent reception), such measures can be obtained by a periodic invocation of the Hayes command AT+CSQ. The results reflected in Figure 4 illustrate the constant coverage of Lausanne and its neighbourhood. Consequently, the GSM network in Switzerland is highly suitable for the broadcast of RTK solutions, since there is no significant degradation of performance compared to use of other transportation media (Figure 5). Nevertheless, particular attention should be paid to the conversion of serial GPS messages to TCP datagrams, as its misconfiguration enables only DGPS-code positioning accuracy.
Figure 3. Collecting GPS corrections via GPRS
Figure 4. GSM quality in poorly served areas in Lausanne
Figure 5. NTRIP results at EPFL (baseline < 1 km)
3. IMU/GPS INTEGRATION
Although the GPS provides very accurate position information (and attitude - when three GPS antennas are used) it would not be safe to fully rely on it. In cities, forests and tunnels the GPS signal is not available; so, another positioning system is needed. An IMU consists of a triad of accelerometers and a triad of gyroscopes. When the outputs of these six sensors are integrated with respect to time, the displacement and attitude are determined; however, these sensors suffer from biases and scale factor errors that cause the solution to degrade fast in time.
To tackle these problems, the GPS and the IMU are integrated in a Kalman Filter; since there are two GPS antennas, the external measurements of the Kalman Filter are GPS derived position and azimuth. This IMU/GPS integration, thus, provides a direct georeferencing for the camera (three position components and three attitude angles) in all circumstances. For details in the IMU/GPS integration, the reader can refer to Schwarz and Wei (2000), Titterton and Weston (1997). The development of real-time IMU/GPS Kalman Filter is not trivial. However, the GPS RTK procedure presented above facilitates one aspect of this integration. The other aspect is the real-time monitoring of reliability and integrity of the solution. This is provided by applying advanced statistical techniques and reliability testing.
The accuracy of the IMU/GPS integration depends on two factors:
- The quality of the IMU
- The duration of the GPS outages.
To test our system we conducted many surveys in Switzerland, where forests and tunnels are common. The GPS outrages in these environments lasted for less than a minute with a very few cases exceeded the one minute. Simulated GPS outages were carried out to study the error in the IMU position. Figure 6 shows the error in the position derived from the stand-alone IMU for one minute.
Figure 6. Error in IMU-derived position
The true multi-base station processing has become a reality in many receivers. In this work, we highly consider the multi-base station processing because of the nature of our mission, where the photobus moves constantly and needs to get the DGPS correction from the nearest reference station. This is done by a training procedure and algorithmic additions that take into account all the reference stations in the vicinity. Before the survey, the reference stations in the vicinity are defined so that the algorithm can switch from one reference station to the other depending on the optimal distance that provides the best DGPS corrections.
4. ENHANCING THE IMAGING COMPONENT BY CMOS TECHNOLOGY
4.1 Introduction of CMOS technology
Over the past five years, there has been a growing interest in CMOS image sensors. Such imagers can be made with standard silicon processes in high-volume foundries. Peripheral electronics, i.e. digital logic, clock drivers, or analog-to-digital converters, are readily integrated with the same fabrication process. To achieve these benefits, the CMOS sensor’s architecture is arranged more like a memory cell or flat-panel display (Figure 7). Each photosite contains a photodiode that converts light to electrons, a charge-to- voltage conversion section, a reset and select transistor and an amplifier section. Overlaying the entire sensor is a grid of metal interconnects to apply timing and readout signals, and an array of column output signal interconnects. The column lines connect to a set of decode and readout (multiplexing) electronics that are arranged by column outside of the pixel array (Mendis et al., 1994). This architecture allows the signals from the entire array, from subsections, or even from a single pixel to be read by a simple X-Y addressing technique.
Figure 7. CMOS structure
4.1 Power consumption
Whereas CCD cameras require numerous chips for the sensor, drivers and signal conditioning, CMOS technology allows the manufacture of imaging devices that can be monolithically integrated as mentioned earlier. The reduced number of parts required has a positive impact on the power consumption while decreasing system size and complexity (Cho et al., 2001).
4.2 Noise and dark current
Fixed Pattern Noise (FPN) and random temporal noise eventually limit the performance of image sensors. FPN is time-dependent and arises from component mismatch due to process variations. Calibration or appropriate electronics can cancel FPN as shown in Figures 8 and 9.
Figure 8. CMOS frame without FPN correction
Figure 9. CMOS frame with FPN correction
The temporal noise includes:
- Dark current shot noise, induced by thermally generated charge carriers.
- Electronic noise including 1/f noise, thermal noise and reset noise.
4.4 Bandwidth and saturation
CCDs rely on a process that can leak charge to adjacent pixels when the CCD registers overflows. Thus, bright light blooms cause unwanted streaks on the image. CMOS architecture is inherently less sensitive to this effect. Moreover, smear that is caused by charge transfer in the CCD under illumination is non-existent with CMOS.
4.5 Introduction of a CMOS camera to Photobus
The Ethercam CMOS camera is a complete vision system that combines the functions of image acquisition and digital processing in a compact form (Figure 10). Interpreted results or raw images can be transmitted remotely to host computers through a 10-Mbit Ethernet connection. A serial RS232 interface and three optoisolated I/O lines allow synchronized image acquisition. Consequently, the connection to the computer hosting the mobile mapping software is simple and without a need a frame-grabber.
Figure 10. The Ethercam
The Ethercam embeds a Linux operating system, which supports high level programming languages and thus a wide range of vision libraries. Keeping the computation tasks within the camera helps its host computer to better focus upon time-critical tasks such as the synchronisation of the GPS-RTK position data with the captured frames.
The imaging sensor mounted on the Ethercam is a monochromatic matrix of VGA size, i.e. 640 ×480 pixels. It presents a dynamic range of six decades (120 dB) as a consequence of the logarithmic response of pixels to light intensity (Fossum, 1997). Such a response implies that relative variations of light intensity (.I/I) are perceived with constant sensitivity over the entire range. This property is particularly useful for the analysis of outdoor scenes where light intensity varies substantially from high sunny conditions (100 000 Lux) to dark shadows (10 Lux).
The previous surveys of the road with the CCD sensors showed that most automatic algorithms of centreline detection are deceived by varying light conditions (Gilliéron et al., 2001). In fact, using fast low-level filtration techniques, such as binarization, reject under-exposed pixels of shadowed areas or promote over-saturated pixels under the direct sunlight. Due to its logarithmic response to illumination and unlike a CCD camera, the Ethercam CMOS sensor allows the reproduction of the outdoor scenes without any imperfections such as blooming, smearing, or time lag (Figure 11). Hence, CMOS reproduction of the reality simplifies the methodology for extracting the pixel coordinates of the road centreline.
Figure 11. Light and shadow on the same frame, as seen by CCD (left) and CMOS (right) cameras.
5. APPLICATIONS OF THE SYSTEM AND CONCLUSIONS
Road management is an essential part of transportation engineering and it requires a fast and accurate road mapping. The system proposed here is an integrated system from which the user can study the road geometry and map the road features, which are considered as two critical characteristics for managing safety requirements and warning procedures (Gontran et al., 2005). We explored two promising solutions to map the road centreline in real-time: Internet-based RTK and a logarithmic CMOS camera. The presented enhancements due to the CMOS-based Ethercam relieved the workstation hosting the mobile mapping software from image grabbing and processing. Consequently, most of the computer time can be dedicated to more critical tasks and the processor can be given further help by a real-time operating system that allows a definition of a hierarchy of tasks. Data storage and visualisa-tion of the trajectory and the heading of Photobus are given a lesser priority than pixel georeferencing while the data synchronisation with GPS time is completed first (Figure 13).
To implement our future real-time mobile mapping system, we chose the RTLinux kernel that is well-established in the academic community. This kernel accomplishes real-time performances by monitoring device drivers, the use of interrupt disabling and virtual memory operations that are sources of unpredictability. In fact, the RTLinux kernel lies between the standard Linux kernel and the hardware, whereas the standard Linux kernel sees the real-time layer as the actual hardware. Theoretically, the user can introduce and set priorities to every task. Consequently, we can achieve correct timing for the processes by deciding on the scheduling algo-rithms, priorities and frequency of execution (Yodaiken, 1999).
Figure 13. Real-time approach for Photobus
Further development will involve a tighter integration of the sensors within the scheduled automatic algorithms of RTLinux. This should lead to the implementation of a quality check of mapped data directly in the field.
REFERENCES
- Cho, K.-B., Krymski, A., Fossum, E., 2001, A micropower self-clocked camera-on-a-chip, Extended programme of the IEEE CCD & Advanced image sensors workshop, Lake Tahoe, USA.
- El Gamal, A., Fowler, B., Min, H., Liu, X., 1998, Modeling and estimation of FPN components in CMOS image sensors, Proceedings of SPIE, volume 3301, pp. 178-185.
- Fossum, E., 1997, CMOS Image Sensors: Electronic Camera-On-a-Chip, IEEE transactions on electron devices, Volume 44, No. 10, October.
- Gilliéron, P.-Y., Skaloud J., Merminod B., Brugger D., 2001, Development of a low cost mobile mapping system for road data base management, Proceedings of the 3 rd Symposium on Mobile Mapping Technology, 3-5 January, Cairo, Egypt.
- Gilliéron, P.-Y., Gontran H., Skaloud, J., 2002, Tests with the system Photobus for road data acquisition, Kinematische Messungen auf Strasse und Schiene, 17-19 September, München, Germany.
- Gontran, H., Gilliéron, P.-Y., Skaloud, J, 2005, Precise Road Geometry for Integrated Transport Safety Systems. ATRS, 5 th Swiss Transport research Conference, March 2005.
- Liu, G. C., 2003. GPS RTK positioning via Internet-based 3G CDMA2000/1X wireless technology. GPS Solutions: Theory, Practice and Applications of Global Positioning Systems including GLONASS and Galileo, Springer-Verlag.
- Mendis, S., Kemeny, S., Gee, R., Pain, B., Kim, Q., Fossum, E.. 1994, Progress in CMOS active pixel sensors, Proceedings SPIE volume 2172, pp. 19-29.
- Schwarz K. P. and M. Wei, 2000, INS/GPS Integration for Geodetic Applications, Lecture Notes ENGO 623. Department of Geomatics Engineering, University of Calgary, Canada, 2000.
- Titterton D. H. and J. L. Weston, 1997, Strapdown Inertial Navigation Technology, Peter Peregrinus Ltd., 1997.
- Weber, G., Dettmering, D., Gebhard, H., 2003, Networked Transport of RTCM via Internet Protocol, International Union of Geophysics and Geodesy General Assembly, 30 June-11 July, Sapporo, Japan.
- Yodaiken, V., 1999, The RTLinux manifesto, Proceedings of the 5th Linux Expo, 18-22 May, Raleigh, USA.
- Zhang, K., Xiao, B., 2003, Current status of low-cost GPS and mobile mapping systems, Proceedings of the Malaysia Geoinformation and Surveying Conference, 9-10 April, Kuching, Malaysia.