“Are we There Yet?” Experiences and Lessons in a GIS Conversion Project
BackgroundJay Johnson City of Tallahassee 300 South Adams Street, A6 Tallahassee, FL 32301 City of Tallahassee (COT) staff has spent considerable effort over the last two years converting their electric utility infrastructure GIS from the VISION platform to ArcFM. The project has been a considerable challenge to both COT staff and to the consultant hired to perform the conversion. This paper summarizes the difficulties encountered during this project and offers some of the lessons learned. COT entered into a contract with the consultant on March 26, 1999. The project schedule called for the entire conversion to be done over seven months. As this paper is being written, some 22 months later, the project is still not finished. The project included conversion of all existing electric GIS data from VISION to ArcFM, creation of custom functionality within ArcFM, and interfacing with an existing custom-built field-based pen computer system. Implicit in this conversion was moving staff from a system they knew and were comfortable with to a newly released system with which they had no experience. The learning curve was steep and because COT runs a lean organization, the work was concentrated on a very few shoulders. The objective of this paper is to distill the experience COT gained from the growing pains of this conversion into a set of observations other organizations can use to improve the efficiency and productivity of similar projects. The difficulties encountered in an inherited project will be discussed, as well as the actual advantages of coming in to “bat clean-up” on a project. The value of establishing detailed expectations and roles within the entire project team should be clear, yet this relatively simple concept was elusive in practice. Structuring of vendor invoicing/payments, managing subcontractors, establishing clear acceptance criteria, and resolving deadlocks are all critical elements of a large conversion project. What roles did partnership play in this project and how did the partnership evolve over the life of the project? How were issues resolved and what could have been done to improve on this process? Where do we go once the conversion is done? In truth, COT staff’s job has only begun once the conversion consultant has completed its work. The question staff keeping ask is, “Are we there yet?” The Nature of Project Failure Recent industry research estimates that 40% of IT projects fail (“IT Project Management Research Findings,” www.techrepublic.com). Why then do we continue to attempt to implement IT projects in the face of such stiff odds? We keep trying because the potential benefits are so large or because our organizational strategy demands aggressive IT implementations just to keep up with the competition. Either way, we are faced with daunting odds against success. Even if we view “failure” as an inability to meet budget or stay on schedule, can we as managers thrive in such an environment? Clearly it is in our best interest to understand the causes of major project failures and plan to reduce or avoid this risk. Failure in the COT electric GIS conversion project stems from several root causes: poor communication within the project team, underestimation of project complexity, and resource limitations. Each of these causes will be discussed. Poor Communication Within the project team The project team was composed of managerial and technical staff from both the conversion consultant and COT. The first instance of failure was perhaps the most fundamental and laid the groundwork for many of the problems that were to follow. The conversion consultant made a verbal commitment that the new system would have “all the functionality that existed in the old system.” Two mistakes occurred here. First, the consultant set itself up for failure by making such a sweeping, non-specific statement. Secondly, COT staff set themselves up by believing it. The two systems in question had numerous technical differences in the ways that they store and process data, making an exact reproduction of existing functionality unlikely if not outright impossible. By making a general claim to preserve all the existing functionality, the consultant encouraged the client to believe that the end product would be essentially identical in operation and function to the initial system. This might not have been what the consultant meant, but it is what the client heard. Once these expectations are set, they form the basis for all future discussions and it is very difficult to change them. Lesson One: Don’t make broad, sweeping claims. If you can’t stop yourself, follow the claims with specific details of exactly what those claims mean. This protects both parties. Lesson Two: Don’t accept broad, sweeping claims at face value. Question how these objectives will specifically be accomplished. During the course of the electric GIS conversion project, there was complete turnover of project management staff on both the consultant and client sides. This created a vacuum of information. Information that had been passed to the consultant was lost, agreements that had been verbalized but never formalized continually came out of the woodwork, and there was a loss of the “big picture” strategy of how to finish the project. The new project team was assembled to finish the project during the summer of 2000 and had great success dealing with numerous specific technical issues, but had greater difficulty dealing with the larger issues of whose responsibility certain tasks should be and how the project as a whole would conclude. A specific example of the problems caused by this lack of continuity within the project team was the interface between the GIS system and a pen-based computer system used for collecting field data. Portions of the GIS would be downloaded into the pen unit to be taken into the field to record changes in the electric infrastructure or to capture new infrastructure. The pen units would then be brought to the office and uploaded into the GIS. Under the old system, this was all accomplished by a highly customized interface that also handled multiple administrative tasks, such as data backups and synchronization of pen units with new software patches. COT staff had received assurance from the consultant’s first project manager that none of the pen unit software would be altered. However, when the “final” solution for interfacing the new GIS with the existing pen units was implemented, the majority of the custom functions had been dropped. Lesson Three: Get agreements and understandings in writing. If you are concerned enough about an issue to bring it up in a meeting, you should get the agreement in writing. Even if your written documentation is no more than an email reiterating the decisions and agreements from a specific meeting or conversation, at least you will have a record to refer to if the manager you had a “handshake agreement” with leaves the project. When the project team was reconstituted during the summer of 2000, there were a seemingly endless number of technical problems on the table. In order to get a handle on these issues and to control the scope of the project the project team used an “Open Issues” list to identify each pending problem, whose responsibility it was, and what action was being taken. This was very helpful in keeping track of the issues in an organized fashion and facilitating the resolution of issues. This open issues list was regularly reviewed during weekly status meetings. As solutions were found, specific problems would be removed from the list. In a complex conversion there are so many changes and issues that arise that a formal process for tracking their resolution is an absolute necessity. Lesson Four: Document problems and issues in an organized manner, assign responsibilities and required actions, and then review this all regularly. Underestimation of Project Complexity A common complaint heard from COT staff was that the consultant didn’t really understand portions of the project. To the consultant, the project was primarily a conversion job, with “some additional work” tacked on (i.e., the pen unit interface to the GIS). To COT staff, that “additional work” represented some of the most complex and essential portions of the entire project. When the consultant began work on the pen interface, problems of significant complexity began to emerge. The consultant didn’t truly understand what the custom code within the pen units was doing, which made it impossible to establish a comprehensive understanding of how the new solution should function. The primary problem came down to a set of tables that was supposed to map attribute values from the GIS into the pen unit and then back to the GIS after the pen unit was returned from the field. This sounds relatively straightforward, but due to the custom nature of the pen unit code and the complex structure of the GIS data model, it was if fact rather obscure. What would have been best accomplished through rigorous analysis of the necessary mapping between the two systems instead was approached from a “best guess” strategy. A subcontractor who believed the mapping was not his responsibility actually constructed the mapping tables to the best of his ability strictly for the purposes of getting enough functionality into the system to test the code he was actually responsible for. These very rough mapping tables were then given to COT , which ended up with the responsibility of trying to repair/enhance the tables with no master mapping scheme to assist them. COT staff worked many hours on this before the project team finally recognized that there was no effective way for this task to conclude in the “hunt and peck” manner it was being conducted. In the meantime, the consultant had declared the project finished; after the problem was elevated to higher management within both organizations, the consultant finally brought a plan for a comprehensive attribute mapping solution to the table. Lesson Five: Demand clearly defined contractor-client responsibilities. There is a tendency among some staff to “do whatever it takes” to get the work done. This in itself is laudable and can go a long way toward moving projects forward. It can also be a trap if responsibility is not clearly defined among other the project team members. This can lead to work being shifted onto the “workhorse” staff, without regard to the effect of this unplanned load on their other responsibilities. Resource Limitations Lesson Six: Dedicate reasonable staffing levels and cross-train, cross-train, cross-train. The consultant also had resource limitations that affected the project. Members of its staff identified in the RFP as possessing unique experience and expertise with the old GIS platform were not put on the project team. As the project schedule continued to slip, the consultant had serious budget concerns that restricted the amount of staff time they could continue to devote to the project. Mid-Project Course Corrections Given the multitude of problems that beset this project, it is not surprising that the project is far behind schedule. To give credit where credit is due, the consultant brought in a new team during the summer of 2000 and this infusion of qualified staff to the project did wonders for bringing the correct technical resources to bear on resolving nagging problems. On the COT side, a new project manager was able to facilitate problem solving primarily by helping the two sides communicate more clearly and with fewer misunderstandings. Some of the mutual frustration with the project was that the consultant felt that project completion should be based on delivery of major components, while COT technical staff had a definition of project success that relied on implemented functionality (i.e. Does the system work?). While this functional definition seemed unambiguous to COT, the consultant felt it left project completion hostage to COT’s judgment. It should be noted that these sorts of basic conflicts between the consultant and the client stem from the consultant’s initial –and unattainable– promise of the new system having all the functionality of the old system. Many of the problems within the project could have been prevented through a simple contract structure. COT’s contract set up payments to the consultant on a calendar basis, rather than on a deliverable basis. This meant that the consultant was paid without regard to whether the deliverables were on time or met acceptable standards. This unfortunately allowed the project to keep moving forward even when deliverables were not completed. As a specific example of the damage caused by this, the “Functional Requirements” document, which should have been completed up-front and should have been used to drive the fundamental design of the project, was actually written and delivered after the fact. It ended up as documentation of what had been done, rather than as a road map for what was to be done. Lesson Seven: Tie project payments to acceptance of project deliverables. The irony in this situation is that, regardless of the good working relationship between the members of the reconstituted project team, the history of the project had created divergent consultant-client expectations that were very difficult to work around and that continued to subtly undermine the resolution of project issues. It is difficult to recover from a project gone astray, both because of the conflict in expectations and, of course, because once the money has been spent to do something the wrong way, it is difficult to find the funds or resources to fix it. Lesson Eight: Create unambiguous functional criteria to define project success. This seems obvious, but too often we focus on the minutiae of product deliverables and ignore the overall functional needs that we are attempting to satisfy. When we buy a new car we not only want all the parts, pieces, and options to be delivered on time, we not only want the parts assembled in the correct relationship, we want the car to run perfectly and come with a full tank of gas. If we buy a new car, we can safely assume all of this. The same cannot be said of IT projects. From COT’s experience, it is clear that it is almost never too late to clarify the criteria that define project success. It may not be easy and there will likely be resistance from some part of the project team, but the sooner you have agreement over what constitutes success, the better off your project will be. The worst time to find that the consultant and the client disagree over the terms of the project is when the final invoice has been submitted. This is particularly important when stepping in as project manager on a partially completed project. We all know there are numerous disadvantages to “inheriting” a project. Perhaps the biggest difficulty in an inherited project is the lack of knowledge of project history. Different team members often seem to have differing recollections about why a certain task was done a certain way. It may be difficult to unravel the reasons behind certain team member behaviors that stem from historical interactions within the team or to understand mindsets within the team that are based on discussions that took place prior to the new manager’s arrival. One actual advantage to inheriting a project is that the project can be analyzed through a fresh perspective and attitude. IT projects that drag on long past their initial schedule can lead to team fatigue. The addition of fresh staff can have a positive effect in reenergizing the project. A manager coming in to the middle of a “troubled” project can give the entire team an opportunity to review where they have been, inventory the current problems, and reassess their strategies for moving forward with the project. In the COT project, one surprising development that indicated an issue that needed to be dealt with, related to team member roles. After some roundabout discussion with the COT GIS technical staff member of the project team it became apparent that this person did not understand her role within the project and in fact questioned whether she would remain with the project once the conversion was completed. This staff position is the sole dedicated GIS support for the electric department, so it was surprising that she would not understand her critical role to the long-term success of the GIS project within the electric department. In addition, the 75% dedicated engineering staff member indicated that after the conversion her long-term involvement in the ongoing GIS would be reduced to 25%. This reemphasizes the importance of knowledge transfer and cross-training to get the GIS staff up to speed on electric-specific business processes and to build the skill sets necessary for this person to take primary responsibility for the electric GIS once the conversion is done. Lesson Nine: Assess staff skills and confirm their understanding of roles within the project. A project difficulty that could have been completely avoided with proper planning was delivery of documentation and source code. When COT staff requested source code, they were informed that the code had been written by subcontractors who were not required to supply source code to the prime contractor; therefore COT was on their own in its attempt to obtain that code. Similarly, when project deliverable documents, such as user guides, were requested in an editable electronic format so that COT staff could enhance the documents, the consultant’s response was that they could not supply the documents due to company policies. Lesson Ten: Specify delivery of completed source code and editable electronic deliverables in the contract. Where are we Now? So what is the status of the COT electric GIS conversion project? In late October 2000 the project hit an impasse. The consultant insisted the project was complete and notified COT that the system was entering warranty; COT still asserted that the system was not operational and couldn’t go to warranty yet. The project managers on the two sides had exhausted their options and could not negotiate a mutually agreeable outcome. Subsequently, this issue was elevated to higher levels of management within both organizations. After a series of conference calls to clearly establish the critical issues, the consultant put together a plan to address COT’s issues and traveled to Tallahassee to present that plan. The technical plan was sound and met with approval from the project team. The consultant did, however, explain that they could only commit an additional 160 hours to the project and that additional work beyond that would be at COT’s expense. COT chose to accept the 160 hours, knowing that completion of the technical plan would require substantially more hours. From the COT project manager’s perspective, the project has been rehabilitated over the last six months and is nearing completion. The primary goal for the project manager at this point is to stay focused on finishing; responsibility for dealing with potential financial or legal repercussions has been passed to other managers further up in the organization. As of January 2001, the COT members of the project team have identified the critical portions of the project on which the consultant’s remaining time will be used; they have also detailed the other resources, both internal and external, that will be required to finish the project. It may be another four to six months, but with a sound technical plan in hand the end of the road is at last in sight. The project team finally has reason to hope that someday they will no longer wonder, “Are we there yet?” | ||
| © GISdevelopment.net. All rights reserved. |