Risk management for laboratory automation projects

2.2.1. Risk analysis 

At the top of Figure 1, the input should be at key stages of the project such as when a project definition document is written, system selection, or before implementation and rollout. From the project plan, the individual tasks can be identified for this portion of the work and analysed for potential risk factors. Using the knowledge and experience of the project team members, risk analysis can be carried out and potential risks identified; alternatively, some of the risk management tables in this tutorial can be used (see Table 1, Table 2, Table 3, Table 4, Table 5, Table 6).

Table 1. Risk factors associated during project definition stages

	Risk factor	Low risk	High risk
	Project scope	Well defined	Undefined
	Project deliverables	Well defined	Undefined
	Benefits of the system	Quantified business impact	Undefined
	System requirements	Straightforward, using standard components and technologies	Complex, using custom components and new technologies
	Personnel providing application knowledge	Knowledge of both IT and user areas	Lack user area knowledge
	Project members who have experience of business area	All project members	None
	Status of documented procedures in user area	Complete and current	Nonexistent or outdated documentation
	Number of computer applications that will interface with the system	None	Two or more computer applications
	Status of these applications	Operational	Under development
	Resource required	<5 resource years	>15 resource years
	Time to develop the system	<12 months	>3 years

Table 2. Risk factors associated with system sponsorship and user commitment

	Risk factor	Low risk	High risk
	Project sponsor	Identified and has a strong user influence	Unknown
	Attitude of user management	Fully support the project	Resistant and skeptical
	Attitude of the users	Understand and support the project	Resistant and skeptical
	Organizational maturity	Able to use the system effectively	Unable to understand rationale for the system or use it
	Relationship of the project to the strategic IT plan	Included in the plan	Not included in the plan

Table 3. Risk factors associated with the impact of the system on the organization

	Risk factor	Low risk	High risk
	The new system	N/A	Replaces an existing one
			A completely new system
	Effect of the system on the IT department of the organization	Little change	Much change
	Procedural changes required by new system	Little change	Much change
	Organizational structures	None required or no changes to existing ones	Not considered
	Policy changes required to support the new system	None	Extensive

Table 4. Risk factors associated with project management

	Risk factor	Low risk	High risk
	Experience of project manager	3 or more projects	No prior experience
	Managing the project	Full time	Part time
	Project team assigned	Full time	Part time
	Experience of team members as a team	All worked together before	All are strangers
	Number of times application/system implemented	More than once	No experience of this application
	Team location	In one place	In several locations

Table 5. Risk factors associated with system selection

	Risk factor	Low risk	High risk
	New or non standard hardware or system software required	No	Yes
	Team has experience of this application/system	Expertise available	Used for the first time
	New language(s) required by the project	None	Used for the first time
	Database used in the application	Well established in the organization	New in the organization
	The system processing	Batch processing	Distributed system
	On-line response required	>7 seconds 90% of time	2 seconds or less 90% of time
	System availability	95%	>99%
	Technology mix (database, network, etc)	Existing or simple architecture	New or complex architecture
	Team's knowledge of the package	Previous experience	No knowledge
	Organization has worked with the vendor	Three or more times before	Never
	The package matches the system requirements	Well (little customization required)	Poor (major customization required)
	Computer department involvement in package selection	High involvement	Not involved
	User department involvement in package selection	High involvement	Not involved
	Vendor reputation	Good	Poor or unknown

Table 6. Risk factors associated with system development and implementation

	Risk factor	Low risk	High risk
	System scope fixed and prioritized	Yes	No
	Changes in organization or policy implemented	Yes	No
	Change control procedures in place	Yes	No
	Scope matches existing or proposed working practices	Yes	No
	Documented roll-out plan available, detailing who, when and where	Yes	No
	Sympathetic users identified for prototyping and testing	Yes	No
	Quality Assurance involved for pre-operation audit and GMP advice	Yes	No
	Development documentation to be produced identified	Yes	No
	Performance of development system matches expectation	Yes	No
	Contingency plan available if performance not good enough	Yes	No
	Focused training planned and agreed with the users	Yes	No
	Documentation available for users, backup and support staff	Yes	No
	Assistance arranged for new users in period after training	Yes	No
	Work schedules altered to cope with post training productivity loss	Yes	No
	Plan for management of user expectations	Yes	No

For any stage of the project, the risk analysis process is to:

2.2.2. Risk evaluation 

The evaluation process is very simple—it asks the question, Does the risk need to be mitigated or not?

If the answer is “no,” then the risk is accepted and nothing further is required. However, if there is a requirement for mitigation, then the risk moves into the next stage of the process: risk control. Typically, only high-risk factors will pass to the next stage; however, this decision depends on the criticality of the project in question.

2.2.3. Risk control 

Once the high-risk tasks have been highlighted, then it is possible to prepare plans and countermeasures to overcome the risk. “Risk Factors during Project Definition and Start-up,” “Evaluation and Selection of the System,” and “Risks Associated during Development and Rollout” discuss the risk and some of the possible approaches to mitigation. Note that it is not always possible to eliminate a risk, as this may be impossible or require too much effort; however, sufficient work needs to be done to ensure that the impact of the risk is managed and is acceptable.

The project manager then implements these approaches within the updated project plan. Milestones of the project can be identified and progress of the project reviewed against these; the same is true of risk factors. Once the progress of the project has been evaluated, this can be fed back into the system for an updated risk analysis. As can be seen, risk analysis is linked very closely with project management, and the two approaches should operate intimately.

2.2.4. Project risk information 

As the project progresses, a body of information is collected. It describes how the project risk has been managed and how effective the approaches have been. It is important that this information is not forgotten or ignored. Reuse, cross-reference, and update the information; it can be used to feed back into the risk cycles as shown in Figure 1. Also, experience from failed projects, discussed briefly in “Learning from Failure,” should also be incorporated in the organization's experience of automation projects as lessons to be learnt for the future.

3.2.1. Defined business benefits 

To avoid premature cancellation of the project due to budget cuts or management change, define and, if possible, quantify the business benefits that the system is anticipated to deliver. Senior management will need this information for project approval if over a preset spending limit, and continued management support is essential for longer-term projects. The advantages or direct benefit to the organization and the user should be outlined. To obtain the information to write this authoritatively, involve experienced staff from the user environment as well as laboratory customers.

The business benefits are useful for defining, in another way, the target of the system to be developed. This definition can be of positive use in helping to make decisions concerning which functions are to be evaluated during selection and development.

3.2.2. System requirements and documentation of current procedures 

At the start of a project, the system requirements are relatively vague and can hide a number of complex technical, procedural, and even organizational issues within them. However, even at this early stage, the requirements of a project and the operations carried out within the target laboratory should give an understanding of the degree of complexity involved. If the system to be implemented appears to be complex, a number of approaches to reduce the risk can be suggested:

If current procedures are documented, these will help define the current practices and system. In contrast, poor, outdated, or no documentation can cause assumptions, perhaps wrong ones, to be drawn and requirements defined from incorrect information. Great care must also be taken not to assume that even if practices are defined, say in standard operating procedures (SOPs), that the current working practices in the laboratory match them. No assumptions should be made in this respect, as these documents may be major works of artistic fiction. Enlist the help of expert users to help define the current system.

It is essential to define current working practices, modify them where necessary, and map them onto a proposed system to help selection.

3.2.3. Knowledge of the project team members and users 

The skills of all of the project team members should be assessed before the start of the project. Clearly, where the IT and laboratory automation members have been involved successfully with similar projects in the past, especially within the same area, there will a high degree of confidence and technical skill. In contrast, a team that is new and has little experience will require team building and technical training, ideally, before the project starts.

Equally, the experience of the user representatives working with the project team members may not be adequate to get a high-performance team working immediately. Therefore, some on-the-job training in project team membership may be appropriate.

An issue of major concern is the degree and depth of understanding and knowledge each member has in the other team members' disciplines. When there is none, a level of common understanding has to be developed. Equally important is the degree of knowledge and understanding the computer staff has in the laboratory. In the author's experience, this understanding takes much effort to acquire but is a worthwhile investment. A corollary is that carefully trained IT staff must be retained by the project; otherwise, momentum will be lost. To reduce risk before the project is fully underway, management has the responsibility to ensure that the team is formed and has the required level of knowledge and understanding to do the job.

3.2.4. Interfaces to other systems 

This aspect is not always identified in project proposals, but for integration to form an efficient organization, IT and automation projects should not exist in a vacuum but interface with each other to provide an integrated information environment. If the system is a standalone, no interfaces with other systems are deemed necessary, and development can proceed unhindered. In contrast, if the system has to interface with one or more systems, this adds complexity and risk. The interfaces, especially the data inputs and outputs, must be carefully defined and documented along with the responsibilities of who does what. This is acceptable if the systems already exist and are functional, as the interfaces are tangible.

Problems can arise if the interfaces are with proposed or partially developed systems, since interfacing with these applications increases the risk assessment. Now, additional time is required to identify where the projects overlap and how they should interface; liaison between projects is essential. Liaison may include sharing project team members, planning inter-project dependencies, or identifying the other project's deliverables.

3.2.5. Resources for project development 

Smaller projects carry less risk. Therefore, to minimize the risk for larger projects, there are a number of measures that can be used to reduce the risk to acceptable proportions:

3.2.6. Project timescales 

In today's organizations, a timescale of 2 to 3 years is a long time; in some companies, this can be the expectancy of an organizational structure or time between mergers. Therefore, if the timescale exceeds this, the project is unlikely to complete before the next change and is very unlikely to bring business benefit. Therefore, projects that last longer than 18 to 24 months are high risk. As described above, reduce the scope or break the project into a number of clearly defined phases. However, in doing this, it is essential that each phase provides defined and quantifiable business benefits of itself, or it is not work doing.

3.3.1. Sponsorship of the project 

The best way of identifying a project sponsor is to ask the question, Who provides the money? Active sponsorship of large projects is important to persuade people to use the new system. A sponsor who is just a figurehead is a green light to wasting a large amount of money, as there will be few questions asked if the project fails to deliver. Senior management's understanding of automation and computer projects is usually based on two premises:

These perceptions must change.

If the project sponsor is not strong, political battles within the organization units under this individual can result in project delays due to a lack of decision or management commitment, especially in large projects. Therefore, to avoid these problems, procedures for resolving disputes should be instigated by the user management.

3.3.2. Attitude and commitment of user management 

The attitude and commitment of the user management is essential for the success of any project. Apparent lack of commitment may indicate that they are unaware of the potential benefits that the system may bring or of plans to change the laboratory's direction. The manager of the user area should be briefed regarding the benefits that the system should deliver and its ability to enhance the business objectives of the specified area.

Winning the hearts and minds of user management is one option. If the project sponsor links a performance bonus to a successful project implementation, this adds the dimension of the wallet or purse to the equation.

3.3.3. Attitude of the users 

Even with total commitment of the project sponsor and user management, users can cause serious problems throughout the whole project by refusing to cooperate during all stages of development. This may be the result of fears of radical change that would result from the operation of the system. Mechanisms for effective communication to representatives of the user community need to be established by regular status reports or meetings that should be continued throughout the development cycle. Concerns of the users should be communicated to the project team and management. If there are organizational impacts of the system (see “Changes in Organizational Structure”), these should be identified and communicated to the users and discussed to obtain a consensus agreement.

The maturity of a user organization to support a LIMS effectively is a factor to be considered early in a project. If, in the judgement of the project team or user management, the organization is not capable of supporting an automated system, then an education programme should be undertaken for the whole user base.

3.3.4. Organizational maturity 

Not often considered when assessing the risks of a laboratory automation or IT project is the capability of the organization to implement the new system. This will vary, but a simple way to find out is to review how successful previous projects of this type have been. Projects that have been an outstanding success, rare but they do occur, will be a pointer towards the successful design and exploitation of automation or IT. It may also indicate that risk taking is encouraged, although this needs to be established by direct evidence of a corporate policy or equivalent.

More often, if this is the third or fourth attempt at a project, it will indicate a poor track record and, therefore, a high-risk project that has to be handled more carefully and in a risk-averse manner.

3.3.5. Relation of the project to the strategic plan 

If a project falls outside of the scope of a strategic automation or IT plan, then the risk increases as the obvious question arises, Why is this project important? Conflicts may result from an unplanned project being given priority and resources over existing ones. It is best to find out the reasons for a new system that is outside of an existing plan. If there has been a change in the business strategy, then the IT strategy should be revised accordingly. The place of the new project and any dependencies among other projects should be identified. A strong and committed project sponsor may be required.

3.4.1. New or replacement system? 

Regardless of whether a new or replacement is considered, both present a high risk; however, the nature of the risks are slightly different and are discussed here. Both approaches impact the user community by requiring it to change; it is the issue of change that presents the risk and, thus, it must be effectively managed.

A replacement system should not present too many problems with respect to the impact on the organization, if, and only if, the replacement simply automates what was automated in the previous one. However, many IT applications do not fully automate the process, and automating the status quo simply perpetuates the problem. Replacement of an existing system should be undertaken as enumerated here:

Design the replacement system with the current business needs and processes in mind, not the old ones. However, a new system tends to impact many areas, including:

To counter these impacts, management should assess the effects of the new system on the organization and the users. Communication of the benefits of the new system to the users should be undertaken, but remember to keep the statements realistic to manage user expectations. This discussion can be achieved in groups or individually.

Involvement of the users in all stages of the project is essential. Areas where this can occur are project planning, analysis, testing prototypes, and implementation. Request input on how to structure and phase the training to use the new system. A champion or champions for the system should be identified and involved throughout the project.

3.4.2. Impact of the system on the IT department 

As systems become highly integrated environments and work in close cooperation over networks, a new system can have different levels of impact. If there is little change in the operation and management of the computer, there will be few problems apart from negotiating the facilities management contract. At the other end of the spectrum, there must be enough capacity or bandwidth on the network to accommodate the anticipated data flows from the laboratory to the server plus sufficient workstations and peripheral devices to access the system. In short, ensure there is sufficient capacity for the new system to operate effectively.

If the IT department has no experience with newly acquired hardware, operating systems, and/or application software, this will have an impact on the support staff and will increase risk accordingly. Organization members should have the skills and experience to run these new methodologies efficiently. If not, they will have to be acquired by training existing staff or recruiting new personnel. The input from operations staff to the project team during evaluation and selection can identify many of these areas. To reduce the risk and aid communication between various applications, the first intent should be to purchase or develop a system that is consistent with the current systems in place within the organization. This aspect will be considered in more detail in “New or Non-Standard System Components.” The author would advocate using corporate standards whenever possible, as this will be the easiest to implement.

Documentation of system procedures, coupled with effective training, to use any new hardware and operating systems must be in place before the system goes live.

3.4.3. Procedural changes required by the new system 

Project risk can be greatly increased with the failure to recognize early in a project the need for any new or revised procedures; thus, plan and implement them rapidly. The current working practices should be reviewed, and the level of the user's commitment to change procedures should be established. If change is resisted, do not implement change via the computer system but change procedures, if possible, by altering the manual ones first. This is a relatively cost-effective route to take, as small modifications can be undertaken easily and rapidly until the new manual procedure is streamlined and effective. Then, overlay the new system on top of the modified manual system. In this way, problems can be resolved with the new procedure without the computer being used as a scapegoat by dissatisfied users.

3.4.4. Changes in organizational structure 

Computer systems have the power to cross functional and organizational boundaries with ease. The failure to recognize and plan for any changes may result in staff not knowing new responsibilities or roles or in disruption occurring from reorganization of organizational units; hence, there will be an increase in risk to the project.

The impact of any organizational changes should be documented clearly during design and development, although it may be alluded to in the project proposal wherever possible. There should be change management of any such changes over a specified time period. Always, if possible, allow time for the changes to settle down before implementing the new application to avoid too much change in a short time period. Again, the communication of the realistic benefits of the new system to the users should be undertaken.

3.4.5. Policy changes to accommodate the new system 

Changes to policies should be identified and controlled by the user management. Since these are not always identified until the detailed design stage or the development stage, any delay in implementing these could delay the operation of the project. The resolution of these policy issues must be made before development can take place.

Policy changes may be the result of the introduction of new technology, organizational changes, or modification of procedures caused by the new system. Therefore, it is important to identify and resolve any policy changes rapidly but not before considering the impact of the changes. If large numbers of policy changes have to be made, there should be a mechanism in place to document and inform all staff of them—a user appointed as a coordinator might be one approach to use here.

3.5.1. Experience of the project manager 

An inexperienced project manager may have difficulty developing an efficient project plan and modifying that plan as the project progresses. Moreover, not all tasks may be identified, or the project plan may not be broken down to a sufficient level to enable accurate scheduling. Taken together, this often results in delays to the project and missed deadlines, with tasks rescheduled or additional ones included, often at short notice. The impact can be damaging to the project, as budgets may be increased, and there may be loss of confidence by the users or cancellation of the project, as benefits have not been obtained in a timely manner.

To counter this, the project manager should be trained in project management techniques. When drawing up the plan, allocate more time for the completion of the tasks to allow for slippage, or allow slack time. Regular reviews of project progress should be set up. To gain from the experience of others, read the status reports and reviews of similar projects completed within the organization.

3.5.2. Full-time project manager 

Depending on the size of the project and the resource available, it is preferable to have a full-time project manager. This avoids conflicts of interest with line management responsibilities and allows the ability to focus on key issues that might not occur if it were a part-time position. In some respects, this is a management decision about the amount of time and resources allocated to a project. However, there is also the onus on the project manager to inform management if he feels overworked when given dual responsibilities.

3.5.3. Full-time project team 

Whilst it is common for the project manager to be allocated full time to the project, the team members are usually allocated on a part-time basis. Here, the line/matrix conflicts outlined in the last section will become apparent as the project competes with the line for the resources of these skilled individuals. In this situation, errors can be made or delays occur that could impact on the project, ordinary work, or both.

To manage this situation effectively, it is important to define accurately the amount of time that a project team member should spend on his respective duties. This will reduce the amount of time available for line work, and, accordingly, the manager of the individuals should negotiate with clients regarding deadlines involving these staff. If specific tasks for the project such as in-house evaluation require an individual's time, this must be negotiated with the supervisor well in advance of the event.

Ideally, the project team member's position description should be changed to reflect the work done on the project so that both the line manager and the project manager can evaluate the individual's overall performance.

3.5.4. Project members operating as a team 

When the project team is composed of members who have not worked together before, some delays may occur in the initial stages of a project. Team members need time to get to know other member's personalities, understand their skills, strengths, and weaknesses, and learn how to work together. Risk arises if the team lacks skills or understanding of the technology involved or the knowledge to complete the project successfully.

To overcome this and reduce the risk, attempt to use staff that has worked together as a team. Working to the strengths of an individual is always preferable to training another member. However, this approach carries the risk that IT and automation skills can often be in short supply, and one individual can often be carrying out several tasks, which can conflict with line duties. A way to transfer skills should be included when feasible and when time and resources allow.

3.5.5. Experience with the application or system 

Often, the majority of project team members from the user areas have little or no experience with a new type of application. Without experience, the team will not have the insight to avoid mistakes or enter blind alleys. Additional time may be required for reviews and revisions of the plan and its execution.

If similar projects have been introduced in the organization, utilize the knowledge from some of the team members as an internal consultancy role. Ensure that more time is allocated to the project to allow for problems.

3.5.6. Multisite projects 

The concept and introduction of a corporate LIMS or an automation project may involve two or more sites. Whilst from the corporate viewpoint, this is an effective use of resources for development and maintenance, and the benefits will be significant over the development of different point solutions at every site, problems will be encountered. By its very nature, a project covering more than one site tends to be larger, more complex, and hence more expensive than one at a single site. Senior management should look carefully at these projects, as a significant proportion of their IT budgets will be involved.

Communication may be difficult, especially if there are time differences involved that are greater than one or two hours. This can be overcome by the use of electronic mail facilities or an Intranet site for common-interest items. Progress updates will need to be regular for all sites and held centrally at one location to ensure control of the overall project.

There may be lack of coordination at the sites where the project manager is not located. Travel, often extensive, will be involved for the manager and several key members. Budgeting of money for this and the associated subsistence is essential. Different sites may have different working practices, policies, and organizations. These typical issues, raised by a standard system, will have to be resolved at the senior management level before much progress can be made. In companies working on a global basis, there will also be cultural differences, methods of working, statutory holidays, and even the length of the lunch break to be taken into account. The timescales for the project will need to be increased to account for these factors. However, the overall benefits to the organization should outweigh these difficulties.

4.1.1. New or non-standard system components 

Increased risk to a project will be incurred if new or non-standard system components are selected for the application. Under this category are included:

The risk in selection of non-standard components is manifested in several ways. The development team and the support staff need to become familiar with the respective components. This may require training that may be extensive as well as costly. The extent of integration between these new components and any existing applications may raise technical problems at the very least. Training to use these packages, and potential delays due to technical problems to be solved, must be an essential element of the project plan.

Establishing contact with the vendor's technical experts for specialist information and advice may be a way of gaining information to reduce risk or to obtain solutions to actual problems experienced. It is preferable to keep to the corporate standards wherever they are established for easier implementation and maintenance.

The choice of packages that do not conform to corporate guidelines must be made carefully:

The choice of the wrong database or development language will have a major impact on the project's ability to deliver the expected benefits.

4.1.2. Type of system and processing 

The greater the complexity of the system, the higher the risk that something will go wrong—this is Murphy's Law of laboratory automation. Management of risk approaches should be adopted to choose the simplest approach consistent with supporting the application effectively. The choice of a pilot system to size the processor, memory, and disc I/O accurately before the installation of an operational system may be one avenue to take. If distributed processing is required, implementing core functions in two locations first could be adopted and preferable to finding the completed package does not work as anticipated. Some applications may require on-line data capture in real time; this requirement may entail having a failure resistant hardware configuration. The need and justification for every requirement should be investigated thoroughly.

4.1.3. Response time 

The faster the response time required by the application and the users, the higher the risk. Failure to meet this performance criterion may result in loss of user involvement and interest. The sizing of hardware components and the design for rapid database searches for urgently required data may be crucial, but remember that not all data may be needed rapidly. Ensure that the computer has the expansion capacity for the next three to five years to cope with increased demand, either with sufficient capacity purchased at first intent or by planned incremental growth.

4.1.4. System availability 

The need for high system availability should be investigated and justified; there is often a stated requirement for 24/7 availability but few systems actually require it (best justified are manufacturing processes for raw materials and active ingredients).

If a high degree of availability is placed upon the system, the supporting hardware and network also need a high level of availability. Therefore, fault-tolerant hardware may be justified in cases of near-100% availability being required. Procedures for identifying and solving problems should be developed, as should effective and rapid contingency plans and disaster recovery procedures—e.g., consideration should be given to spare hardware systems being available to be started up in the case of an unplanned system or IT infrastructure failure.

4.1.5. Technology mix 

The greater the number of technologies that have to be integrated into an application environment, the greater the risk becomes. Wherever possible, keep to the simplest approach that is consistent with the requirements of the application and that meets the needs of the user and organization. Wherever possible, use components, or proven technology, that the organization has knowledge of and has used successfully before. Here, the success rate of the organization in implementing IT and automation projects plays a role. An organization with a successful and innovative track record in implementing projects can probably justify the risk involved with a range of technologies. However, a less-evolved organization should lower its sights and err on the side of caution.

4.2.1. Selecting the system 

When a system or an application package is chosen, there are a number of parties with vested interests within an organization such as the IT Department, the user laboratory, and, in a regulated industry, the quality assurance unit (QAU). Using an effective project team approach, all parties should be represented and have input.

From the IT departments viewpoint, the users may have chosen the wrong package for a number of reasons such as non-standard components, new technology, or the database does not appear to fit the requirements. The input of the IT department should be to check the requirements and package to assess the degree of fit from the IT perspective. It is possible that the package may not meet user expectations, or it may take considerably longer to implement than anticipated from an IT perspective. This can be resolved by ensuring that the package is tested fully with tests that represent functions carried out by the users.

If the IT department, with little input from the users, selects the package, the greatest risk is that the users will reject the system. The users know their own environment the best and appreciate the functions they require. The QAU interest is that the selected system can be validated, and they can use the system to carry out audits effectively.

The closer the package matches the requirements, the less risk incurred by the project. The further the package is from the requirements, the more customization will be required, or the users will have to modify their working practices to use the package. Both instances increase the risk of the project and can lead to excessive time delays or user rejection. It may be appropriate in this instance to consider a custom-designed system rather than a package. Alternatively, redefine the scope of the project and reassess the fit to the modified requirements.

4.2.2. Seduction by technology 

Evaluating a system or application package without a system or user requirements specification is asking for trouble. There is no baseline from which to make a value judgement and is likened by the author to being seduced by technology—it is unknown if the system can actually meet the business needs, as they have not been documented. Therefore, before evaluating systems, it is important to document the requirements and have objective means of evaluating the systems being reviewed.

4.2.3. The vendor 

The project will have increased risk if the organization has no experience dealing with a specific vendor. Without first-hand knowledge of their contract negotiating techniques and their willingness to modify their system (if required), a laboratory could end up with a system that does not support the business and that incurs expensive delays. This risk is increased if the company is new or has only a relatively small number of installations.

To a certain extent, an indication of a vendor's attitude toward existing customers and their problems can be obtained from site visits. However, it is important to remember that a vendor will not usually take potential customers to a site at which they have had many problems. The site most likely to be selected will be one with which the vendor has a positive relationship.

To counter this, it may be prudent to insist that all agreements with the vendor are in writing; this may also be true of statements made by sales personnel who are attempting to win an order. Access to the vendor's technical specialists can build confidence in dealing with a vendor and be the start of a good working relationship. Communications, both formal and informal, should be established, and any issues discussed should be entered into a log as a formal record of progress.

4.2.4. Vendor failure 

This risk covers the failure of a vendor due to either commercial failure of the company or, more often, the withdrawal from a market sector for commercial reasons or changes in business direction. If any of these problems occur, it is important that the organization does not suffer a loss of its investment—both money and time. Contingency plans may be drawn up for the maintenance of the system, at least until a replacement can be found and implemented (possibly a one- to three-year period).

To try and avoid failure of this type before any order has been placed, and preferably during the selection process itself, obtain financial statements from each vendor under consideration. Non-disclosure agreements may be essential to obtain information, especially if it is not part of a published annual report. Key indicators are the length of time the vendor has been in the business and the growth of the company over that period. Within the IT area, many companies may have relatively short track records and may be relatively small. The impact that their product or products have realized in the time they have been available could be used instead. When considering automation, it is more likely that the company will be larger, but this is no guarantee of minimized risk, as some of the largest companies have changed direction and left customers with little or no future direction.

While care is needed in vendor selection, a track record and growth with a successful product is ideal, but these factors should not be used as exclusion criteria against smaller companies that may be emerging with a superior product.

Safeguarding the investment can be achieved by the use of clauses in the purchase contract—for example, all software and documentation should be provided or put into escrow with a third party in the event of failure. Access to source code is a contentious issue, but in the event of corporate failure, this may be the only way of maintaining the system. To protect the laboratory, it may be prudent to include a clause allowing the maintenance of the software by a third party if the vendor cannot or will not fulfil the contract. Incorporating items such as those outlined above is a long and complex process that should be undertaken carefully.

5.1.1. System scope matches laboratory working practices 

When the development starts, the scope should either match the working practices in the laboratory, or changes in the manual practices have been instigated so that they match the new system functions. System credibility can be lost easily amongst the users by an unplanned mismatch of system and working practices. Liaison amongst the user representatives on the project team with the system developers should help to alleviate this problem.

5.1.2. Change-control procedures 

Once the scope has been fixed, there should be change-control procedures set up to debate and approve any additions, deletions, or modifications to the scope. Without change control procedures in place, there is a significant risk in uncontrolled development of a system.

The change control process involves a set of procedures and a review group. The latter can be either a subgroup of the project team or a separate group whose purpose is to review and prioritize any modification of the scope. Submissions, detailing the changes to be made, should be in writing with the business benefit laid out. Change control should avoid the trivial functions being added at the expense of more urgent ones, thus delaying the project. The corollary is that occasionally some important functions are missed from a specification, and this mechanism provides the means to have them authorized for inclusion.

Implementation of change control is also useful when the system is fully operational, as all changes to the system configuration should be proposed and authorized in this manner.

5.1.3. Documentation 

The documentation of the system is a key quality issue. The main document required for the development is the scope describing the functions to be customized. Additionally, an outline of the change-control procedures, draft testing and validation plans, draft procedures for start-up and shut down of the system, and outlines for the user manuals are needed.

Documentation is required for validation of a system, but more importantly, it is essential for the smooth transition from development to operation. The time required writing good quality user and support documentation is usually longer than anticipated. Therefore, the tasks should be started well in advance of when they are required, and enough time should be allowed for the job to be completed, with sufficient quality to be the first line of support for the system.

5.1.4. Involvement of users in prototyping and testing 

Before development starts, the project team should have identified a group of sympathetic users who will be used to test prototypes or functions developed via conventional programming. The users should represent all groups within the laboratory environment. Note the use of the word “sympathetic.” Credibility is easily lost during development by word of mouth and by the actual performance of a system. There is little point in selecting a group of users who do not want the system to succeed or who are skeptical toward the use of automation. What is required is constructive comment and criticism that will allow development of functions to proceed without detrimental comments about the system being made.

5.2.1. Detailed implementation plan available 

Before commencing this phase of work, a detailed plan covering the implementation must be available. Details covered should be as follows:

The aim of the plan is to remove most of the uncertainty involved during implementation and direct resources to where they are most needed and when they are most required.

5.2.2. Training plans agreed 

Once the implementation style has been agreed upon, the training schedule can be developed relatively easily. In the implementation plan, the groups of workers who will be trained to use the system and the order of training should be identified along with the support staff who must be on hand to augment training and solve any problems. Obviously, risk increases dramatically if staff are not trained to use the system.

Many vendors offer standard courses; however, this may not meet the needs of users where the system has been customised from the core system offered for sale. It may be beneficial to consider customizing training courses and holding them on site if there is sufficient demand to do so or the cost benefit is good.

Training is an easy target when it comes to budget cuts; instead of training all system users, only key ones are trained, with the aim of cascading the training from one or two key users to the rest of the user community. This is often a false economy, as the key users may be technically very capable but not professional trainers; skills and knowledge may not be transferred effectively to the key users who are poorly equipped to transfer what they have retained to others. Ensure training is properly budgeted and professionally carried out to guarantee that the organization has a good opportunity to gain the best benefit from its investment in the system.

5.2.3. Implementation delays 

There are a number of possible causes of delays including vendor failure to deliver a package within an agreed time frame or the writing of in-house software is slower than expected. More problematical are instances where the functions of the system do not match the current working practices in the laboratory, necessitating a delay to rewrite software. The lack of suitably qualified staff, either in-house or from a vendor, may impact the project at a crucial time. Regardless of the cause, delays in implementation are frustrating and have a bad effect on morale and a negative impact on the credibility of the system.

Using staff to work on the project in their spare time increases risk due to conflicting interests. It is preferable to have dedicated staff working on a project to ensure implementation in a timely manner.

The easiest way to manage the risk is to have slack or contingency periods built into the project plan. This can be used to offset delays and avoid reissuing the project plan. If they are not required, then the project delivers ahead of schedule.

5.2.4. Poor system performance 

This is a classic reason for failure of projects during implementation: the system was sized either by estimation or by a formula. The overall platform performance is not sufficient to operate the system effectively and provide adequate performance to the users and, ultimately, the laboratory customers. Effectively, the system is useless and unable to perform its function. This can be due to a combination of factors:

There exist a number of approaches to overcome these problems. One is to define the overall workload of the laboratory accurately and define in unambiguous terms what a sample, test, analysis, and result mean within the context of a specific laboratory. This should allow a vendor to size a system more accurately. Note that, however, vendors work on average system sizes. If a laboratory's application is below average, performance should not be affected and may be enhanced. However, if the application is above average, performance will be affected, often quite dramatically.

Visits to existing users are a more practical way of discovering how effective the vendor has been at sizing a system. If this approach is taken, it is imperative that the site visit is to a laboratory in the same industry and, wherever possible, that it uses the same software modules as the vendor is proposing for you. Often it can be very difficult to find a laboratory site that operates even in the same industry as your laboratory, or if one is found, it is located in a different country. However, a number of aspects of site visits are very useful.

Alternatives exist to avoid performance problems. The approach taken in the author's laboratory was to purchase a small development computer system, develop the software, and carry out performance tests that predicted the size of computer system required to support the intended user base. Another is to carry out a performance test on the potential system configuration, and time the responses obtained. A third is to specify the minimum response times required in the contract with any penalties upon failure to achieve them. The author prefers the more practical approach of direct sizing, as it removes an area of uncertainty during the most critical phase of a project. This practical approach to hardware sizing should also eliminate the need to seek additional funds for a processor upgrade or additional discs soon after the system is operational.

5.2.5. Obsolescence 

Given the rapid development and life cycle of hardware and communications components, it will not come as a surprise to find that the system hardware can be replaced in a product line before the organization's depreciation period is completed. If the equipment has been purchased from a recognized supplier, service support should not be a problem but expansion may be. To reduce, but not completely eliminate, this risk, ensure that the development plans of the hardware supplier are known, especially if a proposed hardware system has been available for over two years.