What is Project Evaluation? Stakeholders, Types, Objectives

What is Project Evaluation?

Project evaluation is a process of assessing and measuring the success or effectiveness of a project. It involves collecting and analyzing data to determine whether a project has achieved its intended goals, objectives, and outcomes. The evaluation process can be carried out at various stages of the project lifecycle, such as before, during, or after the project implementation.

Project evaluation typically includes measuring the project’s impact, assessing its cost-effectiveness, identifying strengths and weaknesses, analyzing the project’s sustainability, and making recommendations for improvement. The evaluation can be done using various methods, such as surveys, interviews, focus groups, and data analysis.

Different Stakeholders on Project Evaluation

The views of different stakeholders are as follows:

• Project Team Members: They believe that the submission of the project on time will have a positive impact on the organization and give them valuable experience.
  
  
• Project Manager: He/she has a view that a successful closure of a project leads the organization to earn profits, which, in turn, will lead to promotion or increment in his/her salary.
  
  
• Client or Project Sponsor: He/she believes that post the implementation of the project, the success of the project will be determined resulting in a great impact on the organization’s value.

Therefore, project evaluation can be performed in four ways as listed.

---

Types of Project Evaluation

Let us discuss the types of project evaluation in detail:

• Individual Performance Review
  
• Post-mortem Review
  
• Project Audit
  
• Project Success Evaluation – MOV

Individual Performance Review

An individual performance review is conducted by the project manager for each project team member. Every organization generally has a set of rules for conducting reviews to get the best results. However, the following are the points that every project manager must keep in mind while conducting an individual performance review:

• Initiate the Review With the Evaluation of an Individual’s Performance: The project manager must evaluate the performance of an individual team member with genuine intentions. However, being critical of any concern can make the employee feel offended. Therefore, the manager should begin by asking the team member how he/she rates his/her performance.
  
  The project manager may get critical issues from the member, which can serve as an opportunity for the manager to carry them forward. To avoid any kind of conflict between the manager and the member, it is of utmost importance for the project manager to give constructive feedback to the member on his/her performance.
  
  
• Avoid Comparison of a Team Member With That of the Other: Comparisons always result in counter effects. For example, the team member may inculcate the feeling of jealousy; thereby, finding ways to discredit the person in comparison. The project manager must remember that each person is unique and possesses discrete qualities.
  
  The fact is that no one is perfect; however, identifying the areas of improvement is important. Therefore, the manager and the team member should seek the consent of each other on the improvement areas. Moreover, they should agree upon how improvement in the team member will result in the growth of the organization.
  
  
• Focus on the Behaviour or Actions Instead of the Individual: The project manager must remember that if the areas of improvement are discussed, the focus should be laid on a few behaviors of the member as an example. To better understand it let us take an example. Suppose a team member has a habit of reaching late which makes him/her unable to attend team meetings.
  
  In such a scenario, the manager should not ask blunt questions like what is the reason for his/her laziness. Instead, the project manager can politely convey to the team member that getting late and not attending can affect their career growth. Such a methodology will surely help the team member to improve the aspect of getting late for the office.

Post-mortem Review

After creating the final project report and delivering the presentation, the project manager and the project team should go for a post-mortem review of the project. This review should be conducted before the team gets released from the current project.

This is because it becomes cumbersome to let the people participate post they get engaged in the other projects or they are no longer working for the organization. In addition, with time the team may forget the aspects related to the current project.

The main objective or focus areas of the post-mortem review are as follows:

• Analyze the MOV of the initial project. A few questions need to be answered such as whether or not the MOV of the project was clearly defined and agreed upon by the sponsor or customer. In addition, the project manager must ensure whether or not any change was incorporated into the project. Also, he/she must consider what the chances are that the project scope is achieved.
  
  
• Analyze the project from various aspects such as scope, schedule, budget, and quality objectives. The project manager must answer the following questions:
  
  • How well is the project scope defined in the document?
    
    
  • Did the scope of the project change with time?
    
    
  • What is the effectiveness of the scope management processes?
    
    
  • Did the actual deadline and project cost identical to the planned or scheduled one?
    
    
  • Were the quality objectives laid during the initial phase of the project met?

• Analyze every project deliverable and answer the following questions:
  
  • How effective were the business case, the project charter, and the project plan?
    
    
  • How can the manager or team members improvise these deliverables?

• Analyse various project plans and Project Management Body of Knowledge (PMBOK®) areas. The effectiveness of the project should be reviewed in the following areas:
  
  • Project integration management
    
  • Project scope management
    
  • Project time management
    
  • Project cost management
    
  • Project quality management
    
  • Project human resources management
    
  • Project communications management
    
  • Project risk management
    
  • Project procurement management
    
  • Organizational change management
    
  • Project implementation

The conclusion of the post-mortem review should also be documented as other reviews. The project manager as well as the team should get to know the best areas of the project. In addition, the best practices should be identified and become part of the IT project methodology of the organization.

Project Audit

The internal working of the project is analyzed with the help of individual performance and post-mortem reviews. As discussed earlier, these reviews are conducted between the project manager and the team. To get a narrower perspective on the project, the audit should be done by an external party along with the internal team.

The internal audit team comprises the project manager, the project team, the project sponsor, and other key project stakeholders. This helps in identifying problems, issues, or scope of improvement.

As suggested by Gray and Larson in 2000, the depth of the audit is dependent on the size of the organization, the importance and size of the project, the risks involved, and the problems encountered while working on the project.

The external party should:

• not have a direct engagement or interest in the project
  
  
• be honored and seen as impartial and fair
  
  
• have an interest in listening
  
  
• not have fear of recrimination from special interests
  
  
• perform for the benefit of the organization
  
  
• have sufficient interest in containing project or industry experience
  
  
• be able to document the results of the project audit along with learning and best practices.

Project Success Evaluation – MOV

The MOV, a short form of measurable organization value, is introduced at the beginning of the project. It provides the base for getting a hold of the project and supports many decision points throughout the life cycle of the project. Generally, at end of the project, the MOV cannot be identified.

As mentioned earlier, each stakeholder will have a different and unique perspective towards the fact of whether or not the project is a success. Also, it is crucial to get the value that the project provides to the organization.

This review may be done by many people from both sides such as the project sponsor or client’s organization and the organization or are responsible for carrying out the project.

The following questions are answered while evaluating the project’s success based on the MOV:

• Did the project achieve its MOV?
  
• Was the sponsor/customer satisfied?
  
• Was the project managed well?
  
• Did the project manager and team act professionally and ethically?
  
• What was done right?
  
• What can be done better the next time?

Before evaluating the success of the project, the consulting firm or individuals representing the project should be confident that the information system delivered will not be altered. Generally, when an information system is handed over to the project sponsor, the users or personnel may make changes.

---

Objectives of Project Evaluation

Some projects are divided into phases while others involve only a single phase. If a project includes multiple phases, an evaluation near the end of each phase is done to determine whether to continue the project in the same direction, in a different direction, or to abandon any further activities. Overall evaluation of the project is extremely useful in providing an objective assessment of the performance of the project in terms of cost and time. In a general project, evaluation is done to:

• Determine whether projects are being implemented on an efficient and effective basis. Efficiency shows whether the amount of time put into the project is equal to the expected outcome or the economic rate of return achieved by the project is at least equal to the expected line of investment. Effectiveness evaluates how much of the intended goal for the project was completed and impact measures the outcome of the project.
  
  
• Determine the quality of the project.
  
  
• Provide objective assessment of the performance of the project in terms of cost, time, and other technical parameters. These are also helpful tools in determining how a process could be handled better.
  
  
• Determine the variance in the performance (Budgeted-Actual).
  
  
• Identify the areas of problems (obstacles that may have been inhabited in the development of the project) so that their adverse effects can be minimized.
  
  
• Identify the key strengths within the different areas of the project and areas which could be developed or improved.
  
  
• Identify opportunities before they are lost.
  
  
• Recommend for follow-on work.
  
  
• Ascertain the merits and limitations of the project team.
  
  
• Provide useful feedback for project planning and control.

Thus, the evaluation of projects is done: To test the effectiveness of the project, to keep the project work on the right track, and to trace the problems that occur in the implementation of the projected activities.

---

Evaluation Framework

Projects are goal-oriented open systems. The primary purpose of evaluation in project management is to judge the performance (actual performance is compared with planned performance), reveal areas where the project deviates from goals, and uncover potential problems to be corrected. On-going and ex-post evaluations should address five sets of issues which, of course, should be adapted to the specific reasons for evaluation in particular projects:

• Continued Rationale for the Project
• Project Efficiency
• Project Effectiveness
• Project Effects and Impact
• Lessons Learned From the Project

Continued Rationale for the Project

It refers to the extent to which the project design:

• Continues to reflect the project priorities and relevance of project objectives;
  
• Relates to the policy framework;
  
• Suggests alternate choice; and
  
• Validates the assumptions for the project.

Project Efficiency

Efficiency relates to the activity and component level during project execution and refers to the extent to which:

• Inputs were supplied and managed;
  
• Activities were organized most appropriately; and
  
• The above was done at the least cost to produce the expected outputs.

Project Effectiveness

Effectiveness relates to the purpose and goal levels after completion and should refer to the extent to which:

• The project produced the expected outputs;
  
• The outputs achieved led to project purpose; and
  
• Purpose attainment contributed to the goal.

Project Effects and Impact

Whereas effectiveness relates to the direct project objectives universe, effects and impacts refer to the effects of the project in other areas. Project effects and impacts refer to the planned and unplanned consequences of the project. In general:

• Effects relate to purpose-level planned and unplanned consequences.
  
• Impact relates to goal-level planned and unplanned consequences.

Narrative Summary	Indicators	Means of Verification	Assumptions
Goals
Purpose
Outputs
Inputs

Lessons Learned From the Project

lessons learned are of two types: Operational lessons, which tend to relate efficiency, effectiveness, and rationale; Developmental lessons, which focus on effects and impact. Project evaluation is similar to standard costing and variance analysis in cost accounting.

In standard costing and variance analysis, the actual cost is compared with standard cost (pre-decided cost), difference whether favorable or unfavorable are called variances. These variances are broken into components like price, usage, quantity, etc. and the decision is taken for further action. Similarly to this, project evaluation answers the following questions:

• What was expected,
  
• What has happened?
  
• How good were our initial estimates? How could they have been improved?
  
• Should the project be reformulated? What should have been the position under the circumstances?
  
• Is there a proper utilization of resources? And if not, what can be done for doing so?
  
• Whether the project manager and core team is performing the right task? Action, if required, for penalties or rewards and future guidance.
  
• If the project were to be repeated, what should be done differently?
  
• What did we learn from this project? Where should we go from here?

---

Methodology of Project Evaluation

Despite all the latest tools and techniques and helps from sophisticated automation, there are various examples where actual outcomes on the completion of the projects are wide out of the budgeted outcomes. So, evaluation is immensely important. A project evaluation reviews the project process (phases) and determines the effectiveness of a completed project.

Project evaluation and its related decisions are not only important but also complex topics not to be taken lightly. This varies depending on the complexity of different projects. In general, the methodology of evaluating a project can be as mentioned below:

• Realization of Need
• Setting Up Evaluation Agency
• Specification for Evaluation
• Measurable Milestones and Their Evaluation
• Lessons Learned
• Follow-up Action

Realization of Need

As the first requirement, the planning and execution agency as well as the chief executive of the project should be convinced about the need and importance of conducting the evaluation.

Setting Up Evaluation Agency

This includes deciding who will perform the evaluation. An evaluation agency/team must be set up either on a permanent or ad hoc basis. This team may consist of an internal person or it may be an independent external agency, depending on the capacity and competence required.

Specification for Evaluation

Various phases, programs, and activities of the project should be specified in consultation with the technical specialist in the planning and implementing department of the project. An evaluation team will revisit the organization’s broad mission for the project and will develop reporting processes that measure whether it has been met.

A project with a financial goal is the most easily measurable, while broader impact-related goals may be more difficult to evaluate. Thus, evaluation goes much beyond the current operations of the project. It looks at objectives set at the time of appraisal and tries to find out if the objectives were met or not.

Measurable Milestones and Their Evaluation

The proposed evaluation mechanism usually involves milestones. Milestones may be thought of as the completion of project activities or phases like problem identification, research, technology selection, prototype implementation, end product documentation, project reviews, project reporting, etc.

Measurable and quantifiable aspects of these milestones should be identified and data to be collected should also be determined. Once the milestones have been identified, it is necessary to define how each milestone will be evaluated. This definition involves two components:

The first one is to define the mechanism that will be used to perform the evaluation. This includes how the evaluation will be performed. In this step, a design of the study/ survey should be prepared and the survey may require questionnaires and schedules, selection of samples, etc.

Relative importance has been assigned to each based on how critical each is to demonstrating a satisfactory end product. It is most important to identify those milestones, which are critical to project success, and to assign a high-importance rating. Some of them are as follows:

• Strategies: Project management strategies relate to the detailed actions project participants use to reach project objectives. An examination of the effectiveness of individual tasks and assignments, as measured by their outcome, provides insight into the effectiveness of a project’s strategies.
  
  
• Reaction and satisfaction: A questionnaire or survey to the related parties can be used for measuring satisfaction with the project.
  
  
• Timetables: This evaluation includes the overall project timetable, as well as timetables established for incremental project activities.
  
  
• Network charts: The project manager can analyze the PERT (Project Evaluation and Review Technique) charts and can compare actual project deliverables for project evaluation. Another is CPM (Critical Path Method), which clarifies the critical path of a project, or the most efficient path between project initiation and termination.

The second component is to define the possible results of the evaluation and what constitutes success. For evaluation, essential information should be collected both from primary and secondary sources. Statistical data and other types of quantitative and qualitative information can be used to calculate pre-project and post-project status changes.

For example, Return on Investment (ROI) can be used for quantitative evaluation consideration. ROI is “a performance measure used to evaluate the efficiency of an investment in the project”. Calculating and interpreting ROI within a project include measuring the project’s ultimate cost against the projected and actual returns.

The so collected information is analyzed, conclusions are drawn and the results are presented in the form of reports. These reports must be available in a clear, informative manner to company personnel who may work on the same or similar project in the future.

Periodic progress reports offer several benefits:

• They provide timely information so that corrective action can be initiated to tackle potential problems.
  
• They generate inputs for further activities.
  
• They serve as the basis for calculating variances and explaining variances.

Both quantitative and qualitative evaluation methods have their benefits, though one is usually more appropriate than the other in any given situation. Quantitative evaluation can help remove human bias from a statistic, making it more of a reliable fact than any piece of information gathered qualitatively. Thus, accurate quantitative evaluations can be relied upon as truth.

Qualitative evaluations may also entail truths, but these truths are harder to get at, and evaluators may not agree. Still, qualitative information is invaluable precisely because it involves human interaction and interpretation. Many activities in project phases need this interpretation to decide how to move forward or to judge the source of data.

Lessons Learned

The lessons learned are an important part of any project evaluation. Sample evaluation questions that relate to lessons learned are:

• What went well? It is an indication of what the team already knew, learned, or where it was lucky.
  
  
• What did not go well? It indicates the topics/activities on which improvement is required. What technical and nontechnical knowledge was gained? The technical knowledge gained will provide additional tools to assist team members in their professional advancement. Nontechnical knowledge should be included in the evaluation of the project since they (team members) can contribute/be more efficient in the future.
  
  
• What would be done if the same project were to be repeated? Dissect the project to determine which successful methodology can be used for future projects. For example, if a certain project is very successful due to specific personnel management (e.g. different groups on different facets of the project), the same will also work well for other similar projects.
  
  
• What would be done differently if the project were to be repeated?

Follow-up Action

Several possibilities are available concerning potential follow-on work on the project. Regardless of which approach is chosen, the decision should be based on sound engineering judgment with supporting data and just not personal pride or a ‘gut feeling’. This runs like this –

• Determine whether the project results were accurate. For this review, all the notes, costs, and records are to make sure that every aspect of the procedure used is taken into consideration.
  
  
• Decide on changes, adjustments, or improvements that can be used to improve future projects. First-time projects often set the stage for future projects in a progressive company. These projects will be revised to fit the current scenario.
  
  
• Measure the profitability or success of the project. Short-term as well as long-term results must be taken into consideration.
  
  
• No further activity (stop) -This alternative may be recommended due to any of the following reasons:
  • The financial condition of the company changes.
    
  • Continued activity is neither practical nor cost-effective.
    
  • Certain critical expertise is no longer available.
    
  • The original reasons for the project disappear.

Doing an honest evaluation in each of these areas can significantly help in not making the same mistake a second time. Regular evaluation of projects:

• provide a documented log (report) of experience that may be valuable in improving future decision-making,
  
  
• help in discovering systematic biases in judgment,
  
  
• enable the firm in identifying individuals with superior abilities in planning and forecasting,
  
  
• serve as a useful training ground for promising executives who need broader business experience and exposure
  
  
• help to access whether the decisions taken by the project manager match the intended requirements and
  
  
• induce healthy caution among project sponsors.

A sample outline of an evaluation is shown in appendix-1.

---

Phases of Project Evaluation

Evaluation is a continuous process and an integral part of project management. Projects can be evaluated at various stages of their life cycle. Its evaluation phases are divided into the following parts:

• Ex-Ante Evaluation (Project Appraisal)
• On-going (Con-current or Mid-term) Evaluation
• Project Audit
• Ex-Post Evaluation

Ex-Ante Evaluation (Project Appraisal)

This is the first phase of a new project’s evaluation. It is conducted before the implementation of the project. Ex-ante evaluation includes feasibility study, project appraisal, and policy assessment. Based on the information generated during this phase decision is taken whether to implement, abandon or reformulate the designed project proposal.

On-going (Con-current or Mid-term) Evaluation

Though a lot of effort has been expended in selecting projects, things often go wrong in the implementation phase. This is evident from the frequent time and cost overruns witnessed in practice. Hence it is necessary to exercise strict control on in-progress projects. On-going evaluation is done at any time during the project implementation to ensure that things are progressing according to plan.

The primary purpose of this evaluation is to keep the project on schedule and within budget. It is designed to improve the management of the project to stay on course to meet projected outcomes. A project manager is responsible for conducting this evaluation. Project Audit and variance studies are done to take the relevant decisions: to keep the project on right track or to modify it according to the current scenario etc.

Project Audit

Project audit is the technique of performance evaluation of an ongoing project. In other words, Formal and systematic examination of the performance of an ongoing project as compared to its planned objectives is called a project audit. According to W.S. Turner, “Project audit is defined as a formal and systematic examination of the performance of an ongoing project as per its requirements”.

A project audit is a review of the books of accounts, records of transactions, and financial and other systems maintained by a Project Coordination Unit (PCU) to execute a project. The whole point of a project audit is to get a reality-based accurate picture of a project’s expenditures, schedules, and quality of work.

Project audit systems function independently and report directly to the top management. Project audits must cover the whole spectrum of control. The groundwork for an audit should be laid during the project appraisal phase. It may include the following:

• Project audit systems in an organization may function either on a continuous or a periodic basis. When a project audit goes viz-a-viz project commissioning, it is like a preventive audit and is known as a continuous project audit. On the other hand, a periodic audit system may be performed when the project is just commissioned or when the operations of the project stabilize, or at some other time in the life of the project.
  
  
• The project auditor should be supported by an independent group consisting of executives drawn from different backgrounds like planning, engineering, accountancy, marketing, and so on. All team members as well as the project auditor should be educated on project objectives before they are put on project audit.
  
  
• Audits can vary considerably, depending on the objectives, project activities, and expected reports or outcomes. These can be classified into three categories: (i). Financial Statements Audit: A financial statement audit assesses whether reported disbursements have been made following the relevant financial agreements. This audit ascertains whether individual expenditures, which comprise the totals, are properly authorized, appropriately accounted for, and fully supported by documentation.
  • Compliance Audit: It involves reviewing, testing, and appraising the project’s control procedure, including compliance with laws and regulations to which the project is subjected.
    
    
  • Operational Audit: An operational audit is a more thorough examination of management techniques and performance. It may help to identify implementation issues for prompt remedial actions at the project level.

• The audit report should include the audited financial statements, the auditor’s opinion, a management letter, and any special reports required by the organization. The auditor’s opinion for the ongoing project may be unqualified, qualified, adverse, or a disclaimer of opinion. The report should contain both the results of inquiries (i.e. the true state of the project) and recommendations for dealing with the problems found.

The basic aim of a project audit is to measure the efficiency, effectiveness, and accountability of the project. It is done to find the reasons for uncomfortable symptoms in the project, and answers to the questions posed by the sponsor or senior manager. Project audits in an organization will be able to check the manipulations, pitfalls, malpractices, and mistakes which may go unnoticed and maybe get repeated. Thus:

• A project audit provides a clear picture of the actual status of the project.
  
  
• It helps in discovering systematic biases in judgment.
  
  
• A continuous check-through project audit on the observance of plans, policies, procedures, and practices will ensure management efficiency.
  
  
• A project audit is used to ascertain the factors which might lead to cost and time overruns. It gives credibility to the financial statements and other management reports. Audit of financial aspects of a project identifies weaknesses in internal controls and financial systems and makes recommendations for improvement.
  
  
• It is very useful in identifying problems that might be faced while commissioning the project.
  
  
• It can be used to create a good information base for the proper execution and implementation of the project.

The soundness of an ongoing project of an organization may be evaluated in terms of the following criteria:

• Techniques: Are efficient techniques being employed for planning, implementation, decision-making, and control of the project?
  
  
• Decentralization: Is there meaningful delegation and decentralization which permits decision-making at appropriate levels?
  
  
• Intelligibility: Are the policies, methods of analysis, and procedures understood by different segments of the organization which are involved with the project?
  
  
• Flexibility: does the project system have sufficient flexibility to respond to the dynamic changes in the environment and to permit variations in approaches for projects with differing characteristics?
  
  
• Communication: Are the premises underlying to project communicated to those participating in its process?
  
  
• Control: Are adequate controls being exercised in the implementation phase to ensure that slippages are mitigated?
  
  
• Results: Are the results of the project consistent with the goals of the organization?
  
  
• Review: Is there a systematic review of every phase to provide meaningful feedback for improving the system and its effectiveness?

Ex-Post Evaluation

Each project provides some additional knowledge to the company’s database regarding projects and their management. To be most successful, the company needs to close the loop at various points in the project particularly at its end, to determine success or failure and lessons learned from the project. Without determining these, the company is assured of repeating previous mistakes.

Ex-post evaluation is done after the completion of the project by analyzing the actual against the projected estimates in respect of cost, time, and quality specifications. It covers every aspect of the project starting from project planning, appraisal, decision-making, analysis, implementation, social cost-benefit analysis, etc. This evaluation should have the following attributes: adaptive, long-term, comprehensive, integrated, and cumulative.

Adaptive evaluation is used to continuously assess and adjust project-related decisions within the changing context and conditions. Long-term evaluation monitors impacts that occur over the time scale of several years or more. Comprehensive ex-post evaluation includes the full array of social economic, environmental, and institutional impacts of the project. The integrated ex-post evaluation examines the interactions among these different types of impacts. The cumulative evaluation considers how the impacts of one project are related to the impacts of other projects.

Ex-post evaluation is the assessment of a project after its completion. The basic purpose of doing the ex-post evaluation is to study the project’s impacts and feedback. The overall efficiency, budget specifics, output, and expected impacts of the project are reported in this evaluation. The fundamental objective is to record the findings from such evaluation so that these observations can be of immense help in the future.

The project management cycle starts (initial planning) with the ex-post evaluation of the other similar project. Until the ex-post phase, evaluation processes tend to focus on the input and output levels of the logical framework. The special role of an ex-post evaluation is to take a serious look at the purpose level and begin to analyze project contributions to the goal.

Ex-post evaluation or post-audits assess the actual impacts of completed projects, programs, and policies on the people, environment, and landscapes that are affected by the project. Ex-post evaluation can also help to guide decisions about project design operations, refurbishment, and decommissioning. Well-designed ex-post evaluation can help to draw short and long-term lessons for project and resource management.

Ex-post evaluation can be carried out in two phases: (i) soon after the completion of the project –to find the actual outcomes, variances, and reasons for such variances, (ii) after the lapse of about two–three years since the completion of the project. The project completion report is the basic instrument used for ex-post evaluation. This evaluation can be carried out by questioning the following:

• Whether the technology goals, market share, etc. envisaged in the project are achieved.
  
• What are the reasons for the shortfall in the above-mentioned?
  
• What else could have been done to avoid the shortfall noticed?

---

Observations in Ex-post Evaluation

Though the observations in ex-post evaluation vary due to the complexity, interest of related parties, and volume of different projects, these may be grouped into three categories:

• Successful Projects
• Challenged Projects
• Abandoned Projects

Successful Projects

A sound feasibility report should result in sound and successful projects. Such projects should not only meet all their obligations towards owners, workers, suppliers, government, and society at large but also have sufficient funds to keep them in an adequate state of liquidity. These projects serve the purpose for which they were established and their neighborhood or spillover effects should be positive and a model for further industrialization.

Challenged Projects

Unfortunately, there is a time lag between project appraisal and project evaluation. By ex-post evaluation, there may be project overruns of an alarming magnitude. Project overruns include both actual time overrun and actual cost overrun as compared to the projected time and projected cost respectively. The usual reasons for project over-runs may be like:

• Underestimated project schedule and project cost;
  
• Technological changes leading to obsolescence in technology;
  
• Contractor’s genuine problems;
  
• Adverse site conditions;
  
• Unforeseen difficulties and problems;
  
• Lack of integration in project organization etc.

In such cases, the evaluator conducts in-depth studies to find whether the delays or overruns were unavoidable or not. The evaluator tries to find out if there was any element of dishonesty or inefficiency. A sample outline of an ex-post evaluation report is shown in Appendix-2.

Abandoned Projects

Ex-post evaluation is a systematic determination of a project’s merit, worth, and significance, using criteria governed by a set of standards. For, various reasons, the project may become terminally sick and no amount of bailout can save them. It may be due to bad location, technological obsolescence, fiscal anomalies, or disinterested management. Such projects should be abandoned and resources so released be applied for other useful purposes.

Systematically recorded and analyzed observations in ex-post evaluation can be of invaluable guidance for future projects. It may be useful to develop specimens of model projects and avoid the wastage of resources to a great extent. It is understood that evaluation findings will be harnessed for qualitative gains in future undertakings.

In other words, the point is not to bring past failures into question, but to utilize the lessons that can be gleaned from past successes and failures for a more constructive approach that will lead to being more effective and efficient.

---

Project Appraisal vs. Project Evaluation

Evaluation and appraisal are sometimes used interchangeably due to the following reasons: (i) both mean assessment and (ii) at times they overlap especially when we talk of ex-ante evaluation. But both are different. Project appraisal is the pre-investment evaluation of estimates/forecasts etc.

A project may be financially sound but may be inappropriate for the economy or society. Before taking the decision, project feasibility studies or appraisal reports are prepared by using all the relevant indicators (details are given in Unit 3), which helps in deciding in favor or against such a proposal.

Thus, the appraisal is the process of examining a proposal before a decision is made. It is the discipline of calculating the viability of the project. Evaluation, on the other hand, is the audit and assessment of the actual as against the estimates, based on which the project was launched and completed. It is a review of the actual operations which covers (a) how successful or otherwise it had been and (b) what lessons we learn for the future.

While appraisal contains ‘estimates for the future’, the evaluation is to find the ‘valuation’ of what has happened practically. To be short, appraisal and evaluation are like before-project and project situations respectively.

Barriers in Evaluation

Major barriers in the process of evaluation may be as follows:

• Degree of control: There is always a dilemma regarding the degree of control i.e. too much vs. too little. Too much control may impair the ability of managers and create unnecessary hindrances to efficient performance. On the other hand, too less control may make the evaluation process ineffective and superfluous.
  
  
• Short-term view: evaluators often tend to rely on short-term implications of activities and try to measure the immediate results. Often long-term impacts are ignored.
  
  
• Resistance to evaluation: the evaluation process involves controlling the behavior of the project team, and hence is likely to be resisted by them.
  
  
• Difficulties in measurement: There are certain inherent difficulties in measurement during the process of evaluation. These difficulties mainly relate to the reliability and validity of measurement techniques used for evaluation, inability in getting timely and valid information, lack of quantifiable standards against which performance in every phase of the project life cycle is measured or the control system may be not uniform.

Evaluation methods can prove costly, time-consuming to design and implement and require trained staff to ensure success, reports Louisa Gosling and Mike Edwards in “Toolkits: A Practical Guide to Planning, Monitoring, evaluation and Impact”. The benefits and importance of this technique, however, far outweigh these limitations.

---

Objectives and Essentials of Project Evaluation

If a project is well formulated and thoroughly appraised, good follow-through on the subsequent stages of the project cycle will ensure the achievement of its goals. Evaluation involves a cautious examination of the basic data, assumptions, and methodology used in project preparation; an assessment of the project’s organizational and management aspects; an in-depth review of the work plan, cost estimates, and proposed financing; and finally the validity of the financial, economic and social benefits expected from the project.

Based on these assessments, it can be ascertained whether the project is financially reasonable, technically sound, and viable from the viewpoint of the economy as a whole. The exercise of project evaluation simply means the ‘assessment of a project in terms of its economic, commercial, market, social and financial viability in terms of returns and risk’. This exercise is aimed at determining the viability of a project and sometimes also at reshaping the project to upgrade its viability.

In other words, it aims at sizing up the quality of projects and their long-term profitability. Evaluation of projects by funding institutions is an important exercise for financial institutions and investment companies in credit decisions. The art of project evaluation puts more emphasis on the economic and technical soundness of the project and its earning potential than on the adequacy and liquidity of the security offered.

Hence, the process of evaluation requires a dynamic approach as it is linked with the present and future. Project evaluation is a scientific tool. It follows a specific pattern. Evaluation can be referred to as ex-ante, ongoing, and ex-post.

The effectiveness of a project may be determined in several ways. It may range from a one-off assessment, through a periodic independent review, to an ongoing project management overview. As there exist many possibilities in testing the effectiveness of a project, it will be suitable to use the term project evaluation. The particular project circumstances determine the type of evaluation to be chosen.

---

Need for Project Evaluation

Project evaluation is a practical, constructive, and essential diagnostic tool available for enhancing the capability of the sponsoring organization’s project management team. It helps provide information ranging from an informal inquiry to an extensive analysis of the effectiveness of every aspect of the project process. In the latter context, it can be conducted to ferret out common failings of many project management arrangements.

Thus, project evaluation is helpful in:

• Identification of the strengths of current practices in a project management organization, or on an existing project
  
  
• Establishment of various groups’ perceptions of the organization’s effectiveness in managing projects
  
  
• Examination of the effectiveness of project communication and documentation, and clarification of the relationships between project scope, quality, time, and cost
  
  
• Identification of barriers to better performance, or critical skills needed by project managers or their supporting teams to increase their effectiveness
  
  
• Immediate identification of specific aspects which require improvement and hence speed the achievement of results
  
  
• Facilitation an exchange of ideas, information, problems, solutions, and strategies with project team members, which leads to the development of a plan of action for carrying out improvements
  
  
• Creation of a supportive environment focusing on project success, and the professional growth of project team members

Thus, project evaluation in a timely and favorable manner helps in identifying potential difficulties and bringing them out into the open for appropriate corrective action. Further, it may avoid potential problems altogether, if the concept and timing of project evaluation are built into the project plan from the outset.

• Identifying Projects Whose Evaluation Should Be Conducted
• The intent of Project Evaluation

Identifying Projects Whose Evaluation Should Be Conducted

Project evaluation is most suitable for projects involving a significant risk of potential difficulties. It assumes that the sponsoring organization has recognized the potential for risk on the project, and has, or will include risk management as a standard functional component of the project management effort. The sponsor involves the project evaluator in the project risk identification stage of the project planning phase.

The intent of Project Evaluation

The intent of the project evaluation should be to avoid useless costs in the future. Since avoided costs are not reflected in normal accounting procedures, it is often difficult to justify this additional expenditure until management knows that it is already in trouble.

Thus, the actual extent of an evaluation is determined by the following factors:

• The size and complexity of the project
  
• The extent of the concern that management has with the project
  
• The duration for which the project has been running and the time it is expected to take to complete
  
• Whether or not previous inquiries have already been conducted

It is significant to the success of project evaluation that it be structured based on serving to advance the prospect of success of the project. Simply finding fault and pointing the finger at those responsible does not help.

---

Qualities of Project Evaluator

Project evaluation should be carried out with a certain degree of independence to protect the credibility and reliability of the resulting findings. In other words, it needs to be carried out by a third party. However, the third party may enjoy varying degrees of independence. For example, the evaluation may be carried out by someone from another department or division of the sponsoring organization. On the other hand, a completely independent party commissioned for the purpose may conduct the evaluation.

Irrespective of the way of conducting the project evaluation, it is important to ensure that the persons undertaking the evaluation have no direct personal, financial, or organizational ties with the management of the project. In addition, if the findings are to be convincing, the evaluation must be done by people who are familiar with the technology of the project, and/or who are thoroughly knowledgeable in managing the processes of realizing a capital project.

The objective of the evaluator is to provide consistent and realistic advice to the owner or sponsor’s executive through a professional approach and a degree of independence. The evaluator must be recognized as competent, fair, objective, and thorough.

---

Approach to Project Evaluation

Project management experts are almost unanimous that the project evaluation should be carried out and the subsequent report writing, in a strictly constructive way. If this basic principle is ignored, the morale of the project would be so undermined that it will be hard to obtain the required information for project evaluation purposes, and any potential benefit will be negated.

The project evaluation, whether conducted formally or informally, regularly or one-off, is fundamentally the development of a set of questions and answers found through the examination of data or personal interviews. It provides a current snapshot of the health of the project. In this sense, however, it is just like a company’s annual financial statement. It may possess much interesting information but does not serve its full purpose so long as it is compared either with similar previous reviews or with the governing project management plan.

In developing the set of questions, it needs to be ensured that each question has been cross-referenced to the relevant section of the governing plan or procedure, the potential concerns have been identified, and the persons to whom each question will be addressed have been identified. The questions should also be most conveniently grouped according to the project management function. Essential to the success of the process are three considerations, namely the reporting level, a future orientation (not past), and a perceived net benefit to the members of the project team.

The project evaluator must not be responsible for any of the individuals who may be referenced in the report. It does not mean that the draft findings need not be shared with those involved. They should certainly be. Early acknowledgment of this vital point will be in the interests of accuracy and honesty.

This will also facilitate cooperation among the evaluation team members and early implementation of the recommendations. The final project evaluation report must be presented to those who are in a position to act on the advice or ensure that the recommendations are successfully executed.

---

Methodology and Timing of Evaluation

• Evaluation Methodology
• Timing of Project Evaluation

Evaluation Methodology

The project evaluation process should demonstrate the following:

• Potential problems are being identified earlier than they might otherwise be.
  
  
• The presence of the project evaluator is welcomed by the project team.
  
  
• Practical and timely recommendations for corrective action are being offered.

The methodology involved in the project evaluation involves the following steps:

• Step (i) Establishment of the project evaluation’s goal and scope
  
  
• Step (ii) Acquisition of information
  
  
• Step (iii) Examination and correlation of the information and, in the light of the reviewer’s experience, determination of its relevance, completeness, and reliability
  
  
• Step (iv) Description of conclusions on the current status of the project
  
  
• Step (v) Development of recommendations affecting the future project status
  
  
• Step (vi) Discussion on the preliminary draft of the findings and recommendations with those interviewed, and subsequent modification
  
  
• Step (vii) Presentation of the final results for discussion with those who commissioned the evaluation
  
  
• Step (viii) Discharge of the evaluation team, until recalled

Timing of Project Evaluation

There may be both ‘planned’ and ‘spot’ evaluations. However, a planned project evaluation (concerned parties notified in advance) is much less unapproachable and more positive. This helps in the extraction or retention of information to serve as a basis for the review, with the least interruption to ongoing work.

The other advantage of such evaluation is that management is bound to set standards of conduct and performance, while individuals remain mindful of these standards in the course of their daily activities. In determining the timing for PMA, it is important to relate to the four basic phases in a typical project life cycle.

The first phase of a project involves its conceptualization, including preliminary configuration, technical and economic feasibility, positive and negative social and environmental impacts, and examination of project alternatives. The second phase involves stages in which the technical plans are developed, any required technical feasibility studies are conducted, and the resulting findings provide input to a thorough planning stage.

The third or execution phase of a construction project typically encompasses the stages of detailed design, and procurement of construction services, i.e., tendering and award of contract(s), followed by the major part of the construction. The final or finishing phase of a construction project not only involves the testing and startup of the facility but typically includes training of operating personnel, transfer of responsibility for the facility, release of project resources, and closing of project documentation.

The project evaluation should be planned into the project early in the second phase. The identification of the project evaluator and the establishment of a suitable mandate should be the first step. The evaluation can be carried out with an advantage towards the end of the second phase, which provides an opportunity to verify the various risks involved, and possibly identify additional risks, which can then be provided for in the project brief. The project evaluation activity should usually be stepped up during the several stages of the third phase of the project and, like the various other project activities, trailed off in the final phase.

---

Scope of Project Evaluation

The scope of project evaluation may be as extensive or limited as circumstances indicate. However, it is important to note that there are at least three separate dimensions that may be covered by any evaluation of the project.

Each dimension has equal importance to its outcome and success. The first consideration is related to the technical objectives of the project as represented by its scope and quality parameters, which need specific attention from the management, both in terms of technological content and managing and controlling the development of the content, consistent with the overall objectives of the project.

The second dimension of the project is related to the business management objectives as represented by its time and cost parameters. The third dimension is related to stakeholders’ satisfaction and their collective perception of the success of the project. Depending on the nature of the project, the stakeholders may be many and various.

---

Checklist for Determining Criteria for Project Evaluation

The checklist for project evaluation criteria may be prepared under the following heads:

• Relevance and importance
  
  How substantially does your project support the values, demands, programs, and priorities of the customer (Evaluation of the relevance of project objectives)?
  
  
• Validity of assumptions
  
  How realistic and accurate are these characteristics? (Assessment of the validity of project assumptions)
  
  
• Methodology acceptability and feasibility
  
  How well does the methodology support the project? How acceptable will it be to other project stakeholders? How reasonable and adequate will be its inputs and outputs? (Examination and prove/disprove of the logistic, economical acceptability, and feasibility of your project methodology)
  
  
• Stakeholder competence and credibility
  
  How capable is the project team? What financial power can the sponsor provide to your project? How “deeply” can the customer be involved in the project? (Evaluation of competence and credibility of key stakeholders including the owner, team, and customer)
  
  
  • Failure risk: How sensitive will the project be to changes occurring in the working environment? How can undesirable factors and outcomes be mitigated? (Assessment of the risk and consequences of unexpected results or failure)

---

Components of Evaluation Report

An evaluation report includes the following:

• Executive Summary describing main findings and recommendations on improving the project.
  
  
• The subject of your evaluation, including a brief history and description of the project and its cultural, technical, natural, economic, and social environments.
  
  
• Description of the background of the evaluation, including purpose, methodology, assumptions, limitations, etc.
  
  
• Name of the participants of the project and those who own and use the final product.
  
  
• Confirmation of your project to prioritize the needs and demands of the customer.
  
  
• A brief review of the benefits of your project and how they correlate with costs (cost-benefit analysis).
  
  
• Report on the probability of achieving the expected results and the effectiveness of your project.
  
  
• List of factors that ensure sustainability and compatibility of the project goals.
  
  
• List of factors that ensure the economic and financial sustainability of the project results.
  
  
• Confirmation of appropriateness of technology used within the project.
  
  
• Report on environmental impact (if any)