10.7Measuring Success: Process vs. Outcomes

Failure is the condiment that gives success its flavor.Truman Capote, writer, 1924 - 1984

As with communications planning (see Chapter 9: Strategic Planning for Communications), it is important to define what a successful public participation program will look like as you plan your outreach. There are no consistent methods for defining success in a public participation process for BRT, although there are both quantitative and qualitative methods available for evaluating public involvement. The threshold that defines “success” is dependent on the complex mix of variables including the size, reach, and level of controversy surrounding a given project, the resources available for a project from the organization, the community in question, and the overall intent of the public participation effort.

Criteria for measurement are set by the goals and objectives of the project. While measuring things like numbers of meetings and participants is relatively straightforward, outcomes are more complex to quantify. The International Association for Public Participation offers guidelines for evaluating participatory processes. Basic criteria regarding the process of participation includes whether:

• The public had access to appropriate resources and clear information to allow them to meaningfully participate;
• The purpose of the participation tasks were clearly defined;
• The decision-making process was structured appropriately to allow for and incorporate public input;
• Efforts were cost-effective;
• Views were diverse and representative.

Measuring success of outcomes is trickier to quantify because of the diversity of preferred results. For example, an agency might consider public support or ease of implementation as an appropriate outcome, while the public might consider the extent to which the community can achieve its goals or block decisions as better measures of success. Outcome-based success measures can include:

• Project or decision acceptability;
• Project efficiency;
• Cost avoidance;
• Mutual learning and respect;
• Improved understanding;
• The amount of conflict resolution required;
• The degree of consensus achieved;
• Influence on decision making;
• Participant satisfaction with the results of the process.

Preestablished metrics, including performance indicators, benchmarks, and performance standards, set beforehand and based on key project goals, can help gain up-front agreement on what to measure. These metrics can also be integrated into the project plans. Regardless of the evaluation method used, it is essential to keep evaluation in mind as part of public participation planning from the outset. While some approaches will be more fruitful than others, clearly delineating expectations at the beginning will help determine what needs to be changed as the project develops, and what you can do differently in the future. Public participation, if managed properly, is a gold mine of information and a source of knowledge that cannot be achieved in any other way.