Some artists claim praise is irrelevant in measuring the success of art, but I think it’s quite relevant. Besides, it makes me feel great.Chris Van Allsburg, illustrator and writer, 1949–
An often-neglected, yet critical part of any communications plan is measuring the success of the plan. While measurement is useful for satisfying funding requirements, the most important reason for measuring success is to ensure continuous learning and improvement. Identifying both quantifiable and anecdotal evidence to measure success helps demonstrate the system’s progress to internal audiences such as public transport staff and external audiences such as funders, policymakers, and the media.
The communications planning process should include identifying what “success” looks like for each stated objective. From that definition, a set of benchmarks can be developed to measure against. The plan should include allocating resources necessary to conduct research for reporting progress toward these benchmarks. The purpose of developing a communications strategy is to ensure that key messages are getting to the right audiences and influencing their behavior. It is important to correct course quickly if the strategy isn’t working. Ongoing revision of the communications plan is a reality of nearly every communications effort, and not a sign of failure. Plans should be adapted and modified as often as is necessary.
Measures of success should be a mix of outputs and outcomes. Outputs for example, can include things like positive news coverage, advertising, or meetings held with stakeholders. These should be straightforward to measure and are quantifiable. How many press hits were there? How many public meetings were held?
Outcomes are the changes that occur because of these outputs. For example, did positive press coverage lead to a positive shift in public opinion? Did stakeholders choose to become more involved in the project because they were happy with the meetings they attended? While measuring outcomes is more complex, it is essential to do so regularly to determine if the outputs are having the desired effect.
If, for example, your goal was to get all public transport riders on a specific corridor to use the system, ridership data can be collected. If, however, an objective is more qualitative, such as “build support”, a more sophisticated measurement system, and/or anecdotal information and analysis may be required. To illustrate how to measure success, consider the sample goal and objectives from the beginning of this chapter.
Goal:Improve the quality of urban life with a world-class BRT system.
Objectives:See a 10 percent mode shift toward public transport over two years;
Maintain a safe and reliable system in a state of good repair;
Support economic development along the corridor with a significant increase in street-level retail within a year of the system opening.
As we discussed in the goals and objectives section, the goal is a broad statement of your vision rather than a measurable one, while objectives should be SMART: Specific, Measurable, Attainable, Realistic, and Time-bound. The objectives we’ve set for our broad vision goal should be broken down into metrics to measure progress, and these should be evaluated throughout the process. Some examples of metrics for each objective:
Objective: See a 10 percent mode shift toward public transport over two years.
Setting metrics for modal shift can be complex, but most use some form of regular surveys. Data should be collected before the opening of the system and at regular points throughout the first two years of operation, tracking the number of trips, trip type and characteristics, and mode of travel used.
Objective: Maintain a safe and reliable system in a state of good repair.
This objective is somewhat more complex than the first, but it still fits the SMART requirements if broken down properly. Define what you mean by “safe and reliable” and “state of good repair”. Possible metrics include low or no annual customer injuries, a low percentage or overall number of service delays, or how many days it takes for a repair job to be completed. As this is an ongoing objective, the time bound part of the objective could be met with progress being measured quarterly, monthly, annually, etc.
Objective: Support economic development along the corridor with a significant increase in street-level retail within a year of the system opening.
This objective could be measured in many different ways. The most obvious way would be to compare how many retail outlets exist before the corridor versus how many are open within a year of the BRT. But what this objective is really measuring is how the BRT is leveraging transit-oriented development along the corridor, so the measures will need to take that into account. “Significant increase” should be defined as is realistic for this area. Twenty percent more stores? Fifty percent more pedestrian activity? Also, does street-level retail contribute to a more pleasant environment, with places for people to walk and sit? Or is it taking away from this with compound walls and anti-loitering policies?
Also, since the objective contains “support”, that will also need to be defined. Progress toward the objective can include changes in zoning and land use policy that allow for higher densities along public transport, or tax breaks for desirable businesses opening along the corridor.
In cases where success is not achieved or not clearly determined, having as much data as possible helps to understand why, and what can be done differently in the future to gain the support of the stakeholder group or stave off negative press you may have received. There is no substitute for experience, and a good system for measuring success will help to leverage resources efficiently.