Urban sustainability: learning from evaluation of community plans in Calgary.
Article Type: Report
Subject: Sustainable development (Models)
City planning (Models)
Urban policy (Evaluation)
Authors: Tsenkova, Sasha
Damiani, Rose-Mary
Pub Date: 06/22/2009
Publication: Name: Canadian Journal of Urban Research Publisher: Institute of Urban Studies Audience: Academic Format: Magazine/Journal Subject: Social sciences Copyright: COPYRIGHT 2009 Institute of Urban Studies ISSN: 1188-3774
Issue: Date: Summer, 2009 Source Volume: 18 Source Issue: 1
Topic: Event Code: 290 Public affairs
Product: Product Code: 9107130 Urban Planning Assistance NAICS Code: 92512 Administration of Urban Planning and Community and Rural Development
Geographic: Geographic Scope: Canada Geographic Name: Calgary, Alberta Geographic Code: 1CANA Canada
Accession Number: 229218940
Full Text: Abstract

Various types of planning policy evaluation can occur at different points of policy formation and application. This research paper develops a framework for ex post evaluation of community plans, drawing on the main sustainability goals and objectives of the Sustainable Suburbs Study--a major planning policy document adopted by the City of Calgary in 1995. Due to the number of policies and the overlapping nature of the policy intentions, the evaluation framework consolidates different planning policies into four thematic clusters related to intended outcomes that identify the direction for change. Further, performance in the clusters at the community level is evaluated through a system of indicators, developed to measure the achievement of intended outcomes.

The application of the evaluation framework in four case study communities in Calgary indicates that performance in relation to sustainability goals and objectives has been limited, although planning policies reflect a major shift and new direction for community planning that is more sustainable. Efficiencies and diversity measured through unit densities and housing mix have exceeded minimum targets, however, efficiencies in local and citywide infrastructure were constrained as local employment and the extent of commercial, social and cultural activities in neighbourhood core/nodes do not support a modal shift from the private automobile to walking, cycling or transit use. Accessibility policies have filtered into new community design and development but not at the comprehensive level at which considerable transportation and land use efficiencies would be recognized. Finally, action on environmentally responsible policies was limited due to the lack of integrated design policy promoting waste, water and energy management.

Key words: community plans; sustainable suburban development; ex post evaluation

Resume

Plusieurs types d'evaluation des politiques de planification peuvent se presenter a differents stades de formation et d'application d'une intervention publique. Base sur les buts et les objectifs principaux de durabilite du document Sustainable Suburbs Study--un document important de planification publique adopte par la ville de Calgary en 1995--cet article presente un cadre d'evaluation ex post de ces plans communautaires. Parce qu'un grand nombre de politiques et d'intentions se chevauchent, les differentes politiques ont ete regroupees autour de quatre axes thematiques, en fonction des resultats vises identifiant l'orientation pour le changement. De plus, la performance de chaque groupe est evaluee au niveau communautaire avec un systeme d'indicateurs developpe pour mesurer les resultats projetes.

Le cadre d'evaluation, utilise pour etudier quatre communautes calgareennes, indique une performance limitee quant aux buts et aux objectifs de durabilite, et ce meme si les politiques de planification refletent un changement important et une nouvelle orientation vers une planification communautaire plus durable. Les rendements et la diversite, mesures par unite de densite et d'habitat mixte, ont depasse les cibles minima, mais les rendements des infrastructures locales, et a l'echelle de la ville, se sont averes limites puisque les genres d'emplois, et les activites commerciales, sociales et culturelles dans les centres/noeuds de quartiers, n'appuient pas un changement modal de transport, de l'automobile, a la marche, au velo, ou au transport en commun. Des politiques d'accessibilite se sont retrouves dans les nouveaux designs de developpement communautaire, mais non a des niveaux reconnaissables pour un bon rendement des systemes de transport et de l'exploitation territoriale. Enfin, il y a des limites a l'execution des politiques environnementales responsables, due a l'absence de politique integree pour gerer les dechets, l'eau et l'energie.

Mots cles: plans communautaires; developpement durable des banlieues; evaluation ex post

Monitoring and Evaluation in Community Plans: A Missing Link

Various types of policy evaluation can occur at different points of policy formation and application. Much of the literature regarding planning policy evaluation focuses on ex ante evaluation: the evaluation of the expected outcomes (i.e. costs and benefits, etc.) of policies as a means to choose between various alternatives in the policy formation stage (Bracken 1981; Lichfield et al. 1975). However, ex post evaluation with an emphasis on outcome based performance is rarely carried out in planning practice. The evaluation is concerned with the outcomes of the plan and determines whether the policies should be continued, modified, or terminated (Patton and Sawicki 1986). A number of studies point to the need for continuous monitoring and evaluation of community plans to establish a feedback loop (Barton et al. 2003; City of Calgary 1997). Requirements include: first, the continuous surveying of the conditions produced by the implementation of plans; and second, an evaluation of the survey to enlighten and adapt plans to improve progress and reach initial goals. This cycle in community planning is often hOt accomplished (Kelly and Becker 2000). A lack of monitoring limits the information about the performance of plans, which in turn limits the extent of a comprehensive ex post evaluation. The result of this incomplete feedback loop is a tendency in land use planning to proceed with plans without a proper evaluation of their influence on intended outcomes. Calkins (1979) refers to this lack of evaluation as 'new plan syndrome,' in which plans are updated or redone without regard to the implementation status of the originally prepared plan.

Planning without evaluation can propagate design and development characteristics that may not be achieving the intended plan outcomes, with either positive or negative externalities. "In the planning literature, evaluation is used primarily to understand why planning--planners, planning practice--does what it does instead of whether or not plans are invoked ... the reasons why planning fails are unearthed exhaustively without sufficient empirical grounding in what planning has failed (or succeeded) to do" (Talen 1996, 249). Evaluation of community plans, and ex-post evaluation in particular, propagates and supports informed decisions and can act as a communication tool for describing the impacts and value of particular plans (Seasons 2005). Evaluation can be a means of learning by doing. Consistent evaluation of plan outcomes can clarify what constitutes effective planning practice in substantive terms while adding empirical evidence to theoretical discussions (Talen 1996; Seasons 2005). These benefits of planning evaluation have become increasingly important in the current discourse on sustainable community planning, where a good understanding of successful plan implementation is instrumental in urban growth management (Kim 2005; Roseland 2005). The location, type and form of that growth in cities have an impact on the potential for fiscally, environmentally and socially sustainable practices. Growth management strategies that incorporate sustainable new community development practices can reduce the costs associated with growth and promote more livable communities (Smart Growth Network 2003).

Evaluation of Community Plans and Sustainability

How could one assess the impact of city-wide planning policies on the sustainability of a community? More precisely, how could one show that a suburban community plan has adopted land use principles that guide development towards or away from sustainability?

The planning of sustainable communities represents an important overarching goal and a long-term vision for the future, along with sustainable consumption and production, climate change and natural resource protection. However, despite a growing interest in sustainable community planning, academics and planning practitioners still lack the tools necessary to determine whether and how plans and policies aimed at community sustainability reach their intended goals. The challenge is both theoretical and practical. Sustainable communities have been defined as an aggregate of characteristics including, among others, economic vibrancy and growth, environmental quality and integrity, social cohesion and quality of life, empowerment and governance. (1) Few places have incorporated sustainability across their entire social and economic processes and their physical fabric to establish benchmarks for performance (Barton and Kleiner 2000). Measuring sustainability is also controversial and the metrics often locally defined (Bell and Morse 1999). (2) Despite these issues, some argue that it is important to monitor progress towards sustainability, as people need a reality check to ensure that incremental steps are taken in a desired direction (Hemphill et al. 2002; Innes and Booher 2000). Many authors use sustainable community check lists without clear theoretical and methodological foundations or resort to environmental impact indicators (Barton 2000; Barton et al. 2003; Bell and Morse 2003; Brownhill 2002). (3)

Researchers have examined questions of community plan making and its effectiveness within the context of sustainability, smart growth management and natural hazard mitigation (Conroy and Berke 2004; Burby and Dalton 1994; Dalton and Burby 1994; Nelson 2002). Findings from these studies, particularly regarding the effectiveness of city-wide or state-wide sustainability objectives and policies and their implementation in local community plans, send a mixed message. They document that local governments with stronger planning mandates address issues of growth and environmental planning in a comprehensive manner, although strategies might be different and success in implementation varies. Berke and Conroy (2000) examined thirty comprehensive plans for their adherence to the principles of sustainability, concluding that there was essentially no difference between the substance of plans that are explicitly guided by sustainable development frameworks and those that are not. Evaluations of the implementation of smart growth principles by Nelson (2002), Edwards and Haines (2007), and Talen and Knaap (2003) concluded that the movement has a limited influence on local community plans and actual regulatory changes. Ye et al. (2005) identified six dimensions of smart growth accompanied by a list of elements of smart growth strategies based on commonalities of various definitions. These include natural resource preservation, housing and economic development, transportation choice, and planning for smart growth. The evidence on the breadth and coverage of these strategies in local plans was less conclusive, although limiting outward expansion, increasing densities, providing more mixed-use developments that minimize auto-dependency and emphasizing public transit were generally key elements. Downs (2005) recently outlined formidable obstacles to the implementation of smart growth goals and objectives, including resistance to policy shifts, hostility to high density developments, inefficient public transit policies, and a lack of regional planning and coordination, to name a few.

In summary, the principles of smart growth, new urbanism and sustainable development are part of a planning philosophy dominating much of the planning literature in recent years, but their influence on community plan making and plan implementation might be limited. With the launch of LEED for Neighbourhood Design in the United States (4) a framework for the evaluation of community plan implementation might become an operational way of addressing the gap in planning evaluation in the context of sustainable community planning. The framework includes tested indicators and metrics and allows for continuous evaluation from the development of goals and objectives through the final stages of community plan implementation. This research builds on the conceptual and methodological foundations of studies dealing with ex post evaluation of community plans through sustainability lenses. In the tradition of conformance-based evaluation it seeks to determine how well the plan has been implemented. The evaluation is focused on the relationship between the plan, its policies and the physical development that results. Laurian et al. (2004) suggest that conformance-based evaluation focuses on planning outcomes and assumes a direct relationship between the plan objectives and the outcomes of the plan. Furthermore, it also assumes that policies in the plan are specific enough to guide development and that the degree to which development adheres to, or departs from, these policies can be measured, either qualitatively or quantitatively. Like Edwards and Haines (2007), we argue that in evaluating the effectiveness of sustainable community planning policies, one has to examine both the content and the actual outcomes of plans. This entails the evaluation of implementation tools, zoning and subdivision regulations, and design guidelines in order to establish clear linkages between plan implementation and outcomes and the sustainability of these outcomes.

Objectives and Methodology of the Research

This research attempts to develop a framework for ex post evaluation of community plans, drawing on the main sustainability objectives in a major planning policy document adopted by the City of Calgary in 1995. The Sustainable Suburbs Study (SSS) promoted alternative practices in land use planning to create socially and environmentally responsive communities. The study emphasized three important policy goals central to sustainability:

* Fiscally, the cost of building, operating and maintaining new communities and their supportive infrastructure and services are affordable, having regard to other spending priorities, and will not become a burden on future generations;

* Socially, communities are designed to be socially diverse, adaptable to changing lifestyles and to further the objective of providing all Calgarians with access to affordable housing, education, health care, essential goods, public amenities and services, such that their basic needs are met; and

* Environmentally, communities are designed to minimize air, water, and soil pollution, reduce resource consumption and waste, and protect natural systems that support life. (City of Calgary 1995, 3)

In addition, The City of Calgary has adopted a number of high-level directive policies to promote the implementation of sustainable principles through land use planning. Such principles are embedded in the Calgary Transportation Plan (1995), Calgary Municipal Development Plan (1998), and lmagineCALGARY (2006), the most recent long-range urban sustainability plan. While the SSS was a pivotal document in the history of land use planning in Calgary, promoting citywide sustainability objectives, there has been no formal monitoring of performance and/or evaluation of its implementation to date. As the SSS comes under review, the ex post evaluation process seeks to assess the compliance of new communities in Calgary with the goals, objectives and policies articulated in the SSS, as well as to determine the gap between the SSS policy objectives and the results manifested in a sample of new communities in Calgary. The evaluation centres on compliance of the Area Structure Plans (ASPs)--policy and land use planning documents guiding development in new communities--as well as implementation outcomes manifested in the built environment. The research is designed to assist the analysis and evaluation of the effectiveness and efficiency of planning policies directed at more sustainable growth management in the new communities of Calgary. It has the following objectives:

* To develop a framework for evaluating the implementation of Sustainable Suburbs planning policies; and

* To apply the evaluation framework to a sample of new communities and to assess the success of plan implementation.

The methodology combines quantitative and qualitative methods including a literature review, policy content analysis, case study analysis and key informant interviews. The literature and policy document review inform the development of the evaluation framework and the selection of indicators that are measurable, reliable and manageable. The policy content analysis focuses on policies and performance criteria in the SSS to design an evaluation framework that allows us to determine the effectiveness of the implementation of sustainable design and development strategies in the context of suburban Calgary in the last decade. The case study approach allows the application of the evaluation framework to a sample of four communities representative of development trends. Further, indicator measures and outcome ratings, grouped in several sustainability clusters/themes for each community, are summarized in community report cards. Finally, input from key informant interviews is essential for the selection of conceptually appropriate case studies as well as for the actual implementation of the evaluation framework.

Planning Policy Context

Various levels of land use planning policy influence the development of new communities in Calgary. The Municipal Government Act sets out a framework for statutory policy plans that include: Inter-municipal Development Plans, the Municipal Development Plan (Calgary Plan), Area Structure Plans and Area Redevelopment Plans. Calgary's land use planning policy framework includes different statutory and non-statutory plans schematically presented in Figure 1.

The research is concerned with the planning of new communities and more specifically with the influence of the SSS on the development and implementation of ASPs. This is particularly important in the context of Calgary, given its rates of growth in the last 15 years and the fact that more than two-thirds of this growth is accommodated in the suburbs. (5) These plans establish the general planning framework for new communities, provide the basis for the more detailed levels of planning and address a range of technical matters such as transportation, servicing networks, locations of schools, parks, and commercial sites and the density and types of uses permitted. The ASP is a statutory document that is approved by Council, while the SSS has the status of a Special Study completed by city staff and adopted by Council resolution. It affects the content of ASPs and correspondingly influences the implementation of city-wide goals and objectives through the legally binding requirements for development.

[FIGURE 1 OMITTED]

The SSS was developed to expand on high-level policy that directly relates to suburban growth management such as the Calgary Transportation Plan (1995), which argued for a significant reduction in the vehicle trips generated by new suburbs. Furthermore it responded to Environmental Policy, Principles and Goals (1994), which recommended policies that encourage less automobile use, the conservation of resources, reductions in waste, and low impact development. The SSS was developed during a political climate when provincial funding for infrastructure (transportation, health care, education and family support services) was cut back significantly. This forced municipalities to accept increased responsibilities, and consequently, created a greater burden on the City's revenues. The policies provided means to deal with new municipal fiscal realities while addressing the ever-increasing awareness for community quality of life and environmental issues.

Evaluation Framework

Policy evaluation can occur at various points of policy formation and application. In the evaluation of community plans, the process begins by specifying policy goals and outcomes that are the requirements for achieving those goals (Moursund 1973). The outcomes are physical representations of the goals. Criteria or indicators then need to be specified as a means to determine outcome performance. Indicators tend to be objective and measurable representations of planners' standards and development criteria and they may relate to 'accessibility' (or spatial interactions) and 'space' (e.g. density of development) measures of the built environment (Bracken 1981, 79). For each indicator a target is set to represent the intended outcomes, which is used as the basis for comparison with the real world measures of the plans evaluated (Patton and Sawicki 1986). The development of an evaluation framework to analyse the degree of compliance and/or the performance gap between the SSS policies and development in new communities follows the general rules outlined in the planning literature. The framework content is derived from the hierarchical structure of goals, objectives and policies of the SSS. The goals introduce a long-term vision for the form and function of new suburban communities in Calgary, while the objectives define a strategy for more sustainable planning and design of new communities.

Identification of Major Thematic Clusters and Outcomes

We initiated the process of evaluation framework development by categorizing the goals and objectives of the SSS into three major domains--economic, social and environmental, which represent the three basic elements of sustainability, common to all frameworks. The three major goals identified are:

* Reduce the cost of suburban development;

* Design more livable suburban communities that are accessible to a broad cross-section of society, give people genuine options for housing and mobility, and are adaptable to changing demographics and lifestyles; and

* Reduce environmental impacts through community and building design.

These goals were related to eight major objectives, which were supported by twenty eight clusters of planning policies, some of them with considerable overlap. The planning policies were initially categorized into several thematic categories-community centres and neighbourhood nodes, schools and open space, housing, transportation, accessibility and environment. These were further aligned with the goals and amalgamated in four core domains--efficiency, diversity, accessibility, environmental responsibility. A concise version of the policy matrix with key goals and objectives presented in Table 1 identifies important linkages to planning, design and development policies and the four broad sustainability themes/ clusters used to evaluate plan implementation.

Efficiency The concept of efficiency is expressed in the economics of servicing so that the costs of building, operating and maintaining new communities and their supportive infrastructure and services are affordable and will not become a burden on future generations. According to SSS policies this translates into a more compact urban form that can provide land efficiencies as well as capital cost savings with the slower extension of hard infrastructure. Also, better utilization of services like public transit can decrease the downstream costs of city-wide infrastructure systems by less dependence on private automobile trips outside of the community.

Diversity refers to the opportunities for people of different lifestyles to live and be active in the community. It recognizes that the community needs to be accessible to people of different economic backgrounds and that the daily service, retail and educational needs of various residents must be met by the community's facilities and amenities, this is reflected in policies promoting housing choice and diversity in the local amenites that cater to many household types and lifestyles. The SSS policy framework strives for a land use mix that provides residential, commercial and public uses. The integration and design of community cores and neighbourhood nodes intends to provide public and private activity centres within the community, thus allowing convenient options for people and the possibility of choosing travel modes other than the private automobile.

Accessibility refers to the extent to which the community design promotes the vibrancy and use of community facilities and amenities. Further to the provision of local amenities, accessibility refers to providing desirable design elements that enhance access to and use of the focal points and amenities. A focus on travel options encourages healthier lifestyles through more walking and cycling, improving access to local facilities by means other than the private automobile. This entails enhanced transit service and suburban streetscape design that promotes walking and cycling. Planning policies focus on local public spaces to increase public activity, enhancing the economic vibrancy of local facilities while reducing energy use and air pollution.

Environmental Responsibility means the extent to which the design and development of the community manages natural resource consumption. In a new suburban community context the SSS planning policies encourage high water quality through the integration of natural systems in stormwater management and preserving environmentally significant areas. The design and development advocate responsible resource consumption promoting lifestyles where consumption and waste can be reduced and conservation encouraged.

With the four thematic clusters defined, a matrix of SSS goals, objectives and planning policy priorities was prepared. Further policy content analysis established an alignment of planning policies with the design-based principles of well-known sustainable suburban planning practices promoted by the smart growth network and new urbanism, however with divergent policy targets in some cases (City of Calgary 1997; 2007). Due to the number of policies and the overlapping nature of the policy intentions, the evaluation framework consolidates the twenty six planning policies into outcomes that identify the direction for change and intended results. The outcomes were identified through a validation process with other frameworks for community planning evaluation that have achieved a degree of legitimacy either from planning practice or academic debate. The sixteen outcomes in the framework are closely related to the thematic sustainability clusters, as indicated in Figure 2.

Performance in the clusters was evaluated through a system of indicators, developed to measure the achievement of intended outcomes, based on the performance criteria set out in the SSS. The indicators selected were based on a number of planning evaluation frameworks at the community or neighbourhood level (see for example Smart Growth Network 2003; Strathcona County 2007; US Green Building Counci1 2007). We applied a two-filter process to arrive to a manageable set. First an overlapping or similarity filter was applied, which clarifies whether indicators are overlapping or similar in the sense that they provide the same information. Second, the local sustainability filter refers to sustainability concepts that are explicitly related to planning policies implemented through community plans, The local filter is the result of the definition of sustainability in the SSS and its emphasis on the impact of specific planning measures on new communities. Thus, measures such as 'noise pollution' and 'air pollution' have been excluded as were indicators requiring resident surveys on satisfaction with local area services, design or public transport. In addition, the final framework of outcomes and indicators were discussed with 12 senior level planners involved in the planning and development of new communities in Calgary and the implementation of the SSS to augment some of the indicators.

[FIGURE 2 OMITTED]

It should be acknowledged that the selection of indicators was restricted by data availability, but more importantly considered the criteria of relevance, logical interpretation, reliability and manageability (Wong 2006; Sustainable Seattle 1998). The indicator measures were translated into a rating system representing the degree of performance. The result was a community report card that is user-friendly and simple to use, but fairly robust with a range of quantifiable measures that reflect clear relationships between outcomes, objectives and goals. The indicators required the use of practical methods of data collection or measurement (e.g. content analysis of ASPs and/or personal observation of development patterns in new communities to create accessible data). An effort was made to create a manageable system with sufficient information to complete a thorough analysis, while maintaining a preference for quantitative data to minimize personal bias (Tsenkova 2006). It is recognized that in assigning ratings and in the production of composite scores there is a certain amount of subjective judgment from the analyst (Wong 2006). Clarity on the dimension of the element being ranked and the links with the policy objectives provided the relevance of the indicators and associated ratings.

Indicator Profiles and Indicator Rating Scales

A rating method was applied to the system of indicators to simplify the data into a set of scores. Each indicator was measured and then the result was translated into a rating from zero to three.

0--No performance related to the indicator.

1--Limited performance, in which some progress was evident, however lower than the projected targets.

2--Good performance, meeting or slightly exceeding minimum targets.

3--High performance, exceeding minimum targets.

The rating specifications for each indicator were derived from performance targets set out in the SSS. The partial rating for each outcome was determined from the indicator ratings. Though the number of indicators associated with each outcome varies, the related indicators were assumed to be of equal value in determining the partial rating of outcomes. The community report card presents the degree of plan implementation based on a composite rating. A composite rating, ranging from one to twelve for each sustainability thematic cluster, indicates the extent of policy implementation. The composite rating was derived by adding the partial ratings, from zero to three, of the outcome statements (Damiani 2008). Each outcome statement, in accordance with SSS policies, was weighted equally in the composite rating that evaluates the extent of compliance in the ASP.

Following is a profile of one of the indicators in relation to the intended outcomes to illustrate the approach.

Outcome #2: Increase the Accessibility, Comfort and Safety of Public Transportation.

Indicator: Proportion of dwelling units within 400 metres of the transit network Sustainability Concept: Accessibility to the transit network and stops is an important factor in attracting a significant number of transit riders. Accessibility and convenience of public transit to residents is enhanced by providing transit stops within a reasonable walking distance from dwelling units. Accessible public transit should correlate to increased transit ridership, and in turn lower the City's operating cost per passenger.

Requirement: 85% of dwelling units are within 400 m of a transit stop (City of Calgary 1995, p. 55).

Rating: The rating indicates whether development complies with the explicit accessibility target. For example, if the estimated share is lower than 75%, the rating is 0; if it is between 75 and 84% the rating is 1; between 85 and 95% it is 2; and over 95% gives a rating of 3.

Applying the Evaluation Framework

The evaluation framework was applied to a sample of new communities in Calgary. The case studies were selected with the assistance of senior planners from the City of Calgary and include four of the 22 communities with ASPs approved in the last decade, The case study communities range in size from 247 to 407 ha and are located in the outer suburban territory of Calgary (see Figure 3). McKenzie Towne adopted the new urbanism framework and has been recognized with a Canadian Institute of Planners Award of Excellence as well as by the Urban Land Institute for its unique form of suburban development. According to city planners the other two suburban communities (Evergreen and Bridlewood) have been extensively influenced by SSS policies (City of Calgary 1997). Garrison Woods--as the last case study in the sample--is somewhat different. This infill community of 70 ha is identified as best practice in sustainable neighbourhood design and development in Calgary. It has received recognition through the Award for Excellence for Environmental Design in 1999 from the Alberta Association of the Canadian Institute of Planners and the Best Practice and Comprehensive Planning Award in 2000 from the Real Property Institute of Canada. Development in all case study communities was initiated in 1997 and with the exception of Garrison Woods, which is built out, is an ongoing process. Both McKenzie Towne and Garrison Woods are promoted and built by a single developer.

[FIGURE 3 OMITTED}

The evaluation results are presented in the community report cards with the indicator and the partial and composite ratings (see the example of McKenzie Towne in Table 2). The indicator ratings related to each outcome were averaged to derive the partial rating ranging from zero to three. The composite rating is the total of the partial ratings in each thematic cluster, while the sum of the four sustainability cluster ratings represents the overall performance score. While detailed analysis and interpretation of the community report cards is beyond the scope of this paper, the evaluation summaries identify limited success in the implementation of sustainable planning policies in the four major thematic clusters. McKenzie Towne and Garrison Woods perform better compared to the more traditional suburban communities reaching score of 23 and 24 out of 48.

Efficiency

Land use planning policy has improved land efficiencies through the increase of unit densities in new communities, particularly in McKenzie Towne and Garrison Woods, but has failed to realize economic efficiencies through shared sites and/or buildings (Damiani 2008). Such practices could have increased the intensity of use on sites and of buildings while in turn providing facilities that could enhance local activity by residents. Further, home occupations have not become a significant element of new communities. Residents continue to leave the community for employment purposes. Though access to transit service is adequate according to SSS requirements, transit has not been given priority in core and node design as a comfortable and viable means of transportation. Travel by private automobile continues to be a more attractive option for community residents thus leading to higher cost per capita in transit service and downstream costs on road infrastructure.

[FIGURE 4 OMITTED]

Diversity

The integration of diverse housing options and basic service and retail opportunities within new communities has had relative success. The share of multifamily housing in three of the case studies ranges from 25 to 36%, while Garrison Woods accommodates 88%. Suburban communities remain predominantly single family neighbourhoods (see Figure 4). While in most cases an attempt is made to provide a range of dwelling types to accommodate different types of households, there is a lack of provision of affordable (non-profit and/or assisted) rental housing. Furthermore, opportunities to satisfy basic daily needs--shopping, recreation, work, education--within the community are provided through the land use mix in community cotes and nodes. Such policies in practice have had a limited success with an average of 65% of the residents within walking distance to community amenities. In all communities under review basic commercial needs and schools are concentrated within the cote, while neighbourhood nodes have evolved predominantly as open spaces with low intensity of use (see Figure 5).

Accessibility

Planning policies related to accessibility have had satisfactory performance. Street patterns remain curvilinear and public space along the street does hOt meet the enhancements for a comfortable pedestrian environment. Pedestrians and cyclists are accommodated through segregated pathway systems that provide direct linkages to community focal points. It should be acknowledged that the design of boulevards and major streets has been enhanced in McKenzie Towne and Garrison Woods through extensive sidewalks, trees and traffic calming measures. Rear lanes are commonly used to provide for narrower lots in all new communities under review, although to a more limited extent in Evergreen and Bridlewood. This provides an improved streetscape from the common front-drive garage style single family home.

[FIGURE 5 OMITTED]

The design of the community cores in McKenzie Towne and Garrison Woods represents good practices relative to the SSS policies and a successful attempt to promote pedestrian-oriented design. By contrast, in Evergreen and Bridlewood community core sites are auto-oriented with vast parking areas and access points oriented towards them (see Figure 6). All new communities have significant amounts of open space provided through the 10% dedicated land as municipal reserve. Access to open space areas is very good, but the types of recreational activities remain limited, mainly accommodating passive and children's recreational activities.

[FIGURE 6 OMITTED]

Environmental Responsibility

Performance in this area is extremely limited. Planning policies implemented relate to alternative storm water practices and wetland conservation (see Figure 7). Waste management initiatives remain limited to the inclusion of community recycling bins, while non-renewable energy use was not enhanced through planning policy or any other means.

[FIGURE 7 OMITTED]

Concluding Comments and Implications for Planning

In summary, the research suggests that unit densities and housing type mix reached the minimum targets set out in the SSS. The efficiencies in local and citywide transportation infrastructure intended were not achieved as the employment opportunities, extent of commercial, social and cultural activities, and location and form of density do not support extensive modal shift from the private automobile to walking, cycling or transit use. Action on the environmentally responsible policies of the SSS was extremely limited as integrated design policy promoting waste, water and energy management in ASPs was absent. Overall, elements of SSS policy have filtered into new community design and development but not at the comprehensive level at which considerable transportation and land use efficiencies would be recognized. It has taken a decade and a major shift in Calgary's housing market to implement, and even surpass, some of the policy targets in the SSS, setting a direction for new community design and development that is more sustainable.

The findings are indeed less surprising given the general consensus in the planning literature about the limited effectiveness of city-wide sustainability objectives and policies and their implementation in local community plans (Conroy and Berke 2004; Dalton and Burby 1994). The research confirms the evaluation results of several studies arguing that the implementation of smart growth principles has a limited influence on local community plans and actual regulatory changes (Nelson 2002; Talen and Knaap 2003; Ye et al. 2005). Although the SSS goals, objectives and policies are not necessarily identical to the principles of smart growth and new urbanism, admittedly they strive to achieve very similar outcomes and development patterns that are more sustainable. Overall, so far sustainable suburban development in Calgary might be more of rhetoric than a reality.

The results of the research need to be interpreted in the context of its limitations. First, the evaluation indicators are referenced from a limited number of ASPs. Second, data accessibility is a considerable constraint in the development and application of an indicator-based evaluation framework. The sample communities include only those for which research was compiled for the City of Calgary in 2007, rather than completing a comprehensive review including all communities planned and developed since 1995. Third, the ranking system provides a rapid assessment of a complex subject. Such a framework can conceal detailed information on different elements of sustainability, presenting the potential for misinterpretation (Tsenkova 2006). However, rigorous analysis and interpretation of the ratings provides an added value by converting the evaluation information into knowledge on policy gaps and implementation challenges (Wong 2006). Lastly, due to the long-term timeframe of ASPs and the pace of actual development none of the communities under review is completely built out (with the exception of Garrison Woods). Indicators are based on the actual built form and the anticipated development types and forms derived from the approved ASPs, thus excluding future developments and/or changes. Notwithstanding these limitations, insights gained from the ex post evaluation can inform the policy review and enhance the development of new planning policies and performance standards promoting sustainable patterns of development in Calgary.

Despite growing interest in sustainable community planning, academics and planning practitioners still lack the tools necessary to determine whether and how plans and policies aimed at community sustainability reach their intended goals. In the tradition of conformance-based evaluation, this research addresses the need for ongoing evaluation of community plans to determine how well they were implemented. The evaluation framework is normative, focused on relationships between the plan, its policies and the resulting planning outcomes. Such evaluations are uncommon and planners have hOt developed adequate methods to deal with the wide policy and implementation spectrum covered. The approach makes a contribution in that regard but should not be seen as a comprehensive method, let alone as a universally applicable recipe. One may be critical about the choice of some planning outcomes and/or indicators or of the way they are categorised. The framework is guided by the policy content and criteria advanced in SSS itself, thus using a subset of criteria and indicators suggested by theoretical work on sustainable community planning. The emphasis on evidence-based, quantitative research that draws on a limited set of measurable, quantifiable indicators has obvious limitations for documenting what is actually accomplished to change development patterns in new communities. It does not measure the level of service provided or changes in resident/consumer behaviour, but remains grounded in the regulatory powers of land use policies and design guidelines.

Much more effort is needed in terms of data collection and evaluation methods so that planning outcomes can be adequately evaluated, making the measurement variables explicit and connected to implementation tools to allow cross-sectional comparisons. Academics and planning practitioners need to become actively engaged in a dialogue on the effectiveness of evaluating community plans and on the design of appropriate tools. A related question for future research is whether community plans, as the regulatory planning policy framework, have the capacity to implement city-wide sustainability goals and objectives. To what extent do other factors external to the process affect plan implementation? These are critical issues that will help us evaluate the effectiveness of planning at the community level and its ability to promote more sustainable development patterns in the future.

Acknowledgements

The authors acknowledge the input of Simon Ginn, former Manager of Sustainable Communities at the City of Calgary, and Dick Ebersohn, Team Leader of Sustainable Communities Project, who were instrumental in defining the scope and content of the research. Funding was provided by the City of Calgary through the Cities, Policy & Planning lab at the University of Calgary (http://www.Ucalgary.ca/cities).

References

Barton, H. ed. 2000. Sustainable communities: The potential for eco-neighbourhoods. London: Earthscan Publications Ltd.

Barton, H., M. Grant, and R. Guise. 2003. Shaping neighbourhoods: A guide for health, sustainability and vitality. Canada: Spon Press.

Barton, H., and D. Kleiner. 2000. Innovative eco-neighbourhood projects. In Sustainable communities: The potential for eco-neighbourhoods, ed. H. Barton, 2000, 66-85. London: Earthscan Publications Ltd.

Bell, S., and S. Morse. 1999. Sustainability indicators: Measuring the immeasurable. London: Earthscan Publications Ltd.

Bell, S., and S. Morse. 2003. Measuring sustainability: Learning from doing. London: Earthscan Publications Ltd.

Berke, P. and M. Conroy. 2000. Are we planning for sustainable development? An evaluation of 30 comprehensive plans. Journal of the American Planning Association 66 (1): 21-33.

Bracken, I. 1981. Urban planning methods: Research and policy analysis. London: Methuen & Co. Ltd.

Brownhill, D. 2002. A sustainability checklist for developments. Watford: Construction Research Communications.

Burby, R., and L. Dalton. 1994. Plans can matter! The role of land use plans and state planning mandates in limiting development in hazardous areas. Public Administration Review 54 (3): 229-238.

Calkins, W. 1979. The planning monitor: An accountability theory of plan evaluation. Environment and Planning A 11 (7): 745-758.

City of Calgary. 1995. Sustainable suburbs study: Creating more fiscally, socially and environmentally sustainable communities. Calgary: Planning and Building Department.

--. 1997. Monitoring the effectiveness of the sustainable suburbs study. Calgary: Planning and Building Department.

--. 1998. CFB East community plan. Calgary: Planning and Building Department.

--. 2007. Smart growth rating: Guidebook for the application of a smart growth rating system to outline plan and land use amendment applications (draft). Calgary: Development & Building Approvals.

Conroy, M., and P. Berke. 2004. What makes a good sustainable development plan? An analysis of factors that influence principles of sustainable development. Environment and Planning A 36:1381-1396.

Damiani, R. 2008. Implementing sustainable community policy: An evaluation of the sustainable suburbs study. Calgary: EVDS Cities, Policy & Planning Research series.

Dalton, L., and R. Burby. 1994. Mandates, plans and planners: Building local commitment to development management. Journal of the American Planning Association 60 (4): 444-461.

Downs, A. 2005. Smart growth: Why we discuss it more than we do it. Journal of the American Planning Association 71 (4): 367-387.

Edwards, M., and A. Haines. 2007. Evaluating smart growth implications for small communities. Journal of Planning Education and Research 27: 49-64.

Hemphill, L., S. McGreal, and J. Berry. 2002. An aggregated weighting system for evaluating sustainable urban regeneration. Journal of Property Research 19 (4): 353-373.

Innes, J., and D. Booher. 2000. Indicators for sustainable communities: A strategy building on complexity theory and distributed intelligence. Planning Theory & Practice 1 (2): 173-186.

Kelly, E. D., and B. Becker. 2000. Community planning: An introduction to the comprehensive plan. Washington: Island Press.

Kim, K. 2005. Towards sustainable neighbourhood design: A sustainability evaluation framework and a case study of the Greenwich Millennium Village Project. Journal of Architectural and Planning Research 22 (3): 181-203.

Laurian, L., M. Day, P. Berke, N. Ericksen, M. Backhurst, J. Crawford, and J. Dixon. 2004. Evaluating plan implementation: A conformance-based methodology. Journal of the American Planning Association 70 (4): 471-480.

Lichfield, N., P. Kettle, and M. Whitbread. 1975. Evaluation in the planning process. Toronto: Pergamon Press Inc.

Long, D. and M. Hutchins. 2003. A toolkit of indicators of sustainable communities. London: The Housing Corporation and Liverpool: The European Institute for Urban Affairs, Liverpool: John Moores University.

Moursund, J. P. 1973. Evaluation: An introduction to research design. California: Brooks/Cole Publishing Company.

Nelson, A. 2002. Do we know smart growth when we see it? In Smart growth: Form and consequences, eds. T. Szold, and A, Carbonell, 2002, 82-101. Cambridge, MA: Lincoln Institute of Land Policy.

Patron, C. V., and D. S. Sawicki. 1986. Basic methods of policy analysis and planning. Englewood Cliffs: Prentice-Hall.

Roseland, M. 2005. Toward sustainable communities: Resources for citizens and their governments. Canada: New Society Publishers.

Seasons, M. 2005. Monitoring, evaluation and indicators (MEI). [Presentation at Canadian Institute of Planners Conference 2005]. Retrieved from http://www. cip-icu.ca/English/conference/2005proceedings.htm.

Smart Growth Network. 2003. Getting to Smart Growth II: 100 more policies for implementation. Washington, DC: International City/County Management Association.

Strathcona County. 2007. Emerald Hills Urban Village goals, indicators and targets spreadsheet. Strathcona County: Author. http://www.strathcona.ab.ca/Strathcona/Departments/ Planning+and+Development+Services/Emerald+Hills/SuN+Living+.htm.

Sustainable Seattle. 1998. Indicators of sustainable community 1998. Seattle: Author.

Talen, E. 1996. Do plans get implemented? A review of evaluation in planning. Journal of Planning Literature 10 (3): 248-259.

Talen, E. and G. Knaap. 2003. Legalizing smart growth: An empirical study of land use regulations in Illinois. Journal of Planning Education and Research 22: 345-359.

Tsenkova, S. 2006. Ex-post evaluation of social housing programmes in Finland. Paris: Council of Europe Development Bank.

U.S. Green Building Council. 2007. LEED for neighbourhood development rating system: Pilot version. Washington, DC: Author.

Wong, C. 2006. Indicators for urban and regional planning: The interplay of policy and methods. London: Routledge.

Ye, L., P. Mandpe, and P. Meyer. 2005. What is 'smart growth'--really? Journal of Planning Literature 19 (3): 301-315.

Sasha Tsenkova

Rose-Mary Damiani

Faculty of Environmental Design

University of Calgary

Notes

(1) See Long and Hutchins (2003) for an in-depth discussion.

(2) For example, Kim (2005) uses a case study approach to create a sustainability evaluation framework that measures progress on sustainability indicators at a neighbourhood level. By compiling the findings from site to site, the author develops performance profiles to inform future plans.

(3) White and Ellis (2007) focus on indicators that measure environmental impacts--habitat loss and fragmentation, degradation of water resources and water quality, degradation of air quality, and green house gas emissions/climate change. Indicators are measured at different scales, from an individual home or building to the region, to demonstrate how the development process contributes to or detracts from sustainability.

(4) A Canadian version is expected to be launched before the end of 2009.

(5) Calgary's significant rate of growth results in the absorption of 570 hectares of residential and 120 hectares of industrial land on average each year to accommodate the 16,000 new residents.
Table 1: Sustainability Clusters Derived from SSS Goals and Objectives

Key Goals & Objectives       Planning, Urban        Sustainability
in the SSS Policy            Design & Development   Thematic Cluster
Framework                    Domains                Derived from SSS

Manage City's costs of       Transportation;        Efficiency
accommodating growth;        Housing density;
Use land more efficiently;   Employment
Encourage people to          opportunities
commute b transit

Provide local services and   Community Cores &      Diversity
amenities; Provide more      Nodes; Housing form
housing choice;              and choice

Encourage people to          Transportation; Open   Accessibility
walk and cycle within        Space
the community; Improve
access to local facilities

Protect and integrate        Parks & Open           Environmental
natural systems where        Space; Stormwater      Responsibility
possible; Encourage          Management;
home builders and home       Environmental Issues
buyers to reduce waste
and pollution

Source: Damiani (2008)

Table 2. McKenzie Towne Retort Card

                                               Indicator   Partial
Theme        Outcomes   Indicators              Rating     Rating
Efficiency

#1: Achieve a           Residential Unit           2          2
minimum /gross          Density
density of 3 units /
hectare.

#2: Increase the        Locating transit           1          2
accessibility,          network and stops in
comfort and safety      ASP
of public                                          3
transportation.         Proportion of
                        dwelling units
                        within 400m of
                        transit network

                        Transit Stop Quality       1

#3: Achieve shared      Number of shared use       0          0
use of sites and/or     sites and/or
buildings for           buildings
public
facilities
and services.

#4: Increase home       Number of design           0          0
Diversity               elements tailored
occupations.            for home occupations

#5: Increase            Distance of core           1          2
accessibility to        from regional
mixed-use activity      shopping centre
centres.
                        Ratio of commercial        2
                        development per
                        resident

                        Centrality of Core         3

#6: Achieve a           Mix of public and          3          2
significant mix of      private uses
uses in the
community core and      Mix of public and          1
neighbourhood nodes.    private uses

#7: Increase the        Proportion of              1          2
variety of housing      dwelling units that
types in addition to    are multi-family
single-family type      units
dwellings.
                        Affordable housing         2
                        program

#8: Increase            Proportion of multi-       2          2
accessibility of        family units located
multi-family            within 400m of
development to          commercial
activity centres.       activities

Accessibility

#9: Increase            Proportion of parking      2          3
accessibility for       spaces located to
pedestrians and         side or rear of the
cyclists within         site
activity centres.
                        Proportion of retail       3
                        access points
                        fronting the street

#10: Increase the       Incidence of 4-way         0          1
quality of road         intersections
patterns and
streetscape design      Community through-         0
for pedestrians,        street spacing
cyclists and
transit-users.          Number of traffic          1
                        calming features

                        Proportion of              3
                        residential
                        development with
                        rear lanes

                        Proportion of block        3
                        faces with sidewalks

#11: Increase           Access to open space       3          2
accessibility to
various recreational    Range of outdoor           1
opportunities.          recreational
                        activities

                        Organized community        1
                        involvement in
                        planning &
                        management

#12: Increase the       Presence of                1          1
amount of existing      environmental open
natural systems         space
incorporated
into
the open space plan.

Environmental Responsibility

#13: Increase the       Alternative                2          2
use of alternative      stormwater
methods to              management features
stormwater              integrated in open
management.             space plan

#14: Reduce the         Construction waste         0          1
amount of waste         management
entering landfills
from the                Household and              1
construction process    commercial activity
and homeowners in       waste management
new communities.

#15: Reduce water       Household water            2          2
consumption.            consumption
                        reduction practices

#16: Reduce             Non-renewable energy       0          0
non-renewable energy    reduction practices
consumption.

                                               Composite
Theme        Outcomes   Indicators              Rating
Efficiency

#1: Achieve a           Residential Unit           4
minimum /gross          Density
density of 3 units /
hectare.

#2: Increase the        Locating transit
accessibility,          network and stops in
comfort and safety      ASP
of public
transportation.         Proportion of
                        dwelling units
                        within 400m of
                        transit network

                        Transit Stop Quality

#3: Achieve shared      Number of shared use
use of sites and/or     sites and/or
buildings for           buildings
public
facilities
and services.

#4: Increase home       Number of design
Diversity               elements tailored
occupations.            for home occupations

#5: Increase            Distance of core           8
accessibility to        from regional
mixed-use activity      shopping centre
centres.
                        Ratio of commercial
                        development per
                        resident

                        Centrality of Core

#6: Achieve a           Mix of public and
significant mix of      private uses
uses in the
community core and      Mix of public and
neighbourhood nodes.    private uses

#7: Increase the        Proportion of
variety of housing      dwelling units that
types in addition to    are multi-family
single-family type      units
dwellings.
                        Affordable housing
                        program

#8: Increase            Proportion of multi-
accessibility of        family units located
multi-family            within 400m of
development to          commercial
activity centres.       activities

Accessibility

#9: Increase            Proportion of parking      6
accessibility for       spaces located to
pedestrians and         side or rear of the
cyclists within         site
activity centres.
                        Proportion of retail
                        access points
                        fronting the street

#10: Increase the       Incidence of 4-way
quality of road         intersections
patterns and
streetscape design      Community through-
for pedestrians,        street spacing
cyclists and
transit-users.          Number of traffic
                        calming features

                        Proportion of
                        residential
                        development with
                        rear lanes

                        Proportion of block
                        faces with sidewalks

#11: Increase           Access to open space
accessibility to
various recreational    Range of outdoor
opportunities.          recreational
                        activities

                        Organized community
                        involvement in
                        planning &
                        management

#12: Increase the       Presence of
amount of existing      environmental open
natural systems         space
incorporated
into
the open space plan.

Environmental Responsibility

#13: Increase the       Alternative                5
use of alternative      stormwater
methods to              management features
stormwater              integrated in open
management.             space plan

#14: Reduce the         Construction waste
amount of waste         management
entering landfills
from the                Household and
construction process    commercial activity
and homeowners in       waste management
new communities.

#15: Reduce water       Household water
consumption.            consumption
                        reduction practices

#16: Reduce             Non-renewable energy
non-renewable energy    reduction practices
consumption.
Gale Copyright: Copyright 2009 Gale, Cengage Learning. All rights reserved.