This white paper presents a generalized evaluation framework that can be used for assessing project impacts within the context of transportation-related city projects. In support of this framework, we discuss a selection of metrics and data sources that are needed to evaluate the performance of smart city innovations. We first present a collection of projects and applications from near-term smart city concepts or actual pilot projects underway (i.e., Smart City Challenge, Federal Transit Administration (FTA) Mobility on Demand (MOD) Sandbox, and other pilot projects operating in the regions of Los Angeles, Portland, and San Francisco). These projects are identified and explained in Section 2 of this report. Using these projects as the basis for hypothetical case studies, we present selected metrics that would be necessary to evaluate and monitor the performance of such innovations over time. We then identify the data needs to compute those metrics and further highlight the gaps in known data resources that should be covered to enable their computation. The objective of this effort is to help guide future city planners, policy makers, and practitioners in understanding the design of key metrics 3 and data needs at the outset of a project to better facilitate the establishment of rigorous and thoughtful data collection requirements.

Key findings



Cities do not have to wait for a project to be proposed, funded, and implemented to ask core questions about what data are available to measure common metrics, which are almost always part of an evaluation.



Cities and researchers should also consider the types of data structures that they might need to efficiently address questions. Some data structures need only be established in the form of distributions, while others may require the presence of more detailed and anonymized activity data.



Cities and researchers can do their best to generate key questions of interest at the beginning of an evaluation, which play a major role in guiding decisions about needed supporting data. The sooner these questions are raised (even if they are general), the sooner data needs and structures can be established.



View



View PDF



Behind a hard paywall (paid accounts only)



Behind a soft paywall (limited views allowed until a paid account is required)