Several members of our research team spent years interviewing and spending time with individuals with low incomes who faced significant transportation-related challenges. We saw up close, how problems with transportation shaped people’s everyday lives in profound ways with great great consequences for themselves, their families, and their ability to flourish.
Yet there existed no concept, at that time, that defined what we were seeing.
Existing measurement tools
We looked at the available measurement tools being used to capture the experiences of these individuals, we found many measures were falling short in capturing what we saw on the ground.
For example:
Researchers sometimes looked at whether people had cars to assess whether they could get around adequately. But we found there were many people who had cars who could not get around when they could not pay for gas, insurance, or repairs.
Proximity to bus lines was also not very telling of transportation insecurity. Some people lived close to public transit, but did not take it because they worried about their safety or had small children and found public transportation too cumbersome.
Social networks—friends, family, neighbors, co-workers—were a key way that people got around. No existing measures captured this important, relational dimension of mobility.
Many measures were based on travel behavior (e.g. commute time), yet they missed people who barely got around at all (unmet demand). me.
Therefore, we set out to develop a definition and measure of what we had observed, using these qualitative insights as input. Our hope was that in doing so, we could (1) begin to understand how many people in the U.S. shared the experiences of the small group of people we had come to know (2) better understand the causes and consequences of this condition, and (3), with such knowledge, draw attention to the issue and identify ways to solve it. Defining “transportation insecurity” and developing the Transportation Security Index is the result of these efforts.
Our model: the Food
Security Index
In developing the TSI, we used as our model the Food Security Index (FSI). Developed by the U.S. Department of Agriculture, the FSI is a well-respected, rigorously developed and validated tool used to assess food insecurity in the United States. One of its great strengths is that rather than measuring food insecurity with questions about the “inputs” of food insecurity—like calories consumed or whether people are eating each of the categories of food in the food pyramid—it instead measures food insecurity by asking people whether they have experienced the symptoms of food insecurity as observed in qualitative research. For example: “In the past 12 months, was there a time you skipped a meal because you could not afford food?”
Given the many different modes of transportation people use to get around, and the many destinations where they may go, we thought that developing a measure based on symptoms rather than inputs and destinations was ideal. And, since existing measures were missing the qualitative experience of transportation insecurity as observed in people’s daily lives, we also thought it best to develop new, original questions designed to actually capture the phenomenon of transportation insecurity.
On the surface, potential users of the index may be put off by the fact that the TSI does not include questions that help us understand the inputs of insecurity (i.e an inability to afford gas, no access to public transit). Similarly, they may be put off by the fact that the TSI does not ask people about what destinations they cannot access (i.e. work, the doctor). Yet such design features are actually one of the TSI’s strengths: used in a causal inference framework, people can use the TSI as both an independent and dependent variable enabling the investigation of the many causes and many consequences of this condition. This enables us to understand, for example, whether interventions that alleviate the condition of transportation insecurity—broadly—have a greater impact on improving health and wellbeing than interventions that simply help people get rides to medical-related destinations, alone.
There is a famous saying that “you can’t manage what you don’t measure.” Arguably, one of the most important contributions of the Food Security Index was that it brought attention to the fact that hunger exists in the United States. With attention came further research and policy responses. Our hope is that by developing the TSI, we can do the same for the issue of transportation insecurity as the FSI has done for food insecurity.
Lessons learned by conducting cognitive interviews
In the early stages of creating the TSI, we developed a series of candidate questions for potential inclusion on the TSI and used these questions to conduct 52 cognitive interviews with people in urban, suburban, and rural areas. Cognitive interviews are a technique that enable survey researchers to assess (1) whether respondents understand the question being asked of them, (2) respondents’ confidence in how they have answered a question, and (3) respondents’ ability to recall past information. Our goal was to use findings from the cognitive interviews to refine existing questions, identify burdensome questions that should be dropped from consideration, and identify areas where new questions may need to be developed.
Our findings yielded a number of insights important for understanding transportation insecurity and questions in the TSI. Key among them are:
People do not tie their experiences with transportation insecurity to the financial costs of transportation.
Initially, each of our candidate questions included a reference to the costs of transportation. For example, the question about skipping going places was originally phrased as, “In the past 30 days, how often have you skipped going someplace because you could not afford transportation?” However, cognitive interviews revealed that, for respondents, they did not interpret their experiences with symptoms of transportation insecurity as being tied to the costs of transportation. Instead, they reported skipping trips because, for example, the bus never showed up, or the ride with a friend that they had arranged fell through, or their car needed work and was not drivable. In many of these cases, having more financial resources for transportation would have ameliorated these problems. If respondents could afford a working car of their own, or even had cash for a rideshare, they would not be counting on other, less reliable transportation options. While an outside analyst would reach this conclusion, our respondents did not think in this way: In their eyes, the problem was the unreliable bus, rather than not having money for a car or a rideshare. They thus responded “never” to questions asking if they had skipped trips because they could not afford the transportation they needed.
In light of this finding, we determined that asking respondents to connect the symptom (skipped trips) to a root cause (lack of money) was generating more false negatives than false positives. Indeed, in subsequent rounds of cognitive interviews we conducted on questions without the financial qualifiers, few respondents with adequate financial resources endorsed questions indicating transportation insecurity, while respondents whose financial situations prevented them from getting where they needed to go endorsed the questions we expected them to. Thus, the questions that appear in the index do not have any mention of the costs of transportation as being the sole reason why a symptom was experienced. Cognitive interviews revealed that by removing such language, our revised questions more accurately captured those people who were experiencing transportation insecurity.
Capturing people who have experienced chronic transportation insecurity over long periods of time is difficult.
We interviewed a number of people whose mobility had been severely constrained over long periods of time. When these people were asked questions about whether they had, for example, skipped trips they answered “never.” Yet when we probed further and got a full sense of their transportation situations we learned these people were very transportation insecure. What we came to learn is that not being able to go places easily over long periods of time changed what people thought was possible for themselves such that they don’t think about, in this case, skipping trips because they no longer think about—or try—going places at all. Understanding this, we developed new questions that would capture this group of people specifically.
There are a range of “transportation insecurity” symptoms that are not easily captured by survey questions.
Our first round of candidate questions included questions that tapped into a wide range of symptoms of transportation insecurity. Such questions included those that asked people whether they had been exposed to harsh weather conditions because of problems with transportation, whether they worried about being stopped by the police because of the transportation they were using, whether they encountered dangerous situations because of the transportation they were using (i.e. walking in the street when sidewalks are not available, getting stuck when a car breaks down), among others. Although these were symptoms of transportation insecurity, cognitive interviews revealed that questions that sought to tap into these symptoms were too burdensome to be well comprehended and thus were not generating accurate data. Importantly, though these questions do not appear in the TSI, because the index is designed to identify people experiencing transportation insecurity—not just the symptoms of insecurity itself —the index is in fact capturing people who experience these symptoms.
On shortening the TSI
The original TSI is composed of 16-questions (TSI-16). Understanding that 16 questions is burdensome for respondents and costly to administer, since validating the TSI-16 our team has worked to abbreviate the original index, first to an index composed of 6 questions (TSI-6), then to one composed of 3 (the minimum number of questions needed to measure a latent construct like transportation insecurity). We understand that for many users, a single question TSI would be ideal for its ease and cost effectiveness of use. Accordingly, we have tested a number of single question TSI options. None, however, have performed well when compared to the longer, validated versions of the TSI.
Collaborators
ALEXANDRA K. MURPHY, PhD, sociologist and co-creator of the TSI, is the Associate Director of Social Science Research at Mcity. Murphy is also an Assistant Research Scientist at Poverty Solutions in the Ford School of Public Policy, and a Faculty Associate of the Population Studies Center at the Institute for Social Research, all at the University of Michigan.
ALIX GOULD-WERTH, PhD, sociologist and co-creator of the TSI, is a Research Associate at the University of Michigan’s Population Studies Center. Previously, Gould-Werth served as the Chief Evaluation Officer of the U.S. Department of Labor, was is the former director of family economic security policy at the Washington Center for Equitable Growth, and was a human services researcher at Mathematica Policy Research.
JAMIE GRIFFIN, PhD, survey methodologist and co-creator of the TSI, is an independent survey methodologist and statistical analyst. Previously she worked as an Assistant Research Scientist at the University of Michigan’s Panel Study of Income Dynamics (PSID). Before that worked as a survey researcher at Mathematic Policy Research.
KARINA MCDONALD-LOPEZ, PhD, a sociologist and former graduate student at the University of Michigan, is the Research & Insights at OpenResearch.
NATASHA PILKAUSKAS, PhD, a social demographer and poverty policy researcher, is an Associate Professor of Public Policy at the Gerald R. Ford School of Public Policy and a Research Associate Professor at the Population Studies Center at the Institute of Social Research, both at the University of Michigan.
NICOLE KOVSKI, PhD, a health policy researcher, is currently a Postdoctoral Fellow at the University of Wisconsin-Madison. Prior to this she was a Postdoctoral Fellow at the University of Michigan.
MICHAEL BADER, PhD, a sociologist, is an Associate Professor of Sociology and the Faculty Director of the 21st Century Cities Initiative, both at Johns Hopkins University.
LYDIA WILEDEN, PhD, a Sociologist and former graduate student at the University of Michigan, is an Assistant Research Professor with the Institute for Municipal and Regional Policy at the University of Connecticut’s School of Public Policy.
The study revealed that adults with lower incomes, those with disabilities and people living in urban areas are especially vulnerable to transportation insecurity—and often simultaneously face other hardships.
ALIX GOULD-WERTH
Co-PI Transportation Security Index