There’s a disturbing trend in recent “official” reports on the high speed rail project. Authored by people who either do not understand HSR or who have shown opposition to HSR, these reports take known uncertainties, turn them into controversies, and slap an indefensible but high-profile, overstated conclusion about how their findings suggest huge problems with the HSR project.

We saw this with the Legislative Analyst report, and definitely with the State Auditor’s report.

Now it seems the Berkeley Institute for Transportation Studies’ report on HSR ridership has fallen into the same trap. If you actually read through the entire document, you see that it primarily consists of a long and in-depth debate between the report’s authors and Cambridge Systematics over the finer points of ridership modeling theory. In that sense it is a very typical academic document, where two groups with different views on theories battle it out in technical reports.

Despite the fact that the ITS authors generally commend CS on their modeling, and focus their criticisms on specific aspects of the modeling that they disagree with, the ITS report jumps to its own unsupportable conclusion:

it is not possible to predict whether the proposed high-speed rail system will experience healthy profits or severe revenue shortfalls

The problem is that the ITS report makes the same error they blame CS for making in the study – reaching a conclusion that isn’t supported by the evidence. As with the State Auditor’s report, the CHSRA is also crying foul on this conclusion. CEO Roelof van Ark wrote in a letter included in the report package that “this is an extraordinary statement for which we find no foundation in the Draft Report.”

Further, the report appears to suffer from another common error in official assessments of the HSR project – a belief that the status quo in California will last indefinitely, and that the HSR project’s claims that it will produce significant shifts in traveler behavior should be greeted with skepticism. There is a mountain of evidence that suggests ridership will quickly shift to bullet trains, as in Spain and Taiwan. But the ITS folks don’t seem to consider this as a possibility.

Part of the problem is that the ITS authors are not unbiased observers. The lead author for ITS, Samer Madanat, also authored a extremely flawed and misleading anti-HSR report last fall. In fact, some of the same errors Madanat made in that report crop up here, and are used by the ITS authors to claim that the HSR ridership study is flawed.

Let’s take a closer look at the report. ITS recognizes that the study generally follows accepted principles:

We are, for the most part, satisfied with their responses and agree that their work on this project meets generally accepted standards for travel demand modeling. We are, however, concerned about the impact of some of Cambridge Systematics’ modeling decisions on the reliability of the forecasts based upon these models.

In other words, the ITS authors did not find any evidence that disproves the HSR ridership study. Instead, they took issue with some of the modeling choices made. It’s not matter of them finding clear errors in the ridership study, but of a difference of opinion among experts on how the study should have been conducted. That’s an important distinction that is likely to be lost in the media reporting on the study, and a distinction likely to be ignored by project opponents.

One of the criticisms ITS makes of the study is on how inter-region business travel is handled:

On Page 3-2 of the “model system development” report, the market segments are defined. According to this definition, all business or commuting trips were assigned to peak conditions. This is potentially a serious problem.

Cambridge Systematics’ response to this comment is that they have followed common practice in regional travel demand modeling. While we don’t doubt that this is the case, this fact is irrelevant to the subject, which is an interregional travel demand model. For interregional travel, quite a few business trips are made in the off-peak periods. For example, the California Travel survey shows over 25% of business trips in the off-peak periods.

The effect of this assignment is that a measurement error is added to the level of service attributes for off-peak business travelers. Even if this measurement error is totally random it will lead to biased and inconsistent parameter estimates. This measurement error is in fact not random – off-peak business travelers face better levels of service for auto and worse levels of service for other modes.

In fact, the “serious error” is on the part of ITS. They are assuming here, quite openly and blatantly, that HSR travelers will behave exactly like existing travelers.

There is plenty of reason to believe that’s not how it will work. Business travelers currently travel off-peak for airlines because of the travel time involved, because last-minute fares tend to be cheaper for off-peak travel, and because getting to and from airports is difficult at peak times due to traffic.

HSR would provide qualitatively different choices. If your destination is downtown LA and you’re leaving from the Bay Area, a peak trip becomes quite plausible. If you live in, say, Oakland, you can take BART to Transbay Terminal or SJ Diridon and hop on an HSR train and be in downtown LA by 9 or 10 in the morning, and make your return trip around 4 or 5 PM, getting home around 6 or 7 depending on your final destination.

The ITS authors need to defend their assertion that HSR travelers will behave exactly the same as air travelers. They can’t just assume that’s the case. We need to see evidence it’s the case before we can accept this criticism.

Here’s Cambridge Systematics’ response on the peak travel issue:

The ITS Draft Report claims that the use of peak period service levels for all business/commute travel is “potentially a serious problem” for an interregional travel demand model. We disagree with this conclusion. In fact, this approach is reasonable and acceptable for both regional and interregional modeling.

The reviewers acknowledge that it is standard practice in model development to assign business and commute trips during the peak period. This convention simply reflects the fact that the majority of business travelers and commuters travel during the peak period and face peak-period levels of service. This pattern holds both for regional and interregional travel as the reviewers note. Therefore, using peak-period travel impedances for the business and commute travel market is a reasonable approach for a planning-level model for interregional travel.

The ITS report also repeats a flawed statement made by Madanat in the fall 2009 study he authored. To refresh your memory, here’s what he said about travel time:

Travelers heading to Los Angeles from San Francisco, for example, will consider the time it takes to go to and from airports at each end of the trip, versus the time spent getting to a high-speed rail station. Time spent on the line-haul portion of the trip (actual flying or riding time) is more productive than the access and egress portions. But if access and egress times from HSR stations are as long and onerous as those for air, passengers will save time by driving to an airport instead.

“High-speed rail trades unproductive access and egress time for productive line-haul time,” explained Madanat. That is advantageous to travelers, and they are willing to spend an extra hour or more in line-haul time if egress and access time are diminished. Air travel between some cities in Japan has become nonexistent, thanks to the ease of traveling by high-speed rail.

Here was my reply: I’m sorry, but Madanat is just plain wrong here. The unproductive access and egress time belongs entirely to airplanes, at least in California. He does not appear to have included the ridiculous security theater involved in air travel that adds up to a half hour to travel times. TSA recommends people arrive two hours before a domestic flight. Add in the travel to LA-area airports, none of which have good mass transit connections (whereas LAUS is the hub of the entire Southern California mass transit network), and it is not conceivable to me that HSR is at a disadvantage in terms of travel times. If anything it is likely to have an advantage, or would be comparable, which is all it really needs to be.

Again, we can look at reality to demonstrate the point: if HSR was such a bad deal, why does the Acela have half the market share on the Northeast Corridor? Madanat apparently didn’t speak to actual Acela users:

Barry Ginsberg of Deer Park, N.Y., boarded an Acela train after a meeting in Washington.

“It’s a lot less hassle and more comfortable,” Ginsberg says. “When you figure how much in advance you have to get to the airport, it’s a lot more convenient.”

Note that Ginsberg is a business traveler. Anyhow, Madanat’s team makes a similar claim in the ITS report, this one about headways:

Thus, in the final models, the headway coefficient was constrained by the modelers to be equal to the in-vehicle travel time.

The modelers’ expectation would be reasonable if this was an urban travel demand model, but it is incorrect in the present context. This is because, in inter-city travel where headways are longer, passengers do not arrive randomly at the stations or airports, but rather according to the vehicles’ schedule, which implies that the waiting time cannot be assumed to be equal to half the headway. There is therefore no reason to expect the coefficients on travel time and waiting time to be equal. Indeed, a recent study by Adler, for example, concluded that air travelers place 4-5 times as much weight on travel time as on the time difference between their ideal departure time and when service is available.

It has been argued that if service headways are sufficiently low, high speed rail travelers may indeed use the system in a manner similar to some urban transit riders, arriving at stations randomly and waiting for the next trains. For such travelers, constraining the waiting time coefficient to equal that for travel time may be appropriate. It is clearly inappropriate for air travelers, however.

This is merely an assertion by Madanat that is unproven. He’s basically saying that if headways are low, people won’t just show up at a station and board a train, they’ll schedule around when the HSR trains are operating. But by 2035 there will likely be significant demand for HSR trains, and headways could very well be frequent enough that it would be feasible to essentially drop by the station and grab the next available train.

In fact, HSR travelers can be expected to behave differently than air travelers – because they already are behaving differently. See the above example about Madanat’s line-haul times: an interview with an Acela rider showed Madanat didn’t know what he was talking about in terms of how intercity travelers use HSR and understand its differences from air travel. In fact, there is some indication that people living on the NEC already use Acela in the form of an urban system, dropping in at a station to grab the next available train.

Here’s Cambridge Systematics’ response on this issue:

The reviewers contend that constraining this coefficient led to “bias in the model forecast” because the headways for interregional service are much longer than for urban travel, resulting in different arrival patterns for travelers at HSR stations compared to urban transit systems. We disagree with the assertion that planned headways for California HSR are substantially different than for urban rail service. Accordingly, we believe that the treatment of sensitivity to wait times and headway is reasonable and does not introduce any biases.

The frequency of air and rail service has an impact on the time that travelers expect to wait at a terminal or a station, and on the convenience with which they are able to travel close to their desired departure time. Therefore, two separate components are used to reflect the impact of service frequency on travelers’ choice behavior.
Based on observed data and expert input, average wait times of 55 minutes were established for air travelers and 15 minutes for HSR and rail travelers. Similarly, separate terminal processing times of 18 to 24 minutes were established for air travelers and 3 to 12 minutes for HSR and rail travelers. The sensitivity to wait and terminal time is twice as high as the sensitivity to travel time, consistent with the literature and practice.

Beyond this traditional “wait-time” component, the sensitivity to headways was introduced as an additional component to reflect travelers’ anticipated reaction to schedule convenience.

The proposed HSR service offers a new paradigm of interregional service. The proposed HSR headways are more comparable to the best urban rail services in the U.S. rather than current intercity air and passenger rail services. In this context, the value that was used for the headway coefficient was debated during the model estimation and validation process, and a value consistent with urban rail systems was determined to be appropriate given the planned frequencies of the California HSR system.
During calibration of the original model, there was an overestimation of air trips in markets with low frequency of air service and an underestimation of air trips in markets with high- frequency air service.
The merits of different potential interpretations and values for the headway coefficient were documented and discussed during the peer review process. The con- straint on the coefficient was deemed to be a more reasonable approach than introducing higher alternative-specific constants that would have a greater impact on model sensitivity.

Finally, it also should be noted that the short headways and corresponding short wait times account for a small portion of the interregional air and high-speed rail travel times in this study. As a result, the impact of using different assumptions on coefficient values will be correspondingly small.

In other words, Madanat is in error in assuming that HSR travelers will behave like air travelers. I agree with Cambridge Systematics on this point.

Another claim ITS made was about airport choice:

The modeling approach does not explicitly consider the choice of airport or station. Given the trip origin and destination, the model determines the airport or stations that would be used for access and egress “taking into account the level of service of the access and egress modes and the frequency of service at each station and airport ….” (See Appendix A). Such “all-or-nothing” assignment is behaviorally unrealistic since, depending on their desired travel schedule, access/egress modes and other factors, travelers may choose different stations.

The failure to consider station choice is important when comparing ridership for the Altamont and Pacheco corridors. In the Altamont alternative, trips between South Bay locations and Southern California must be exclusively assigned to either a station on the San Jose or the San Francisco lines, and the frequency of service will be accordingly reduced. In reality, travelers in these markets could choose either line, depending on which is most convenient for their particular travel schedule. Failure to consider this possibility means that the inconvenience of the split schedule assumed for the Altamont alternative is exaggerated. The problem is compounded by the inflated value of the headway coefficient, as pointed out earlier. Correcting this deficiency would reduce, although probably not eliminate, the projected ridership difference between the two alignments.

As you can tell, this is intended to sow fear, uncertainty and doubt about the Pacheco choice. But Cambridge Systematics reacted very strongly against this point:

The ITS Draft Report contends that the adopted process for assigning travelers to individual airports and rail stations is “behaviorally unrealistic”. The reviewers contend that the absence of a more elaborate modeling structure for airport/station choice “has a substantive impact on the comparisons of ridership for the Altamont and Pacheco corridors.” We disagree with both points. Further, we believe that a more elaborate “airport/station choice model” is not critical for meeting the objectives of the model development and application work that has been conducted, nor for accurately distinguishing the ridership and revenue potential between the Altamont and Pacheco corridors.

The model currently uses a network-based method that assigns an airport or rail station to all travelers originating from a specific zone. The rule that is used is based on evaluating paths from each origin zone to alternative airports and rail stations. The attractiveness of each path reflects the access modes that are available, the level of access service they offer, and the frequency of air and rail service available at each airport and rail station.

An airport/station-choice model would allow the allocation of a proportion of travelers to different nearby airports and rail stations. However, such an approach would have, at most, a minimal effect on Altamont’s ridership, and then only for a few zones in the study area.

Cambridge Systematics then goes on to provide an example from the SF Bay Area, showing that the only part of the region where there’s any real uncertainty over how riders would behave is in the Menlo Park to Sunnyvale region. The full explanation is on pages 51-52 of the report PDF, and is worth reading to see why the ITS report is flawed to make this criticism.

There are other points where ITS and CS disagree on the details. And that’s typical of an academic discussion where smart people have different interpretations of the same data and how it should be analyzed. God knows I’ve sat through my share of these discussions in my graduate studies in US history, where there are very strong debates and disputes over how to use historical sources.

So it’s important to understand what this report means – and what it does not mean. It does not say that the HSR ridership numbers are proven to be incorrect. What ITS is saying is that their view of proper modeling indicates that the numbers might or might not be right. But CS has their own view on this, backed up by other observers and econometrists, who say the numbers are valid. This is not a case of someone disproving something, it’s a case of different views on how modeling should be done.

Of course, the real question here is the political reaction. We know the media is going to report on this the same way that they reported on the State Auditor’s report, despite its flaws – as a sign that the HSR project is in trouble. The media, unfortunately, is used to framing government as flawed and error-prone. They’re not so used to questioning the assumptions of those that make such criticisms.

It’s always been my view that the state legislature will order new ridership studies. That could be a blessing for HSR backers, since the current ridership study uses surveys conducted in 2005. A survey conducted in 2010 or 2011, after the rise in oil prices and the new, higher level of passenger rail ridership around the state, would probably show higher projected ridership.

Whether that means a delay for the HSR project or not is an open question. Surely we can expect the HSR opponents to demand new EIRs to be conducted. But that doesn’t seem necessary, since the EIRs were certified for a high level of ridership. A lower level means less environmental impact within the same ROW.

Ultimately, this report shows the desperate need to build up knowledge of how HSR works around the world – and across the country – in our state institutions, from the LAO to the State Auditor to our colleges and universities. Most transportation studies experts are too deeply locked into a view that suggests the 20th century will last forever. As we know, that’s not the case. It’s time for these experts to catch up, and stop writing flawed reports.