Comparison of empirical Bayes and full Bayes approaches for before-after road safety evaluations

The empirical Bayes (EB) approach has now gained wide acceptance among researchers as the much preferred one for the before-after evaluation of road safety treatments. In this approach, the before period crash experience at treated sites is used in conjunction with a crash prediction model for untreated reference sites to estimate the expected number of crashes that would have occurred without treatment. This estimate is compared to the count of crashes observed after treatment to evaluate the effect of the treatment. This procedure accounts for regression-to-the-mean effects that result from the natural tendency to select for treatment those sites with high observed crash frequencies. Of late, a fully Bayesian (FB) approach has been suggested as a useful, though complex alternative to the empirical Bayes approach in that it is believed to require less data for untreated reference sites, it better accounts for uncertainty in data used, and it provides more detailed causal inferences and more flexibility in selecting crash count distributions. This paper adds to the literature on comparing the two Bayesian approaches through empirical applications. The main application is an evaluation of the conversion of road segments from a four-lane to a three-lane cross-section with two-way left-turn lanes (also known as road diets). For completeness, the paper also summarizes the results of an earlier application pertaining to the evaluation of conversion of rural intersections from unsignalized to signalized control. For both applications, the estimated safety effects from the two approaches are comparable.     

The importance of confounding in observational before-and-after studies of road safety measures

This paper discusses the importance of confounding in observational before-and-after studies of road safety measures. The importance of the approach taken to controlling for confounding factors is shown by means of examples. It is shown that the size of the effect on accidents attributed to a road safety measure can be profoundly affected by which confounding factors are controlled for in an evaluation study, and the way this is done. Simple before-and-after studies, not controlling for any confounding factors should never be trusted and are likely to overstate the effects of road safety measures.     

Analytic choices in road safety evaluation: Exploring second-best approaches

Conducting rigorous before-and-after studies is essential for improving knowledge regarding the effects of road safety measures. However, state-of-the-art approaches like the empirical Bayes or fully Bayesian techniques cannot always be applied, as the data required by these approaches may be missing or unreliable. The choice facing researchers in such a situation is to either apply “second-best” approaches or abstain from doing an evaluation study. An objection to applying second-best approaches is that these approaches do not control as well for confounding factors as state-of-the-art approaches. This paper explores the implications of choice of study design by examining how the findings of several evaluation studies made in Norway depend on choices made with respect to:

1.

Using the empirical Bayes approach versus using simpler approaches;     

Association between setting quantified road safety targets and road fatality reduction

Since the 1980s, many countries have set quantified road safety targets as part of their comprehensive road safety strategies. This paper studies the association between the setting of quantified road safety targets and road fatality reduction over the past two decades. Both aggregate and disaggregate levels of hypothesis tests are performed on data from 14 countries that set road safety targets during the period 1981-1999. A before-and-after analysis of the treatment and comparison groups is conducted, and the results show that the majority of countries with quantified road safety targets experienced a reduction in road fatalities in this period. Overall, the establishment of quantified road safety targets is found to have an appreciable association with an improvement in road safety.     

Empirical Bayes before-after safety studies: lessons learned from two decades of experience and future directions

The empirical Bayes (EB) methodology has been applied for over 20 years now in conducting statistically defendable before-after studies of the safety effect of treatments applied to roadway sites. The appeal of the methodology is that it corrects for regression to the mean and traffic volume and other changes not due to the measure. There is, therefore, a natural tendency to put a stamp of approval on any study that uses this methodology, and to assume that the results can then be used in specifying crash modification factors for use in developing treatments for hazardous locations, or in designing new roads using tools such as the interactive highway safety design model (IHSDM). At the other extreme are skeptics who suggest that the increased sophistication and data needs of the EB methodology are not worth the effort since alternative, less complex methods can produce equally valid results. The primary objective of this paper is to capitalize on experience gained from two decades of conducting EB studies around the world to illustrate that the EB methodology, if properly undertaken, produces results that could be substantially different and less biased than those from more conventional types of studies. A secondary objective is to emphasize that caution is needed in assessing the validity of studies undertaken with the EB methodology and in using these results for providing crash modification factors. To this end, a number of issues that are critical to the proper conduct and interpretation of EB evaluations are raised and illustrated based on lessons learned from recent experience with these studies. These include: amalgamating the effects on different crash types; the specification of the reference/comparison groups; and accounting for traffic volume changes. Current and future directions, including the improvements offered by a full Bayes approach, are discussed.     

An update on the association between setting quantified road safety targets and road fatality reduction

This note is intended to rectify estimates provided in a previous paper (Wong et al., 2006) of the short-term effectiveness in terms of road fatality reduction of the setting of quantified road safety targets using data from 14 OECD countries during the period 1980–1999. This work is important in measuring the association between target setting and road safety improvement, because such targets are intended to serve as a useful tool to motivate timely road safety measures by the road authorities and others. The estimates to be rectified were based on before-and-after analysis using a comparison group of countries for each country that had set a target. This note first provides a correction to the qualification test for the inclusion of a country in any particular comparison group. It then presents the numerical effects of this correction on the estimates of the effectiveness of setting quantified road safety targets, both in individual countries and across the whole group of countries that set targets in the relevant period. Finally, impacts on the findings of the previous paper are discussed, with the conclusion that the changes in those of the numerical estimates that are affected do not alter the main message of the paper.     

Investigating the effects of the fixed and varying dispersion parameters of Poisson-gamma models on empirical Bayes estimates

Traditionally, transportation safety analysts have used the empirical Bayes (EB) method to improve the estimate of the long-term mean of individual sites; to correct for the regression-to-the-mean (RTM) bias in before-after studies; and to identify hotspots or high risk locations. The EB method combines two different sources of information: (1) the expected number of crashes estimated via crash prediction models, and (2) the observed number of crashes at individual sites. Crash prediction models have traditionally been estimated using a negative binomial (NB) (or Poisson-gamma) modeling framework due to the over-dispersion commonly found in crash data. A weight factor is used to assign the relative influence of each source of information on the EB estimate. This factor is estimated using the mean and variance functions of the NB model. With recent trends that illustrated the dispersion parameter to be dependent upon the covariates of NB models, especially for traffic flow-only models, as well as varying as a function of different time-periods, there is a need to determine how these models may affect EB estimates. The objectives of this study are to examine how commonly used functional forms as well as fixed and time-varying dispersion parameters affect the EB estimates. To accomplish the study objectives, several traffic flow-only crash prediction models were estimated using a sample of rural three-legged intersections located in California. Two types of aggregated and time-specific models were produced: (1) the traditional NB model with a fixed dispersion parameter and (2) the generalized NB model (GNB) with a time-varying dispersion parameter, which is also dependent upon the covariates of the model. Several statistical methods were used to compare the fitting performance of the various functional forms. The results of the study show that the selection of the functional form of NB models has an important effect on EB estimates both in terms of estimated values, weight factors, and dispersion parameters. Time-specific models with a varying dispersion parameter provide better statistical performance in terms of goodness-of-fit (GOF) than aggregated multi-year models. Furthermore, the identification of hazardous sites, using the EB method, can be significantly affected when a GNB model with a time-varying dispersion parameter is used. Thus, erroneously selecting a functional form may lead to select the wrong sites for treatment. The study concludes that transportation safety analysts should not automatically use an existing functional form for modeling motor vehicle crashes without conducting rigorous analyses to estimate the most appropriate functional form linking crashes with traffic flow.     

The range of replications technique for assessing the external validity of road safety evaluation studies

This paper introduces a simple statistical technique that can be used to assess the external validity of road safety evaluation studies. External validity refers to the possibility of generalising the results of research to other contexts than those in which it was made. There are several aspects of external validity. Two aspects that are often of interest concern the applicability of the results of road safety evaluation studies across countries and time. Can the results of studies made in one or more countries be applied in countries where studies have not been made? Can the results of studies made many years ago still be applied? The technique introduced in this paper is designed to provide support in answering these questions. The technique evaluates the stability of research results in time and space. The technique is based on cumulative meta-analysis and produces statistics that show the consistency of study results in time and space (across countries). The range of replications denotes the span of time and countries in which studies have been made. The idea is that if the results of studies are stable throughout the range of replications, one may have greater confidence in their external validity than if the results of research vary in time and between countries. The technique is illustrated by means of numerical examples.     

Prediction of road accidents: A Bayesian hierarchical approach

In this paper a novel methodology for the prediction of the occurrence of road accidents is presented. The methodology utilizes a combination of three statistical methods: (1) gamma-updating of the occurrence rates of injury accidents and injured road users, (2) hierarchical multivariate Poisson-lognormal regression analysis taking into account correlations amongst multiple dependent model response variables and effects of discrete accident count data e.g. over-dispersion, and (3) Bayesian inference algorithms, which are applied by means of data mining techniques supported by Bayesian Probabilistic Networks in order to represent non-linearity between risk indicating and model response variables, as well as different types of uncertainties which might be present in the development of the specific models.     

Assessing the validity of road safety evaluation studies by analysing causal chains

This paper discusses how the validity of road safety evaluation studies can be assessed by analysing causal chains. A causal chain denotes the path through which a road safety measure influences the number of accidents. Two cases are examined. One involves chemical de-icing of roads (salting). The intended causal chain of this measure is: spread of salt --> removal of snow and ice from the road surface --> improved friction --> shorter stopping distance --> fewer accidents. A Norwegian study that evaluated the effects of salting on accident rate provides information that describes this causal chain. This information indicates that the study overestimated the effect of salting on accident rate, and suggests that this estimate is influenced by confounding variables the study did not control for. The other case involves a traffic club for children. The intended causal chain in this study was: join the club --> improve knowledge --> improve behaviour --> reduce accident rate. In this case, results are rather messy, which suggests that the observed difference in accident rate between members and non-members of the traffic club is not primarily attributable to membership in the club. The two cases show that by analysing causal chains, one may uncover confounding factors that were not adequately controlled in a study. Lack of control for confounding factors remains the most serious threat to the validity of road safety evaluation studies.     

Mixing Bayes and empirical Bayes inference to anticipate the realization of engineering concerns about variant system designs

Mixing Bayes and Empirical Bayes inference provides reliability estimates for variant system designs by using relevant failure data - observed and anticipated - about engineering changes arising due to modification and innovation. A coherent inference framework is proposed to predict the realization of engineering concerns during product development so that informed decisions can be made about the system design and the analysis conducted to prove reliability. The proposed method involves combining subjective prior distributions for the number of engineering concerns with empirical priors for the non-parametric distribution of time to realize these concerns in such a way that we can cross-tabulate classes of concerns to failure events within time partitions at an appropriate level of granularity. To support efficient implementation, a computationally convenient hypergeometric approximation is developed for the counting distributions appropriate to our underlying stochastic model. The accuracy of our approximation over first-order alternatives is examined, and demonstrated, through an evaluation experiment. An industrial application illustrates model implementation and shows how estimates can be updated using information arising during development test and analysis.     

Models relating traffic safety with road environment and traffic flows on arterial roads in Addis Ababa

This paper presents the study carried out to develop accident predictive models based on the data collected on arterial roads in Addis Ababa. Poisson and negative binomial regression methods were used to relate the discrete accident data with the road and traffic flow explanatory variables. Significant accident predictive models were found with a number of significant explanatory variables. The results show that the existing inadequate road infrastructure and poor road traffic operations are the potential contributors of this ever-growing challenge of the road transport in Addis Ababa. The results also indicate that improvements in roadway width, pedestrian facilities, and access management are effective in reducing road traffic accidents.     

The contribution of on-road studies of road user behaviour to improving road safety

For over 40 years transport safety researchers have been using methods of vehicle instrumentation to gain greater insights into the factors that contribute to road user crash risk and the associated crash factors. In the previous decade in particular the widespread availability of lower cost and more advanced methods of vehicle instrumentation and recording technologies are supporting the increasing number of on-road research studies worldwide. The design of these studies ranges from multi-method studies using instrumented test vehicles and defined driving routes, to field operational tests, through to much larger and more naturalistic studies. It is timely to assess the utility of these methods for studying the influences of driver characteristics and states, the design and operation of the road system, and the influences of in-vehicle technologies on behaviour and safety for various road user groups. This special issue considers the extent to which on-road studies using vehicle instrumentation have been used to advance knowledge across these areas of road safety research. The papers included in this issue illustrate how research using instrumented test vehicles continues to generate new knowledge, and how the larger scale United States and European naturalistic and field operational test studies are providing a wealth of data about road user behaviour in real traffic. This is balanced with a number of studies that present methodological developments in data collection and analysis methods that, while promising, need further validation. The use of on-road methods to accurately describe the behaviours occurring in everyday real-world conditions, to quantify risks for safety critical events, and an improved understanding of the factors that contribute to risk, clearly has huge potential to promote further road trauma reductions.     

Costs and benefits to Sweden of Swedish road safety research

This paper summarises a study designed to answer the following question: what are the benefits to Swedish society of road safety research in Sweden funded by the Swedish Transport Research Council and the programme for vehicle safety research during the period 1971-2004? The paper starts by discussing whether research can answer this question at all and explains why a well-controlled study was not feasible. A case study approach was selected, and five major research projects were examined in detail for the purpose of trying to estimate their effects on road safety. Estimates of safety effects were developed for four of the projects, indicating that road safety measures that were at least to some extent based on the findings of the research projects have made major contributions to reducing the number of road accident fatalities in Sweden. The estimates are not analytically rigorous and should be treated as qualified guesses only. Causal inferences are not possible. Nevertheless, if taken at face value, they show that the benefits to society of road safety research are large and outweigh by a wide margin the costs of the research, and of the road safety measures developed as a result of research. Thus, even if the estimated safety benefits exaggerate the true effects, the benefits of applied road safety research are likely to be greater than the costs of conducting this research and implementing road safety measures developed by research.     

The impact of red light cameras on safety in Arizona

Red light cameras (RLCs) have been used in a number of US cities to yield a demonstrable reduction in red light violations; however, evaluating their impact on safety (crashes) has been relatively more difficult. Accurately estimating the safety impacts of RLCs is challenging for several reasons. First, many safety related factors are uncontrolled and/or confounded during the periods of observation. Second, “spillover” effects caused by drivers reacting to non-RLC equipped intersections and approaches can make the selection of comparison sites difficult. Third, sites selected for RLC installation may not be selected randomly, and as a result may suffer from the regression to the mean bias. Finally, crash severity and resulting costs need to be considered in order to fully understand the safety impacts of RLCs.

Recognizing these challenges, a study was conducted to estimate the safety impacts of RLCs on traffic crashes at signalized intersections in the cities of Phoenix and Scottsdale, Arizona. Twenty-four RLC equipped intersections in both cities are examined in detail and conclusions are drawn. Four different evaluation methodologies were employed to cope with the technical challenges described in this paper and to assess the sensitivity of results based on analytical assumptions. The evaluation results indicated that both Phoenix and Scottsdale are operating cost-effective installations of RLCs: however, the variability in RLC effectiveness within jurisdictions is larger in Phoenix. Consistent with findings in other regions, angle and left-turn crashes are reduced in general, while rear-end crashes tend to increase as a result of RLCs.     

Safety effects of section control - An empirical Bayes evaluation

The safety effects of section control were investigated at 14 sites in Norway. A before–after study was conducted with the empirical Bayes method in order to control for regression to the mean (RTM). Effects of trend, volumes, speed limit changes and speed cameras at some of the sites in the before period are controlled for as well. For injury crashes a non-significant reduction by 12% was found. The number of killed or severely injured was found to be significantly reduced by 49% at the section control sites. The results indicate that the crash reductions in tunnels (most of which are undersea tunnels with section control on steep downhill segments) are at least of the same magnitude as on open roads. The results are consistent with findings from speed measurements, although the crash reductions are larger than one would expect as a result from the speed reductions. Downstream of the section control sites (up to 3 km in each direction) injury crashes were found to be significantly reduced by 46%. The number of KSI downstream of the section control sites is too small for drawing any conclusions. It is concluded that section control is effective in reducing both speed and crashes, especially serious crashes, and that spillover effects (crash reductions at non-enforcement sites) are more likely to occur than crash migration. The size of the effects that were found should be interpreted with caution because of the relatively short after periods for some of the sites and the sensitivity of the results to the outcomes of individual crashes.     

Validation of a Full Bayes methodology for observational before-after road safety studies and application to evaluation of rural signal conversions

The objective of the study on which the paper is based was to explore the application of fully Bayesian methods for before-after road safety studies. Several variations of the methodology were evaluated with a simulated dataset in which hypothetical treatments with no safety effect were randomly assigned to high accident locations to mimic the common site selection process in road jurisdictions. It was confirmed that the fully Bayesian method by estimating no safety effect can account for the regression-to-the-mean that results from this biased site selection process. The fully Bayesian method was then applied to California rural intersection data to evaluate the safety effect of conversion from stop to signalized control. The results were then compared with those from the empirical Bayesian method, currently the accepted approach for conducting unbiased before-after evaluations. This comparison was generally favorable in that FB can provide similar results as EB.     

The influence of the infrastructure characteristics in urban road accidents occurrence

This paper summarizes the result of a study regarding the creation of tools that can be used in intervention methods in the planning and management of urban road networks in Portugal.     

Uncertainty assessment of the road safety index

Composite indicators aggregate domain-specific information in one index, on the basis of which countries can be assigned a relative ranking. Recently, the road safety community got convinced of the policy supporting role of indicators in terms of benchmarking, target setting and selection of measures. However, combining the information of a set of relevant risk indicators in one index presenting the whole picture turns out to be very challenging. In particular, the rank of a country can be largely influenced by the methodological choices made during the composite indicator building process. Decisions concerning the selection of indicators, the normalisation of the indicator values, the weighting of indicators and the way of aggregating can influence the final ranking. In this research, it is shown that the road safety ranking of countries differs significantly according to the selected weighting method, the expert choice and the set of indicators. From these three input factors, the selection of the set of indicators is most influential. A well considered selection of indicators will therefore establish the largest reduction in ranking uncertainty. With a set of appropriate indicators, the proposed framework reveals the major sources of uncertainty in the creation of a composite road safety indicator.