From data to probability densities without histograms

When one deals with data drawn from continuous variables, a histogram is often inadequate to display their probability density. It deals inefficiently with statistical noise, and binsizes are free parameters. In contrast to that, the empirical cumulative distribution function (obtained after sorting the data) is parameter free. But it is a step function, so that its differentiation does not give a smooth probability density. Based on Fourier series expansion and Kolmogorov tests, we introduce a simple method, which overcomes this problem. Error bars on the estimated probability density are calculated using a jackknife method. We give several examples and provide computer code reproducing them. You may want to look at the corresponding figures 4 to 9 first.

Program summary

Program title: cdf_to_pdCatalogue identifier: AEBC_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AEBC_v1_0.htmlProgram obtainable from: CPC Program Library, Queen's University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 2758No. of bytes in distributed program, including test data, etc.: 18 594Distribution format: tar.gzProgramming language: Fortran 77Computer: Any capable of compiling and executing Fortran codeOperating system: Any capable of compiling and executing Fortran codeClassification: 4.14, 9Nature of problem: When one deals with data drawn from continuous variables, a histogram is often inadequate to display the probability density. It deals inefficiently with statistical noise, and binsizes are free parameters. In contrast to that, the empirical cumulative distribution function (obtained after sorting the data) is parameter free. But it is a step function, so that its differentiation does not give a smooth probability density.Solution method: Based on Fourier series expansion and Kolmogorov tests, we introduce a simple method, which overcomes this problem. Error bars on the estimated probability density are calculated using a jackknife method. Several examples are included in the distribution file.Running time: The test runs provided take only a few seconds to execute.