Questions and Discussion Topics
The following is a (partial) list of questions that were considered
during the INT program. We
strongly encourage participants and others to add their own questions
(and answers) as well.
(Participants are also welcome to join the github
project.
To do so, please send us your github username.)
General questions:
- What do Bayesian techniques offer that frequentist
statistics do not?
- Also, what kinds of problems ill-suited for
Bayesian or frequentist approaches?
- What is the modern view of the conflict (if any)
between Bayesian and frequentist statistics?
- What are the best references (e.g., texts or
pedagogical reviews) for introductory Bayesian
statistics and for advanced topics?
- As we compile lists: What are we missing? Are there
more modern versions?
- What are the common or subtle pitfalls that novices to
Bayesian methods fall into?
- What are we likely unaware of on the frontier of
(Bayesian) statistical methods?
- D. Furnstahl: In
interacting with applied mathematicians I've found that
physicists are often using the Numerical Recipes version
of numerical methods, while the state-of-the-art is one
or two generations more advanced. What are the analogs
for statistics?
- A Steiner: I'm currently using Goodman and Weare
(2010)'s affine-invariant MCMC. Is there any way
to do better? I'd like to get more accurate results
with fewer samples. Will Metropolis-Hastings methods
be superior if I have a sufficiently accurate
proposal distribution?
Parameter estimation, model calibration, and
model selection
- What is the difference between model calibration and
parameter estimation?
- How should one do basic regression analysis?
- The old-school theoretical physics way is to do a
least-squares fit with adding penalty terms for
theoretical errors (which could be from the model or
from the numerical method used to calculate the model)
in quadrature to the data errors.
- When the theoretical systematic uncertainty is not
known, one often determines the overall scale by
requiring \( \chi^2/\mathrm{dof} = 1 \)
(Birge factor). How is this done in
Bayesian statistics?
- When should a nuclear model with systematic theory
errors have a statistical distribution of residuals?
- What are appropriate Bayesian priors?
- A. Steiner: How does one deal with the
ambiguity created by heteroscedasticity? E.g.
if we have two types of data points in a
\( \chi^2 \) fit, how do we decide the
relative theoretical uncertainty between
the two types?
- What approximations or techniques are useful for
reducing computational cost?
- What is Approximate Bayesian Computation?
- What method should I use for calculating the evidence
or odds ratios?
- e.g., simulated annealing, nested sampling, analytic
approximations, ...
- What are the pros and cons?
- How do we propagate theoretical uncertainties (e.g.,
from truncations of an expansion or limitations of a physics
model) to calculations of physics observables?
Priors:
- What is Bayesian model checking and how can it be used
to minimize or validate the influence of priors?
- What are other ways to validate priors?
- How does empirical Bayes work and when is it useful
(or dangerous)?
- How do we choose priors for systematic errors in
physics?
- E.g., what general guidance is there?
- What range of priors should I consider?
- How does one choose a "non-informative" prior?
Software:
- What should we know about MCMC sampling algorithms and
software?
- MCMC programs are often a black box to physicists.
- What are recommended implementations for different
types of physics applications?
- Are there parallelized versions?
- What are the pitfalls or "tricks" in using MCMC?
- Should one use more than one algorithm?
- Autocorrelations in MCMC
- A. Steiner: I'm using the method outlined
here
similar to the acor program used in
emcee.
- What are good programs for visualization (e.g., of
projected posteriors)?
- What are the best software options for Python, C++, R, ...
Other topics:
- Inconsistent data (or model)
- Outliers
- Model and uncertainty extrapolation
- Empirical Bayes
- Emulation
-
A Steiner: In nuclear astrophysics, in order to
perform a proper uncertainty quantification, we need two
things: (i) the correlations between masses in the
Atomic Mass Evaluation, and (ii) the correlations
between parameters in popular mass models (e.g. FRDM).
How do we get those?
- A. Steiner: What can be understood from the
analogy between a particle propagator and a conditional
probability distribution? Or does the fact that the
former is defined over complex numers spoil the analogy?