The Chennai Institute of Mathematical Sciences held its 50th birthday party in Pondicherry, India, 4-8 January 2013, organized by Ronojoy Adhikhari and Rahul Siddharthan. This was a lively meeting attended by statisticians, physicists, biologists, climate scientists, computer scientists and others, united by an interest in applying Bayes Theorem in solving all kinds of scientific problems — and divided, as usual, by the many possible interpretations of Bayes Theorem. I look forward to the day(?) when Bayesians can find consensus, not over what in particular probabilities may be, but over the fact that they may be diverse things. Acknowledging objectivity needn't come at the price of abandoning subjectivity (see, e.g., David Lewis's "A subjectivist's guide to objective chance" in R. Jeffrey (ed) Studies in Inductive Logic and Probability, vol III, 1980).

In any case, there were many interesting presentations and discussions, including, among many others: Devinder Sivia (Oxford) presenting Bayesian methods of data analysis, Rajesh Rao (Washington) describing recent Bayesian models of brain function, Erik van Nimwegen (Basel) using Bayesian networks to predict protein contacts, Balaji Rajagopalan (Colorado) analysing climate change with extreme value models. I gave talks on Bayesian network modeling, causal discovery of Bayesian nets, and discretization. Most of these were filmed and will be made available on Youtube. When that happens, I'll update this post.

The Science Slam final will be held in few days in Cologne. The English-speaking world needs something like this as well! (see the list of countries at wikipedia.) What is the Science Slam?

The Science Slam offers students and researchers an opportunity to present their research projects in an entertaining 10-minute show on stage.

In contrast to a Poetry Slam any sort of aid is allowed: Power point, props or live experiments are welcome. When the Science Slam ends, the audience decides which Slammer goes home the winner.

The aim of Science Slam is to encourage scientists to present their work in a clear and easily understandable way. At the same time, the entertaining lectures for non-specialist audiences give people the chance to get infected by the enthusiasm of the slammers for their projects. Although research is the focus here, the scientific value of the lecture plays a subordinate role. Rather, the emphasis is on communication, and on showing the public what young scientists are devoting their energies to.

Bad science comes in a number of varieties, at least including the following:

Sloppy science. This might include poor experimental design, poor measurements, slovenly reasoning, insufficient power in one's tests, failure to blind experimenters or subjects, etc. Presumably, the intentions are right, but the execution is wrong.

Pseudo-science. This is fake science. The fakery may be intentional or unintentional. For example, cultists may intentionally generate some large-scale fantasy, while their followers unsuspectingly take it seriously. If the pseudo-scientific methods employed have the look and feel of science, then this is due to simulation or accident, and not due to the proper employment of scientific methods. For Karl Popper, demarcating real from pseudo-science was a kind of mission. He proposed a "falsificationist" criterion: that theories which were (or could be) protected from any possible contrary evidence were non-scientific. Unfortunately, this could never quite be made to work; there are no logical limits to what can be defended, or not, since, as Quine put it, all of our ideas are tied together in a "Web of Belief" (Quine and Ullian, 1978). Still, Popper was certainly on to something: those, such as climate change deniers, who spin excuses and rationalizations no matter what the evidence may be good propagandists, but they are not good scientists.

Cheats. This is also fake science, but most likely not with a view to promoting a false story about the world, but instead a false story about the researcher.

Ben Goldacre's book Bad Science (Fourth Estate, 2009) treats miscreants and violators of scientific method primarily in the first two categories. Being a journalist (and MD) he, perhaps naturally, focuses largely on the aberrations and violations perpetrated by journalists. On his account, they've done quite a lot of damage. For example, around 2005 there were repeated scandals in the UK concerning rampant MRSA in UK hospitals, but the findings were all traceable to a single lab, "the lab that always gives positive results". Apparently, journalists responded to that description by anticipatory salivation, rather than anxious palpitation. It's a ludicrous, and sad, story.

For newcomers to scientific or medical research, Goldacre's book is an entertaining, accessible introduction to a host of issues you will need to know about: experimental design, bias in statistics, cheating by pharmaceutical companies in research and in advertising, the silliness of homeopathy, how we fool ourselves into believing what we want to believe and what measures can be taken to minimize our own foolishness.

For those well versed in these kinds of issues, the book, while a good source of anecdotes, is just a little disappointing. It's important to provide accessible accounts of science and method, but Goldacre goes just a bit far in dumbing things down, in my opinion. Popular science writers should not be assuming that their readers are idiots. He proposes as his motto: "Things are a little more complicated than that". Indeed, they are. Still, on the whole, this is a good and positive contribution to the public understanding of science.

(17 Nov 2012) I think perhaps I was a bit too negative at the end of the note above. Goldacre's book can be seen as an extended plea for a more evidence-oriented treatment of science journalism and, in particular, as a protest against the view that science is just too complicated for ordinary folk to understand — a view which he rightly condemns for promoting appeals to authority for arbitrating scientific disputes, rather than appeals to evidence. The result is a serious dumbing down of public policy debates, including a tendency to portray all sides of a scientific dispute as having equal support, because all sides can call upon any number of "experts". This message certainly needs to be spread. The quality of public debate about topics that concern science is very poor indeed.

The proceedings of our recent workshop on applying Bayesian networks to real-world problems will be coming out soon (a preliminary version is available for on-line viewing here). The workshop was co-located with the 28th Conference on Uncertainty in Artificial Intelligence (UAI 2012), on Catalina Island, California on August 18, 2012.

Bayesian networks are by now a well-established technology for reasoning under uncertainty, supported by numerous mature academic and commercial software tools. They are being applied in many domains, for example, environmental and ecological modelling, bioinformatics, medical decision support, many types of engineering, robotics, military, financial and economic modelling, education, forensics, emergency response, and surveillance. This workshop solicited submissions describing real-world applications, whether as stand-alone BNs or BNs embedded in larger software systems. We suggested authors address the practical issues involved in developing the applications, such as knowledge engineering methodologies, elicitation techniques, defining and meeting client needs, validation processes and integration methods, as well as software tools to support these activities.

The resultant workshop included presentations on a good variety of applications, including oil drilling, managing river catchments, analysing HIV mutations, gang violence, and understanding students' reading comprehension. Many of the applications responded to a workshop theme by illustrating models of temporal reasoning, using dynamic Bayesian networks (DBNs), continuous-time Bayesian networks (CTBNs) and partially observable MDPs (POMDPs).

The workshop demonstrated an active and growing community of modellers taking what were until recently research techniques for Bayesian modelling and applying them to solving a diverse range of important problems in the wider community.

The ABNMS will meet in Wollongong in the last week of November 2012. For details see their conference website. Submissions of abstracts are due by the end of August.

A Bayesian evaluation of the evidence, old and new, for the existence of a God of the sort the Abrahamic religions postulate reveals that there really isn't any: on the contrary, such evidence as can be found is very strongly against such a being. In my new book Objecting to God (Cambridge, 2011) I employ Bayesian probability to counter many 'pro-God' arguments in the recent literature, particularly those bits of it discussing the alleged extreme improbability of fine-tuning and the development of complex life-forms. In particular, the "Anthropic Argument" for the existence of a God is no more compelling than its underlying "Anthropic Principle", which I show to be fallacious.

Not only do the Abrahamic religions lack any credible evidential foundation, but their influence is largely malign, embodying codes of ethics both primitive and repressive. In my book I argue on the contrary for a humanitarian ethics based on a more modern version of Aristotle's notion of eudaimonia. Another novel feature of my book is its drawing a parallel between the logico-mathematical paradoxes of the late nineteenth and early twentieth centuries and the ancient theological paradoxes arising from the notion of an omniscient, omnipotent, perfectly good deity. I show how Tarski's celebrated theorem(s) on the indefinability of truth refutes the postulate of omniscience. I also present a critical discussion of Richard Dawkins's well-known attempt to prove that the hypothesis of God is itself extremely improbable.

Colin Howson is a Professor of Philosophy at the University of Toronto and Emeritus Professor in the Philosophy Department at the London School of Economics. For a more detailed and careful presentation of these ideas read his book Objecting to God (Cambridge University Press, 2011).

This single day workshop is an excellent forum for presenting and hearing about real-world applications of Bayesian networks. It follows the 28th Int. Conference on Uncertainty in AI, the premier conference for presentation of research on Bayesian technology (Aug 15-17th). The call for papers is now out, with submission deadline May 5th (with a week’s extension very likely!).

The aim of the workshop is to foster discussion and interchange about novel contributions that can speak to both the academic and the larger research community. Accordingly, we seek submissions also from practitioners and tool developers as well as researchers. We welcome submissions describing real world applications, whether as stand-alone BNs or where the BNs are embedded in a larger software system. We encourage authors to address the practical issues involved in developing real-world applications, such as knowledge engineering methodologies, elicitation techniques, defining and meeting client needs, validation processes and integration methods, as well as software tools, including visualization and user interaction techniques to support these activities.

We particularly encourage the submission of papers that address the workshop theme of temporal modeling. Recently communities building dynamic Bayes networks (DBNs) and partially observable MDPs (POMDPs) are coming to realize that they are applying their methods to identical applications. Similarly POMDPs and other probabilistic methods are now established in the field of Automated Planning. Stochastic process models such as continuous time Bayes networks (CTBNs) should also be considered as part of this trend. Adaptive and on-line learning models also fit into this focus.

Minimum message length (MML) inference is a computational implementation of Bayesian inference, an information-theoretic means of finding high posterior probability hypotheses, devised by Chris Wallace and David Boulton around 1968 (see Wallace's history of MML). MML seeks to minimise a two-part message length , where encodes a hypothesis and some relevant evidence (data). So long as coding follows the principles developed by Claude Shannon, so that the codes enforce the efficiency principle that message lengths , then minimising the MML message length is trivially equivalent to maximising posterior probability:

Since during this sequence we have multiplied by -1, we have also switched from minimising a message length to maximising a probability. And at the end, since and differ only by a positive multiple (see Bayes' theorem), maximising one is the same as maximising the other.

This foundation for minimum message length inference is quite elementary, so the fact that it was not in use before 1968 may be a little surprising. It is probably partly due to limits on computational capacity inhibiting Bayesian statistics and the related dominance of frequentist methods. That there remains any debate about computational Bayesianism, however, is even more surprising.

The application of Bayes' theorem is straightforward for discrete (multinomial) variables governed by a probability function. But consider a problem in which one or more variables are continuous, rather than discrete. Can Bayes' theorem apply?

Any continuous attribute (variable) can only be measured to some limited accuracy, .

So, every datum that is possible under a model (theory, hypothesis) has a probability that is strictly greater than zero, and not just a probability density.

Any continuous parameter of a model can only be inferred (estimated) to some limited precision.

So, every parameter estimate that is possible under a prior has a probability that is strictly greater than zero, and not just a probability density.

So, in continuous empirical domains both the data and the model spaces have a natural discretisation and Bayes' theorem can always be applied.

However, this is not to say that it is easy to go and make MML work in any given application; in fact it can be quite difficult. After the self evident observations above, a lot of hard work on efficient encodings, search algorithms, code books, invariance, Fisher information, fast approximations, robust heuristics, adaptations to specific problems, and all the rest, remained to be done. Fortunately, MML has been made to work in many general and useful applications, including, but not limited to:

Since Bayesians without Borders will in significant part be about Bayesian networks and their uses, in this post I will introduce them to newcomers to the technology.

Bayesian networks (BNs) are an increasingly popular technology for representing and reasoning about problems in which probability plays a role. A Bayesian network is a directed, acyclic graph whose nodes represent random variables and arcs represent direct dependencies. The arcs often, but not always, also represent direct causal connections between the variables. The nodes pointing to are called its parents and collectively are denoted . The relationship between variables is quantified by conditional probability tables (CPTs) associated with each node, namely . The CPTs together compactly represent the full joint distribution. Users can set the values of any combination of nodes in the network that they have observed. This evidence, , propagates through the network, producing a new posterior probability distribution for each variable in the network. There are a number of efficient exact and approximate inference algorithms for performing this probabilistic updating, providing a powerful combination of predictive, diagnostic and explanatory reasoning.

I will illustrate with the design of a BN for a simplified version of a real ecological problem, modeling native fish populations in Victoria. Problem: A local river with tree-lined banks is known to contain native fishpopulations, which need to be conserved. The river passes through croplands and is susceptible to drought conditions. Rainfall helps native fish populations by maintaining water flow, which increases habitat suitability as well as connectivity between different habitat areas. However, rain can also wash pesticides that are dangerous to fish from the croplands into the river. What we want to do is build a BN adequate for modeling this system.

The first step is to decide what the variables of interest are, which will become the nodes in the BN. The abundance of native fish directly depends only on the level of pesticide in the river and the river flow, hence Native Fish Abundance — a so-called "leaf node" — has only those two parent nodes. RiverFlow depends on how much rain falls in a given year (Annual Rainfall), and how much of that water ends up in the river, which means it depends also on the long term Drought Conditions. The amount of pesticide in the river (Pesticide in River) depends on Pesticide Use and whether there is enough rain (Annual Rainfall) to wash it into the river. Finally, the condition of the trees on the river bank depends only on the long term drought and more recent rainfall.

This graphical structure captures these causal interactions:

In consultation with an ecologist we might build the CPTs (i.e., eliciting the parameters from the ecologist), as in these example tables:

Note that Pesticide Use (just "Pesticides" in the table) and Annual Rainfall are so-called "root nodes" with no parents, so there is only a single probability distribution for each, whereas for nodes with parents there is a conditional distribution for each possible instantiation of its parents.

The CPT for the Native Fish Abundance node shows the possible combinations of values for the parent nodes (Pesticides and River Flow), and a probability distribution of the resultant Native Fish Abundance, over the three levels, High, Medium and Low. We can see that the best conditions for the fish are Low levels of pesticide and Good River Flow (.8, .15, 0.05), while the worst are High pesticide use and Poor River Flow (.01, .10, .89). Note also that there may well be other factors in play, such as the presence of exotic predators, or disease, that are not represented explicitly by nodes in the BN. The effects of these are averaged over in the CPTs. They are reflected, for example in the 0.05 probability that native fish abundance is Low, even under the best pesticide and river flow conditions.

Now that we have the BN structure and its parameters, the BN can be used for reasoning. That is, we can instantiate different possible scenarios by updating the values of particular nodes and then updating the BN, using one of the many Bayesian network programs around, such as Netica. First, here is the network with no evidence:

Without making any observations, this BN tells us that the most likely state of the native fish is Low abundance (57.8%), though the Tree Condition is most likely Good (53.3%). If we add observations of the root nodes in the BN, when there is High pesticide use, above average rainfall and no drought conditions, we get:

This reasoning is predictive, from cause to effect. In this scenario, the prediction is that the Native Fish Abundance will improve, due to the River Flow being Good, despite the increased Pesticide in River. Alternatively, the BN can be used for diagnosis, by entering evidence for the Native Fish Abundance leaf node:

Comparing to the no evidence case, we can see that it is less likely that the pesticide use was high, less likely there have been drought conditions, and more likely that rainfall has been above average. Finally, we can use the BN in any arbitrary combination of diagnostic and predictive reasoning; here with evidence entered for both a cause (Pesticide Use being High) and an effect (Native Fish Abundance being High), resulting in (fairly small) changes to the distributions for all the other nodes:

Here I have briefly described and illustrated the usual knowledge engineering process of building Bayesian networks. There is, of course, a great deal more to it when building a real network of any complexity, which you can read about in depth in our book Bayesian Artificial Intelligence. Some of these, including causal discovery algorithms for learning BNs from sample data, will also be discussed in future posts in this blog.