We will be holding our next Introduction to BNs workshop at the Mount Albert Research Centre in Auckland, New Zealand, on May 13-14. We still have some places available, so if you would like to attend, please register at:
There you will also find a draft workshop schedule with an overview of the topics covered. If you are unable to make these dates, please email me indicating your interest in attending at another time.
The DTCA is set to take effect at the beginning of 2015, criminalizing much of the research into high tech and applied science in Australia, whether conducted in universities or industry. I have summarized its main features and impacts on the university sector (see "Australia's act of Intellectual Terrorism" below). For impacts on industry see the website victimsofdsto.
I have started a new blog, BayesianWatch, on which I will post (occasionally) on Bayesian argumentation theory and practice. The differences from this blog are: this is the official blog of Bayesian Intelligence, while BayesianWatch is my personal blog; this blog is about Bayesian technology and methods, while the other is about argument analysis, with a Bayesian orientation. Inevitably, there will be some cross talk, however; for example, on that blog you will find a response to the Sally Clark post here.
On February 13-14 BI will run its Introduction to BNs workshop at Monash University, Caulfield Campus. We still have places free, so if you would like to attend, please register at the following site:
There you will find a draft workshop schedule with an overview of the topics covered. If you are unable to make these dates, please email me (steven.mascaro@bayesian-intelligence.com) indicating your interest in attending at another time.
[Editor's note: These ideas were prepared by Yung in support of her participation in a forecasting project and describe techniques she found helpful for predicting specific world events within specific time frames. The context is one where specific possible outcomes of political, economic, etc. events were described, and probability ranges for those outcomes had to be supplied within a few days. The requirement was to provide predictions as accurate and precise as possible without succumbing to over- or under-confidence. I think these ideas are of general interest.]
Favour the status quo (this seems to me more like an empirically derived tip – if you do this you’ll come out ahead in the long run)
Flip the question around
Use reference classes
Prepare lines of retreat (what would make me change my mind about this?) [this is equivalent to think of reasons/evidence that make it unlikely – a sort of correction for overconfidence]
Abandon sunk costs
Consider your loss function (this is more about strategic hedging in response to how the Brier score is computed)
These ideas are, in part, about the psychological aspects of making the prediction, but he doesn’t discuss strategies for finding information, reconciling or evaluating information, forming mental models etc.
Tools I use:
Understand the conditions required for resolution of question.
Iteration (rapid prototyping). Search and scan articles quickly to build up some mental model of what the situation is, who the actors are and understand any processes required for resolution of the question (point 3 below) – go quick, build a crude mental model then refine as more material is encountered/assimilated.
Understand any processes required for resolution of question. E.g. standard operating protocol for declaration of disease status of OIE member (World Organization of Animal Health); process of granting of EU member candidacy; voting and veto powers re UN Security Council, process for obtaining full negotiating member status at TPP (Trans-Pacific Partnership), etc.
If a question involves non-English speaking players make sure to search beyond the elite news sources (e.g. BBCNews, Guardian, Newsdaily, New York Times, Economist, WSJ) for primarily English speakers (e.g. Al-Jazeera, Al-Monitor, Kurdistan news, Ahram online etc.).
Consider whether the uncertainty associated with the question is reducible or irreducible. Uncertainty may be reducible if it’s due to a lack of knowledge or understanding. If I suspect this is the case, I try to find a knowledgeable interpreter/analyst or try to construct Reference Classes and identify plausible Base Rates. Indicators of knowledge: knowledge of historical context, logical argumentation, ability to articulate reasons, provide reasons supported by evidence, ability to recognise and acknowledge where and when things aren’t known (e.g. Aurelan George Mulgan on Japan TPP; Reidar Visser on Iran). If the uncertainty is irreducible (and I consider things like oil, stock and currency prices to fall in this category) then don’t bother looking for more info.
Use Reference Classes, try to estimate Base Rate from any available data and use that (see Kahneman: Thinking Fast, Thinking Slow).
Use Bueno de Mesquita’s ‘factors’ (Position, Influence, Salience, Flexibility/Resolve, see below) to think through the interests of actors involved.
Understand the status quo and consider favouring the status quo (empirically, we would expect the status quo to persist).
Another empirically robust finding from interest group politics: well-organised private interests nearly always win out over diffuse, generally disorganized public interests.
Try and articulate Reasons that would make an event LIKELY and and Reasons that would be it UNLIKELY (this requires adequate research or analysis or reasoning to furnish or construct reasons; will often rely on tally for informing prediction).
Keep looking out for new info till just before the deadline and update estimates accordingly.
Consciously consider our tendency towards What You See IS All There Is (WYSIATI) – this tendency is misleading! Remember that the capacity for unpredictable, unforeseeable events or chains of events is ever present, particularly with long time lines and adjust forecasts accordingly (e.g. UN Sec Council resolution on Mali).
Note: Bruce Bueno de Mesquita is a political scientist who claims to have built a system using Game Theory for making predictions (http://www.predictioneersgame.com/) I think he overreaches and he’s been criticised for not being transparent about his methods. I agree. Nevertheless, I find his key factors for assessing actor interests, stakes, power and influence useful.
Relative potential influence - the relative potential ability of each player to persuade other stakeholders to adjust their approach to the issue to be more in line with the influencer’s perspective. Resources - the potential ability each player has to persuade other stakeholders to support a point of view on the issue in question. The ability to persuade may be derived from holding a position of authority, being an expert, commanding a large budget, or any other factor that makes others listen to someone.
Policy position - position preferred by each stakeholder on the issue, taking constraints into account. This position is not likely to be the outcome the stakeholder expects or is prepared to accept, nor is it likely to be what the player wants in his or her heart of hearts. It is the position the stakeholder favors or advocates within the context of the situation. When a player’s position has not been articulated, it is best thought of as the answer to the following mind experiment: If the stakeholder were asked to write down his or her position, without knowing the values being written down by other stakeholders, what would he or she write down as the position he or she prefers on the issue continuum? To place a numeric value on the position, the investigator must first have defined the issue continuum. The continuum will either have a natural numeric interpretation, such as the percentage of uninsured on health care to be covered under a new policy or the analyst will need to develop numeric values that reflect the relative degree of difference across policy stances that are not inherently quantitative. It is important that the numerical values assigned to different positions (and they can range between any values) reflect the relative distance or proximity of the different solutions to one another. An easy way to turn player preferences on an issue into numeric values is to place each player on the issue continuum you defined, locating then at the point on the continuum that reflects the policy they support. Then, use a ruler to measure how far each player is from one end of the line that reflects the range of choices. Let the left-hand end of the line equal 0. Then each other point on the line is simply its distance from 0 on the ruler.
Salience - assesses how focused a stakeholder is on the issue. Its value is best thought of in terms of how prepared the stakeholder is to work on the issue when it comes up rather than some other issue on his or her plate. Would the stakeholder drop everything else to deal with the issue? Would the player work on it on a weekend day, come back from vacation, etc.? The more confidently it can be said that this issue takes priority over other matters in the stakeholder’s professional life (or personal life if the issue is about a personal or family matter), the higher the salience value.
Flexibility/Resolve - evaluates the stakeholder’s preference for reaching an agreement as compared to sticking to his or her preferred position even if it means failing to reach an agreement.
In October 2012 the Australian parliament passed the Defence Trade Controls Act. The stated purposes of the act are unobjectionable: implementing the prior Australia-United States Defense Trade Cooperation Treaty, simplifying defence-related trade between Australia, the US and the UK, and tightening the regulation of intangible transfers of military goods, reflecting the growth of the internet in communications. Unfortunately, these good intentions have led the Australian government to adopt an extraordinarily broad definition of military goods and to impose an impossibly harsh regulatory regime on activities concerning them, to the point that what is today ordinary academic research into, for example, Bayesian network technology may tomorrow become a criminal activity subject to 10 years imprisonment. It is an act that can legitimately be called intellectual vandalism. Indeed, some academics have already abandoned long-pursued research projects out of fear of the Act being used for retribution, making it a means of intellectual terrorism.
The Defence and Strategic Goods List
One support for the act is the Defence and Strategic Goods List (DSGL). The DSGL lists goods targeted for regulation that are strictly military and also many goods which could be put to military uses, so-called dual-use goods. These latter include a huge variety of things, including: bacilli and viruses, plant pathogens, metallic alloys, carbon fibres, neural computers, optical computers, vector and other high-performance computers, optical telecommunications, signal processors, fault-tolerant systems,1 image processing, cryptography, and, of most interest to us, robotics. The DSGL explicitly incorporates all software which is designed or modified for the "development, production or use" of the goods. This means that all software that may be used to drive robots are covered by the list and by the Act using the list: all software that can be used for intelligent decision making and analysis, which clearly means all of Bayesian network technology, not to mention all of computational statistics, data mining and artificial intelligence generally.
The Act's intent is to regulate foreign access to new research in military and dual-use goods. While all of the dual-use goods could potentially be used in military applications and controlling such uses is a worthwhile objective, all of them also are used in non-military business, industry and governance, and these civilian uses are vital to the economy and prosperity of the nation. Therefore, a reasonable balance must be struck between supporting research in these areas and controlling access to the research. As the Act stands, it is clear to us that these matters are not in balance.
Australia Prepares to Eat its Brains
A key feature of the Act is that it requires prior permission to communicate new research to a foreign national in any of the nominated areas. This includes, but is hardly limited to, publishing research in academic journals. As many submissions on the bill to the Senate Committee on Foreign Affairs, Defence and Trade point out, this requirement implies that the Department of Defence (DoD) would need sufficient expertise across all of the domains listed in the DSGL to judge whether or not proposed research would require a permit. Plausibly, this is a level of expertise which the DoD does not have, nor will ever have. But the difficulties with the Act go far beyond the need to hire thousands of experts to make permit judgments upon research and education.
Obtaining prior approval for each research project and, possibly, each research communication would put an end to a very large amount of research activity in Australia, directing researchers, students and subsequent economic activity elsewhere. Permission would be required to publish across a huge range of areas under active research in the university sector. Without a clear opportunity to publish, most academics would choose not to undertake research projects in these areas, meaning that ARC and NHMRC projects would not even begin without prior approval from the Department of Defence. The Act provides no guidance as to how the DoD should make judgments about research which has yet to be conducted. Many ongoing collaborations with overseas institutions would need separate approvals or would cease. Since about half of enrolled postgraduate students are foreign nationals, for the most part without permanent residency, the requirement for prior permission to communicate new research to them would also cripple postgraduate coursework.
A minor point is that there seems to be some unclarity about whether or not foreign students can be taught anything on the DSGL without prior permission. The Chubb Steering Group's (more on them below) report on its 12 April 2013 meeting states that "the Australian export control system only regulates transfers from a person inside Australia to a person outside Australia". However, the Defence Department's Explanatory Memo for the Bill states that communications are regulated "if the supply is from an Australian person to a foreign person regardless of their geographical location" and "the provision occurs in Australia to a foreign person". Re-skimming the Act itself (96 pages of gripping prose) unfortunately failed to reveal who's right. Still, the considerations immediately below mean this issue is well and truly dominated by other problems.
PhD and Masters theses have always been published by the institutions granting them; in consequence of the Act, either this practice would have to be abandoned, and the contents of theses in the nominated areas suppressed — which would certainly give Australian research a unique voice in the world — or prior approval for publishing as-yet uncreated new research would have to be obtained. Note that this restriction would apply to all postgraduate thesis work in dual-use domains, not just that involving foreign students.
As a crude index of how the Act unaltered will impact the activities of my own organization, the Faculty of IT at Monash University, I surveyed all masters level coursework units (5000 series units, ignoring thesis units) for topics proscribed by the Act (i.e., found in the DSGL). Note that it is an imperative that researchers who teach (which describes myself and almost all of my teaching colleagues at Monash) should incorporate new research into their own teaching. Of necessity, therefore, if a topic matches a dual-use good, it is regulated by the Act. By my count 18 out of 61 classes would come under the provisions of the Act, requiring prior permission for each international student enrolled. (The DoD Explanatory Memorandum makes it clear that individual permission is required — short of Ministerial intervention!) The main areas impacted are: security, software usable for controlling robots, telecommunications technology and high-performance computing. No doubt a similar proportion of our degree by research topics would have to be pre-approved or abandoned. The disruption to Monash FIT’s educational activities will be severe.
The Strengthened Export Controls Steering Group to the Rescue
Amendments to the Bill were proposed by the Senate Foreign Affairs, Defence and Trade Committee in October 2012 which would appear to be sufficient protection for the ongoing research and educational activities of the tertiary sector. Alternative amendments might be derived from the UK Export Controls Act, which again protect public research and education. The corresponding US International Traffic in Arms Regulations also contain a corresponding exclusion, Section 120.11. The Australian Act contains no such protection (despite the extraordinary gobbledygook produced by the US Ambassador to Australia on the subject; see item 8 on the Committee's submission page). The accompanying Explanatory Memorandum does list some exemptions, relying upon the DSGL. The DSGL exempts technology that is in the public domain, required for patents or is basic scientific research. The Department of Defence reckons that this is sufficient protection for tertiary education and research: "With the exemptions ... Defence anticipates that these controls will apply only to very specialised and high-end research" (Explanatory Memo). There are a few substantial problems with this thinking, however. The administrative burden of deciding what is already public and what is not, covering individual research publications over something like 1/3 of Australia's engineering and IT work, would probably suck up a significant percentage of Australian GDP forever. Actual scientific research, while building upon what is in the public domain lives and grows according to what it adds that is new, and so it will not generally be public domain. However, according to the Defence Explanatory Memo (footnote 16), basic scientific research excludes research "directed towards a specific practical aim or objective." So, presumably university research is in the clear. What perhaps eludes Defence experts is that a very large amount of research at universities does have a practical aim or objective. Indeed, the endless waves of "Excellence of Research in Australia"-like programs that regularly inundate academic life here are always trying to pump out information about the wider impact of research on the community. Academics are constantly being badgered to prove that their research has near-term practical impacts and objectives. I suppose an unintended salutary effect of the Bill could be to discourage academics from focusing too much on the immediate benefits of their work! Be that as it may, probably most university research comes with intended applications, and so is covered by the legislation.
In consequence researchers right now must reckon with the possibility that the communication of their efforts, whether in teaching, supervision or publication, may result in 10 years in prison.
Many parties to the discussions, such as Universities Australia and the National Tertiary Education Union, have noted the disparity in protections offered by the UK and the US compared with that on offer in the DTCA for Australians. The consequence of their raising their concerns has been a suspension of enforcement of the Act for two years and the founding of the Strengthened Export Controls Steering Group, headed by Australia's Chief Scientist Ian Chubb, to investigate and test the consequences of the Act with pilot studies. The Steering Group has roughly equal representation from Defence (and Defence industry) and academia. They are empowered to make recommendations. Some people appear to think this answers all of the concerns raised above about the legislation. But until concrete recommendations are made, and rather more importantly, acted upon by Parliament, the university sector is operating in the dark. We do not know which of our research and pedagogical activities will in the near future be prosecuted as criminal. Given that now simply posting classified information on the internet in the public interest is being trumpeted as "assisting the enemy" (Julian Assange, Edward Snowden), it is not much of a stretch to imagine the publication of research on ordinary, but dual-use, technology soon being called assisting the enemy as well.
Three outcomes seem now to be possible: (1) the law may be revised to accord with the corresponding legislation in the US and the UK; (2) the law may be left as is, but its provisions not enforced with regard to most research of a non-military nature; (3) the law may be left as it is, and its provisions enforced in a wide-ranging way. If the first option is realized, then the research and teaching environment will remain much as it has been, and the Australian nation may get on about its business. If option (3) is realized, a large portion of Australian research will simply stop. The tertiary education sector will be massively disrupted and its international reputation will collapse, outside of fields which would appear to be free of the regulations implied by the Act, such as the Arts. Much of engineering, the physical and medical sciences, and information technology would be finished in Australia. Option (2) implies less immediate impact. However, the continuing possibility and threat that an administration has only to change its mind about enforcement would have a significant chilling effect on many researchers. The choice of research area would have to weigh the opportunity for administrative interference and the ultimate potential for criminal sanctions should a researcher or her or his research rub the DoD or the government of the day in the wrong way. Over the long term, research and education in Australia would be severely crippled.
Bayesian Intelligence finds that the unamended Defence Trade Controls Act 2012:
Fails to provide the same safeguards for research and educational activity to be found in the corresponding legislation enacted in the UK and the USA
Will handicap research and tertiary educational activities across a wide range of domains in engineering, information technology, science and medicine
Is likely to lead to a diversion of world-class researchers, students and projects outside of Australia
The introduction of a two-year transition period in the final Act defers, but leaves unaltered, these bad consequences. In view of these probable deleterious effects, Bayesian Intelligence, as a company dependent upon new research in a dual-use technology, strongly urges that the legislation be amended as soon as possible to incorporate similar protections to those in the corresponding legislation of the UK and the US.
1 CORRECTION: Originally, I had the following parenthesis here: "(including computers with error-detecting memories, which long ago became nearly universal)". However, I have in the meantime noticed that the DSGL has an explicit exclusion for this case, in Note a of 4A003.a on p. 165. My apology for not noticing this in the first place.
From the mid-1990s we have developed and enhanced causal discovery software for learning causal Bayesian networks from sample data using an MML measure. We have now made linear CaMML available after some years of absence, when we exclusively focused on developing discrete CaMML. Discrete CaMML has been available for some years via Rodney O'Donnell's github web page, but we link into it now through our "Software" page. The user interfaces for these programs are not exactly user friendly; however, a GUI has been developed for discrete CaMML and should become available reasonably soon.
I have a lot of respect for Crikey, the online Australian newsletter. They report on a lot of things other media outlets won't touch, especially bias and disinformation in those other media outlets. But the other day, while reading a piece by Bernard Keane on the inconsistency of Tony Abbott's rejection of a carbon tax, after his having previously advocated one, I read:
Insistence that the planet is not getting warmer — or, as Abbott until recently insisted, is getting slightly cooler — has become more difficult to maintain publicly, despite the faulty logic of linking weather to climate. (B Keane, Crikey, 5 Feb.)
It is certainly the case that weather is not the same as climate. This is pretty clearly revealed by the fact that many global warming deniers are weather forecasters, whereas hardly any climatologists are deniers. (NB: there are a lot more forecasters than climatologists!) But denying a link between weather and climate is simply absurd. The relation between climate, the prevailing weather in a region and season, and the weather itself, on any given occasion, is stochastic: the climate system, plus specific, highly variable, conditions together determine the specific weather. That establishes a kind of probabilistic dependency —i.e., a link —which is widely recognized in society.
For example, only ignorant people or fools now deny that smoking causes lung cancer. This is so despite the fact that many smokers never get lung cancer. Some of them die too soon from other causes, such as emphysema. But many smoke contentedly for decades with no sign of the cancer showing up. Lawyers for tobacco companies used to point this out in trying to make the case that specific complainants had no basis for complaint, because their lung cancers might have been amongst those caused by pesticide exposure or smog or a stray cosmic ray striking a susceptible cell in the lung. But these defences have been abandoned, with even tobacco companies accepting some culpability for the disease in specific cases. The situation is analogous with weather and climate change. Was Katrina specifically due to global warming? Well, obviously not in its totality; but global warming heating the Gulf of Mexico likely contributed to its intensity. Specific events will never be entirely attributable to a broad-scale change, because the broad-scale change will never be entirely responsible for a specific event in all its specificity. Denying a linkage on that basis, however, is a nonsense. An accumulation of extreme weather events, and a statistical assessment showing that the probability of their extremity without global warming is rapidly vanishing, will eventually silence those who claim weather tells us nothing about climate.
Probabilistic dependencies are real. The link they establish, in fact, is just that between a stochastic hypothesis and the evidence which confirms it. Denying such a link is tantamount to denying the statistical foundations of empirical science.
If you want to "sound reasonable" by making some concession or other to global warming deniers, then you should do so by reporting something that is factual, rather than counterfactual. You can point out that many deniers have good dress sense, or sometimes use grammatical sentences, for example. Buying into their dogma about weather versus climate change can all too easily turn into buying into their rejection of science.
