Causal Reinforcement Learning under Bareinboim at Columbia Univ.

Author: Robert R. Tucci

Elias Bareinboim

I recently wrote a popular article on Causal AI. (link)  See also this nice list of recent Causal AI references by R. Guo at github. Note that Causal AI is almost exclusively the province of Bayesian networks (bnets), not of their more puny brethren, neural networks (NNs). Indeed, Causal AI is based on Judea Pearl’s theory of causality which is expressed via bnets, not NNs.

Something that I didn’t mention in my popular article because I didn’t know it at the time is that there is a group at Columbia Univ. headed by Elias Bareinboim (EB) dedicated to Causal AI. EB got his PhD at UCLA under Judea Pearl, and now works at Columbia Uni.

In the past, EB et al have worked on integrating Pearl causality and sensor fusion. More recently, EB’s group at Columbia is working on integrating Pearl causality and Reinforcement Learning (RL), what they call Causal RL. EB’s group has a great website documenting their progress in Causal RL. Much of that work is only 1 or 2 years old. Their Causal RL website does not list any github presence so I assume they haven’t released any major Causal AI  software packages yet.

In the past, the RL and bnet communities have seen very little overlap between their endeavors, but this has been changing recently. In fact, I recently wrote a chapter for my github book “Bayesuvius” explaining past RL work in terms of bnets. Pearl causality is not used in my chapter because I am only reviewing past RL algorithms and none of those have taken Pearl causality into account.

I believe Causal AI will become an exploding AI fad in the future (it already is in some rarefied levels of the AI community), and deservedly so.

Since quantum bnets are a superset of classical bnets, Quantum Causal AI and Quantum Causal RL will someday be an active field too.

Go to Source