Autonomous Behavior Selection For Self-driving Cars Using Probabilistic Logic Factored Markov Decision Processes

ABSTRACT

We propose probabilistic logic factored Markov decision processes (PL-fMDPs) as a behavior selection scheme for self-driving cars. Probabilistic logic combines logic programming with probability theory to achieve clear, rule-based knowledge descriptions of multivariate probability distributions, and a flexible mixture of deductive and probabilistic inferences. Factored Markov decision processes (fMDPs) are widely used to generate reward-optimal action policies for stochastic sequential decision problems. For evaluation, we developed a simulated self-driving car with reliable modules for behavior selection, perception, and control. The behavior selection module is composed of a two-level structure of four action policies obtained from PL-fMDPs. Three main tests were conducted focused on the selection of the appropriate actions in specific driving scenarios, and the overtaking of static obstacle vehicles and dynamic obstacle vehicles. We performed 520 repetitions of these tests. The self-driving car completed its task without collisions in 99.2% of the repetitions. Results show the suitability of the overall self-driving strategy and PL-fMDPs to construct safe action policies for self-driving cars.

Acknowledgements

The authors would like to thank Sergio Yahir Hernandez-Mendoza for his generous support in conducting part of the tests in this work, and the reviewers for their insightful and interesting comments and feedback.

Data availability statement

The source code of this work (that includes the simulated environment, the self-driving system, and the four PL-fMDPs), and a video recording showing a run of the system are freely available at: https://github.com/mnegretev/AutonomousBehaviorSelection2023.

Disclosure statement

No potential conflict of interest was reported by the author(S).

Supplemental data

Supplemental data for this article can be accessed online at https://doi.org/10.1080/08839514.2024.2304942.

Notes

1. An atomic formula (or atom, for short) has the form “$\mathit{a}(t_1,\dots ,t_n)$,” for $\mathit{n}\ge 0$, where $\mathit{a}$ is the identifier of the atom, and each argument $t_i$ for $\mathit{i}=1$ to $\mathit{n}$ is a term (that is, a variable, a constant, or a compound term). An atom is grounded when none of its arguments are variables or when they do not contain variables.

2. Iverson bracket function $\left[\left[\cdot \right]\right]$ evaluates to 1 if the propositional condition enclosed in the brackets holds, and it evaluates to 0 otherwise.

3. In this document, prime notation is employed to differentiate between post-action state variables $\mathit{X}{{\mathit{ }}^{\mathrm{\prime }}}_1,..,\mathit{X}{{\mathit{ }}^{\mathrm{\prime }}}_{\mathit{n}}$ and pre-action state variables $X_1,..,X_n$.

4. Lowercase letters are used to denote state fluents, rather than uppercase letters for state variables as in the previous section, in order to adhere to the standard definition of atoms in Prolog.

5. Wherever used after its introduction in an MDP-ProbLog program, a pre-action (resp. post-action) state fluent is identified by adding a value 0 (resp. 1) as its first parameter.

6. Notice that free_NE, free_NW, and success are used twice in the hierarchy, so they are counted only once for the single PL-fMDP.

Additional information

Funding

This work was partially supported by UNAM-DGAPA under grant TA101222 and AI Consortium - CIMAT-CONAHCYT.