News Details

Seminar by Dr. Mohammed Abouhaeaf, 19 Nov 2014


Solutions of Dynamic Graphical Games Using Reinforcement Learning Techniques
Wednesday, 19 November, 2014
From 2:10 PM - 3:10 PM
Location    22-132

Dr. Mohammed Abouheaf
Assistant Professor, Systems Engineering Department

Dynamic graphical games represent a new class of multi-agent discrete-time dynamic games. In graphical games, the information flow between agents is prescribed by a communication graph structure. For that reason a local performance index is defined for each agent that depends only on the local information available to this agent. The Discrete Hamiltonian mechanics, distributed multi-agent control, optimal control theory, game theory, and reinforcement learning techniques are brought together to formulate and solve the multi-agent graphical games. Ideas from cooperative control are used to attain synchronization among the agents’ dynamics to the leader dynamics. Novel coupled Hamilton Jacobi equations and coupled Bellman equations are developed for this class of the dynamic graphical games. The relationship between the coupled Hamilton Jacobi equations and the coupled Bellman equations is shown. The graphical game Bellman equations are shown to be equivalent to certain graphical game Hamilton Jacobi Bellman equations. Nash solutions are given in terms of solutions to a set of coupled discrete-time Hamilton-Jacobi-bellman equations. Reinforcement learning algorithms are developed to solve the dynamic graphical games online in real-time. Approximate Dynamic Programming techniques namely Heuristic Dynamic Programming is extended to solve the dynamic graphical games. Online and offline adaptive learning structures are implemented using actor-critic network structures to solve these dynamic graphical games.

Mohammed Abouheaf received his B.Sc. and M.Sc. degrees in Electronics and Communication Engineering, Mansoura College of Engineering - Egypt in 2000 and 2006 respectively.   He worked as an assistant lecturer with the Air Defense College  (2001-2002). He worked as a Planning Engineer for the maintenance department, Suez Oil Company (SUCO) from 2002-2004.   He worked as an Assistant Lecturer with the Electrical Engineering Department, Aswan College of Energy Engineering from 2004 - 2008. He received his PhD degree in Electrical Engineering, University of Texas at Arlington (UTA) in 2012.   He worked as Adjunct Faculty at the Electrical Engineering Department at the same university from 2012 – 2013.  Before that he was also a member of the Advanced Controls and Sensor Group (ACS) and the Energy Systems Research Center (ESRC) from 2008 - 2012.  Currently, he is an Assistant Professor in the Systems Engineering Department at King Fahd University of Petroleum and Minerals (KFUPM. His research interests include optimal control, adaptive control, reinforcement learning, fuzzy systems, game theory, microgrids, and economic dispatch.