1. The simplest actor‑critic (QAC)2. Advantage actor‑critic (A2C)2.1. Baseline invariance2.2. The algorithm of advantage actor‑critic3. Off‑policy actor‑critic3.1. Illustrative examples3.2. Importance sampling3.3. The theorem of off‑policy policy gradient3.4. The algorithm of off‑policy actor‑critic4. Deterministic actor‑critic (DPG)4.1. The theorem of deterministic policy gradient4.2. The algorithm of deterministic actor‑critic参考文献本文是一篇学习笔记内容全部源自于以下视频https://www.bilibili.com/video/BV1Pz5C6iE3X/?p6spm_id_from333.1007.top_right_bar_window_history.content.clickvd_source44ed90827c8f67247cab0ab288133c80