框架
mmagic：
https://github.com/open-mmlab/mmagic
综述
World Models for Autonomous Driving: An Initial Survey
The
JEPA model
aims to construct mapping relationships between different inputs in the encoding space by minimizing input information and prediction errors.
Towards Knowledge-driven Autonomous Driving
Embodied AI
is a facet of intelligence emphasizing the direct interaction between an intelligent system and its environment, involving perception, understanding, and action.
扩散模型
Diffusion model
扩散模型 - Diffusion Model【李宏毅2023】_哔哩哔哩_bilibili
Diffusion扩散模型大白话讲解，看完还不懂？不可能！
多模态预训练CLIP
Diffusion论文：Denoising Diffusion Probabilistic Models
StableDiffusion论文：High-Resolution Image Synthesis with Latent Diffusion Models
Diffusers
stable-diffusion-v1-5权重地址：runwayml/stable-diffusion-v1-5 at main
pytorch手写Diffusion Model：The Annotated Diffusion Model
Stable Diffusion with Diffusers
手写扩散模型-diffuers介绍_哔哩哔哩_bilibili
世界模型
GAIA-1（2023.9.29）
MAGICDRIVE（2024.1.26）
论文地址：MAGICDRIVE: STREET VIEW GENERATION WITHDIVERSE 3D GEOMETRY CONTROL
github：https://github.com/cure-lab/MagicDrive
Drive WM（2023.11.29）
https://github.com/BraveGroup/Drive-WM?tab=readme-ov-file
MUVO（2023.11.23）
多模态生成
DriveDreamer（2023.11.27）
https://github.com/JeffWang987/DriveDreamer
https://drivedreamer.github.io/
DriveDreamer-2: LLM-Enhanced World Models for Diverse Driving Video Generation（2024.3.11）
多视角视频生成
WorldDreamer
https://world-dreamer.github.io/
OccWorld: Learning a 3D Occupancy World Model for Autonomous Driving
Think2Drive: Efficient Reinforcement Learning by Thinking in Latent World Model for Quasi-Realistic Autonomous Driving (in CARLA-v2)
Driving with LLMs
DRIVEVLM: The Convergence of Autonomous Driving and Large Vision-Language Models（2024.2.25）
DriveVLM集成了场景描述、场景分析和分层规划的思维链（CoT）模块
VLM计算量大，提出DriveVLM Dual，将DriveVLM的优势与传统自动驾驶流水线协同混合
据说可以部署在orin芯片上，但猜测只是在orin上跑通，还没有大规模部署到车上
DriveVLM识别了一个倒塌的树木，并给出了稍微向右偏移的驾驶决策
Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving
论文地址：Driving with LLMs: Fusing Object-Level Vector Modality for Explainable Autonomous Driving
github：https://github.com/wayveai/Driving-with-LLMs
ADriver-I: A General World Model for Autonomous Driving
