Jiaqi Wang 王佳琦

Research Scientist
Shanghai AI Laboratory

E-mail: wjqdev@gmail.com

Biography

Jiaqi Wang is a Research Scientist at Shanghai AI Laboratory. Before that, he received his Ph.D. at Multimedia Laboratory (MMLab) of The Chinese University of Hong Kong (CUHK), supervised by Prof. Dahua Lin. He also worked closely with Prof. Chen Change Loy.

His research interests focus on Multimodal Learning, Visual Perception, and AI Content Creation in both 2D and 3D open worlds.

We have opening positions for research interns (undergraduate, master, Ph.D. and gap year students are all welcomed) and full-time researchers. Please drop me an e-mail (wjqdev@gmail.com) if you are interested.

News

[2024-04] We are hosting V3Det Challenge jointly with Visual Perception via Learning in an Open World Workshop at CVPR 2024.
[2024-04] We release InternLM-XComposer2-4KHD, a pioneering large vision-language model handling resolutions from 336 pixels to 4K HD.
[2024-02] Four paper accepted to CVPR 2024 (Two Spotlights).
[2024-01] We release InternLM-XComposer2, a groundbreaking vision-language large model excelling in free-form text-image composition and comprehension.
[2023-09] We release InternLM-XComposer, a large foundation model for advanced text-image comprehension and composition.
[2023-09] One paper accepted to AAAI 2024.
[2023-07] One paper accepted to Siggraph Asia 2023.
[2023-07] One paper accepted to ICCV 2023. (One Oral)
[2023-02] One paper accepted to ICML 2023.
[2023-02] Four papers accepted to CVPR 2023. (One Award Candidate)
[2023-01] One papers accepted to ICLR 2023. (One Spotlight)
[2022-10] Two papers accepted to AAAI 2023. (One Oral)
[2022-07] One paper accepted to NeurIPS 2022.
[2022-02] One paper accepted to CVPR 2022.
[2021-11] Received his Ph.D. from MMLab and Join Shanghai AI Laboratory
[2021-10] One paper accepted to TIP.
[2021-09] One paper accepted to NeurIPS 2021.
[2021-04] One paper accepted to TPAMI.
[2021-03] One paper accepted to CVPR 2021.
[2020-07] One paper accepted to ECCV 2020 (One Spotlight).
[2019-11] 1st prize in COCO 2019 Object Detection Challenge (without external data).
(Team Leader, Team: MMDet)
[2019-07] One paper accepted to ICCV 2019 (One Oral).
[2019-02] Two papers accepted to CVPR 2019.
[2018-09] MMDetection is released. See Technical Report.
[2018-09] 1st prize in COCO 2018 Object Detection Challenge. (Team: MMDet)
[2018-02] One paper accepted to CVPR 2018.
[2017-08] Awarded Hong Kong Ph.D. Fellowship Scheme (HKPFS) and Join MMLab.

Publications

InternLM-XComposer2-4KHD: A Pioneering Large Vision-Language Model Handling Resolutions from 336 Pixels to 4K HD

In Arxiv 2024. Xiaoyi Dong *, Pan Zhang *, Yuhang Zang*, Yuhang Cao, Bin Wang, Linke Ouyang, Songyang Zhang, Haodong Duan, Maosong Cao, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Zhe Chen, Xinyue Zhang, Wei Li, Jingwen Li, Wenhai Wang, Kai Chen, Conghui He, Xingcheng Zhang, Jifeng Dai, Yu Qiao, Dahua Lin, Jiaqi Wang# (* equal contribution, # corresponding author)

Long-CLIP: Unlocking the Long-Text Capability of CLIP

In Arxiv 2024. Beichen Zhang,Pan Zhang, Xiaoyi Dong, Yuhang Zang, Jiaqi Wang

RAR: Retrieving And Ranking Augmented MLLMs for Visual Recognition

In Arxiv 2024. Ziyu Liu *, Zeyi Sun *, Yuhang Zang, Wei Li, Pan Zhang, Xiaoyi Dong, Yuanjun Xiong, Dahua Lin, Jiaqi Wang

SongComposer: A Large Language Model for Lyric and Melody Composition in Song Generation

In Arxiv 2024. Shuangrui Ding *, Zihan Liu *, Xiaoyi Dong, Pan Zhang, Rui Qian, Conghui He, Dahua Lin, Jiaqi Wang (* equal contribution)

DualFocus: Integrating Macro and Micro Perspectives in Multi-modal Large Language Models

In Arxiv 2024. Yuhang Cao, Pan Zhang, Xiaoyi Dong, Dahua Lin, Jiaqi Wang

InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Model

In Arxiv 2024. Xiaoyi Dong *, Pan Zhang *, Yuhang Zang*, Yuhang Cao, Bin Wang, Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, Wenwei Zhang, Yining Li, Hang Yan, Yang Gao, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, Jiaqi Wang# (* equal contribution, # corresponding author)

GPT4Point: A Unified Framework for Point-Language Understanding and Generation

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024, as Spotlight. Zhangyang Qi*, Ye Fang*, Zeyi Sun*, Xiaoyang Wu, Tong Wu, Jiaqi Wang#, Dahua Lin, Hengshuang Zhao# (* equal contribution, # corresponding authors)

OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024, as Spotlight. Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, Nenghai Yu

Alpha-CLIP: A CLIP Model Focusing on Wherever You Want

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024. Zeyi Sun*, Ye Fang*, Tong Wu, Pan Zhang, Yuhang Zang, Shu Kong, Yuanjun Xiong, Dahua Lin, Jiaqi Wang (* equal contribution)

InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Composition

In Arxiv 2023. Pan Zhang *, Xiaoyi Dong *, Bin Wang, Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuangrui Ding, Songyang Zhang, Haodong Duan, Wenwei Zhang, Hang Yan, Xinyue Zhang, Wei Li, Jingwen Li, Kai Chen, Conghui He, Xingcheng Zhang, Yu Qiao, Dahua Lin, Jiaqi Wang# (* equal contribution, # corresponding author)

OneLLM: One Framework to Align All Modalities with Language

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2024. Jiaming Han, Kaixiong Gong, Yiyuan Zhang, Jiaqi Wang, Kaipeng Zhang, Dahua Lin, Yu Qiao, Peng Gao, Xiangyu Yue

ShareGPT4V: Improving Large Multi-Modal Models with Better Captions

In Arxiv 2023. Lin Chen*, Jinsong Li*, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, Dahua Lin (* equal contribution)

HyperDreamer: Hyper-Realistic 3D Content Generation and Editing from a Single Image

In SIGGRAPH Asia 2023, conference track. Tong Wu*, Zhibing Li*, Shuai Yang*, Pan Zhang, Xingang Pan, Jiaqi Wang, Ziwei Liu, Dahua Lin (* equal contribution)

VIGC: Visual Instruction Generation and Correction

In AAAI 2024. Bin Wang, Fan Wu, Xiao Han, Jiahui Peng, Huaping Zhong, Pan Zhang, Xiaoyi Dong, Weijia Li, Wei Li, Jiaqi Wang, Conghui He

V3Det: Vast Vocabulary Visual Detection Dataset

In IEEE International Conference on Computer Vision (ICCV) 2023, as Oral. A Detection dataset with more than 13,000 Categories,
Jiaqi Wang*, Pan Zhang*, Tao Chu*, Yuhang Cao*, Yujie Zhou, Tong Wu, Bin Wang, Conghui He, Dahua Lin (* equal contribution)

OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023, Award Candiate
Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Jiawei Ren, Liang Pan, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, Dahua Lin, Ziwei Liu

UPop: Unified and Progressive Pruning for Compressing Vision-Language Transformers

In International Conference on Machine Learning (ICML) 2023
Dachuan Shi, Chaofan Tao, Ying Jin, Zhendong Yang, Yuan Chun#, Jiaqi Wang#
(# corresponding authors)

BUOL: A Bottom-Up Framework with Occupancy-aware Lifting for Panoptic 3D Scene Reconstruction From A Single Image

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Tao Chu, Pan Zhang, Qiong Liu, Jiaqi Wang

Multi-level Logit Distillation

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Ying Jin, Jiaqi Wang#, Dahua Lin
(# corresponding author)

Dense Distinct Query for End-to-End Object Detection

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2023
Shilong Zhang, Xinjiang Wang, Jiaqi Wang, Jiangmiao Pang, Chengqi Lyu, Wenwei Zhang, Ping Luo, Kai Chen

Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstruction

In International Conference on Learning Representations (ICLR) 2023, as Spotlight
Tong Wu, Jiaqi Wang#, Xingang Pan, Xudong Xu, Christian Theobalt, Ziwei Liu, Dahua Lin#
(# corresponding authors)

Self-supervised Action Representation Learning from Partial Spatio-Temporal Skeleton Sequences

In AAAI Conference on Artificial Intelligence (AAAI) 2023, as Oral
Yujie Zhou, Haodong Duan, Anyi Rao, Bing Su, Jiaqi Wang

Semi-Supervised Semantic Segmentation via Gentle Teaching Assistant

In Advances in Neural Information Processing Systems (NeurIPS) 2022
Ying Jin, Jiaqi Wang#, Dahua Lin
(# corresponding author)

LAVT: Language-Aware Vision Transformer for Referring Image Segmentation

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2022
Zhao Yang*, Jiaqi Wang*, Yansong Tang, Kai Chen, Hengshuang Zhao, Philip H.S. Torr
(* equal contribution)

Few-Shot Object Detection via Association and DIscrimination

In Advances in Neural Information Processing Systems (NeurIPS) 2021
Yuhang Cao, Jiaqi Wang#, Ying Jin, Tong Wu, Kai Chen, Ziwei Liu, Dahua Lin
(# corresponding author)

Texture Memory-Augmented Deep Patch-Based Image Inpainting

In IEEE Transactions on Image Processing (TIP) 2021
Rui Xu, Minghao Guo, Jiaqi Wang, Xiaoxiao Li, Bolei Zhou, Chen Change Loy

CARAFE++: Unified Content-Aware ReAssembly of FEatures

In IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI) 2021
Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, Dahua Lin

Seesaw Loss for Long-Tailed Instance Segmentation

In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) 2021
Jiaqi Wang, Wenwei Zhang, Yuhang Zang, Yuhang Cao, Jiangmiao Pang, Tao Gong,
Kai Chen, Ziwei Liu, Chen Change Loy, Dahua Lin

Side-Aware Boundary Localization for More Precise Object Detection

In European Conference on Computer Vision (ECCV) 2020, as Spotlight
Jiaqi Wang, Wenwei Zhang, Yuhang Cao, Kai Chen, Jiangmiao Pang, Tao Gong, Jianping Shi, Chen Change Loy, Dahua Lin

CARAFE: Content-Aware ReAssembly of FEatures

In IEEE International Conference on Computer Vision (ICCV) 2019, as Oral
Jiaqi Wang, Kai Chen, Rui Xu, Ziwei Liu, Chen Change Loy, Dahua Lin

Region Proposal by Guided Anchoring

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019
Jiaqi Wang*, Kai Chen*, Shuo Yang, Chen Change Loy, Dahua Lin (* equal contribution)

Hybrid Task Cascade for Instance Segmentation

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2019
Kai Chen, Jiangmiao Pang, Jiaqi Wang, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jianping Shi, Wanli Ouyang, Chen Change Loy, Dahua Lin

Optimizing Video Object Detection via a Scale-Time Lattice

In IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2018
Kai Chen, Jiaqi Wang, Shuo Yang, Xingcheng Zhang, Yuanjun Xiong, Chen Change Loy, Dahua Lin

Projects

MMDetection: Open MMLab Detection Toolbox and Benchmark

As Co-author and Core Developer

Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang, Chen Change Loy, Dahua Lin

Sep 9, 2018

MMDetection: Open MMLab Detection Toolbox and Benchmark

Honors and Awards

1st prize in COCO Object Detection Challenge (without external data) (Team Leader, Team: MMDet), 2019
1st prize in COCO Object Detection Challenge (Team: MMDet), 2018
Hong Kong Ph.D. Fellowship Scheme (HKPFS), 2017
National Scholarship, 2016
National Scholarship, 2015
National Scholarship, 2014