Yiming Zheng (郑壹铭)

About Me

I am currently a Research Assistant at the Center for Future Media Lab at the University of Electronic Science and Technology of China (UESTC). I am also the leader of the Matrix Studio at Tianjin University of Science and Technology (TUST). I am passionate about computer science and technology, with a particular interest in multimodal, parameter-efficient fine-tuning (PEFT), and large language models (LLMs).

Education

University of Electronic Science and Technology of China
Research Assistant, Center for Future Media Lab
July 2024 – Present

Matrix Studio of TUST
Group Leader, Official Workshop
June 2022 – Present

Tianjin University of Science and Technology
Major: Computer Science and Technology
Ranking: 3/120
September 2021 – Present
Tianjin, China

Research Interests

• Multimodal
• Parameter-Efficient-Fine-Tuning (PEFT)
• Large Language Models (LLMs)

Achievements

• First Prize Scholarship and Outstanding Student (2022, 2023)
• First Prize, Tianjin Regional, Contemporary Undergraduate Mathematical Contest in Modeling
• National Excellence Award, Blue Bridge Cup Algorithm Competition
• Third Prize, National Computer Application Competition, North China

Projects

SMP 2023 ChatGLM Financial Large Model Challenge

• Constructed a RAG system using financial report data, achieving a top 5.9% rank.
• Developed a keyword-based question analysis system and fine-tuned the model with Ptuning V2.
• Led regular strategy meetings, explored open-source RAG strategies, divided tasks, and collaboratively coded.

Fine-Tuning LLaMA2 for Japanese Language

• Achieved a 20% performance improvement on the Japanese benchmark (JGLUE).
• Collaborated with Japan Ltd, leveraging Transformers library and DeepSpeed for distributed training.
• Resolved training stagnation issues, conducted comprehensive parameter fine-tuning, and continued pre-training.

Skills

• Programming Languages & Tools: PyTorch, Transformers, Shell Script, Numpy, Pandas, Matplotlib
• Model Fine-Tuning & Training: Transformers with DeepSpeed, utilizing the ZeRO strategy
• Parameter-Efficient Fine-Tuning (PEFT): Techniques like LoRA and its variants
• Development Environment: Experienced in Linux development and working as an MLOps