Photo placeholder

Zhangyu Wang

I am a student in the B.S./M.S. (Progressive Degree Program) in Computer Science at the University of Southern California. I am fortunate to be working as a research intern at the Stanford Vision and Learning Lab, advised by Prof. Jiajun Wu, and at USC LiraLab, advised by Prof. Erdem Bıyık. Before that, I had the pleasure of working with Prof. Gaurav S. Sukhatme at the USC Robotic Embedded Systems Laboratory.

My research interest focuses on robot learning, specifically on leveraging existing knowledge in pretrained Foundation Models and VLMs for generalizable manipulation and scalable learning or data collection methods.

Outside of research, I enjoy playing badminton.

Research

My work centers on robot learning: helping robots acquire precise, generalizable manipulation skills by combining scalable data collection with knowledge from pretrained foundation models. To this end, my research includes:

News

Show more...Show less...
  • [Sep 2025] I will be working as a research intern at USC Robotic Embedded Systems Laboratory, advised by Prof. Gaurav Sukhatme.
  • [May 2023] Our paper Offset-Guided Attention Network for Room-Level Aware Floor Plan Segmentation is accepted to IEEE Access.

Publications (* indicates equal contribution; † indicates equal advising)

Cut-GS: Generating Hollow and Solid Interiors for 3D Gaussian Splatting

Sen Deng, Yidan Feng, Zhangyu Wang, Chun-Ka Wong, Yao Jie Xie, Zhengzhe Liu, Jing Qin

CVPR 2027 · In submission

Dosesteer: Steering diffusion-based dose prediction models towards clinical targets via Dual-Response Guidance

Shupeng Cheng, Jing Jia, Jincheng Wu, Zhangyu Wang, Gao Wang, Maohe Jiang, Chi Xu, Feiyang Wang, Guangyang Wang, Ruishan Liu

CVPR 2027 · In submission

Projects (* indicates equal contribution)

Side-by-side insertion rollouts: the baseline stops at the force limit without recovering, while the learned-field model applies a small corrective offset and completes the insertion. A plot below compares the model's offsets with baseline and model contact forces over time.

Self-Supervised Precise Agentic Manipulation Ongoing

Project Lead · Advised by Prof. Erdem Bıyık and Prof. Jiajun Wu

Stanford SVL & USC LiraLab · Aug 2026 – Present

An agentic, VLM-based pipeline for precise manipulation that learns from real interaction observations and actively corrects its actions when keypoint or target estimates are inaccurate. It reaches ~80% success on the ManiSkill PlugCharger task, compared with ~60% for the baseline.

Example results for an orange and a watermelon, from the input exterior image to generated interior cross-section renderings.

Zero-Shot 3D Model Interior Generation

Project Lead · Advised by Prof. Zhengzhe Liu

Dec 2025 – Apr 2026

Feedforward generation of 3D food models with interiors, so that cut-open views render realistic cross-sections for foods with varied exterior shapes. Distilling score distillation sampling (SDS) knowledge into a feedforward model generates an interior in under one second, instead of slow per-object optimization.

Real2Sim reconstruction for the Push-T task: a green T-shaped object model beside a red point cloud.

Real2Sim-Guided Model Predictive Control for Few-Shot Tabletop Manipulation

Research Intern · Advised by Prof. Gaurav S. Sukhatme

USC Robotic Embedded Systems Laboratory · Aug 2025 – Aug 2026

An MPC pipeline with a custom reward model for few-shot tabletop manipulation. Its Real2Sim stage combines VLMs, Grounded SAM, TRELLIS, and FoundationPose to reconstruct common tabletop tasks (Push-T, cube picking, peg insertion) and update object poses in real time; zeroth-order optimization recovers mass and friction from video with 70%+ accuracy.

Predicted V50% isodose boundary mask shown in three orthogonal slice views.

Post-Training Optimization for High-Precision Isodose Line Prediction

Research Intern / Project Lead · Advised by Prof. Ruishan Liu

May 2025 – May 2026

Dose predictions from state-of-the-art models are not yet accurate enough at isodose lines for clinical use. Post-training methods for head-and-neck dose prediction improve HD95 on the V50%, V12Gy, and V80% isodose lines over the baseline.

Seven-step pipeline: LLM-generated species keywords, NCBI Taxonomy query, GEO accession retrieval, Selenium crawling of GSE metadata, PMID extraction, PMCID conversion and open-access PDF download, and LLM-based extraction of microbe, gene, metabolite, and KEGG fields, with human expert QC.

LLM-Guided Machine-Learning-Ready Metabolic Knowledge Graph Construction

Research Intern · Advised by Prof. Zheng Chen

Zheng Research Group, Osaka University · May 2025 – Jul 2026

Combines LLMs with expert knowledge to build a machine-learning-friendly metabolism dataset linking more than 20,000 papers to their KEGG pathways and GEO results. Expert verification of data quality is underway; paper in progress.

Protein CLIP

Training CLIP with Small Batch Size: A Method to Leverage Memory-Heavy Structural Data

Zhangyu Wang

CSCI 467 Course Project · Spring 2025 / Paper (PDF) / Code & data

Brings memory-heavy 3D protein structures into CLIP-style contrastive learning, where they force small batch sizes. Residual Loss Incorporation and Random Row-wise CLIP (with sigmoid and combined objectives), built on a trainable GearNet structure encoder, lower the minimum validation loss from 4.8 to 3.8–4.3 compared with a pre-embedded ProteinCLIP baseline; downstream protein-protein interaction performance remains an open problem.

EEG/EMG Signal-Based Control

Project Lead · Advised by Prof. Zheng Chen

Zheng Research Group, Osaka University · Mar 2026 – Jun 2026

Uses the correlation between EEG and EMG signals during actions to extract subject-agnostic features for EEG/EMG-guided manipulation, applying a learnable wavelet transform to EMG and exploring filters for noisy EEG signals.

Education & Experience

Education

Experience

Honors & Awards

Skills