Li Liu
Li Liu is a Ph.D. candidate in Computer Science and Engineering at UC Santa Cruz, advised by Prof. Leilani H. Gilpin. Li studies spatial intelligence in multimodal models and the agents built on them: how models perceive space, build an internal map of it, and act when what they see is not enough. Li has interned three times as an Applied Scientist at Amazon and is on the 2027 job market for Research Scientist, Applied Scientist, and ML Engineer roles. Email: lliu112@ucsc.edu. Google Scholar. CV.
- Perceive: vision-language models answer even when an image cannot support an answer; Right This Way (NeurIPS 2024) trains them to say how to move the camera instead.
- Represent: language models read spatial layouts far better than they build them; learning to build layouts improves their spatial reasoning (Learning to Draw ASCII, COLM 2026).
- Act: TouchingSpace, a real-time voice-and-touch agent on Gemini native audio that grounds its answers in a map, built from zero and tested with 14 blind and low-vision adults.
Projects
- Directional Guidance for Vision-Language Models (Right This Way, NeurIPS 2024)
- Spatial Reasoning in LLMs through Layout Construction (Learning to Draw ASCII, COLM 2026)
- Real-Time Multimodal Agent Grounded in Touch (TouchingSpace, Under review, 2026)
- Grounding Place Names in 3D-Printed Tactile Maps (Tactile Map Annotation, ASSETS 2026 Demo)
- Human-AI Collaboration with Foundation Models: A Survey (Human-AI Collaboration Survey, ACM TIST 2026)
- Human-in-the-Loop Pattern Discovery in Unlabeled Conversations (Pattern Insights Explorer, WWW 2026 Demo, Amazon)
- Agent Skills as Typed, Inspectable Data (Skills on Board, Under review, 2026)
- Experiential Search over User Reviews (SER, Amazon internship, 2024)
- Multi-View Self-Supervised Pre-Training for 3D Medical Segmentation (SwinMM, MICCAI 2023)
- Efficient Conformal Prediction with Shrunken Centroids (CPSC, IEEE TIM 2022)
- Cross-User sEMG Domain Adaptation (Cross-User Taste Recognition, IEEE TIM 2022)
- Data Augmentation for Label-Scarce Electronic Noses (Boost AI Power, IEEE Sensors Journal 2021)
- Preventing Catastrophic Overfitting in FGSM Adversarial Training (Bag of Tricks for FGSM-AT, arXiv 2022)
- Multi-Scale CNN for EEG Taste Recognition (EEG-MSRNet, IEEE TIM 2023)
- sEMG-to-Speech Silent Speech Interface (Silent Speech Interface, IEEE TIM 2023)
- EEG-EMG Fusion for Taste Recognition (Bi-FusionNet, ICASSP 2023)
- Conformal Risk Triage for COVID-19 from CT (Tri-Light Warning System, EJR Open 2024)
- Taste Intensity Recognition from Facial sEMG (Taste Intensity from Facial sEMG, Sensors 2021)
- Temporal-Spectral Feature Extraction for Electronic Noses (Herbal Medicine Origins, arXiv 2021)
- Light-Responsive Kinetic Pavilion (Lotus Pavilion, Undergraduate design, Zhejiang University)
Publications
- Learning to Draw ASCII Improves Spatial Reasoning in Language Models. Shiyuan Huang, Li Liu, Jincheng He, Leilani H. Gilpin. Conference on Language Modeling (COLM), 2026. Link
- From Anonymous Shapes to Named Places: A Tool for Braille and Place-Semantic Annotation of Tactile Maps. Li Liu, Ashmita Dua, Jiaming Qu, David T. Lee, Leilani H. Gilpin. ASSETS, Posters and Demonstrations, 2026. Link
- Pattern Discovery with Wide-Lens Analysis and Sharp-Focus Validation. Li Liu, Omar Alonso, Giorgio Ballardin. The Web Conference (WWW), Demo Track, 2026. Link
- Skills on Board: Maintaining Agent Skills as Typed Data. Li Liu, Jiaming Qu, Zhiyu Chen, Pei Zhang, Omar Alonso. Under review, 2026. Link
- “I didn’t know how to read a map, but now I can”: TouchingSpace, an Audio-Haptic Map for Blind and Low-Vision Readers. Li Liu, Yihe Wang, Jiaming Qu, Ashmita Dua, David T. Lee, Leilani H. Gilpin. arXiv preprint, 2026. Link
- Right This Way: Can VLMs Guide Us to See More to Answer Questions?. Li Liu, Diji Yang, Sijia Zhong, Kalyana Suma Sree Tholeti, Lei Ding, Yi Zhang, Leilani H. Gilpin. Neural Information Processing Systems (NeurIPS), 2024. Link
- A Tri-Light Warning System for Hospitalized COVID-19 Patients: Credibility-Based Risk Stratification for Future Pandemic Preparedness. Chuanjun Xu, Qinmei Xu, Li Liu, Mu Zhou, Zijian Xing, Zhen Zhou, Danyang Ren, Changsheng Zhou, Longjiang Zhang, Xiao Li, Xianghao Zhan, Olivier Gevaert, Guangming Lu. European Journal of Radiology Open, 2024. Link
- A Survey on Human-AI Collaboration with Large Foundation Models. Vanshika Vats, Marzia Binta Nizam, Minghao Liu, Ziyuan Wang, Richard Ho, Mohnish Sai Prasad, Vincent Titterton, Sai Venkat Malreddy, Riya Aggarwal, Yanwen Xu, Lei Ding, Jay Mehta, Nathan Grinnell, Li Liu, Sijia Zhong, Devanathan Nallur Gandamani, Xinyi Tang, Rohan Ghosalkar, Celeste Shen, Rachel Shen, Nafisa Hussain, Kesav Ravichandran, James Davis. ACM Transactions on Intelligent Systems and Technology, 2026. Link
- SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation. Yiqing Wang, Zihan Li, Zihao Wei, Jieru Mei, Li Liu, Chen Wang, Shengtian Sang, Alan Yuille, Cihang Xie, Yuyin Zhou. MICCAI, 2023. Link
- Bimodal Fusion Network for Basic Taste Sensation Recognition from Electroencephalography and Electromyography. Han Gao, Shuo Zhao, Huiyan Li, Li Liu, You Wang, Ruifen Hu, Jin Zhang, Guang Li. ICASSP, 2023. Link
- Basic Taste Sensation Recognition from EEG Based on Multi-Scale Convolutional Neural Network with Residual Learning. Han Gao, Shuo Zhao, Huiyan Li, Li Liu, Hengyang Wang, You Wang, Zhiyuan Luo, Jin Zhang, Guang Li. IEEE Transactions on Instrumentation and Measurement, 2023. Link
- Silent Speech Interface with Vocal Speaker Assistance Based on Convolution-augmented Transformer. Huiyan Li, Yihao Liang, Han Gao, Li Liu, You Wang, Daqi Chen, Zhiyuan Luo, Guang Li. IEEE Transactions on Instrumentation and Measurement, 2023. Link
- CPSC: Conformal Prediction With Shrunken Centroids for Efficient Prediction Reliability Quantification and Data Augmentation, a Case in Alternative Herbal Medicine Classification With Electronic Nose. Li Liu, Xianghao Zhan, Xikai Yang, Xiaoqing Guan, Rumeng Wu, Zhan Wang, Zhiyuan Luo, You Wang, Guang Li. IEEE Transactions on Instrumentation and Measurement, 2022. Link
- Unsupervised Cross-User Adaptation in Taste Sensation Recognition Based on Surface Electromyography with Conformal Prediction and Domain Regularized Component Analysis. Hengyang Wang, Xianghao Zhan, Li Liu, Asif Ullah, Huiyan Li, Han Gao, You Wang, Ruifen Hu, Guang Li. IEEE Transactions on Instrumentation and Measurement, 2022. Link
- Bag of Tricks for FGSM Adversarial Training. Zichao Li, Li Liu, Zeyu Wang, Yuyin Zhou, Cihang Xie. arXiv preprint, 2022. Link
- Boost AI Power: Data Augmentation Strategies With Unlabeled Data and Conformal Prediction, a Case in Alternative Herbal Medicine Discrimination With Electronic Nose. Li Liu, Xianghao Zhan, Rumeng Wu, Xiaoqing Guan, Zhan Wang, Wei Zhang, Mert Pilanci, You Wang, Zhiyuan Luo, Guang Li. IEEE Sensors Journal, 2021. Link
- Classifying Herbal Medicine Origins by Temporal and Spectral Data Mining of Electronic Nose. Li Liu, Xianghao Zhan, Ziheng Duan, Yi Wu, Rumeng Wu, Xiaoqing Guan, Zhan Wang, You Wang, Guang Li. arXiv preprint, 2021. Link
- Quantitatively Recognizing Stimuli Intensity of Primary Taste Based on Surface Electromyography. Hengyang Wang, Dongcheng Lu, Li Liu, Han Gao, Rumeng Wu, Yueling Zhou, Qing Ai, You Wang, Guang Li. Sensors, 2021. Link
This site needs JavaScript for the full experience.