# Li Liu

> Li Liu is a Ph.D. candidate in Computer Science and Engineering at UC Santa Cruz, advised by Prof. Leilani H. Gilpin. Li studies spatial intelligence in multimodal models and the agents built on them: how models perceive space, build an internal map of it, and act when what they see is not enough. Li has interned three times as an Applied Scientist at Amazon and is on the 2027 job market for Research Scientist, Applied Scientist, and ML Engineer roles.

Homepage: https://leolee7.github.io/ · Email: lliu112@ucsc.edu · Google Scholar: https://scholar.google.com/citations?user=hLHzhqkAAAAJ · GitHub: https://github.com/LeoLee7 · CV: https://leolee7.github.io/files/Li_Liu_resume.pdf

## Research
- Perceive: vision-language models answer even when an image cannot support an answer; Right This Way (NeurIPS 2024) trains them to say how to move the camera instead.
- Represent: language models read spatial layouts far better than they build them; learning to build layouts improves their spatial reasoning (Learning to Draw ASCII, COLM 2026).
- Act: TouchingSpace, a real-time voice-and-touch agent on Gemini native audio that grounds its answers in a map, built from zero and tested with 14 blind and low-vision adults.

## Projects
- [Directional Guidance for Vision-Language Models](https://leolee7.github.io/#project-right-this-way) (Right This Way, NeurIPS 2024): A new task, directional guidance, with an open 821-sample human-labeled benchmark and a label-free synthetic data pipeline. Fine-tuning raises the accuracy of open VLMs by up to 16 percentage points, and the best fine-tuned model exceeds GPT-4o.
- [Spatial Reasoning in LLMs through Layout Construction](https://leolee7.github.io/#project-ascii) (Learning to Draw ASCII, COLM 2026): Across six LLMs, accuracy in reading ASCII layouts exceeds accuracy in constructing them by 13 to 27 percentage points. Training on construction more than doubles construction accuracy, improves text-only spatial reasoning, and transfers to three external benchmarks.
- [Real-Time Multimodal Agent Grounded in Touch](https://leolee7.github.io/#project-touchingspace) (TouchingSpace, Under review, 2026): People explore a place by hand on a laptop trackpad and ask a voice agent open questions about it. Built from zero with a blind co-designer and studied with 14 blind and low-vision adults.
- [Grounding Place Names in 3D-Printed Tactile Maps](https://leolee7.github.io/#project-tactile-annotation) (Tactile Map Annotation, ASSETS 2026 Demo): A web tool that grounds place names in a 3D-printed tactile map: a sighted helper clicks a footprint, matches it to OpenStreetMap, and places print-safe Braille on the shape. Each label is a record linking print geometry to place semantics, so it can also drive an audio readout.
- [Human-AI Collaboration with Foundation Models: A Survey](https://leolee7.github.io/#project-hai-survey) (Human-AI Collaboration Survey, ACM TIST 2026): A survey of human-AI collaboration with large foundation models, covering human-guided model development, collaboration patterns, governance, and high-stakes applications.
- [Human-in-the-Loop Pattern Discovery in Unlabeled Conversations](https://leolee7.github.io/#project-pattern-insights) (Pattern Insights Explorer, WWW 2026 Demo, Amazon): A modular framework that pairs wide-lens views of where candidate patterns cluster with sharp-focus checks of whether they are meaningful, for unlabeled data without ground truth.
- [Agent Skills as Typed, Inspectable Data](https://leolee7.github.io/#project-skills-on-board) (Skills on Board, Under review, 2026): A system that turns a free-text SKILL.md into a typed meta file, rendered as a board for people and compiled into a structured skill for agents. On SkillLearnBench, agents use half the output tokens at equal or better task performance. Built during my internship at Amazon.
- [Experiential Search over User Reviews](https://leolee7.github.io/#project-ser) (SER, Amazon internship, 2024): A search system that turns customer reviews into search facets, each linked to the review sentence behind it, for needs catalogs do not record. On recipes it finds 18% more facet matches than keyword search.
- [Multi-View Self-Supervised Pre-Training for 3D Medical Segmentation](https://leolee7.github.io/#project-swinmm) (SwinMM, MICCAI 2023): Masked multi-view pre-training with Swin Transformers for accurate and data-efficient 3D medical image segmentation.
- [Efficient Conformal Prediction with Shrunken Centroids](https://leolee7.github.io/#project-cpsc) (CPSC, IEEE TIM 2022): Conformal prediction with shrunken centroids, which quantifies how reliable each prediction is at lower cost than k-nearest-neighbor conformal prediction.
- [Cross-User sEMG Domain Adaptation](https://leolee7.github.io/#project-taste-xuser) (Cross-User Taste Recognition, IEEE TIM 2022): Domain regularized component analysis and conformal prediction that let taste recognition models trained on one person work on others without new labels.
- [Data Augmentation for Label-Scarce Electronic Noses](https://leolee7.github.io/#project-boost-ai) (Boost AI Power, IEEE Sensors Journal 2021): A systematic comparison of data augmentation strategies for electronic noses with scarce labels, including a new ensemble inductive conformal prediction method.
- [Preventing Catastrophic Overfitting in FGSM Adversarial Training](https://leolee7.github.io/#project-fgsm) (Bag of Tricks for FGSM-AT, arXiv 2022): Simple tricks, such as masking partial pixels, that prevent catastrophic overfitting in single-step FGSM adversarial training.
- [Multi-Scale CNN for EEG Taste Recognition](https://leolee7.github.io/#project-taste-eeg) (EEG-MSRNet, IEEE TIM 2023): A multi-scale fully convolutional network with residual learning that recognizes basic taste sensations from EEG.
- [sEMG-to-Speech Silent Speech Interface](https://leolee7.github.io/#project-silent-speech) (Silent Speech Interface, IEEE TIM 2023): A Mandarin silent speech interface that converts surface EMG from silent speakers into audio, with help from paired EMG and audio of vocal speakers.
- [EEG-EMG Fusion for Taste Recognition](https://leolee7.github.io/#project-taste-bimodal) (Bi-FusionNet, ICASSP 2023): A bimodal network that fuses complementary EEG and EMG features to recognize basic taste sensations.
- [Conformal Risk Triage for COVID-19 from CT](https://leolee7.github.io/#project-trilight) (Tri-Light Warning System, EJR Open 2024): A conformal-prediction warning system that sorts hospitalized COVID-19 patients into high, uncertain, and low risk from CT and clinical data.
- [Taste Intensity Recognition from Facial sEMG](https://leolee7.github.io/#project-taste-intensity) (Taste Intensity from Facial sEMG, Sensors 2021): Quantitative recognition of the intensity of the five primary tastes from facial surface electromyography.
- [Temporal-Spectral Feature Extraction for Electronic Noses](https://leolee7.github.io/#project-herbal-origins) (Herbal Medicine Origins, arXiv 2021): Temporal and spectral feature extraction that lets an electronic nose distinguish the geographic origins of herbal medicines.
- [Light-Responsive Kinetic Pavilion](https://leolee7.github.io/#project-lotus-pavilion) (Lotus Pavilion, Undergraduate design, Zhejiang University): A light-responsive pavilion inspired by West Lake in Hangzhou, built from modular units that open to sunlight like lotus petals.

## Publications
- Learning to Draw ASCII Improves Spatial Reasoning in Language Models. Shiyuan Huang, Li Liu, Jincheng He, Leilani H. Gilpin. Conference on Language Modeling (COLM), 2026. https://arxiv.org/abs/2604.14641
- From Anonymous Shapes to Named Places: A Tool for Braille and Place-Semantic Annotation of Tactile Maps. Li Liu, Ashmita Dua, Jiaming Qu, David T. Lee, Leilani H. Gilpin. ASSETS, Posters and Demonstrations, 2026. https://doi.org/10.1145/3797867.3841235
- Pattern Discovery with Wide-Lens Analysis and Sharp-Focus Validation. Li Liu, Omar Alonso, Giorgio Ballardin. The Web Conference (WWW), Demo Track, 2026. https://doi.org/10.1145/3774905.3793105
- Skills on Board: Maintaining Agent Skills as Typed Data. Li Liu, Jiaming Qu, Zhiyu Chen, Pei Zhang, Omar Alonso. Under review, 2026. https://youtu.be/G_XJpzWnH_c
- “I didn’t know how to read a map, but now I can”: TouchingSpace, an Audio-Haptic Map for Blind and Low-Vision Readers. Li Liu, Yihe Wang, Jiaming Qu, Ashmita Dua, David T. Lee, Leilani H. Gilpin. arXiv preprint, 2026. https://arxiv.org/abs/2609.36404
- Right This Way: Can VLMs Guide Us to See More to Answer Questions?. Li Liu, Diji Yang, Sijia Zhong, Kalyana Suma Sree Tholeti, Lei Ding, Yi Zhang, Leilani H. Gilpin. Neural Information Processing Systems (NeurIPS), 2024. https://proceedings.neurips.cc/paper_files/paper/2024/file/efe4e50d492fedc0dfd2959f3320a974-Paper-Conference.pdf
- A Tri-Light Warning System for Hospitalized COVID-19 Patients: Credibility-Based Risk Stratification for Future Pandemic Preparedness. Chuanjun Xu, Qinmei Xu, Li Liu, Mu Zhou, Zijian Xing, Zhen Zhou, Danyang Ren, Changsheng Zhou, Longjiang Zhang, Xiao Li, Xianghao Zhan, Olivier Gevaert, Guangming Lu. European Journal of Radiology Open, 2024. https://www.sciencedirect.com/science/article/pii/S2352047724000583
- A Survey on Human-AI Collaboration with Large Foundation Models. Vanshika Vats, Marzia Binta Nizam, Minghao Liu, Ziyuan Wang, Richard Ho, Mohnish Sai Prasad, Vincent Titterton, Sai Venkat Malreddy, Riya Aggarwal, Yanwen Xu, Lei Ding, Jay Mehta, Nathan Grinnell, Li Liu, Sijia Zhong, Devanathan Nallur Gandamani, Xinyi Tang, Rohan Ghosalkar, Celeste Shen, Rachel Shen, Nafisa Hussain, Kesav Ravichandran, James Davis. ACM Transactions on Intelligent Systems and Technology, 2026. https://doi.org/10.1145/3841472
- SwinMM: Masked Multi-view with Swin Transformers for 3D Medical Image Segmentation. Yiqing Wang, Zihan Li, Zihao Wei, Jieru Mei, Li Liu, Chen Wang, Shengtian Sang, Alan Yuille, Cihang Xie, Yuyin Zhou. MICCAI, 2023. https://arxiv.org/abs/2307.12591
- Bimodal Fusion Network for Basic Taste Sensation Recognition from Electroencephalography and Electromyography. Han Gao, Shuo Zhao, Huiyan Li, Li Liu, You Wang, Ruifen Hu, Jin Zhang, Guang Li. ICASSP, 2023. https://ieeexplore.ieee.org/document/10095117
- Basic Taste Sensation Recognition from EEG Based on Multi-Scale Convolutional Neural Network with Residual Learning. Han Gao, Shuo Zhao, Huiyan Li, Li Liu, Hengyang Wang, You Wang, Zhiyuan Luo, Jin Zhang, Guang Li. IEEE Transactions on Instrumentation and Measurement, 2023. https://ieeexplore.ieee.org/abstract/document/10138514/
- Silent Speech Interface with Vocal Speaker Assistance Based on Convolution-augmented Transformer. Huiyan Li, Yihao Liang, Han Gao, Li Liu, You Wang, Daqi Chen, Zhiyuan Luo, Guang Li. IEEE Transactions on Instrumentation and Measurement, 2023. https://ieeexplore.ieee.org/document/10121584
- CPSC: Conformal Prediction With Shrunken Centroids for Efficient Prediction Reliability Quantification and Data Augmentation, a Case in Alternative Herbal Medicine Classification With Electronic Nose. Li Liu, Xianghao Zhan, Xikai Yang, Xiaoqing Guan, Rumeng Wu, Zhan Wang, Zhiyuan Luo, You Wang, Guang Li. IEEE Transactions on Instrumentation and Measurement, 2022. https://ieeexplore.ieee.org/abstract/document/9667498
- Unsupervised Cross-User Adaptation in Taste Sensation Recognition Based on Surface Electromyography with Conformal Prediction and Domain Regularized Component Analysis. Hengyang Wang, Xianghao Zhan, Li Liu, Asif Ullah, Huiyan Li, Han Gao, You Wang, Ruifen Hu, Guang Li. IEEE Transactions on Instrumentation and Measurement, 2022. https://ieeexplore.ieee.org/document/9777907
- Bag of Tricks for FGSM Adversarial Training. Zichao Li, Li Liu, Zeyu Wang, Yuyin Zhou, Cihang Xie. arXiv preprint, 2022. https://arxiv.org/abs/2209.02684
- Boost AI Power: Data Augmentation Strategies With Unlabeled Data and Conformal Prediction, a Case in Alternative Herbal Medicine Discrimination With Electronic Nose. Li Liu, Xianghao Zhan, Rumeng Wu, Xiaoqing Guan, Zhan Wang, Wei Zhang, Mert Pilanci, You Wang, Zhiyuan Luo, Guang Li. IEEE Sensors Journal, 2021. https://ieeexplore.ieee.org/abstract/document/9505688
- Classifying Herbal Medicine Origins by Temporal and Spectral Data Mining of Electronic Nose. Li Liu, Xianghao Zhan, Ziheng Duan, Yi Wu, Rumeng Wu, Xiaoqing Guan, Zhan Wang, You Wang, Guang Li. arXiv preprint, 2021. https://arxiv.org/abs/2104.06640
- Quantitatively Recognizing Stimuli Intensity of Primary Taste Based on Surface Electromyography. Hengyang Wang, Dongcheng Lu, Li Liu, Han Gao, Rumeng Wu, Yueling Zhou, Qing Ai, You Wang, Guang Li. Sensors, 2021. https://www.mdpi.com/1424-8220/21/21/6965