Kevin Yang

yangk at berkeley dot edu // Scholar // GitHub // Curriculum Vitae

I am a research scientist at Scaled Cognition. I am broadly interested in structured planning, control, and alignment/instruction-tuning methods for improving the quality and safety of natural language generation, especially for long context.

I completed my PhD at UC Berkeley, advised by Professor Dan Klein. I previously did a Master's at MIT, advised by Professor Regina Barzilay.

publications

RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment
Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, Yuandong Tian
arxiv 2023; to be submitted soon
We propose a new method for simulating preference data in RLHF alignment pipelines based on generating preference pairs from two contrasting prompts, with strong downstream performance on three diverse alignment tasks and multiple LLaMA model scales.
Citation
```
@article{yang2023rlcd,
                        title={RLCD: Reinforcement Learning from Contrast Distillation for Language Model Alignment},
                        author={Yang, Kevin and Klein, Dan and Celikyilmaz, Asli and Peng, Nanyun and Tian, Yuandong},
                        journal={https://arxiv.org/abs/2307.12950},
                        year={2023}
                      }
                    
```
DOC: Improving Long Story Coherence With Detailed Outline Control
Kevin Yang, Dan Klein, Nanyun Peng, Yuandong Tian
ACL 2023
We improve coherence in several-thousand-word-long stories by constructing a more detailed outline and improving the generator's ability to stay faithful to that outline. Humans prefer DOC to our previous Re3 system by a wide margin in both automatic and interactive generation.
Citation
```
@article{yang2022doc,
                        title={DOC: Improving Long Story Coherence With Detailed Outline Control},
                        author={Yang, Kevin and Klein, Dan and Peng, Nanyun and Tian, Yuandong},
                        journal={arXiv preprint arXiv:2212.10077},
                        year={2022}
                      }
                    
```

Modular Visual Question Answering via Code Generation
Sanjay Subramanian, Medhini Narasimhan, Kushal Khangaonkar, Kevin Yang et al.
ACL 2023
We improve performance on visual question answering tasks requiring multi-step reasoning by synthesizing programs to compose logical reasoning steps, including calling sub-tools.
Citation

@article{subramanian2023modular,
                        title={Modular Visual Question Answering via Code Generation},
                        author={Subramanian, Sanjay and Narasimhan, Medhini and Khangaonkar, Kushal and Yang, Kevin and Nagrani, Arsha and Schmid, Cordelia and Zeng, Andy and Darrell, Trevor and Klein, Dan},
                        journal={arXiv preprint arXiv:2306.05392},
                        year={2023}
                      }

PREADD: Prefix-Adaptive Decoding for Controlled Text Generation
Jonny Pei, Kevin Yang, Dan Klein
Findings of ACL 2023
We propose a simple, no-training-required approach for modulating the control strength exerted through prompting, by contrasting the logit distributions induced by two contrasting prompts. We achieve strong results on toxic output mitigation, bias reduction, and sentiment control.
Citation
```
@article{pei2023preadd,
                        title={PREADD: Prefix-Adaptive Decoding for Controlled Text Generation},
                        author={Pei, Jonathan and Yang, Kevin and Klein, Dan},
                        journal={arXiv preprint arXiv:2307.03214},
                        year={2023}
                      }
                    
```

Predicting Compound Activity from Phenotypic Profiles and Chemical Structures
Nikita Moshkov, Tim Becker, Kevin Yang, Peter Horvath et al.
Nature Communications 2023
We investigate the effectiveness of three different data sources—-chemical structures, imaging data, and gene-expression profiles—-for predicting compound activity in laboratory assays.
Citation

@article{moshkov2023predicting,
                        title={Predicting compound activity from phenotypic profiles and chemical structures},
                        author={Moshkov, Nikita and Becker, Tim and Yang, Kevin and Horvath, Peter and Dancik, Vlado and Wagner, Bridget K and Clemons, Paul A and Singh, Shantanu and Carpenter, Anne E and Caicedo, Juan C},
                        journal={Nature Communications},
                        volume={14},
                        number={1},
                        pages={1967},
                        year={2023},
                        publisher={Nature Publishing Group UK London}
                      }

Re3: Generating Longer Stories With Recursive Reprompting and Revision
Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein
EMNLP 2022
We automatically generate coherent stories, with a consistent overarching plot, by simulating the human writing process and repeatedly reinjecting relevant context into large pretrained language models via structured prompting. The stories are 2000+ words, or even up to 7500.
Citation
```
@article{yang2022re3,
                        title={Re3: Generating Longer Stories With Recursive Reprompting and Revision},
                        author={Yang, Kevin and Tian, Yuandong and Peng, Nanyun and Klein, Dan},
                        journal={Empirical Methods in Natural Language Processing},
                        year={2022}
                      }
                    
```

Automated Crossword Solving
Eric Wallace*, Nicholas Tomlin*, Albert Xu*, Kevin Yang*, Eshaan Pathak*, Matthew Ginsberg, Dan Klein
ACL 2022
We create a system for automatically solving crossword puzzles, and achieve superhuman performance for the first time.
Citation

@article{wallace2022automated,
                        title={Automated Crossword Solving},
                        author={Wallace, Eric and Tomlin, Nicholas and Xu, Albert and Yang, Kevin and Pathak, Eshaan and Ginsberg, Matthew and Klein, Dan},
                        journal={ACL},
                        year={2022}
                      }

Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentation
Kevin Yang, Olivia Deng, Charles Chen, Richard Shin, Subhro Roy, Benjamin Van Durme
Findings of ACL 2022
We propose a data augmentation scheme for low-resource semantic parsing in complex realistic environments, which simultaneously maintains user privacy.
Citation

@article{yang2022addressing,
                        title={Addressing Resource and Privacy Constraints in Semantic Parsing Through Data Augmentation},
                        author={Yang, Kevin and Deng, Olivia and Chen, Charles and Shin, Richard and Roy, Subhro and Van Durme, Benjamin},
                        journal={ACL},
                        year={2022}
                      }

Multi-Objective Optimization by Learning Space Partitions
Yiyang Zhao, Linnan Wang, Kevin Yang, Tianjun Zhang, Tian Guo, Yuandong Tian
ICLR 2022
We propose a space-partitioning search algorithm for finding the Pareto frontier in multi-objective optimization problems.
Citation

@article{zhao2021multi,
                        title={Multi-objective Optimization by Learning Space Partitions},
                        author={Zhao, Yiyang and Wang, Linnan and Yang, Kevin and Zhang, Tianjun and Guo, Tian and Tian, Yuandong},
                        journal={International Conference on Learning Representations},
                        year={2021}
                      }

Learning Space Partitions for Path Planning
Kevin Yang*, Tianjun Zhang*, Chris Cummins, Brandon Cui et al.
NeurIPS 2021
We propose a path planning method inspired by a theoretical analysis of search space partitioning, and show strong performance on difficult multimodal, long-horizon path planning problems.
Citation

@article{yang2021learning,
                            title={Learning Space Partitions for Path Planning},
                            author={Yang, Kevin and Zhang, Tianjun and Cummins, Chris and Cui, Brandon and Steiner, Benoit and Wang, Linnan and Gonzalez, Joseph E and Klein, Dan and Tian, Yuandong},
                            journal={NeurIPS},
                            year={2021}
                        }

FUDGE: Controlled Text Generation with Future Discriminators
Kevin Yang and Dan Klein
NAACL 2021
We propose a simple, flexible and highly effective method for controlling generation toward desired attributes using lightweight classifiers.
Citation

@article{yang2021fudge,
                            title={FUDGE: Controlled Text Generation With Future Discriminators},
                            author={Yang, Kevin and Klein, Dan},
                            journal={NAACL},
                            year={2021}
                          }

A Streaming Approach for Efficient Batched Beam Search
Kevin Yang, Violet Yao, John DeNero, Dan Klein
EMNLP 2020
We propose an efficient batching strategy for variable-length decoding on GPU architectures, demonstrating substantial speedups over existing fixed-width and variable-width beam searches.
Citation

@article{yang2020streaming,
                        title={A streaming approach for efficient batched beam search},
                        author={Yang, Kevin and Yao, Violet and DeNero, John and Klein, Dan},
                        journal={Empirical Methods in Natural Language Processing},
                        year={2020}
                        }

Improving Molecular Design by Stochastic Iterative Target Augmentation
Kevin Yang, Wengong Jin, Kyle Swanson, Regina Barzilay, and Tommi Jaakkola
ICML 2020
We use a simple and theoretically motivated self-training approach guided by an external property predictor to significantly improve upon state-of-the-art approaches in molecular design.
Citation

@article{yang2020improving,
                        title={Improving molecular design by stochastic iterative target augmentation},
                        author={Yang, Kevin and Jin, Wengong and Swanson, Kyle and Barzilay, Regina and Jaakkola, Tommi},
                        journal={International Conference on Machine Learning},
                        year={2020}
                      }

Uncertainty Quantification Using Neural Networks for Molecular Property Prediction
Lior Hirschfeld, Kyle Swanson, Kevin Yang, Regina Barzilay, and Tommi Jaakkola
JCIM 2020
We comprehensively evaluate and compare several approaches for uncertainty estimation in neural models on molecular property prediction tasks.
Citation

@article{hirschfeld2020uncertainty,
                        title={Uncertainty quantification using neural networks for molecular property prediction},
                        author={Hirschfeld, Lior and Swanson, Kyle and Yang, Kevin and Barzilay, Regina and Coley, Connor W},
                        journal={Journal of Chemical Information and Modeling},
                        volume={60},
                        number={8},
                        pages={3770--3780},
                        year={2020},
                        publisher={ACS Publications}
                      }

A Deep Learning Approach to Antibiotic Discovery
Jonathan Stokes, Kevin Yang, Kyle Swanson, Wengong Jin et al.
Cell 2020
We use computational property prediction models to screen drug databases for potential antibiotic activity, and discover new antibiotics with novel mechanisms of action which are effective even against bacteria which are resistant to commonly used antibiotics.
Citation

@article{stokes2020deep,
                        title={A deep learning approach to antibiotic discovery},
                        author={Stokes, Jonathan M and Yang, Kevin and Swanson, Kyle and Jin, Wengong and Cubillos-Ruiz, Andres and Donghia, Nina M and MacNair, Craig R and French, Shawn and Carfrae, Lindsey A and Bloom-Ackerman, Zohar and others},
                        journal={Cell},
                        volume={180},
                        number={4},
                        pages={688--702},
                        year={2020},
                        publisher={Elsevier}
                      }

Analyzing Learned Molecular Representations for Property Prediction
Kevin Yang*, Kyle Swanson*, Wengong Jin, Connor Coley et al.
JCIM 2019
We carefully benchmark models for molecular property prediction on both public and proprietary industry datasets. In addition, we introduce a new variant of message-passing neural networks and demonstrate consistently strong performance that significantly improves over existing baselines on many datasets.
Citation

@article{yang2019analyzing,
                        title={Analyzing learned molecular representations for property prediction},
                        author={Yang, Kevin and Swanson, Kyle and Jin, Wengong and Coley, Connor and Eiden, Philipp and Gao, Hua and Guzman-Perez, Angel and Hopper, Timothy and Kelley, Brian and Mathea, Miriam and others},
                        journal={Journal of Chemical Information and Modeling},
                        volume={59},
                        number={8},
                        pages={3370--3388},
                        year={2019},
                        publisher={ACS Publications}
                      }

Learning Multimodal Graph-to-Graph Translation for Molecular Optimization
Wengong Jin, Kevin Yang, Regina Barzilay, and Tommi Jaakkola
ICLR 2019
We introduce an encoder-decoder architecture for molecular optimization that operates directly on the molecular graph, and demonstrate that it significantly outperforms string-based baselines as well as pre-existing SOTA.
Citation

@article{jin2018learning,
                        title={Learning multimodal graph-to-graph translation for molecular optimization},
                        author={Jin, Wengong and Yang, Kevin and Barzilay, Regina and Jaakkola, Tommi},
                        journal={International Conference on Learning Representations},
                        year={2018}
                      }

Thanks Mohit and Eric for the website template