Research Interests

To maintain robustness and control in adopting advanced black-box neural technologies, such as foundation models, domain-specific conversational systems can gain advantages from employing interpretable modular designs. This involves breaking down tasks into logically abstracted or theory-grounded sub-modules aligned with our desired level of control. My research focues on addressing the challenges:

• Modularized NLP. To connect each module and make a coherent system, my research studies deep structured prediction for extracting symbolic representations from text (Dissertation’22, CoNLL’19, ACL’19, NAACL’21, IWSDS’23) and conditional generation with augmented-memory, and symbolic constraings(DSTC7’19, ACL’19, BEA’23). I especially interested in managing the uncertain symbolic/declarative controls via neural-symbolic mechanism, such as discrete latent variable optimization via variational inference, continuous relaxation etc, such as cross-framework parsing with both explicit and implicit alignments(CoNLL’19), and rhetorically controlled poetry generation(ACL’19).
• Learning with Inductive Biases. Beyond the expensive supervised data in many domains~(e.g., psychotherapy, teaching, tutoring, small-group collaboration), we investigate learning with domain-specific knowledges as inductive biases, such as graph-based parsing via anchoring analysis(CoNLL’19), database workload characterization via self-supervised learning (VLDB’22), zero-shot dialogue state tracking via description-driven learning and supplementary pretraining (NAACL’21).
• Deployment in Real-world Scenarios. Unexpected, complex dialogue scenarios can potentially span infinitely many topics, states and environmental settings(ACL’23). We study robustness issues when deploying dialogue system in our daily life, such as in-classroom AI partner (AIAIC’23), noisy speech (UMAP’23), multi-party, multi-modal dynamics (IWSDS’23), and conversational simulation.

News

• 11/2023: In Fall 2023, I taught NLP class~(CSCI-LING 5832) with James Martin. I newly created course materials on LLMs, In-Context Learning, Dialogue Generation, etc.
• 05/2023: Our paper on Question Generation accepted to BEA’23
• 05/2023: A short paper on “Mind the Gap between the Application Track and the Real World” got accepted to ACL’23
• 04/2023: Our paper on “A Comparative Analysis of Automatic Speech Recognition Errors in Small Group Classroom Discourse” got accepted to UMAP’23.
• 03/2023: #FirstGrant My research proposal on conversational simulation on small-group discussion got awarded by iSAT Trainee Grant.
• 02/2023: Our paper on AI agent for Jigsaw Classrooms got accepted on AIAIC’23.
• 12/2022: Our paper on Dependency Dialog Act got accepted on IWSDS’23.
• 12/2022: Invited Talk on Database Workload Characterization work at Microsoft’s Gray Systems Lab. Slides.
• 08/2022: I joined NSF AI Institute for Student-AI Teaming (iSAT) as a post-doctoral researcher.
• 06/2022: New preprint on visual analysis of neural network pruning.

Selected Publications

• E. Margaret Perkoff, Abhidip Bhattacharyya, Jon Cai, and Jie Cao. 2023. Comparing Neural Question Generation Architectures for Reading Comprehension. 18th Workshop on Innovative Use of NLP for Building Educational Applications, 2023.
  • BibTeX

    @article{qg-bea23,
      title = {Comparing Neural Question Generation Architectures for Reading
          Comprehension},
      author = {Perkoff, E. Margaret and Bhattacharyya, Abhidip and Cai, Jon and Cao, Jie},
      journal = {18th Workshop on Innovative Use of NLP for Building Educational
          Applications, 2023},
      year = {2023}
    }
    

  • Paper
• Ananya Ganesh, Jie Cao, E. Magerate Perkoff, Rosy Southwell, Martha Palmer, and Katharina Kann. 2023. Mind the Gap between the Application Track and the Real World. Proceedings of the 61th Annual Meeting of the Association for Computational Linguistics, 2023.
  • BibTeX

    @article{ananya-acl23,
      title = {Mind the Gap between the Application Track and the Real World},
      author = {Ganesh, Ananya and Cao, Jie and Perkoff, E. Magerate and Southwell, Rosy and Palmer, Martha and Kann, Katharina},
      journal = {Proceedings of the 61th Annual Meeting of the Association for
          Computational Linguistics, 2023},
      year = {2023}
    }
    

  • Paper
• Jie Cao, Ananya Ganesh, Jon Cai, Rosy Southwell, Magerate Perkoff, Michael Regan, Katharina Kann, James Martin, Martha Palmer, and Sideny D’Mello. 2023. A Comparative Analysis of Automatic Speech Recognition Errors in Small Group Classroom Discourse. Proceedings of the 31st ACM Conference on User Modeling Adaptation and Personalization (ACM UMAP 2023).
  • BibTeX

    @article{cao-umap23,
      title = {A Comparative Analysis of Automatic Speech Recognition Errors in Small
        Group Classroom Discourse},
      author = {Cao, Jie and Ganesh, Ananya and Cai, Jon and Southwell, Rosy and Perkoff, Magerate and Regan, Michael and Kann, Katharina and Martin, James and Palmer, Martha and D'Mello, Sideny},
      journal = {Proceedings of the 31st ACM Conference on User Modeling Adaptation
          and Personalization (ACM UMAP 2023)},
      year = {2023}
    }
    

  • Paper
• Jie Cao, Rachel Dickler, Marie Grace, Alessandro Roncone, Leanne Hirshfield, Marilyn Walker, and Martha Palmer. 2023. Designing an AI Partner for Jigsaw Classrooms. Workshop on Language-Based AI Character Interation with Children.
  • BibTeX

    @article{cao-jigsaw23,
      title = {Designing an AI Partner for Jigsaw Classrooms},
      author = {Cao, Jie and Dickler, Rachel and Grace, Marie and Roncone, Alessandro and Hirshfield, Leanne and Walker, Marilyn and Palmer, Martha},
      journal = {Workshop on Language-Based AI Character Interation with Children},
      year = {2023}
    }
    

  • Paper
• Jon Cai, Brendan D. King, Margaret Perkoff, Shiran Dudy, Jie Cao, Marie Grace, Natalia Wojarnik, Ganesh Ananya, James Martin, Martha Palmer, Marilyn Walker, and Jeffrey Flanigan. 2022. Dependency Dialogue Acts — Annotation Scheme and Case Study. The 13th International Workshop on Spoken Dialogue Systems Technology.
  • BibTeX

    @article{jon-dda2022,
      title = {Dependency Dialogue Acts — Annotation Scheme and Case Study},
      author = {Cai, Jon and King, Brendan D. and Perkoff, Margaret and Dudy, Shiran and Cao, Jie and Grace, Marie and Wojarnik, Natalia and Ananya, Ganesh and Martin, James and Palmer, Martha and Walker, Marilyn and Flanigan, Jeffrey},
      journal = {The 13th International Workshop on Spoken Dialogue Systems Technology},
      year = {2022}
    }
    

  • Paper
• Jie Cao. 2022. Inductive Biases for Deep Linguistic Structured Prediction with Independent Factorization. Available from ProQuest Dissertations & Theses A&I;ProQuest Dissertations & Theses Global. (2777357718).
  • BibTeX

    @article{dissertation-proquest,
      title = {Inductive Biases for Deep Linguistic Structured Prediction with
          Independent Factorization},
      author = {Cao, Jie},
      journal = {Available from ProQuest Dissertations & Theses A&I;ProQuest
          Dissertations & Theses Global. (2777357718)},
      year = {2022}
    }
    

  • Paper
• Zhimin Li, Shusen Liu, Xin Yu, Kailkhura Bhavya, Jie Cao, Diffenderfer James Daniel, Peer-Timo Bremer, and Valerio Pascucci. 2022. "Understanding Robustness Lottery": A Comparative Visual Analysis of Neural Network Pruning Approaches. arXiv preprint arXiv:2206.07918.
  • BibTeX

    @article{vis-pruning2022,
      title = {"Understanding Robustness Lottery": A Comparative Visual Analysis of
          Neural Network Pruning Approaches},
      author = {Li, Zhimin and Liu, Shusen and Yu, Xin and Bhavya, Kailkhura and Cao, Jie and Daniel, Diffenderfer James and Bremer, Peer-Timo and Pascucci, Valerio},
      journal = {arXiv preprint arXiv:2206.07918},
      year = {2022}
    }
    

  • Paper
• Debjyoti Paul*, Jie Cao*, Feifei Li, and Vivek Srikumar. 2021. Database workload characterization with query plan encoders. Proceedings of the VLDB Endowment, 15(4):923–935.
  • BibTeX

    @article{cao2021dbqencoder,
      title = {Database workload characterization with query plan encoders},
      author = {{Debjyoti Paul}* and {Jie Cao}* and Li, Feifei and Srikumar, Vivek},
      journal = {Proceedings of the VLDB Endowment},
      volume = {15},
      number = {4},
      pages = {923--935},
      year = {2021},
      publisher = {VLDB Endowment}
    }
    

  • Paper
• Jie Cao and Yi Zhang. 2021. A Comparative Study on Schema-Guided Dialogue State Tracking. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 782–796.
  • BibTeX

    @inproceedings{cao2021comparative,
      title = {A Comparative Study on Schema-Guided Dialogue State Tracking},
      author = {Cao, Jie and Zhang, Yi},
      booktitle = {Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies},
      pages = {782--796},
      year = {2021}
    }
    

  • Paper
  • Poster
• Jie Cao, Yi Zhang, Adel Youssef, and Vivek Srikumar. 2019. Amazon at MRP 2019: Parsing Meaning Representations with Lexical and Phrasal Anchoring. In Proceedings of the Shared Task on Cross-Framework Meaning Representation Parsing at the Conference on Natural Language Learning(CoNLL), pages 138–148.
  • BibTeX

    @inproceedings{cao2019amazon,
      title = {Amazon at MRP 2019: Parsing Meaning Representations with Lexical and
          Phrasal Anchoring},
      author = {Cao, Jie and Zhang, Yi and Youssef, Adel and Srikumar, Vivek},
      booktitle = {Proceedings of the Shared Task on Cross-Framework Meaning
          Representation Parsing at the Conference on Natural Language
            Learning(CoNLL)},
      pages = {138--148},
      year = {2019}
    }
    

  • Paper
• Jie Cao, Michael Tanana, Zac Imel, Eric Poitras, David Atkins, and Vivek Srikumar. 2019. Observing Dialogue in Therapy: Categorizing and Forecasting Behavioral Codes. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
  • BibTeX

    @inproceedings{cao2019observing,
      title = {Observing Dialogue in Therapy: Categorizing and Forecasting Behavioral Codes},
      author = {Cao, Jie and Tanana, Michael and Imel, Zac and Poitras, Eric and Atkins, David and Srikumar, Vivek},
      booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
      year = {2019},
      location = {Florence, Italy}
    }
    

  • Paper
  • Slides
• Zhiqiang Liu, Zuohui Fu, Jie Cao, Gerard de Melo, Yik-Cheung Tam, Cheng Niu, and Jie Zhou. 2019. Rhetorically Controlled Encoder-Decoder for Modern Chinese Poetry Generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
  • BibTeX

    @inproceedings{rhetorical-poetry2019,
      title = {Rhetorically Controlled Encoder-Decoder for Modern Chinese Poetry Generation},
      author = {Liu, Zhiqiang and Fu, Zuohui and Cao, Jie and {de Melo}, Gerard and Tam, Yik-Cheung and Niu, Cheng and Zhou, Jie},
      booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
      year = {2019},
      location = {Florence, Italy}
    }
    

  • Paper
• Shuo, Sun*, Yik-Cheung Tam*, Jie Cao*, Canxiang Yan, Zuohui Fu, Cheng Niu, and Jie Zhou. 2019. End-to-end Gated Self-attentive Memory Network for Dialog Response Selection. In AAAI DSTC7 Workshop (Equal Contribution).
  • BibTeX

    @inproceedings{jie2019dstc,
      title = {End-to-end Gated Self-attentive Memory Network for Dialog Response Selection},
      author = {{Shuo, Sun}* and {Yik-Cheung Tam}* and {Jie Cao*} and {Canxiang Yan} and {Zuohui Fu} and {Cheng Niu} and {Jie Zhou}},
      booktitle = {AAAI DSTC7 Workshop (Equal Contribution)},
      year = {2019},
      location = {Honolulu, American}
    }
    

  • Paper
  • Poster
• Xijiang Ke, Hai Jin, Xia Xie, and Jie Cao. 2015. A distributed SVM method based on the iterative MapReduce. In Semantic Computing (ICSC), IEEE International Conference on, pages 116–119. IEEE.
  • BibTeX

    @inproceedings{ke2015distributed,
      title = {A distributed SVM method based on the iterative MapReduce},
      author = {Ke, Xijiang and Jin, Hai and Xie, Xia and Cao, Jie},
      booktitle = {Semantic Computing (ICSC), IEEE International Conference on},
      pages = {116--119},
      year = {2015},
      organization = {IEEE}
    }
    

  • Paper
• Xia Xie, Jie Cao, Hai Jin, Xijiang Ke, and Wenzhi Cao. 2012. JRBridge: A framework of large-scale statistical computing for R. In Services Computing Conference (APSCC), IEEE Asia-Pacific, pages 27–34. IEEE.
  • BibTeX

    @inproceedings{xie2012jrbridge,
      title = {JRBridge: A framework of large-scale statistical computing for R},
      author = {Xie, Xia and Cao, Jie and Jin, Hai and Ke, Xijiang and Cao, Wenzhi},
      booktitle = {Services Computing Conference (APSCC), IEEE Asia-Pacific},
      pages = {27--34},
      year = {2012},
      organization = {IEEE}
    }
    

  • Paper

Research Experience

• [09/2022 - Now ] Postdoctoral Research Associate at NSF AI Institute for Student-AI Teaming(iSAT), CU Boulder.
• [08/2015 - 08/2022] Research Assistant at Utah NLP Lab, Univeristy of Utah, Salt Lake City
• [06/2020 - 12/2020] Applied Scientist Intern at AWS AI, Amazon Lex, Remote
  • Our paper on schema-guided dialog got accepted by NAACL 2021.
• [06/2019 - 09/2019] Applied Scientist Intern at AWS AI, Amazon Lex, Seattle
  • In CoNLL shared task MRP 2019, over 16 teams, our system on cross-framework meaning representation parsing ranked 1st in AMR parsing task, 5th in UCCA, 6th and 7th in PSD and DM tasks. Spotlight Talk
• [05/2018 - 08/2018] Research Intern at Tecent, WechatAI, Palo Alto
  • Our dialogue system based Gated Attentive Memory Network ranked Top 2 in DSTC7, and got accepted by AAAI 2019 DSTC7 workshop.
• [09/2008 - 03/2012] Research Assistant at CGCL Lab, Huazhong University of Science and Technology, Wuhan
  • I worked closely with Prof. Xia Xie and Prof. Hai Jin. My research interests are widely around Xen, Xen-ARM virtualization, and distributed computing. We study equipping R language with JVM-based large scale distributed statistical infrastructure, such as Hadoop, Spark.

Work Experience

• [10/2014 - 07/2015] Assistant Researcher, SOHU RDC Lab, Beijing
  • Hadoop, Spark, Data migration, Data security, Distributed machine learning
• [07/2013 - 06/2014] Senior Software Engineer, ZUN CLUB (Startup), Beijing
  • Heterogeneous data intergration, Hotel recommendation system.
• [03/2012 - 06/2013] Software Engineer, Baidu, Beijing
  • Voice Assistant, Mobile Search, Speed optimization, Mobile Anti-Attack
• [08/2010 - 05/2011] Software Engineer Intern, Alibaba, Hangzhou
  • KV Storage, MySQL, Database Replication, Real-time Computing, Distributed Pub/Sub Data Pipeline.

Teaching & Mentoring

• Fall 2023, U of Colorado Boulder, Instructor, co-teaching (CSCI-LING 5832) with James Martin.
• 2019-2020, U of Utah, Mentoring Tarun Sunkaraneni, Bachelor Thesis on ‘Transformer-based Observers in Psychotherapy’
• Fall 2018, U of Utah, TA for CS 6350 Machine Learning
• Spring 2019, U of Utah, TA for CS 6355 Structured Prediction
• Fall 2016, U of Utah, TA for CS 6350 Machine Learning
• 2007-2008, HUST, Leader for Algorithm & Game Team in a student innovation organization, Unique Studio

Academic Service

• PC Member / Reviewers for MRP’2019, BEA’23, NLP4ConvAI’23, AmericasNLP’23, SLaTE’23, ACL’20-23, EMNLP’20-23, NAACL’21, EACL’21,24, COLING’(20,22,24), AAAI’19-24, ACL Rolling Review’21-23, AIED’23, EDM’24

Honors and Awards

• [2023] iSAT Trainee Grant 2023 on Conversational Simulation on Small-group Discussion.
• [2019] CoNLL Shared Task, Cross-framework meaning representation parsing, ranked 1st(over 16 teams) for AMR parsing task.
• [2018] DSTC7 track1, ranked 2nd for both advising and ubuntu in subtask 5(with external knowledge)
• [2015] Our system ‘Talking Geckos’ winned 1st in a question-answering competition during Fall 2015 NLP class.
• [2010] VMware Cloud Computing Innovation Cup, Top 50
• [2009] Google Andriod Innovative Idea Sharing Award
• [2007] “Computer World” Magazine Scholarship (50 students per year in China)
• [2007] Microsoft ImagineCup
  • Algorithm Challenge, Top 50
  • Visual Gaming Contest(Project Hoshimi), Top 2 in China, 18th in world final.
• [2006] HUST ACM Programming Contest, Top 3