CAIT: A Syntactic Parsing Toolkit for Child–Adult In Teractions
Open Research objectives/practices
-
Making the outputs of research freely available
-
Increasing the transparency of research processes and methodologies
-
Using open collaborative methods and tools
-
Capitalizing on unique open aspects of open educational resources
-
Promoting and facilitating Open Research and Education practices
Introduction
Together with my team members, I developed CAIT, a syntactic annotation toolkit for child-directed and child-produced speech, a data domain that most NLP tools are not designed to handle well, since they're trained and tuned on adult, written text. CAIT bundles three components: a dependency parser trained on the gold-standard UD_English-CHILDES annotations, a part-of-speech tagger, and an utterance-level construction tagger that classifies child-adult interactions into linguistically meaningful categories (such as formulaic expressions, wh-questions, or imperatives). We originally built this pipeline to support our own research on child language acquisition, but rather than keeping it as an internal tool, we decided to release the full pipeline openly, so that other researchers working with the same domain would have access to it. Open practices addressed:
-
Making research outputs freely accessible ->The parser, the POS tagger, the construction tagger, and all training and testing code are hosted on a public GitHub repository with documentation on how to install, run, and apply the toolkit components. The models themselves are also released separately on Hugging Face for easy download. The paper describing CAIT is open access and links directly to both the repository and the model page, so anyone reading the paper can immediately access and reuse the tools.
-
Increasing transparency of research processes and methodologies -> We didn't just release the final models, we documented and shared the training and evaluation code, the different model configurations we tested, and the evaluation scores for each, including where the parser performs worse. As part of this, we carried out a post hoc error analysis of the best parser's mistakes, which surfaced a number of inconsistencies in the underlying UD_English-CHILDES gold annotations. We reported these to the dataset's curators, who acknowledged them and plan to incorporate corrections into a future release of the data. The transparency of our work is feeding directly back into a shared community resource.
-
Open collaborative methods and tools -> CAIT relies entirely on open-source Python libraries throughout its implementation, and adopts Universal Dependencies, a widely used, community-maintained formalism for consistent syntactic annotation across languages. The paper describing CAIT went through the peer-review process at CoNLL, a leading venue for theoretically and cognitively motivated computational linguistics research, and was accepted and presented there.
-
Promoting and facilitating Open Research practices -> Beyond releasing the tool, we actively worked to spread awareness of it so it would actually get used. We presented CAIT at a local conference, at a conference in Nijmegen, and, through a collaborator, at CONLL in San Diego (California), specifically to reach researchers who work with this kind of data and encourage them to use the toolkit when needed. I have also brought the tool into teaching. As a TA for a course in the Information Science Bachelor's program at the Faculty of Arts, I introduced CAIT and its accompanying paper during a lecture on dependency parsing, using it as a concrete example of Universal Dependencies and open access tool ready to use for the students.
Motivation
No such tool existed in the literature for child-adult interactions. Available parsers were trained and tuned for adult writtent, text, making them unsuitable for this domain. We initially trained our own parser simply to answer specific questions in my PhD research. But we realized other researchers in the NLP and Language
Acquisition communities would likely face the same gap, so we extended the work; comparing different parsers, adding a full error analysis and a case study, and releasing everything open source so the wider community could reuse and build on it.