Publications

OcWikiDialects: A Wikipedia Dataset With Rich Metadata for Occitan Dialect Identification

Occitan is a Romance language spoken mostly in the South of France and characterised by rich dialectal variation, which can pose problems for certain NLP tools. This shortfall is largely attributable to the scarcity of dialect-annotated corpora, in a context where linguistic classification within the Occitan dialect continuum is still debated and major nomenclatures, such as ISO 639, fail to provide granular codes for varieties below the generic “Occitan” label. In this paper, we introduce OcWikiDialects, a new dataset comprising articles from the Occitan Wikipedia. The corpus features rich metadata, including dialect labels, and is segmented at both paragraph and sentence levels. Combined with previously released datasets, we explore approaches for Occitan dialect identification by training three types of model on up to 8 labels: linear SVM classifiers based on word and character n-grams, FastText classifiers based on pretrained vectors, and BERT-based neural classifiers adapted through fine-tuning. Evaluations across in- and out-of-domain test sets demonstrate the substantial impact of our new dataset for the task. However, a peak macro-averaged F1 score of 58.15 underscores persistent challenges for underrepresented Occitan varieties, supported by our per-dialect analysis. Code, dataset and models are available: https://github.com/DEFI-COLaF/OcWikiDialects.

Français Tirailleur and Tây Bồi: Institution-Driven Pidginization?

For decades, the pidgin-creole life cycle hypothesis was considered the standard account for the development of both pidgins and creoles. Key aspects of this account include the belief that pidgins emerge from communicative necessity in multilingual environments, while creoles are the outcomes of the stabilization of pidgins, especially via nativization. This account has been critiqued from various directions, with French-based Creoles often serving as the crux of the debate. For these languages, a major point of contention has been the similarity between the languages of the Americas and the Indian Ocean despite the apparent absence of a stable precursor pidgin. For example, McWhorter (2000) defends the life cycle model by hypothesizing that French- based Creoles ultimately descend from an unattested 17th century slave castle pidgin. In contrast, Mufwene (2020) maintains that pidgins mostly formed after the development of creoles during the during a colonial transitional period in Africa, Asia, and the Pacific. Despite these canonical debates regarding French-based Creoles, French-based pidgins remain underexplored. While Skirgård (2013) has compiled and analyzed a corpus of Français Tirailleur, a variety closely associated with West African soldiers, there is still debate about whether it represents a “real” contact variety from the late 19th and early 20th centuries Parkvall (2018) or a stereotypical projection of otherness (Vigouroux (2017). In contrast, there is more consensus that Tây Bồi, or Vietnamese Pidgin French, developed in French Indochina during roughly the same time period and fell out of use when the French military presence ended. Although Français Tirailleur and Tây Bồi share many features, they are often assumed to be mostly independent developments, and to our knowledge, no extensive comparison has been conducted. In this work, we first review descriptions of Tây Bồi and Français Tirailleur to highlight their close structural affinity, Then, we introduce textual data showing that they were largely consolidated by 1890. Finally, we revisit some contemporary accounts of language education in French colonies. Taken together, these documents suggest that, far from being spontaneous solutions to communicative barriers, Français Tirailleur and Tây Bồi were heavily shaped by top- down instruction of ostensibly simplified French. Rather than disproving the existence of pidginized varieties, the codevelopment of regionalized Frenches in West Africa and Southeast Asia underscores the contradictions and instability of thecolonial-era two-track approach to language education.