llm-token LID In development
I’m working on llm-token LID, a Python library for token-level language identification in multilingual and code-switched text. It supports multiple LLM providers, custom language labels and switch-point analysis.
Papers in progress, research tools and my doctoral work in multilingual NLP.
I’m working on llm-token LID, a Python library for token-level language identification in multilingual and code-switched text. It supports multiple LLM providers, custom language labels and switch-point analysis.
My doctoral research develops evaluation methods that capture how NLP systems really perform on code-switched text, with a particular focus on Nigerian Pidgin-English. The work brings sociolinguistic theory and machine-learning practice together to make AI systems that include the people who speak this way.