An Introduction to "Typhoon" The Market's Most Capable Thai Large Language Model (Thai LLM) by SCB 10X

Over the past year, numerous innovations and new AI models have been launched, including the rise of the Large Language Model (LLM), which has been popularized by the fastest-growing application in history, ChatGPT. However, a notable drawback is the predominance of English-based LLMs, leaving behind Thai and other Southeast Asian languages that have limited resources and data for effective language model development. This has led to lagging development for LLM-based applications in these low-resource languages.

In response to this challenge, SCB 10X has released the "Typhoon" series of Large Language Models optimized for Thai. When evaluated on standardized high school Thai language tests like O-NET, TGAT, TPAT, and A-Level, and industry-specific tests such as IC for investment consultants, Typhoon-7B outperforms other Thai LLMs currently available in the market. This development represents a significant step forward in the development of Thai language models.

Typhoon-7B is a Thai LLM that offers strong Thai language performance at higher efficiency than globally developed multilingual models that support Thai. It performs comparably to GPT 3.5 on standardized tests while being 2.6 times more efficient than GPT-4 when processing Thai tokens. 

The model is based on the industry-leading open-source Mistral-7B model but also includes 5,000 Thai words. It addresses the particular difficulties faced by languages with limited resources through optimized tokenization, focused data preparation, and continual training. Through these techniques, Typhoon can provide strong Thai language performance when compared to LLMs with significantly more parameters at higher efficiency. The next frontier in Typhoon’s development is to improve its fluency in Thai so that it can better serve Thai users.

SCB 10X has released the Typhoon-7B pre-trained model as open-source under the Apache 2.0 license, with the goal of  facilitating community development of Thai LLMs in the future. As a pre-trained model, it offers a strong foundation for additional fine-tuning for specific use cases.

Developers and researchers can download the pre-trained version of Typhoon at https://huggingface.co/scb10x/typhoon-7b/. Access to the instruction-tuned version of Typhoon will soon be available through the Typhoon API. Sign up for the waiting list at https://opentyphoon.ai. The Typhoon technical report is available at https://arxiv.org/abs/2312.13951.

Please visit https://arxiv.org/abs/2312.13951 to read the technical report on the Typhoon software development and assessment released by the Typhoon team.