Language Model

Large classroom sizes in computing education are making it crucial to use automation for student success. Automated feedback generation tools are becoming increasingly popular for their ability to rapidly analyze and test. Among these, large language models (LLMs) like GPT-3 are showing promise. However, concerns about their accuracy, reliability, and ethical implications do exist. Historically, the…

Transformer models have ushered in a new era of Natural Language Processing (NLP), but their high memory and computational costs often pose significant challenges. This has fueled the search for more efficient alternatives that uphold the same performance standards but require fewer resources. While some research has been conducted on Linear Transformers, the RWKV model,…

Artificial Intelligence (AI) relies on broad data sets sourced from numerous global internet resources to power algorithms that shape various aspects of our lives. However, there are challenges in maintaining data integrity and ethical standards, as the data often lacks proper documentation and vetting. The core issue is the absence of robust systems to guarantee…

Large language models (LLMs) are crucial to processing extensive data quickly and accurately. Instruction tuning plays a vital role in enhancing their reasoning abilities and preparing them to solve new, unseen problems. However, the acquisition of high-quality instruction data on a large scale presents a significant challenge. Traditional methods that rely heavily on human input…