Deep learning is continuously evolving with attention mechanism playing an integral role in improving sequence modeling tasks. However, this method significantly bogs down computation with its quadratic complexity, especially in hefty long-context tasks such as genomics and natural language processing. Despite efforts to enhance its computational efficiency, existing techniques like Reformer, Routing Transformer, and Linformer…
First-year students in MIT's Social and Engineering Systems (SES) doctoral program, Eric Liu and Ashley Peake, have a unique interest in investigating housing inequality issues. At the MIT Policy Hackathon, they had an opportunity to apply their knowledge and conduct research on these real-world issues. The hackathon, organized by students in the IDSS and TPP,…
Artificial intelligence (AI) is becoming more sophisticated, requiring models capable of processing large-scale data and providing precise, valuable insights. The aim of researchers in this field is to develop systems that are capable of continuous learning and adaptation, ensuring relevance in dynamic environments.
One of the main challenges in developing AI models is the issue of…
The shift towards renewable energy sources and increased consumer demand due to electric vehicles and heat pumps has significantly influenced the electricity generation landscape. This shift has also resulted in a grid that is subject to fluctuating inputs, thus necessitating an adaptive power infrastructure. Research suggests that bus switching at the substation can help stabilize…
Natural Language Processing (NLP) involves computers understanding and interacting with human language through language models (LMs). These models generate responses across various tasks, making the quality assessment of responses challenging. However, as proprietary models like GPT-4 increase in sophistication, they often lack transparency, control, and affordability, thus prompting the need for reliable open-source alternatives.
Existing…
Researchers have introduced an innovative algorithm known as CIPHER that optimizes large language models (LLMs) by interpreting user feedback edits. LLMs are becoming increasingly popular for a range of applications, with developers constantly enhancing the capabilities of these models. However, one of the key challenges is the alignment and personalization of these models to specific…
Multitask learning (MLT) is a method used to train a single model to perform various tasks simultaneously by utilizing shared information to boost performance. Despite its benefits, MLT poses certain challenges, such as managing large models and optimizing across tasks.
Current solutions to under-optimization problems in MLT involve gradient manipulation techniques, which can become computationally…
First year students, Eric Liu and Ashely Peake, from the Social and Engineering Systems (SES) doctoral program at MIT Institute for Data, Systems, and Society (IDSS), are researching housing inequality issues. They used a policy hackathon organized by IDSS as an opportunity for hands-on research. The annual event gathers participants from around the world to…
Arrays and lists form the basis of data structures in programming, fundamental concepts often presented to beginners. First appeared in the 1957 Fortran and still vital in languages like Python today, arrays are popular due to their simplicity and versatility, allowing data to be organized in multidimensional grids. However, dense arrays, while performance-driven, do not…
Large Language Models (LLMs) have enjoyed a surge in popularity due to their excellent performance in various tasks. Recent research focuses on improving these models' accuracy using external resources including structured data and unstructured/free text. However, numerous data sources, like patient records or financial databases, contain a combination of both kinds of information. Previous chat…
Machine learning is a growing field that develops algorithms to allow computers to learn and improve performance over time. This technology has significantly impacted areas like image recognition, natural language processing, and personalized recommendations. Despite its advancements, machine learning faces challenges due to the opacity of its decision-making processes. This is especially problematic in areas…
Large Language Models (LLMs) signify a major stride in artificial intelligence with their strong natural language understanding and generation capabilities. They can perform plenty of tasks ranging from powering virtual assistants to generating substantial content and conducting profound data analysis. Nevertheless, one obstacle LLMs face is generating factually correct responses. Often, due to the wide…