Stronger Models are NOT Stronger Teachers for Instruction Tuning

This research paper investigates the impact of different language models (LLMs) used as "teachers" to generate synthetic responses for instruction tuning. The authors demonstrate a surprising phenomenon they call the "Larger Models' Paradox," where larger and supposedly "stronger" teacher models do not always lead to improved instruction-following abilities in smaller base models. They propose a novel metric called Compatibility-Adjusted Reward (CAR) to better predict the effectiveness of teacher models, taking into account the compatibility between the teacher and the base model being fine-tuned. The study challenges the common assumption that larger LLMs are always better teachers and suggests that a more nuanced understanding of compatibility is needed for successful instruction tuning.

This research paper investigates the potential for large language models (LLMs) to self-improve in long-context reasoning, which involves processing and understanding complex information spread across long stretches of text. The authors propose a novel approach called SEALONG that leverages the LLMs' ability to generate multiple outputs for a given question and then scores these outputs using a method called Minimum Bayes Risk (MBR). The MBR approach prioritizes outputs that align better with each other, thereby filtering out outputs that might be incorrect or hallucinatory. SEALONG then uses these high-scoring outputs for further training, either through supervised fine-tuning or preference optimization. The authors demonstrate through extensive experiments that SEALONG significantly improves the long-context reasoning performance of LLMs without requiring expert model annotations or human labeling.

LLaMA-Mesh: Unifying 3D Mesh Generation with Language Models, introduces a new method for generating 3D models using large language models (LLMs). The authors address the challenge of tokenizing 3D mesh data for LLMs by representing the mesh data as plain text using the OBJ file format, a standard text-based format for 3D models. This approach allows for direct integration with LLMs without modifying the vocabulary or tokenizers, minimizing additional training overhead. The study then introduces LLAMA-MESH, a fine-tuned LLaMA model that can generate 3D meshes from textual prompts, produce interleaved text and 3D mesh outputs, and understand and interpret 3D meshes. LLAMA-MESH achieves comparable mesh generation quality to models trained from scratch while maintaining strong text generation abilities, demonstrating the potential for LLMs to become universal generative tools for multiple modalities.

The researchers introduce LLaVA-o1, a vision language model designed to perform structured reasoning by breaking down problem-solving into four distinct stages: summary, caption, reasoning, and conclusion. They compiled a new dataset, LLaVA-o1-100k, and proposed a stage-level beam search method to improve model performance during inference. Experimental results demonstrate that LLaVA-o1 outperforms existing open-source and even some closed-source models on multimodal reasoning benchmarks, emphasizing the effectiveness of its structured reasoning approach.

The BlueLM-V-3B, a multimodal large language model (MLLM) designed specifically for mobile devices. The researchers address the challenges of deploying large models on mobile phones, such as limited memory and processing power, by implementing a novel algorithm and system co-design approach. This includes a dynamic resolution scheme that optimizes image processing and a token downsampler that reduces the number of image tokens to improve inference speed. The paper emphasizes BlueLM-V-3B's superior performance compared to other models of similar size and its high deployment efficiency on mobile devices.

CORAL, a novel benchmark dataset for evaluating Retrieval-Augmented Generation (RAG) systems in a multi-turn conversational setting. The authors highlight the limitations of existing datasets in assessing conversational RAG and detail CORAL's unique features, including open-domain coverage, knowledge intensity, free-form responses, topic shifts, and citation labeling. They explain how CORAL is derived from Wikipedia, automatically converting its content into conversational formats, and outline the three core tasks it supports: conversational passage retrieval, response generation, and citation labeling. The authors present a unified framework for evaluating conversational RAG methods and report on experiments conducted on CORAL, showcasing the performance of different conversational search and generation models.

This research paper surveys small language models (SLMs) and explores their applications, design, training, and model compression techniques. The authors explain that while large language models (LLMs) have proven effective, their resource demands have led to the development of SLMs, which are more efficient and can be deployed on a wider range of devices. The paper examines various techniques to optimize SLMs, including lightweight model architectures, efficient self-attention mechanisms, and model compression strategies such as pruning, quantization, and knowledge distillation. The authors discuss the challenges associated with SLMs, such as hallucination, bias, and energy consumption, and offer suggestions for future research. The goal of this work is to provide a comprehensive resource for researchers and practitioners working with small language models.

This research explores whether transformers, a type of neural network architecture, can learn to reason implicitly over knowledge. The authors find that transformers can learn to reason implicitly, but only through a phenomenon called grokking, where training extends far beyond overfitting. The study investigates two reasoning types: composition and comparison. They find that while the transformers generalize well on in-distribution examples for both types, they struggle with out-of-distribution generalization for composition but succeed for comparison. Through mechanistic analysis of the model’s internals, they discover that different circuits are formed during grokking for each reasoning type, which explains the varying levels of systematicity. The authors also demonstrate the potential of parametric memory for complex reasoning tasks with large search spaces, showing that a fully grokked transformer can achieve near-perfect accuracy, while state-of-the-art LLMs with non-parametric memory fail.

This research paper details the development of Llama 3, a large language model with improved capabilities in language understanding, code generation, mathematical reasoning, and multimodality. The paper emphasizes the importance of high-quality data, scaling up compute power, and using simple, efficient methods to achieve optimal results. It also explores the integration of vision and speech capabilities into Llama 3, highlighting the benefits of a compositional approach. The paper concludes with a discussion of safety measures implemented in Llama 3 to mitigate potential risks and ensure responsible use of the model.

Unlike traditional Multi-Layer Perceptrons (MLPs), which have fixed activation functions on nodes, KANs have learnable activation functions on edges. This seemingly simple change allows KANs to outperform MLPs in terms of accuracy and interpretability, particularly for small-scale artificial intelligence and scientific tasks. The text explores the mathematical foundations of KANs, highlighting their ability to overcome the curse of dimensionality and achieve faster neural scaling laws than MLPs. Additionally, the text showcases KANs' potential for scientific discovery by demonstrating their effectiveness in uncovering mathematical relations in knot theory and identifying phase transition boundaries in condensed matter physics.