[AINews] We Solved Hallucinations • ButtondownTwitterTwitter

Compute and Hardware Improvements

• GPT-2 training cost has decreased dramatically due to improvements in compute hardware, software, and data quality.
• FlashAttention-3 released with increased speed and efficiency.
• Hopper GPUs enable major speedups with new hardware features.

LLM Evaluation and Benchmarking

• Synthetic data may not be helpful for vision tasks.
• Avocado360 benchmark introduced for evaluating VLMs.
• Lynx model announced for LLM hallucination detection.

LLM Applications and Frameworks

• Runway AI using AI for automating tasks and scaling with AI capabilities.
• LangGraph incorporating human-in-the-loop feedback.
• Qdrant and LlamaIndex collaboration for advanced RAG architecture.

Memes and Humor

• Giffmana's tweet about love for Thinkpads.

The AI Discord Recap section provides insights into various advancements and community discussions related to AI technologies. Highlights include advancements in LLM models, such as FlashAttention-3 and the upcoming Llama 3 multimodal model. Open-source AI developments like AuraFlow and Cohere Toolkit are discussed, along with community collaborations for training techniques, knowledge sharing, and hardware benchmarking. The section also covers discussions on object detection models, vision models, LLM finetuning toolkits, and AI tool affordability within different Discord communities specialized in AI technologies.

Modular (Mojo 🔥) Discord

• LLVM Creator's Chronicles: Insights into Mojo's creation are discussed after a video interviewing LLVM creator became accessible on YouTube.
• Mojo's REPL Iteration: Debate surrounds Mojo REPL's lack of immediate output for expressions and comparisons to Python's REPL behavior, with suggestions to submit requests through GitHub.
• Max Website Makeover Embraces Clarity: Modular's MAX framework shines on a revamped website, emphasizing performance and ease of use provided by Mojo language.
• GPU Gains with Mojo's MAX: Discussion on writing custom GPU kernels using Mojo within MAX for better performance.
• Datatype Discrepancies in MAX Model Execution: Issue with MAX Model execution highlighting the need for precision in model execution parameters.

Unsloth AI (Daniel Han) Discord

• Gemini Soars with Token Expansion: Gemini 1.5 Pro's new features excite AI developers, with unlimited JSON capacity.
• FlashAttention Sprints on Hopper GPUs: FlashAttention-3 promises efficient Hopper GPU utilization with substantial FLOPs leverage for Hopper users.
• TF-ID Models Eye Vision-Language Tasks: Yifei Hu unleashes TF-ID models for vision-language tasks with training code, dataset, and weights under the MIT License.
• CodeGeeX4 Clips GPT's Wings: CodeGeeX4-ALL-9B model outshines GPT-3.5 and GPT-4 in code generation capabilities.
• Meta's Anticipated LLaMA 3 Debut: Excitement builds for Meta Platform's LLaMA 3 release on July 23, potentially reshaping hardware preferences for AI application deployment.

Nous Research AI Discord

• OpenAI Teases Doctoral Disruption: OpenAI hints at forthcoming models with problem-solving adeptness equating to a doctoral degree.
• Anthropic's AI Prognosis: Predictions on future AI Safety Levels, with concerns raised about potential global risks.
• Community Doubts OpenAI's Strategy: Skepticism in the community about OpenAI's strategic release patterns.
• C's the Day with GPT-2: Karpathy demonstrates efficient replication of GPT-2 (1.6B) using llm.c for large-scale language model training.
• Safety in Simplicity with C++: Safetensors.cpp debuts as a zero-dependency C++ library for LibTorch, streamlining model data processes.

A dive into the logistics of self-hosting 400B parameter models unveils the necessity of around 400GB VRAM, steering the conversation towards resource availability and favoring API usage when proprietary hardware falls short. This predicament places a brighter spotlight on hyperscalers for GPU rental, especially when proprietary data is not a concern, as gleaned from the tech community's dissection of hosting intricacies and API benefits.

Members discussed the technical and emotional loss associated with various GPU models, such as the 1060 3GB and the potential replacements like the A6000 for better rendering capabilities. Budget constraints led to the consideration of options like Facebook Marketplace salvages and freelancing for extra funds. Additionally, discussions included the cost and utility of A100 GPUs for training, with recommendations for backprop.co and free Google Colab T4 instances for more economical usage. Members faced challenges in training Diffusion models, using LoRa on cheaper GPUs and the complications of full finetuning on A100 due to cost. HF Updates were shared, including GGUF support in transformers and integration with KerasNLP models. Members also engaged in a light-hearted conversation about the hypothetical impact of cheese on servers, invoking humor around fondue and GPUs.

The HuggingFace community continues to see exciting developments in various areas. New models like RT-DETR and Hiera Vision Transformer are outperforming previous models, leading to more accessible fine-tuning and training workflows. Efficient use of VRAM with offloading in the Aura Flow Transformer model is now possible, enabling users to work with limited resources efficiently. Additionally, community involvement is emphasized for the AuraFlow model to ensure continuous improvements. In the CUDA MODE channels, discussions range from beginner tips on using Google Colab for GPU access to advanced debates on matrix-matrix multiplication optimizations and tensor subclass support.

Members engaged in discussions regarding challenges and strategies related to fine-tuning models. Techniques such as decaying learning rate and handling multiple datasets for a single model were explored. Unsloth recommended utilizing continued pretraining notebooks for introducing new languages efficiently and managing VRAM effectively.

The discussions in the Unsloth AI channel revolve around various topics related to AI training and models. Users share insights on training local models, handling different sequence lengths, training data requirements, model parameter discrepancies, and resources for RAG systems. Additionally, they discuss the release of TF-ID models for vision-language tasks and MovieChat for enhancing video interaction. The channel also features conversations about the importance of learning to learn, evaluating programming models using Tetris, and the potential of continued pretraining. Throughout the discussions, users also share useful links to documentation and academic papers for further exploration.

GPT-4Chan and TruthfulQA benchmark debate

Discussion surfaced around GPT-4Chan being a SOTA on TruthfulQA before the arrival of ChatGPT, as highlighted by a relevant tweet. Members generally agreed that benchmarks like TruthfulQA and HellaSwag are unreliable, while benchmarks such as MT-Bench and AGI Eval are more accurate indicators of performance.

Jsonnet’s mixed reception in configuration tasks

A user expressed strong mixed feelings about Jsonnet, highlighting its challenge of lacking a comprehensive toolchain for debugging and testing, yet praising its clean implementation. The discussion elaborated on the general difficulty of configuration languages, with Jsonnet being considered the least bad due to its clean design, although not widely adopted or adequately supported.

London AI meetups generally disappointing

Several members voiced dissatisfaction with London AI meetups, noting that they often cater to a more general tech crowd rather than offering in-depth technical discussions. It was suggested that university seminars and research conferences like ICML and ICLR could offer more substantive content for those seeking deep, technical conversations on AI.

The section discusses the efficiency of Logprob for evaluating confidence ratings in document enrichment, the value of Langgraph in state management for iterative steps and parallel processes, a presentation on PDF to Markdown tools by vikp, an upcoming session on RAG Architectures scheduled for 3/15/2024, and comparisons with XState for app logic management. It also covers proposals for a memorable acronym '3E', optimal use of FAISS and Chroma for large datasets, concerns about reembedding documents in LangChain agents, and the use of OpenAI Vector Store in LangChain for efficient document retrieval.

This section discusses various hardware-related topics in the LM Studio channel on Discord. Users debate the value of different NVIDIA GPUs for AI, rumors about the NVIDIA 5090 VRAM, comparisons between V100 compute nodes and 3090 setups, and the performance of ARM computers with Large Language Models (LLMs). The discussions cover factors like memory bandwidth, generational jumps in TFLOPs, compute node budgets, system speed, and cost-effectiveness for AI use cases.

A Twitter user highlighted a speculative report on OpenAI's revenue based on chatbot summaries of public sources. They provided a link to a more credible report by The Information. In another discussion, OpenAI's progress towards reaching human-level AI capabilities was mentioned. The ChatGPT maker believes it's on the first level, which is conversational AI. Additionally, there was a proposal to create roadmaps similar to the 2024 H2 plans by the PyTorch Team for better development pathways in the AI field.

This section covers technical discussions on topics such as function calling on Gemini models, updating integration packages, indexing code libraries, using RAG for reviewing spec documents, agent invocation issues, speed improvement in function calls, GUI upgrades, self-hosted ML telemetry solutions, OpenAI API key requests for chatbot projects, credit balance inquiries, LLM matmul mechanisms, LLM arena for dataset quality, and a discussion on covering multiple areas like recommendation systems, information retrieval, and retrieval-augmented generation.