[AINews] That GPT-4o Demo • ButtondownTwitterTwitter

AI Twitter Recap

Claude 3 Opus provided the recap of AI Twitter, noting work on clustering and flow engineering with Haiku.

Gemma 2 Release by Google DeepMind

• Model Sizes and Training: Google DeepMind introduced Gemma 2 in 9B and 27B parameter sizes, trained on 13T and 8T tokens respectively, using SFT, Distillation, RLHF, and Model Merging on Google TPUv5e.
• Performance: Gemma 2 models showcase class-leading performance and efficient single TPU host operation.
• Availability: Gemma 2 is available on Kaggle and Hugging Face, written in Keras 3 and compatible with TensorFlow, JAX, and PyTorch.
• Safety: Google DeepMind followed strict internal safety protocols towards pre-training data filtering and bias mitigation.

Meta LLM Compiler Release

• Capabilities: Meta LLM Compiler, based on Meta Code Llama, offers additional code optimization and compiler features.
• Availability: Released under a permissive license for research and commercial use on Hugging Face.
• Potential: LLM Compiler aims to achieve near-perfect code optimization, potentially transforming coding efficiency standards.

This section covers a variety of discussions and updates from the AI Discord community. It includes topics such as model performance optimization, fine-tuning challenges, AI applications, memes and humor related to AI, updates on Gemma 2 models, practical AI integration, collaborations, and advancements in datasets and benchmarking. The discussions range from Gemma 2 models outperforming competitors, challenges in prompt engineering, open-source AI developments, training insights, ethical considerations in AI, and practical AI integration issues faced by the community.

Perplexity AI Discord experienced discussions around pricing adjustments for Enterprise Pro offerings, frustrations over RAG performance and access to larger models like Gemma 2. Security protocols were emphasized, and discussions included exploring Android 14 enhancements and potential issues with Minecraft's mechanics. The community shared insights on various tech topics, including Android 14, Linux performance, Robot Skin, sustainable construction inspired by oysters, and criticisms of Minecraft's repair mechanics. The channel also delved into the slow progress of AI agents, ethical considerations around data usage, skepticism towards scaling for AGI, and reflections on AI's impact on global affairs.

Engineers explored building agentic RAG services with LlamaIndex, discussing the process of creating vector indexes and transforming them into query engines. The LlamaIndex community is abuzz about Jina's newest reranker, hailed as their most effective to date. Troubleshooting LlamaIndex's embedding challenges involves factors such as node weights and model mismatches contributing to suboptimal outcomes. Additionally, debates are ongoing on enhancing entity relationship detection and resolving issues with Claude and OpenAI keys. This section also touches on optimizations for batch and parallel index loading to accelerate large file handling.

Gemma-2 Updates:

• New pre-quantized 4-bit versions of Gemma-2-27B and 9B uploaded with faster downloads and less VRAM fragmentation.
• Discussions on using Windows vs Linux for AI development, HF's Tiktoker-like Evaluation System, challenges of large datasets, and AI video generation with ExVideo.

Random Discussions on Unsloth AI:

• Highlighting Gemma 2 technical aspects compared to other models.
• Debate on Knowledge Distillation, inference frameworks, Gemma-2-9B finetuning, and comparisons between Unsloth and large models.

Help Discussions on Unsloth AI:

• Topics include training LM Head for Swedish, inference configuration, VRAM management, LoRA support, and pretraining issues.

Community Collaboration on Unsloth AI:

• Requests for compute power for Toki Pona LLM, Oracle Cloud credits, Oracle platform limitations, adapting Unsloth colabs for Oracle, and comparing Kubeflow.

HuggingFace Announcements:

• Release of Gemma-2 models in the Transformers library, highlighting their efficiency and architecture enhancements.

HuggingFace General Discussions:

• Topics cover FSS in Elixir, Gemma 2 GPT-4 parameters chat, new image retrieval system, visual learning models, and queries on HuggingFace tools.

• Seek YOLO for Web Automation Tasks: Member looking for an efficient method to use YOLO for webpage automation tasks.
• Exploring Efficient SAM Deployment: User seeking advice on deploying SAM models effectively.
• Mask Former Fine-Tuning Challenges: Member facing difficulties in fine-tuning Mask Former model for image segmentation.
• Designing Convolutional Neural Networks: User expressing confusion over determining parameters for CNNs.

Users have reported concerns over the performance of Gemma 9B compared to similar models like Phi-3, particularly on LM Studio. Development efforts are ongoing to address these issues, with functional improvements expected soon. Additionally, discussions in the LM Studio models chat covered topics such as discontent over the context limit of Gemma-2 models, suggestions for supporting a storytelling model, inquiries about Gemma 2 and Deepseek Coder V2 Lite status in LM Studio, and debates on the best models in the 7b~9b category.

The section covers various discussions from the Stability.ai channel, including newbie inquiries on prompt engineering, discussions on the deterministic nature of neural networks, an unsolved semiotics paper inquiry, troubles with Automatic1111, debates about the Cascade channel, specifics about model training nuances and tools, dissatisfaction with Discord management changes, and moments of humor with YouTube video sharing and jokes.

A member questions the status of Chinchilla scaling as an immutable law of nature, suggesting the need for discussions on alternative scaling models. Another member argues for the legitimacy of power law scaling but acknowledges its relevance under specific conditions. There is a debate on whether terms like 'scaling law' should be replaced with 'scaling heuristic' to reflect their provisional nature. Key papers such as 'Parameter Counts in Machine Learning' and 'Adlam 2021 on Scaling Laws' are referenced in discussions about understanding and modeling scaling laws. Practical aspects like data selection methods and their impact on training efficiency are also explored, emphasizing the complexities of predicting the future impact of data on model performance.

Introducing MMLU-SR Dataset to lm_eval:

A member introduced a new dataset, MMLU-SR, designed to challenge LLMs' reasoning abilities through symbol replacement and inquired about adding it to lm_eval. After creating and submitting a PR, they received a prompt response for review. arxiv.org/abs/2406.15468v1

MedConceptsQA Benchmark Addition:

A member requested a review for their PR that adds the MedConceptsQA benchmark aimed at medical concepts question answering. This open-source benchmark features questions of various complexities. github.com/EleutherAI/lm-evaluation-harness/pull/2010

Custom YAML Config Debugging:

A member sought help to run a custom YAML configuration for an evaluation using the harness. They received debugging advice and managed to resolve their issue after identifying and fixing a task name conflict.

Perplexity AI announced reduced pricing for their Enterprise Pro version for philanthropic organizations like schools and nonprofits to support societal and educational development. Users discussed issues with Perplexity's RAG mechanism leading to poor outputs, Claude 3 Opus usage limits, security concerns, intermittent context issues, and VPN access problems. The section also mentioned discussions on Android 14 insights, RDP, sustainable innovations, Linux performance, and concerns about Minecraft's repair mechanics.

OpenRouter announced several updates including price cuts on popular models such as cognitivecomputations/dolphin-mixtral-8x22b (10% reduction), openchat/openchat-8b (20% reduction), and meta-llama/llama-3-70b-instruct (3.5% drop). Discussions in the OpenRouter community covered topics like OpenRouter's moderation strictness compared to AWS and Anthropic, Opus availability issues, troubleshooting GitHub authentication for seamless pushes, API discrepancies with Gemini models, and evaluating LLM APIs. Additionally, Gemma 27B model performance was both praised and met with skepticism, while users reported issues with GPT-4 and 4O models and a shift to using Claude for improved functionality. In the LangChain AI community, topics ranged from CSV and Pandas DataFrame agent issues, LangGraph and Human-in-the-Loop capabilities, to building RAG with Matryoshka Embeddings. Members also shared projects like a no-code Chrome extension for LangChain, a new AI content marketplace called Dappier, and a Data Analyst Agent developed using Cohere and LangChain.

This section discusses various topics related to LLM Finetuning by Hamel and Dan. It includes a member's research on evaluating an evals workshop, seeking a JSONL data editor tool, and generating structured summaries from patient records. Additionally, concerns about Predibase credits expiration and OpenAI credit delays are addressed. Members also discuss port completion for Tinygrad, FPGA-based systems for energy-efficient robotics, and the utility of JSON/YAML in Tinygrad. Furthermore, discussions about Shapetracker capabilities and model storage mechanisms in Tinygrad are highlighted.

OpenAI's early adopters mainly utilize the platform for AI persona local apps, language finetuning, and SQL models. Additionally, specific use cases include requests for over-time Elo data in the chatbot arena, observations about the 'pack' catching up, and information about an upcoming webinar on building an Enterprise-Scale Feature Store with Featureform and Databricks.