Skip to content

localai 3.5.0 #236075

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
BrewTestBot merged 2 commits into from
Sep 4, 2025
Merged

localai 3.5.0 #236075

BrewTestBot merged 2 commits into from
Sep 4, 2025

Conversation

@BrewTestBot
Copy link
Member

@BrewTestBot BrewTestBot commented Sep 3, 2025

Created by brew bump

Created with brew bump-formula-pr.

release notes 

  

   



🚀 LocalAI 3.5.0



Welcome to LocalAI 3.5.0! This release focuses on expanding backend support, improving usability, refining the overall experience, and keeping reducing footprint of LocalAI, to make it a truly portable, privacy-focused AI stack. We’ve added several new backends, enhanced the WebUI with new features, made significant performance improvements under the hood, and simplified LocalAI management with a new Launcher app (Alpha) available for Linux and MacOS.


TL;DR – What’s New in LocalAI 3.5.0 🎉


    

  • 🖼️ Expanded Backend Support: Welcome to MLX!  mlx, mlx-audio, mlx-vlm are now all available in LocalAI. We also added support to WAN for video generation, and a CPU and MPS version of the diffusers backend! Now you can generate and edit images from MacOS or if you don't have any GPU (albeit slow).
    • 

  • ✨ WebUI Enhancements: Download model configurations, a manual model refresh button, streamlined error streaming during SSE events, and a stop button for running backends. Models now can also be imported and edited via the WebUI.
    • 

  • 🚀 Performance & Architecture: Whisper backend has been rewritten in Purego with integrated Voice Activity Detection (VAD) for improved efficiency and stability. Stablediffusion also benefits from the Purego conversion.
    • 

  • 🛠️ Simplified Management: New LocalAI Launcher App (Alpha) for easy installation, startup, updates, and access to the WebUI.
    • 

  • ✅ Bug Fixes & Stability: Resolutions to AMD RX 9060XT ROCm errors, libomp linking issues, model loading problems on macOS, CUDA device detection improvements, and more.
    • 

  • Enhanced support for MacOS: whisper, diffusers, llama.cpp, MLX (VLM, Audio, LLM), stable-diffusion.cpp will now work on MacOS!
    • 



What’s New in Detail


🚀 New Backends and Model Support


We've significantly expanded the range of models you can run with LocalAI!


    

  • mlx-audio: Bring text to life with Kokoro’s voice models on MacOS with the power of MLX!. Install with the mlx-audio backend. Example configuration:

    backend: mlx-audio
name: kokoro-mlx
parameters:
  model: prince-canuma/Kokoro-82M
  voice: "af_heart"
  known_usecases:
    - tts
    

    • 

  • mlx-vlm: Experiment with the latest VLM models. While we don't have any models in the gallery, it's really easy to configure, see Add mlx-vlm mudler/LocalAI#6119 for more details.

    name: mlx-gemma
backend: mlx-vlm
parameters:
  model: "mlx-community/gemma-3n-E2B-it-4bit"
template:
  use_tokenizer_template: true
known_usecases:
- chat
    

    • 

  • WAN: Generate videos with Wan2.1 or Wan 2.2 models using the diffusers backend, supporting both I2V and T2V.  Example configuration:

    name: wan21
f16: true
backend: diffusers
known_usecases:
  - video
parameters:
  model: Wan-AI/Wan2.1-T2V-1.3B-Diffusers
diffusers:
  cuda: true
  pipeline_type: WanPipeline
  step: 40
options:
    - guidance_scale:5.0
    - num_frames:81
    - torch_dtype:bf16
    

    • 

  • Diffusers CPU and MacOS Support: Run diffusers models directly on your CPU without a GPU or with a Mac! This opens up LocalAI to a wider range of hardware configurations.
    • 



✨ WebUI Improvements


We've added several new features to make using LocalAI even easier:


    

  • Download Model Config: A "Get Config" button in the model gallery lets you download a model’s configuration file without installing the full model. This is perfect for custom setups and easier integration.
    • 

  • Manual Model Refresh: A new button allows you to manually refresh the on-disk YAML configuration, ensuring the WebUI always has the latest model information.
    • 

  • Streamlined Error Handling: Errors during SSE streaming events are now displayed directly to the user, providing better visibility and debugging information.
    • 

  • Backend Stop Button: Quickly stop running backends directly from the WebUI.
    • 


Screenshot From 2025-08-15 22-25-52

    

  • Model import and edit: Now models can be edited and imported directly from the WebUI.
    • 


Screenshot 2025-08-14 at 22-28-59 LocalAI - Import Model
Screenshot 2025-08-14 at 22-28-47 LocalAI - Edit Model gpt-oss-20b

    

  • Installed Backend List: Now displays installed backends in the WebUI for easier access and management.
    • 



🚀 Performance & Architecture Improvements


    

  • Purego Whisper Backend: The Whisper backend has been rewritten in Purego for increased performance and stability. This also includes integrated Voice Activity Detection (VAD) for detecting speech.
    • 

  • Purego Stablediffusion: Similar to Whisper, Stablediffusion has been converted to Purego, improving its overall architecture and enabling better compatibility.
    • 



🛠️ Simplified Management – Introducing the LocalAI Launcher (Alpha)


We're excited to introduce the first version of the LocalAI Launcher! This application simplifies:


    

  • Installation
    • 

  • Startup/Shutdown
    • 

  • Updates
    • 

  • Access to the WebUI and Application Folder
    • 


Screenshot From 2025-08-26 11-46-35
Screenshot From 2025-08-26 11-46-50
Screenshot From 2025-08-26 11-46-21

Please note: The launcher is in Alpha and may have bugs.  The macOS build requires workarounds to run due to binaries not yet signed, and specific steps for running it are needed: https://discussions.apple.com/thread/253714860?answerId=257037956022#257037956022.


✅ Bug Fixes & Stability Improvements


    

  • AMD RX 9060XT ROCm Error: Fixed an issue causing errors with AMD RX 9060XT GPUs when using ROCm. This error, "ROCm error: invalid device function", occurred because of device function incompatibility.  The fix involves updating the ROCm image and ensuring the correct GPU targets are specified during compilation.  Recommended kernel versions and verification steps for GPU detection are available [here](link to troubleshooting doc if created).
    • 

  • libomp Linking: Resolved a missing libomp.so issue on macOS Docker containers.
    • 

  • macOS Model Loading: Addressed a problem where models could not be loaded on macOS. This was resolved by bundling necessary libutf8 libraries.
    • 

  • CUDA Device Detection: Improved detection of available GPU resources.
    • 

  • Flash Attention: Set auto for flash_attention in llama.cpp, allowing the system to optimize performance.
    • 



Additional Improvements


    

  • System Backend: Added a new "system" backend path (LOCALAI_BACKENDS_SYSTEM_PATH or via command-line arguments) defaulting to /usr/share/localai/backends. This allows specifying a read-only directory for backends, useful for package management and system-wide installations.
    • 

  • P2P Model Sync: Implemented automatic synchronization of installed models between LocalAI instances within a federation. Currently limited to models installed through the gallery, and configuration changes are not synced. Future improvements will address these limitations.
    • 

  • Diffusers Image Source Handling: Enhanced image source selection in the diffusers backend, prioritizing ref_images over src for more robust loading behavior.
    • 

  • Darwin CI Builds: Added support for building some Go-based backends (Stablediffusion and Whisper) on Darwin (macOS) in the CI pipeline.
    • 



🚨 Important Notes


    

  • Launcher (Alpha): The LocalAI Launcher is in its early stages of development. Please report any issues you encounter.  The MacOS build requires additional steps due to code signing.
    • 

  • Model Configuration Updates: Changes to model configuration files are not currently synchronized when using P2P model sync.
    • 



The Complete Local Stack for Privacy-First AI



  

    
      
        LocalAI Logo
        
LocalAI

      
    		
    
      
The free, Open Source OpenAI alternative. Acts as a drop-in replacement REST API compatible with OpenAI specifications for local AI inferencing. No GPU required.

      
Link: https://github.com/mudler/LocalAI

    		
  

  

    
      
         LocalAGI Logo
         
LocalAGI

      
    		
    
      
A powerful Local AI agent management platform. Serves as a drop-in replacement for OpenAI's Responses API, supercharged with advanced agentic capabilities and a no-code UI.

      
Link: https://github.com/mudler/LocalAGI

    		
  

  

    
      
         LocalRecall Logo
         
LocalRecall

      
    		
    
      
A RESTful API and knowledge base management system providing persistent memory and storage capabilities for AI agents. Designed to work alongside LocalAI and LocalAGI.

      
Link: https://github.com/mudler/LocalRecall

    		
  




Thank you! ❤️


A massive THANK YOU to our incredible community and our sponsors! LocalAI has over 35.000 stars, and LocalAGI has already rocketed past 1100+ stars!


As a reminder, LocalAI is real FOSS (Free and Open Source Software) and its sibling projects are community-driven and not backed by VCs or a company. We rely on contributors donating their spare time and our sponsors to provide us the hardware! If you love open-source, privacy-first AI, please consider starring the repository, contributing code, reporting bugs, or spreading the word!


Full changelog :point_down:




:point_right: Click to expand :point_left: 


What's Changed


Bug fixes :bug:



Exciting New Features 🎉



🧠 Models



📖 Documentation and examples



👒 Dependencies



Other Changes





New Contributors



Full Changelog: mudler/LocalAI@v3.4.0...v3.5.0

View the full release notes at https://github.com/mudler/LocalAI/releases/tag/v3.5.0.

@github-actions github-actions bot added go Go use is a significant feature of the PR or issue bump-formula-pr PR was created using `brew bump-formula-pr` labels Sep 3, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 4, 2025

🤖 An automated task has requested bottles to be published to this PR.

Caution

Please do not push to this PR branch before the bottle commits have been pushed, as this results in a state that is difficult to recover from. If you need to resolve a merge conflict, please use a merge commit. Do not force-push to this PR branch.

@github-actions github-actions bot added the CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. label Sep 4, 2025
@BrewTestBot BrewTestBot added this pull request to the merge queue Sep 4, 2025
Merged via the queue into main with commit dfe208f Sep 4, 2025
22 checks passed
@BrewTestBot BrewTestBot deleted the bump-localai-3.5.0 branch September 4, 2025 00:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bevanjkay bevanjkay bevanjkay approved these changes

@github-actions github-actions[bot] github-actions[bot] approved these changes

Assignees

No one assigned

Labels

bump-formula-pr PR was created using `brew bump-formula-pr` CI-published-bottle-commits The commits for the built bottles have been pushed to the PR branch. go Go use is a significant feature of the PR or issue

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@BrewTestBot @bevanjkay