More LocalAI LocalAI is the free, Open Source OpenAI alternative. LocalAI act as a drop-in replacement REST API that’s compatible with OpenAI API specifications for local inferencing. It allows you to run LLMs, generate images, audio (and not only) locally or on-prem with consumer grade hardware, supporting multiple model families and architectures. Overview | LocalAI documentation GitHub: https://github.com/mudler/LocalAI   Xinference Xorbits Inference (Xinference) is an open-source platform to streamline the operation and integration of a wide array of AI models. With Xinference, you’re empowered to run inference using any open-source LLMs, embedding models, and multimodal models either in the cloud or on your own premises, and create robust AI-driven applications. Welcome to Xinference! — Xinference GitHub:  https://github.com/xorbitsai/inference   NVIDIA NIM Explore the latest community-built AI models with an API optimized and accelerated by NVIDIA, then deploy anywhere with NVIDIA NIM inference microservices. NVIDIA NIM for Deploying Generative AI | NVIDIA Doc:  Introduction - NVIDIA Docs Models:  google / gemma-7b YT:  Self-Host and Deploy Local LLAMA-3 with NIMs - YouTube text-generation-webui A Gradio web UI for Large Language Models. 只能執行本地模型，不支援外部模型 API。 支援以下多重功能的 AI 平台 Chat Fine-Tune Model Multiple model backends: Transformers, llama.cpp (through llama-cpp-python), ExLlamaV2, AutoGPTQ, AutoAWQ, GPTQ-for-LLaMa, QuIP#. OpenAI-compatible API server with Chat and Completions endpoints 教學 GitHub:  https://github.com/oobabooga/text-generation-webui   GitHub:  https://github.com/Atinoda/text-generation-webui-docker   大型語言模型LLMs課程教學 課程大綱 (三) - HackMD YOUTUBE [ 啟動 TextGen ] YOUTUBE [ 上架大型語言模型 ] YOUTUBE [ 指派AI人設 ] YOUTUBE [ 模型微調 ] YOUTUBE [ 上架微調模型 ] 程式碼  Z01_TextGen_Colab.ipynb 預設密碼在程式碼裡面 (account:nchc password:nchc) 請自行修改 koboldcpp GitHub: https://github.com/LostRuins/koboldcpp   單機版運行/免安裝/無相依姓套件 支援 GGUF 模型 支援 OuteTTS (Text-To-Speech), Whisper (Speech-To-Text), 文字/圖像生成 內建 KoboldAI Lite UI