Text Generation WebUI

Text Generation WebUI osFoundry community catalog में एक app है। oobabooga का text-generation-webui — local language models को run + experiment करने का swiss army knife। llama.cpp (GGUF), transformers, ExLlamaV2, AWQ + GPTQ quantizations, साथ ही एक built-in OpenAI-compatible API server को support करता है। सबसे-extensible local LLM UI: parameter presets, character cards, persona-driven chat, notebook + chat + instruct modes, LoRA fine-tuning के लिए training tab। CPU mode bundled (इस host पर GPU नहीं)।

विवरण

Workspace: osfoundry
Category: AI
मूल्य: Free
Access: Community

Features

Load + chat with GGUF (llama.cpp) / HuggingFace Transformers / ExLlamaV2 / AWQ / GPTQ models
Three modes: Chat (assistant) / Instruct (single-turn) / Notebook (free-form completion)
OpenAI-compatible API server bundled — drop-in for any OpenAI SDK client
Parameter presets + samplers (mirostat, dynamic temperature, DRY, smoothing factor, ...) — the deepest sampler knobs in the OSS LLM world
Character cards + persona system shared with SillyTavern format
LoRA training tab for fine-tuning (CPU mode is feasible but slow)

Documentation

Documentation को upstream project द्वारा अंग्रेज़ी में मेंटेन किया जाता है।

# Text Generation WebUI

## Drop in a model

The container ships with no models. Get a GGUF or HuggingFace model into `/data/models/`:

```
curl -L -o /data/models/llama-3.1-8b-instruct.gguf \
  https://huggingface.co/.../resolve/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf
```

Or in the web UI → **Model** tab → **Download model or LoRA** → paste the HF repo path (e.g. `unsloth/Llama-3.2-1B-Instruct-GGUF`).

## Load + chat

1. **Model** tab → pick a downloaded model from the dropdown → **Load**
2. **Chat** tab → start chatting

## Three modes

- **Chat** — turn-based assistant
- **Instruct** — single-turn instruction following
- **Notebook** — free-form completion / story writing

## OpenAI-compatible API

With `--api` flag (default), an OpenAI-compatible endpoint is at port 7860/v1. Use as drop-in OpenAI for any client:

```python
from openai import OpenAI
client = OpenAI(base_url='https://<your-public-url>/v1', api_key='none')
```

## Character cards

Drop SillyTavern-format PNG cards into `/data/characters/`. They show up under Chat → Character.

## CPU mode caveat

This container is CPU-only. 7B Q4 models run at 2-6 tokens/sec on 2 vCPU. Use small quantized models for usable speed; 30B+ models will be too slow for interactive use.

osFoundry में Text Generation WebUI का उपयोग कैसे करें

एक क्लिक में Text Generation WebUI को अपने workspace में install करें, फिर अपने stack के लिए prompts, tools, या configuration को customise करने के लिए इसे osStudio में fork करें। आपके workspace का कोई भी सदस्य वहीं से आगे बढ़ा सकता है जहाँ आपने छोड़ा था।

Community से अन्य apps

CRM — Contacts, deals, और pipeline tracking के साथ customer relationship management।
Kanban Board — एक Trello-style kanban and project board के साथ cards, boards, calendar and table views, and per-board properties. Powered by Focalboard (standalone personal server). Embedded SQLite एक persistent volume पर.
Helpdesk — SLA tracking के साथ ticket triage और customer support inbox।
Page Builder — Visual drag-and-drop page builder के साथ sections, themes, SEO, and publishing
Website Builder — Multi-page website builder के साथ CMS collections, global navigation, footer, themes, and publishing
Storefront — Product catalog, cart, और checkout के साथ e-commerce storefront।