差異處

這裏顯示兩個版本的差異處。

--- tech:ollama [2024/05/11 23:33] – [使用 docker compose 快速安裝測試] jonathan
+++ tech:ollama [2024/12/18 18:34] (目前版本) – [Open WebUI 設定 Gemini API] jonathan
@@ 行 1: / 行 1: @@
-====== Ollama + Open-WebUI  ======
+====== Ollama + Open-WebUI ======
   * 安裝環境 VM 規格 :
     * vCPU : 8 ([host] i7-11700 支援 avx2)
     * RAM : 8G
     * SSD : 64G
-  * OS : Ubuntu 20.04
+  * OS : Alpine 3.19 + docker compose
 ===== 使用 docker compose 快速安裝測試 =====
@@ 行 17: / 行 17: @@
       - ./ollama/ollama:/root/.ollama
     container_name: ollama
+    environment:
+      - OLLAMA_NUM_PARALLEL=4
+      - OLLAMA_MAX_LOADED_MODELS=4
     pull_policy: always
     tty: true
@@ 行 22: / 行 25: @@
     networks:
       - ollama-docker
+    #deploy:
+    #  resources:
+    #    reservations:
+    #      devices:
+    #        - driver: nvidia
+    #          count: 1
+    #          capabilities: [gpu]
   ollama-webui:
@@ 行 37: / 行 47: @@
       - host.docker.internal:host-gateway
     restart: unless-stopped
+    networks:
+      - ollama-docker
+  pipelines:
+    image: ghcr.io/open-webui/pipelines:main
+    container_name: pipelines
+    ports:
+      - "9099:9099"
+    volumes:
+      - ./ollama/pipelines:/app/pipelines
+    extra_hosts:
+      - "host.docker.internal:host-gateway"
+    restart: always
     networks:
       - ollama-docker
@@ 行 50: / 行 73: @@
   * Ollama API 網址 - http://Server-IP:11434
+==== Open WebUI 設定 pipeline 使用雲端 LLM API ===
+  * 採用 pipeline 機制 - https://docs.openwebui.com/pipelines/pipes
+  - 設定 **連線** 新增 **http://host.docker.internal:9099** API Key: **0p3n-w3bu!** ++ 看畫面| \\ {{:tech:螢幕擷取畫面_2024-11-13_070742.png|}}++
+=== Anthropic API ===
+  - **管線(pipeline)** 從 ++GitHub URL 安裝| \\ Repo - https://github.com/open-webui/pipelines/tree/main/examples/pipelines \\ https://raw.githubusercontent.com/open-webui/pipelines/refs/heads/main/examples/pipelines/providers/anthropic_manifold_pipeline.py \\ {{:tech:螢幕擷取畫面_2024-11-13_071123.png|}}++
+  - ++下拉模型就可以選到 anthropic/claude 多個模型| \\ {{:tech:螢幕擷取畫面_2024-11-13_071537.png|}}++
+  - ++將不想使用的模型隱藏| \\ {{:tech:螢幕擷取畫面_2024-11-15_144257.png|}}++
+=== Gemini API ===
+  - **管線(pipeline)** 從 ++GitHub URL 安裝| \\ Repo - https://github.com/open-webui/pipelines/tree/main/examples/pipelines \\ https://raw.githubusercontent.com/open-webui/pipelines/refs/heads/main/examples/pipelines/providers/google_manifold_pipeline.py++
+==== Model =====
+  * [[https://ollama.com/ycchen/breeze-7b-instruct-v1_0|ycchen/breeze-7b-instruct-v1_0]]
+  * [[https://ollama.com/library/llama3|llama3]]
+  * [[https://ollama.com/library/codestral|codestral]]
+  * 下載語法 Exp. ycchen/breeze-7b-instruct-v1_0 <cli>
+docker exec ollama ollama pull ycchen/breeze-7b-instruct-v1_0
+</cli>
+==== 評估安裝環境運行 Model 的效能 =====
+  * Exp. phi3 <cli>
+pve-ollama-221:~# docker exec ollama ollama list
+NAME                                            ID              SIZE    MODIFIED
+phi3:14b                                        1e67dff39209    7.9 GB  5 days ago
+codestral:latest                                fcc0019dcee9    12 GB   5 days ago
+yabi/breeze-7b-32k-instruct-v1_0_q8_0:latest    ccc26fb14a68    8.0 GB  5 days ago
+phi3:latest                                     64c1188f2485    2.4 GB  5 days ago
+pve-ollama-221:~# docker exec -it ollama ollama run phi3 --verbose
+>>> hello
+ Hello! How can I assist you today? Whether it's answering a question, providing information, or helping with a task, feel free to ask.
+total duration:       3.119956759s
+load duration:        905.796µs
+prompt eval count:    5 token(s)
+prompt eval duration: 101.53ms
+prompt eval rate:     49.25 tokens/s
+eval count:           32 token(s)
+eval duration:        2.889224s
+eval rate:            11.08 tokens/s
+</cli>
 ===== VM 內直接安裝與測試 Ollama =====
   * 安裝程序<cli>