背景

概念

Flowise

Flowise 是一个开源的用户界面可视化工具，它允许用户通过拖放的方式来构建自定义的大型语言模型（LLM）流程。
Flowise基于LangChain.js，是一个非常先进的图形用户界面，用于开发基于LLM的应用程序。Flowise还支持Docker和NodeJS，可以在本地或者在容器中运行。如果有需求修改相关代码，可以本地部署运行。

Langflow

Langflow 是一个用于创建和运行语言模型流程的网络应用程序。
Langflow是一个动态图，每个节点都是一个可执行单元¹。它的模块化和交互式设计促进了快速的实验和原型制作。Langflow还提供了一种无代码的AI生态系统，可以无缝地与您的团队熟悉和喜欢的工具和堆栈集成。

Localai

LocalAI是一个免费的、开源的OpenAI替代品。它是一个与OpenAI API规范兼容的REST API，允许你在本地或者在本地网络上使用消费级硬件运行大型语言模型（LLM）。它支持多个与ggml格式兼容的模型系列。

1.Flowise安装部署

git

https://github.com/FlowiseAI/Flowise

官网

FlowiseAI – Build LLMs Apps Easily

docker部署

[root@master docker]# git clone https://github.com/FlowiseAI/Flowise.git[root@master langchain_wk]# lsFlowise[root@master langchain_wk]# cd Flowise/[root@master Flowise]# lsartillery-load-test.ymlbabel.config.js CODE_OF_CONDUCT-ZH.mdCONTRIBUTING-ZH.mdDockerfileLICENSE.mdpackages README-ZH.mdassets CODE_OF_CONDUCT.mdCONTRIBUTING.mddockerimagespackage.jsonREADME.mdturbo.json[root@master Flowise]# clear[root@master Flowise]# lsartillery-load-test.ymlbabel.config.js CODE_OF_CONDUCT-ZH.mdCONTRIBUTING-ZH.mdDockerfileLICENSE.mdpackages README-ZH.mdassets CODE_OF_CONDUCT.mdCONTRIBUTING.mddockerimagespackage.jsonREADME.mdturbo.json[root@master Flowise]# cd docker/[root@master docker]# lsdocker-compose.ymlDockerfileREADME.md[root@master docker]# ls -alh总用量 24Kdrwxr-xr-x 2 root root 4.0K 11月 21 13:21 .drwxr-xr-x 9 root root 4.0K 11月 21 13:21 ..-rw-r--r-- 1 root root988 11月 21 13:21 docker-compose.yml-rw-r--r-- 1 root root441 11月 21 13:21 Dockerfile-rw-r--r-- 1 root root675 11月 21 13:21 .env.example-rw-r--r-- 1 root root 1.3K 11月 21 13:21 README.md[root@master docker]# cp .env.example .env[root@master docker]# vim .env[root@master docker]#

修改.env配置文件

PORT=3090DATABASE_PATH=/root/.flowiseAPIKEY_PATH=/root/.flowiseSECRETKEY_PATH=/root/.flowiseLOG_PATH=/root/.flowise/logs# NUMBER_OF_PROXIES= 1# DATABASE_TYPE=postgres# DATABASE_PORT=""# DATABASE_HOST=""# DATABASE_NAME="flowise"# DATABASE_USER=""# DATABASE_PASSWORD=""# FLOWISE_USERNAME=user# FLOWISE_PASSWORD=1234# FLOWISE_SECRETKEY_OVERWRITE=myencryptionkeyDEBUG=true# LOG_LEVEL=debug (error | warn | info | verbose | debug)# TOOL_FUNCTION_BUILTIN_DEP=crypto,fs# TOOL_FUNCTION_EXTERNAL_DEP=moment,lodash# LANGCHAIN_TRACING_V2=true# LANGCHAIN_ENDPOINT=https://api.smith.langchain.com# LANGCHAIN_API_KEY=your_api_key# LANGCHAIN_PROJECT=your_project

docker-compose启动

[root@master docker]# docker-compose up -dWARNING: The FLOWISE_USERNAME variable is not set. Defaulting to a blank string.WARNING: The FLOWISE_PASSWORD variable is not set. Defaulting to a blank string.WARNING: The DATABASE_TYPE variable is not set. Defaulting to a blank string.WARNING: The DATABASE_PORT variable is not set. Defaulting to a blank string.WARNING: The DATABASE_HOST variable is not set. Defaulting to a blank string.WARNING: The DATABASE_NAME variable is not set. Defaulting to a blank string.WARNING: The DATABASE_USER variable is not set. Defaulting to a blank string.WARNING: The DATABASE_PASSWORD variable is not set. Defaulting to a blank string.WARNING: The FLOWISE_SECRETKEY_OVERWRITE variable is not set. Defaulting to a blank string.WARNING: The LOG_LEVEL variable is not set. Defaulting to a blank string.WARNING: Found orphan containers (supabase-rest, supabase-studio, supabase-edge-functions, supabase-auth, realtime-dev.supabase-realtime, supabase-imgproxy, supabase-storage, supabase-meta, supabase-db, supabase-kong) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.Pulling flowise (flowiseai/flowise:)...latest: Pulling from flowiseai/flowise96526aa774ef: Pull complete3130715204cf: Pull completeb06de8ab1c4f: Pull complete90ef3ffc5156: Pull complete018fd88f5f14: Pull completee405bd0a6e44: Pull complete6678d95b4ce0: Pull completef0b6d32b7e1f: Pull complete754763d6e863: Pull completedb86a2a89c8a: Pull completeDigest: sha256:348211f573335adca3fdcfa1d62345eadee65e19cc3d83301f96b0359f31dc03Status: Downloaded newer image for flowiseai/flowise:latestCreating docker_flowise_1 ... done[root@master docker]# docker-compose logs -fAttaching to docker_flowise_1flowise_1| 2023-11-21 05:30:05 [INFO]: Starting Flowise...flowise_1| 2023-11-21 05:30:06 [INFO]: ⚡️ [server]: Flowise Server is listening at 3090flowise_1| 2023-11-21 05:30:06 [INFO]:[server]: Data Source has been initialized!

这里不多做flowise的教程，可以参考相关官方文档。

2.Localai 部署

注意：这里默认用的是cpu，localai主推用CPU，LocalAI具有Rust后端，内存效率高，体积小,并且后端模型部署gRpc,至于gpu部署，需要不同的nvidia cuda版本，可根据自己的cuda版本拉取相关镜像，然后重新build镜像；

build镜像会非常，最好使用代理或更换国内相关系统镜像源，rust源，go源，pip源；

# docker images | grep nvcr.io/nvidia/cudanvcr.io/nvidia/cuda11.2.1-cudnn8-devel-ubuntu20.04 b74328ac303f 5 months ago7.72GBnvcr.io/nvidia/cuda11.4.2-cudnn8-devel-ubi8260b885046fb 5 months ago9.76GBnvcr.io/nvidia/cuda11.4.0-cudnn8-devel-centos7 32f9c23be6b7 5 months ago9.69GBnvcr.io/nvidia/cuda11.4.2-cudnn8-runtime-ubi816112a9b5038 5 months ago3.89GB

这里用的11.2.1-cudnn8-devel-ubuntu20.04；

GPU的部署和编译相对比较麻烦，可参考官方文档。

GPU编译方法

make BUILD_TYPE=cublas build

启动方法

# ./local-ai --debug=true --f16=true --autoload-galleries=true3:17PM INF Starting LocalAI using 4 threads, with models path: /build/models3:17PM INF LocalAI version: v1.40.0-22-g991ecce (991ecce00462db955934ec1293954b7d61c58ad2)3:17PM DBG Model: bert-embeddings (config: {PredictionOptions:{Model:bert-MiniLM-L6-v2q4_0.bin Language: N:0 TopP:0 TopK:0 Temperature:0 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:bert-embeddings F16:false Threads:0 Debug:false Roles:map[] Embeddings:true Backend:bert-embeddings TemplateConfig:{Chat: ChatMessage: Completion: Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:0 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:}})3:17PM DBG Model: thebloke__wizardlm-13b-v1-0-uncensored-superhot-8k-ggml__wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_k_m.bin (config: {PredictionOptions:{Model:wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_K_M.bin Language: N:0 TopP:0.7 TopK:80 Temperature:0.2 Maxtokens:0 Echo:false Batch:0 F16:false IgnoreEOS:false RepeatPenalty:0 Keep:0 MirostatETA:0 MirostatTAU:0 Mirostat:0 FrequencyPenalty:0 TFZ:0 TypicalP:0 Seed:0 NegativePrompt: RopeFreqBase:0 RopeFreqScale:0 NegativePromptScale:0 UseFastTokenizer:false ClipSkip:0 Tokenizer:} Name:thebloke__wizardlm-13b-v1-0-uncensored-superhot-8k-ggml__wizardlm-13b-v1.0-superhot-8k.ggmlv3.q4_k_m.bin F16:false Threads:0 Debug:false Roles:map[] Embeddings:false Backend: TemplateConfig:{Chat:wizardlm-chat ChatMessage: Completion:wizardlm-completion Edit: Functions:} PromptStrings:[] InputStrings:[] InputToken:[] functionCallString: functionCallNameString: FunctionsConfig:{DisableNoAction:false NoActionFunctionName: NoActionDescriptionName:} FeatureFlag:map[] LLMConfig:{SystemPrompt: TensorSplit: MainGPU: RMSNormEps:0 NGQA:0 PromptCachePath: PromptCacheAll:false PromptCacheRO:false MirostatETA:0 MirostatTAU:0 Mirostat:0 NGPULayers:0 MMap:false MMlock:false LowVRAM:false Grammar: StopWords:[] Cutstrings:[] TrimSpace:[] ContextSize:1024 NUMA:false LoraAdapter: LoraBase: LoraScale:0 NoMulMatQ:false DraftModel: NDraft:0 Quantization: MMProj: RopeScaling: YarnExtFactor:0 YarnAttnFactor:0 YarnBetaFast:0 YarnBetaSlow:0} AutoGPTQ:{ModelBaseName: Device: Triton:false UseFastTokenizer:false} Diffusers:{PipelineType: SchedulerType: CUDA:false EnableParameters: CFGScale:0 IMG2IMG:false ClipSkip:0 ClipModel: ClipSubFolder:} Step:0 GRPC:{Attempts:0 AttemptsSleepTime:0} VallE:{AudioPath:}})3:17PM DBG Extracting backend assets files to /tmp/localai/backend_data ┌───────────────────────────────────────────────────┐ │ Fiber v2.50.0 │ │ http://127.0.0.1:8080 │ │ (bound on host 0.0.0.0 and port 8080) │ │ │ │ Handlers ............ 74Processes ........... 1 │ │ Prefork ....... DisabledPID ............. 10297 │ └───────────────────────────────────────────────────┘

MODEL GALLERY 包含了huggingface的项目，如果下载模型，可能回被墙，至于如何解决，请自行解决。

下载模型

[192.168.12.82]:62589 200 - POST /models/apply3:41PM DBG Checking "bert-MiniLM-L6-v2q4_0.bin" exists and matches SHA3:41PM DBG File "bert-MiniLM-L6-v2q4_0.bin" already exists and matches the SHA. Skipping download3:41PM DBG Written config file /build/models/bert-embeddings.yaml

编码测试

3.Flowise文档知识库召回应用

转载请注明出处！