目录

一、实验

1. 环境

2.Python代码实现获取文件

3.Python代码实现创建文件

4.Python代码实现更新文件

5.GitLab更新库文件与运行流水线

6.ArgoCD 完成前端项目应用发布

二、问题

1.Python获取GitLab指定仓库文件报错

2. K8S master节点运行Python代码报错


一、实验

1. 环境

(1)主机

表1 主机

主机架构版本IP备注
master1K8S master节点1.20.6192.168.204.180

jenkins slave

(从节点)

argocd2.9.3192.168.204.180:31767
node1K8S node节点1.20.6192.168.204.181
node2K8S node节点1.20.6192.168.204.182
jenkins

jenkins主节点

2.414.2192.168.204.15:8080

gitlabrunner

(从节点)

harbor私有仓库1.2.2192.168.204.15
python2.7.5系统自带
gitlabgitlab 主节点12.10.14192.168.204.8:82

jenkins slave

(从节点)

sonarqube9.6192.168.204.8:9000

2.Python代码实现获取文件

(1)GitLab官网查询通过API操作获取raw文件

Repository files API | GitLab

curl --header "PRIVATE-TOKEN: " "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fmodels%2Fkey%2Erb/raw?ref=main"

(2)GitLab 创建TOKEN

(3)GitLab查看项目编号

(4)Postman测试完成,转换成Python代码

import requestsurl = "http://192.168.204.8:82/api/v4/projects/22/repository/files/deployment.yaml/raw" />

(3)python实现获取GitLab指定仓库deplyment.yaml文件

import requestsimport jsonclass GitlabUtil():def __init__(self):self.gitlab_url = "http://192.168.204.8:82/api/v4"self.headers = {'PRIVATE-TOKEN': 'Z6RKxDgK7ort7i9K6f6p'}def http_req(self,method,apiUrl,data={}):url = "{0}/{1}".format(self.gitlab_url,apiUrl)response = requests.request(method,url,headers=self.headers,data=data)return response.textdef write_file(self,content,filePath):with open(filePath,'w') as f:f.write(content)def get_repo_file(self,projectId,filePath,branch,targetFile):apiurl = "projects/{0}/repository/files/{1}/raw" />

(4)运行Python代码(Windows11 Python环境为3.8)

(5)本地生成deployment.yaml

(6)K8S master节点同样运行Python代码进行测试(Python环境为2.7)

1)创建目录及python文件# mkdir pygitlabtest# cd pygitlabtest/# vim test.py# ls2)查看版本# python --version3)运行代码# python test.py 

3.Python代码实现创建文件

(1)GitLab官网查询通过API操作在指定仓库创建文件

curl --request POST --header 'PRIVATE-TOKEN: ' \ --header "Content-Type: application/json" \ --data '{"branch": "main", "author_email": "author@example.com", "author_name": "Firstname Lastname", "content": "some content", "commit_message": "create a new file"}' \ "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fproject%2Erb"

(2)Postman测试完成,转换成Python代码

import requestsimport jsonurl = "http://192.168.204.8:82/api/v4/projects/22/repository/files/demo.yaml"payload = json.dumps({"branch": "master","content": "Hello World","commit_message": "commmit by autorobot"})headers = {'PRIVATE-TOKEN': 'Z6RKxDgK7ort7i9K6f6p','Content-Type': 'application/json'}response = requests.request("POST", url, headers=headers, data=payload)print(response.text)

(3)GitLab查看项目生成了demo.yaml文件

(4)python实现在GitLab指定项目创建demo02.yaml文件

# -*- coding: utf-8 -*-import requestsimport jsonimport base64class GitlabUtil():def __init__(self,projectId):self.gitlab_url = "http://192.168.204.8:82/api/v4"self.gitlab_token = 'Z6RKxDgK7ort7i9K6f6p'self.projectId = projectIdself.encoding = "base64"def http_req(self,method,apiUrl,headers,data):url = "{0}/{1}".format(self.gitlab_url,apiUrl)response = requests.request(method,url,headers=headers,data=data)return response.textdef write_file(self,content,filePath):with open(filePath,'w') as f:f.write(content)def get_repo_file(self,filePath,branch,targetFile):apiurl = "projects/{0}/repository/files/{1}/raw" />

(5)运行Python代码(Windows11 Python环境为3.8)

(6)GitLab查看项目生成了demo2.yaml文件

4.Python代码实现更新文件

(1)GitLab官网查询通过API操作在指定仓库更新文件

curl --request PUT --header 'PRIVATE-TOKEN: ' \ --header "Content-Type: application/json" \ --data '{"branch": "main", "author_email": "author@example.com", "author_name": "Firstname Lastname", "content": "some content", "commit_message": "update file"}' \ "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fproject%2Erb"

(2)python实现在GitLab指定项目更新demo02.yaml文件

# -*- coding: utf-8 -*-import requestsimport jsonimport base64class GitlabUtil():def __init__(self,projectId):self.gitlab_url = "http://192.168.204.8:82/api/v4"self.gitlab_token = 'Z6RKxDgK7ort7i9K6f6p'self.projectId = projectIdself.encoding = "base64"def http_req(self,method,apiUrl,headers,data):url = "{0}/{1}".format(self.gitlab_url,apiUrl)response = requests.request(method,url,headers=headers,data=data)return response.textdef write_file(self,content,filePath):with open(filePath,'w') as f:f.write(content)# 下载文件def get_repo_file(self,filePath,branch,targetFile):apiurl = "projects/{0}/repository/files/{1}/raw" />

(3)运行Python代码(Windows11 Python环境为3.8)

(4)GitLab查看项目更新了提交信息

5.GitLab更新库文件与运行流水线

(1)查看GitLab共享库

(2)新建流水线文件ui.gitlabutil.yaml

(3)复制raw格式

(4)在GitLab devops03-devops-env 环境库项目添加CI配置文件路径

(5)查看前端项目devops03-devops-ui 修改Dockerfile,注释本地CI流水线文件,避免后续运行错误

(6)查看前端项目目录下的index.html文件

(7) GitLab共享库新建目录util及GitLabUtil.py文件,用来封装python类

(8)GitLabUtil.py文件代码

import requestsimport jsonimport base64import sysclass GitlabUtil():def __init__(self,projectId):self.gitlab_url = "http://192.168.204.8:82/api/v4"self.gitlab_token = 'Z6RKxDgK7ort7i9K6f6p'self.projectId = projectIdself.encoding = "base64"def http_req(self,method,apiUrl,headers,data):url = "{0}/{1}".format(self.gitlab_url,apiUrl)response = requests.request(method,url,headers=headers,data=data)return response.textdef write_file(self,content,filePath):with open(filePath,'w') as f:f.write(content) def get_repo_file(self,filePath,branch,targetFile):apiurl = "projects/{0}/repository/files/{1}/raw" />

(9)修改流水线文件ui.gitlabutil.yaml

include: - project: 'devops03/devops03-gitlabci-lib' ref: master file: "/jobs/CI.yaml"workflow:rules:#新建分支永远不执行- if: $CI_PIPELINE_SOURCE == "web"#允许手动触发when: always- if: $CI_COMMIT_BEFORE_SHA == "0000000000000000000000000000000000000000"when: never# 其他情况永远执行- when: always#取消每个阶段自动下载代码,即全局关闭作业代码下载variables:GIT_CHECKOUT: "false"## 全局关闭作业代码下载PROJECT_TYPE: "npm" ## 定义项目类型BUILD_SHELL: "npm run build" ## 构建命令TEST_SHELL: "echo test" ## 测试命令ARTIFACT_PATH: "dist/**"## 制品路径 # TEST_REPORTS: "target/surefire-reports/TEST-*.xml" ## 测试报告stages:- build- sonarscan- dockerbuild- releasefilepipelineInit:extends: - .pipelineInitcibuild:before_script: - "npm install"extends: - .cibuildreleasefile:tags:- buildstage: releasefilescript:- curl"http://192.168.204.8:82/devops03/devops03-gitlabci-lib/-/raw/master/utils/GitLabUtil.py" -o GitLabUtil.py -s- python GitLabUtil.py getfile "22" "deployment.yaml" "master" "deployment.yaml"- ls -l- imageName=192.168.204.15/${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME}:${CI_COMMIT_SHA}- sed -i 's#__PORT__#80#g' deployment.yaml- sed -i "s#__APPNAME__#${CI_PROJECT_NAME}#g" deployment.yaml- sed -i "s#__NAMESPACE__#${CI_PROJECT_NAMESPACE}#g" deployment.yaml- sed -i "s#__IMAGENAME__#${imageName}#g" deployment.yaml- python GitLabUtil.py updatefile "22" "deployment.yaml" "master" "${CI_PROJECT_NAME}%2f${CI_COMMIT_BRANCH}.yaml"#sonarscan:# extends:# - .sonarscan#pushartifact:# extends:# - .pushartifactdockerbuild:extends: - .dockerbuild

(10)共享库完整目录

(11)GitLab 前端项目运行流水线

(12)完成

(13)查看各阶段日志

(14)环境库项目显示更新master.yaml文件

(15) 查看master.yaml文件(已同步更新端口号、命名空间、项目名称及镜像名称)

6.ArgoCD 完成前端项目应用发布

(1)K8S查看集群状态

# kubectl get node

(2)K8S master节点另开一个终端用watch命令观察pod变化

# watch -n 1 "kubectl get pod -n devops03"

(3)外部测试访问

# curl http://devops03-devops-ui.devops.com:31291

(4)K8S 删除命名空间devops03

# kubectl delete ns devops03

(5) 观察pod变化

(5)ArgoCD 查看已有项目

(6)ArgoCD 删除已有项目

(7)GitLab环境库注释其他的yaml文件

(8)ArgoCD 创建application (手动策略、自动创建命名空间、指定仓库与集群)

(9)填写配置

(10)完成创建

(11)点击 SYNCHRONIZE (同步)

(12)观察pod变化

(13)ArgoCD 观察pod变化

(14)查看Harbor仓库镜像文件

(15)K8S node节点连接Harbor拉取镜像

# docker login -u admin -p Harbor12345 192.168.204.15 # docker pull 192.168.204.15/devops03/devops03-devops-ui:RELEASE-1.1.7 # docker logout 192.168.204.15

node1 节点

node2节点

(16)观察pod变化

(17)ArgoCD 再次观察pod变化

(18)外部测试访问

# curl http://devops03-devops-ui.devops.com:31291

二、问题

1.Python获取GitLab指定仓库文件报错

(1)报错

(2)原因分析

函数名错误

(3)解决方法

修改函数名称。

修改前:

修改后:

2. K8S master节点运行Python代码报错

(1)报错

(2)原因分析

encoding不是有效的关键词。

(3)解决方法

去掉encoding。

修改前:

修改后:

成功:

3. GitLabCI 运行流水线报错

(1)报错

(2)原因分析

行尾缺少双引号

(3)解决方法

添加双引号。

成功: