目录
一、实验
1. 环境
2.Python代码实现获取文件
3.Python代码实现创建文件
4.Python代码实现更新文件
5.GitLab更新库文件与运行流水线
6.ArgoCD 完成前端项目应用发布
二、问题
1.Python获取GitLab指定仓库文件报错
2. K8S master节点运行Python代码报错
一、实验
1. 环境
(1)主机
表1 主机
主机 | 架构 | 版本 | IP | 备注 |
master1 | K8S master节点 | 1.20.6 | 192.168.204.180 | jenkins slave (从节点) |
argocd | 2.9.3 | 192.168.204.180:31767 | ||
node1 | K8S node节点 | 1.20.6 | 192.168.204.181 | |
node2 | K8S node节点 | 1.20.6 | 192.168.204.182 | |
jenkins | jenkins主节点 | 2.414.2 | 192.168.204.15:8080 | gitlabrunner (从节点) |
harbor私有仓库 | 1.2.2 | 192.168.204.15 | ||
python | 2.7.5 | 系统自带 | ||
gitlab | gitlab 主节点 | 12.10.14 | 192.168.204.8:82 | jenkins slave (从节点) |
sonarqube | 9.6 | 192.168.204.8:9000 |
2.Python代码实现获取文件
(1)GitLab官网查询通过API操作获取raw文件
Repository files API | GitLab
curl --header "PRIVATE-TOKEN: " "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fmodels%2Fkey%2Erb/raw?ref=main"
(2)GitLab 创建TOKEN
(3)GitLab查看项目编号
(4)Postman测试完成,转换成Python代码
import requestsurl = "http://192.168.204.8:82/api/v4/projects/22/repository/files/deployment.yaml/raw" />
(3)python实现获取GitLab指定仓库deplyment.yaml文件
import requestsimport jsonclass GitlabUtil():def __init__(self):self.gitlab_url = "http://192.168.204.8:82/api/v4"self.headers = {'PRIVATE-TOKEN': 'Z6RKxDgK7ort7i9K6f6p'}def http_req(self,method,apiUrl,data={}):url = "{0}/{1}".format(self.gitlab_url,apiUrl)response = requests.request(method,url,headers=self.headers,data=data)return response.textdef write_file(self,content,filePath):with open(filePath,'w') as f:f.write(content)def get_repo_file(self,projectId,filePath,branch,targetFile):apiurl = "projects/{0}/repository/files/{1}/raw" />
(4)运行Python代码(Windows11 Python环境为3.8)
(5)本地生成deployment.yaml
(6)K8S master节点同样运行Python代码进行测试(Python环境为2.7)
1)创建目录及python文件# mkdir pygitlabtest# cd pygitlabtest/# vim test.py# ls2)查看版本# python --version3)运行代码# python test.py
3.Python代码实现创建文件
(1)GitLab官网查询通过API操作在指定仓库创建文件
curl --request POST --header 'PRIVATE-TOKEN: ' \ --header "Content-Type: application/json" \ --data '{"branch": "main", "author_email": "author@example.com", "author_name": "Firstname Lastname", "content": "some content", "commit_message": "create a new file"}' \ "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fproject%2Erb"
(2)Postman测试完成,转换成Python代码
import requestsimport jsonurl = "http://192.168.204.8:82/api/v4/projects/22/repository/files/demo.yaml"payload = json.dumps({"branch": "master","content": "Hello World","commit_message": "commmit by autorobot"})headers = {'PRIVATE-TOKEN': 'Z6RKxDgK7ort7i9K6f6p','Content-Type': 'application/json'}response = requests.request("POST", url, headers=headers, data=payload)print(response.text)
(3)GitLab查看项目生成了demo.yaml文件
(4)python实现在GitLab指定项目创建demo02.yaml文件
# -*- coding: utf-8 -*-import requestsimport jsonimport base64class GitlabUtil():def __init__(self,projectId):self.gitlab_url = "http://192.168.204.8:82/api/v4"self.gitlab_token = 'Z6RKxDgK7ort7i9K6f6p'self.projectId = projectIdself.encoding = "base64"def http_req(self,method,apiUrl,headers,data):url = "{0}/{1}".format(self.gitlab_url,apiUrl)response = requests.request(method,url,headers=headers,data=data)return response.textdef write_file(self,content,filePath):with open(filePath,'w') as f:f.write(content)def get_repo_file(self,filePath,branch,targetFile):apiurl = "projects/{0}/repository/files/{1}/raw" />
(5)运行Python代码(Windows11 Python环境为3.8)
(6)GitLab查看项目生成了demo2.yaml文件
4.Python代码实现更新文件
(1)GitLab官网查询通过API操作在指定仓库更新文件
curl --request PUT --header 'PRIVATE-TOKEN: ' \ --header "Content-Type: application/json" \ --data '{"branch": "main", "author_email": "author@example.com", "author_name": "Firstname Lastname", "content": "some content", "commit_message": "update file"}' \ "https://gitlab.example.com/api/v4/projects/13083/repository/files/app%2Fproject%2Erb"
(2)python实现在GitLab指定项目更新demo02.yaml文件
# -*- coding: utf-8 -*-import requestsimport jsonimport base64class GitlabUtil():def __init__(self,projectId):self.gitlab_url = "http://192.168.204.8:82/api/v4"self.gitlab_token = 'Z6RKxDgK7ort7i9K6f6p'self.projectId = projectIdself.encoding = "base64"def http_req(self,method,apiUrl,headers,data):url = "{0}/{1}".format(self.gitlab_url,apiUrl)response = requests.request(method,url,headers=headers,data=data)return response.textdef write_file(self,content,filePath):with open(filePath,'w') as f:f.write(content)# 下载文件def get_repo_file(self,filePath,branch,targetFile):apiurl = "projects/{0}/repository/files/{1}/raw" />
(3)运行Python代码(Windows11 Python环境为3.8)
(4)GitLab查看项目更新了提交信息
5.GitLab更新库文件与运行流水线
(1)查看GitLab共享库
(2)新建流水线文件ui.gitlabutil.yaml
(3)复制raw格式
(4)在GitLab devops03-devops-env 环境库项目添加CI配置文件路径
(5)查看前端项目devops03-devops-ui 修改Dockerfile,注释本地CI流水线文件,避免后续运行错误
(6)查看前端项目目录下的index.html文件
(7) GitLab共享库新建目录util及GitLabUtil.py文件,用来封装python类
(8)GitLabUtil.py文件代码
import requestsimport jsonimport base64import sysclass GitlabUtil():def __init__(self,projectId):self.gitlab_url = "http://192.168.204.8:82/api/v4"self.gitlab_token = 'Z6RKxDgK7ort7i9K6f6p'self.projectId = projectIdself.encoding = "base64"def http_req(self,method,apiUrl,headers,data):url = "{0}/{1}".format(self.gitlab_url,apiUrl)response = requests.request(method,url,headers=headers,data=data)return response.textdef write_file(self,content,filePath):with open(filePath,'w') as f:f.write(content) def get_repo_file(self,filePath,branch,targetFile):apiurl = "projects/{0}/repository/files/{1}/raw" />
(9)修改流水线文件ui.gitlabutil.yaml
include: - project: 'devops03/devops03-gitlabci-lib' ref: master file: "/jobs/CI.yaml"workflow:rules:#新建分支永远不执行- if: $CI_PIPELINE_SOURCE == "web"#允许手动触发when: always- if: $CI_COMMIT_BEFORE_SHA == "0000000000000000000000000000000000000000"when: never# 其他情况永远执行- when: always#取消每个阶段自动下载代码,即全局关闭作业代码下载variables:GIT_CHECKOUT: "false"## 全局关闭作业代码下载PROJECT_TYPE: "npm" ## 定义项目类型BUILD_SHELL: "npm run build" ## 构建命令TEST_SHELL: "echo test" ## 测试命令ARTIFACT_PATH: "dist/**"## 制品路径 # TEST_REPORTS: "target/surefire-reports/TEST-*.xml" ## 测试报告stages:- build- sonarscan- dockerbuild- releasefilepipelineInit:extends: - .pipelineInitcibuild:before_script: - "npm install"extends: - .cibuildreleasefile:tags:- buildstage: releasefilescript:- curl"http://192.168.204.8:82/devops03/devops03-gitlabci-lib/-/raw/master/utils/GitLabUtil.py" -o GitLabUtil.py -s- python GitLabUtil.py getfile "22" "deployment.yaml" "master" "deployment.yaml"- ls -l- imageName=192.168.204.15/${CI_PROJECT_NAMESPACE}/${CI_PROJECT_NAME}:${CI_COMMIT_SHA}- sed -i 's#__PORT__#80#g' deployment.yaml- sed -i "s#__APPNAME__#${CI_PROJECT_NAME}#g" deployment.yaml- sed -i "s#__NAMESPACE__#${CI_PROJECT_NAMESPACE}#g" deployment.yaml- sed -i "s#__IMAGENAME__#${imageName}#g" deployment.yaml- python GitLabUtil.py updatefile "22" "deployment.yaml" "master" "${CI_PROJECT_NAME}%2f${CI_COMMIT_BRANCH}.yaml"#sonarscan:# extends:# - .sonarscan#pushartifact:# extends:# - .pushartifactdockerbuild:extends: - .dockerbuild
(10)共享库完整目录
(11)GitLab 前端项目运行流水线
(12)完成
(13)查看各阶段日志
(14)环境库项目显示更新master.yaml文件
(15) 查看master.yaml文件(已同步更新端口号、命名空间、项目名称及镜像名称)
6.ArgoCD 完成前端项目应用发布
(1)K8S查看集群状态
# kubectl get node
(2)K8S master节点另开一个终端用watch命令观察pod变化
# watch -n 1 "kubectl get pod -n devops03"
(3)外部测试访问
# curl http://devops03-devops-ui.devops.com:31291
(4)K8S 删除命名空间devops03
# kubectl delete ns devops03
(5) 观察pod变化
(5)ArgoCD 查看已有项目
(6)ArgoCD 删除已有项目
(7)GitLab环境库注释其他的yaml文件
(8)ArgoCD 创建application (手动策略、自动创建命名空间、指定仓库与集群)
(9)填写配置
(10)完成创建
(11)点击 SYNCHRONIZE (同步)
(12)观察pod变化
(13)ArgoCD 观察pod变化
(14)查看Harbor仓库镜像文件
(15)K8S node节点连接Harbor拉取镜像
# docker login -u admin -p Harbor12345 192.168.204.15 # docker pull 192.168.204.15/devops03/devops03-devops-ui:RELEASE-1.1.7 # docker logout 192.168.204.15
node1 节点
node2节点
(16)观察pod变化
(17)ArgoCD 再次观察pod变化
(18)外部测试访问
# curl http://devops03-devops-ui.devops.com:31291
二、问题
1.Python获取GitLab指定仓库文件报错
(1)报错
(2)原因分析
函数名错误
(3)解决方法
修改函数名称。
修改前:
修改后:
2. K8S master节点运行Python代码报错
(1)报错
(2)原因分析
encoding不是有效的关键词。
(3)解决方法
去掉encoding。
修改前:
修改后:
成功:
3. GitLabCI 运行流水线报错
(1)报错
(2)原因分析
行尾缺少双引号
(3)解决方法
添加双引号。
成功: