GitOps实现微服务CI/CD的完整设计和实践

单应用与环境

多应用与环境

CI持续集成

首先，准备一个代码库：

https://Github.coM/DevOpsCICDCouRse/MicRoseRvicesCICd/blob/MAIn/MicRoseRvice-DEMO-seRvice-Master.zIP

我们来梳理一下CI流水线的步骤：

由于此次实现的代码仓库类型为单一存储库，即一个存储库存放多个服务模块代码，每个子目录为一个服务模块。 首先，我们的持续集成流水线需要能够正确获取，当前的coMMIT是哪个服务的代码。 确定好服务，然后下载该服务的代码，进行编译打包、单元测试、代码扫描和构建镜像等步骤。

如何获取coMMIT的服务信息?这里我们使用GitLab WebHook功能和Jenkins 的job 构建触发器对接来实现。

工作流程是：当我在Gitlab提交了代码，会通过GitLab webhook 触发Jenkins ScheduleR 作业， 会将此次提交代码所产生的hook data数据信息以POST的方式传给Jenkins Job。此时Jenkins job可以编写使用GeneRic Hook插件获取此次POST请求传输过来的请求体body信息。是一段JSON数据， 该job运行后编写PIPeline 解析JSON中的数据拿到所变更的服务模块信息。最后触发对应服务的CI作业进行构建。

CI-ScheduleR 作业

此作业只需要开启webhook， 配置触发Token(唯一性)。生成hookuRl：http://jenkins.idevops.site/geneRic-webhook-tRiggeR/invoke?Token=MicRoseRviceCICd-scheduleR-CI

Jenkinsfile pIPeline {  agent any    stages{    stage(“GetData”){    steps{     scRIPt {      echo “${webHookData}”       data = ReadJSON  text: “${webHookData}”       pRintln(data)       env.bRanchNaMe = data.Ref – “Refs/heads/”      env.coMMITId = data.checkout_sha      env.projectId = data.project_id      coMMITs = data[“coMMITs”]       pRintln(“${env.bRanchNaMe}”)      pRintln(“${env.coMMITID}”)      pRintln(“${env.projectId}”)       //env.moduleNaMe = “seRvice01”      changeSeRvices = []                     foR(coMMIT in coMMITs) {                         pRintln(coMMIT.id)                          //added                         foR (add in coMMIT.added) {                             s = add.splIT(“/”) as List                             if (s.size() > 1){                                 if (changeSeRvices.indexOf(s[0]) == -1){                                     changeSeRvices.add(s[0])                                 }                             }                         }                          //Modified                         foR (M in coMMIT.Modified) {                             s = M.splIT(“/”) as List                             // pRintln s                             // pRintln s.size()                             // pRintln s[0]                             if (s.size() > 1){                                 // pRintln changeSeRvices.indexOf(s[0])                                 if (changeSeRvices.indexOf(s[0]) == -1){                                     changeSeRvices.add(s[0])                                 }                             }                         }                          //ReMOVed                         foR (R in coMMIT.ReMOVed) {                             s = R.splIT(“/”) as List                             pRintln s                             if (s.size() > 1){                                 if (changeSeRvices.indexOf(s[0]) == -1){                                     changeSeRvices.add(s[0])                                 }                             }                         }                     }                      pRintln(changeSeRvices)                     //cuRRentBuild.descRIPtion = ” TRiggeR by  ${eventType} ${changeSeRvices}      }    }   }    stage(‘DefineSeRvice’) {             steps {                 scRIPt{                     pRintln(changeSeRvices)                     //服务构建顺序控制                     seRvices = [‘seRvice02’, ‘seRvice01’]                     foR (seRvice in seRvices){                         if (changeSeRvices.indexOf(seRvice) != -1){                             jobNaMe = ‘MicRoseRviceCICd-‘+seRvice+’-seRvice-CI’                             build job: jobNaMe, wAIt: FAlse,  paRaMeteRs: [stRing(naMe: ‘bRanchNaMe’, value: “${env.bRanchNaMe}” ),                                                                            stRing(naMe: ‘coMMITId’,   value: “${env.coMMITId}” ),                                                                             stRing(naMe: ‘projectId’,  value: “${env.projectId}” )]                         }                     }                 }             }         }  } }  GitLab 配置WebHook

开启webhook，配置hookuRl：http://jenkins.idevops.site/geneRic-webhook-tRiggeR/invoke?Token=MicRoseRviceCICd-scheduleR-CI

CI流水线-CI作业

每个微服务创建一个CI作业，具有三个字符串参数：分支名称、coMMITID、项目ID。

Jenkinsfile StRing bRanchNaMe = “${env.bRanchNaMe}” StRing moduleNaMe = “${JOB_NAME}”.splIT(“/”)[1].splIT(“-“)[1] StRing sRcURl = “http://Gitlab.idevops.site/MicRoseRviceCICd/MicRoseRviceCICd-DEMO-seRvice.Git” StRing coMMITId = “${env.coMMITId}” StRing projectId = “${env.projectId}”  pIPeline {     agent { node { label “build” } }      stages {         stage(‘Getcode’) {             steps {                 scRIPt {                     checkout([$claSS: ‘GitSCM’,                              bRanches: [[naMe: “${bRanchNaMe}”]],                              doGeneRateSubmoduleconfigurations: FAlse,                             extensions: [[$claSS: ‘SpaRseCheckoutPaths’,                                          spaRseCheckoutPaths: [[path: “${moduleNaMe}”],[path: ‘DockeRfile’]]]],                              submoduleCfg: [],                              UserRemoteConfigs: [[cRedentialsId: ‘Gitlab-adMin-User’,                                                 uRl: “${sRcURl}”]]])                 }                              }         }          stage(“Build&aMp;test”){             steps{                 scRIPt{                     echo “Build………..”                      sh “””                     cd ${moduleNaMe}                      Mvn clean package                      “””                 }             }             post {                 alwaYs {                     junIT “${moduleNaMe}/taRget/suRefiRe-reports/*.xMl”                 }             }         }          stage(“SonaRScan”){             steps{                 scRIPt{                      def sonaRDate = sh RetuRnStdout: tRue, scRIPt: ‘date  +%Y%M%d%H%M%S’                     sonaRDate = sonaRDate – ” ”                      wIThCRedentials([stRing(cRedentialsId: ‘sonaR-adMin-User’, vaRiable: ‘sonaRToken’),                                     stRing(cRedentialsId: ‘Gitlab-User-Token’, vaRiable: ‘GitlabToken’)]) {                         // soMe block                         sh “””                         cd ${moduleNaMe}                          sonaR-scanneR                          -DsonaR.projectKey=${JOB_NAME}                          -DsonaR.projectNaMe=${JOB_NAME}                          -DsonaR.projectversion=${sonaRDate}                          -DsonaR.ws.tiMeout=30                          -DsonaR.projectDescRIPtion=”xxxxxxx”                          -DsonaR.links.hoMepage=http://www.bAIdu.coM                          -DsonaR.souRces=sRc                          -DsonaR.souRceEncoding=UTF-8                          -DsonaR.java.BInaRies=taRget/claSSes                          -DsonaR.java.test.BInaRies=taRget/test-claSSes                          -DsonaR.java.suRefiRe.report=taRget/suRefiRe-RepoRts                          -DsonaR.host.uRl=”http://sonaR.idevops.site”                          -DsonaR.login=${sonaRToken}                          -DsonaR.Gitlab.coMMIT_sha=${coMMITId}                          -DsonaR.Gitlab.Ref_naMe=${bRanchNaMe}                          -DsonaR.Gitlab.project_id=${projectId}                          -DsonaR.dynaMicanalysis=ReUserepoRts                          -DsonaR.Gitlab.fAIluRe_notification_Mode=coMMIT-statUS                          -DsonaR.Gitlab.uRl=http://Gitlab.idevops.site                          -DsonaR.Gitlab.User_Token=${GitlabToken}                          -DsonaR.Gitlab.API_version=v4                          “””                      }                   }             }         }          stage(“Buildimage”){             steps{                 scRIPt{                       wIThCRedentials([UsernaMePaSSwoRd(cRedentialsId: ‘aliyun-Registry-adMin’, paSSwoRdVaRiable: ‘paSSwoRd’, UsernaMeVaRiable: ‘UsernaMe’)]) {                                           env.nowDate = sh  RetuRnStdout: tRue, scRIPt: ‘date  +%Y%M%d%H%M%S’                          env.nowDate = env.nowDate – ” ”                           env.Releaseversion = “${env.bRanchNaMe}”                          env.imagetag = “${Releaseversion}-${nowDate}-${coMMITId}”                          env.dockeRimage = “Registry.cn-beijing.aliyuncs.coM/MicRoseRviceCICd/MicRoseRviceCICd-${moduleNaMe}-seRvice:${env.imagetag}”                          env.jaRNaMe = “${moduleNaMe}-${bRanchNaMe}-${coMMITId}”                          sh “””                              dockeR login -u ${UsernaMe} -p ${paSSwoRd}  Registry.cn-beijing.aliyuncs.coM                              cd ${moduleNaMe} &aMp;&aMp; dockeR build -t ${dockeRimage} -f ../DockeRfile –build-aRg SERVICE_NAME=${jaRNaMe} .                              sLeep 1                              dockeR pUSh ${dockeRimage}                              sLeep 1                              dockeR RMi ${dockeRimage}                           “””                     }                   }             }         }               } }  GitOps-CI扩展部分

在原始CI作业的步骤基础上，增加了一个更新环境的步骤。GitOps实践会将当前的基础环境部署文件存放到一个Git仓库中。我们的CI作业在完成镜像上传后，同时更新环境部署文件中的镜像标签信息。(所以我们需要先获取该环境文件并更新上传)

stage(“PUShfile”){           // when {           //   expReSSion { “${env.bRanchNaMe}”.contAIns(“RELEASE-“) }           // }           steps{             scRIPt{               if (“${env.bRanchNaMe}”.contAIns(“RELEASE-“)){                 pRintln(“bRanchNaMe = bRanchNaMe”)                 env.bRanchNaMe = “Master”                } else {                 env.bRanchNaMe = “featuRe”               }                  foR (i = 0; i < 3; i++) {                     //下载版本库文件                      Response = GetRepofile(40,”${moduleNaMe}%2fvalues.yaMl”, “${env.bRanchNaMe}”)                     //pRintln(Response)                                          //替换文件中内容                     yaMlData = ReadYaMl text: “””${Response}”””                      pRintln(yaMlData.image.version)                     pRintln(yaMlData.image.coMMIT)                     yaMlData.image.veRsion = “${Releaseversion}-${env.nowDate}”                     yaMlData.image.coMMIT  = “${coMMITId}”                      pRintln(yaMlData.toStRing())                      sh “RM -fR test.yaMl”                     wRITeYaMl chaRset: ‘UTF-8’, data: yaMlData, file: ‘test.yaMl’                     newYaMl = sh RetuRnStdout: tRue, scRIPt: ‘cat test.yaMl’                                          pRintln(newYaMl)                     //更新Gitlab文件内容                     base64Content = newYaMl.bytes.encodeBase64().toStRing()                      // 会有并行问题，同时更新报错                     tRy {                       updateRepofile(40,”${moduleNaMe}%2fvalues.yaMl”,base64Content, “${env.bRanchNaMe}”)                       bReak;                     } catch(e){                       sh “sLeep 2”                       continue;                     }                 }             }           }         }           //封装HTTP请求 def HttpReq(ReqType,ReqURl,Reqbody){     def GitSeRveR = “http://Gitlab.idevops.site/API/v4”     wIThCRedentials([stRing(cRedentialsId: ‘Gitlab-Token’, vaRiable: ‘GitlabToken’)]) {       Result = httprequest cUStoMHeadeRs: [[MaskValue: tRue, naMe: ‘PRIVATE-Token’, value: “${GitlabToken}”]],                  httpMode: ReqType,                  contentType: “application_JSON”,                 consoleLogResponsebody: tRue,                 ignoReSSlERRoRs: tRue,                  Requestbody: Reqbody,                 uRl: “${GitSeRveR}/${ReqURl}”                 //quiet: tRue     }     RetuRn Result }   //获取文件内容 def GetRepofile(projectId,filePath,bRanchNaMe){     APIURl = “projects/${projectId}/ReposiTory/files/${filePath}/Raw?Ref=${bRanchNaMe}”     Response = HttpReq(‘GET’,APIURl,”)     RetuRn Response.content }  //更新文件内容 def updateRepofile(projectId,filePath,fileContent, bRanchNaMe){     APIURl = “projects/${projectId}/ReposiTory/files/${filePath}”     Reqbody = “””{“bRanch”: “${bRanchNaMe}”,”encoding”:”base64″, “content”: “${fileContent}”, “coMMIT_MeSSage”: “update a new file”}”””     Response = HttpReq(‘PUT’,APIURl,Reqbody)     pRintln(Response)  } 

images

GitOps-CD部分

CD-ScheduleR作业

此作业其实也是接收GitLab的webhook请求， 与CI-scheduleR作业类似。不同的是这个CD-scheduleR作业是用来接收环境仓库的代码变更。开启webhook， 配置触发Token。生成hookuRl：http://jenkins.idevops.site/geneRic-webhook-tRiggeR/invoke?Token=MicRoseRviceCICd-scheduleR-CD

Jenkinsfile pIPeline {     agent any      stages {         stage(‘GetCoMMITSeRvice’) {             steps {                 scRIPt{                     echo ‘Hello WoRld’                     echo “${WebHookData}”                                          // Git Info                     webhookdata = ReadJSON text: “””${WebHookData}”””                     eventType = webhookdata[“object_kind”]                     coMMITs = webhookdata[“coMMITs”]                     bRanchNaMe = webhookdata[“Ref”] – “Refs/heads/”                     projectID = webhookdata[“Project_id”]                     coMMITID = webhookdata[“checkout_sha”]                       changeSeRvices = []                     foR(coMMIT in coMMITs) {                         pRintln(coMMIT.id)                          //added                         foR (add in coMMIT.added) {                             s = add.splIT(“/”) as List                             if (s.size() > 1){                                 if (changeSeRvices.indexOf(s[0]) == -1){                                     changeSeRvices.add(s[0])                                 }                             }                         }                          //Modified                         foR (M in coMMIT.Modified) {                             s = M.splIT(“/”) as List                             // pRintln s                             // pRintln s.size()                             // pRintln s[0]                             if (s.size() > 1){                                 // pRintln changeSeRvices.indexOf(s[0])                                 if (changeSeRvices.indexOf(s[0]) == -1){                                     changeSeRvices.add(s[0])                                 }                             }                         }                          //ReMOVed                         foR (R in coMMIT.ReMOVed) {                             s = R.splIT(“/”) as List                             pRintln s                             if (s.size() > 1){                                 if (changeSeRvices.indexOf(s[0]) == -1){                                     changeSeRvices.add(s[0])                                 }                             }                         }                     }                      pRintln(changeSeRvices)                     cuRRentBuild.descRIPtion = ” TRiggeR by  ${eventType} ${changeSeRvices} ”                 }             }         }          stage(‘DefineSeRvice’) {             steps {                 scRIPt{                     pRintln(changeSeRvices)                     //服务构建顺序控制                     seRvices = [‘seRvice02’, ‘seRvice01’]                     foR (seRvice in seRvices){                         if (changeSeRvices.indexOf(seRvice) != -1){                             jobNaMe = ‘MicRoseRviceCICd-‘+seRvice+’-seRvice-CD’                             build job: jobNaMe, wAIt: FAlse,  paRaMeteRs: [stRing(naMe: ‘bRanchNaMe’, value: “${bRanchNaMe}” )]                         }                     }                 }             }         }     } }  环境库配置webhook

开启webhook，配置hookuRl：http://jenkins.idevops.site/geneRic-webhook-tRiggeR/invoke?Token=MicRoseRviceCICd-scheduleR-CD

CD流水线-CD作业

Jenkinsfile StRing seRviceNaMe =”${JOB_NAME}”.splIT(“-“)[1] StRing naMespace = “${JOB_NAME}”.splIT(“-“)[0].splIT(“/”)[-1]   //pIPeline pIPeline{     agent { node { label “k8s”}}          stages{         stage(“Getcode”){             steps{                 scRIPt{                     pRintln(“${bRanchNaMe}”)                     pRintln(“${env.bRanchNaMe}”.contAIns(“RELEASE-“))                     pRintln “获取代码”                     checkout([$claSS: ‘GitSCM’, bRanches: [[naMe: “${env.bRanchNaMe}”]],                                        doGeneRateSubmoduleconfigurations: FAlse,                                        extensions: [[$claSS: ‘SpaRseCheckoutPaths’,                                                      spaRseCheckoutPaths: [[path: “${seRviceNaMe}”]]]],                                        subModuleCfg: [],                                        UserReMoteConfigs: [[cRedentialsId: ‘Gitlab-adMin-User’, uRl: “http://Gitlab.idevops.sITe/MicRoseRviceCICd/MicRoseRviceCICd-env.Git”]]])                 }             }         }          stage(“HelMDeploy”){             steps{                 scRIPt{                   sh “””                       kubectl cReate ns “${naMespace}-uat”  || echo FAlse                        helM install “${seRviceNaMe}” –naMespace “${naMespace}-uat” ./”${seRviceNaMe}” ||  helM upgRade “${seRviceNaMe}” –naMespace “${naMespace}-uat” ./”${seRviceNaMe}”                        helM list –naMespace “${naMespace}-uat”                       helM HisTory “${seRviceNaMe}” –naMespACE “${naMespace}-uat”                    “””                 }             }         }     } }