やりたいこと
JenkinsからKubernetesでPodを起動して処理を行うが、その際のvolumeのマウント先をパラメータビルドで動的に変更したい
結論
yamlの中にenv.XXXなどで指定しておいてもうまくいかない
指定する前に文字列として置換してからしてしないとうまくいかない
Jenkinsのパラメータ化設定
できない例1(${env.DATADIR})
コンテナの起動まで試行するがyamlの解析でエラー
jnlpのConnectまでは成功するが、
実際の処理を行うPod(下記ではscrapy)がCreateContainerErrorを繰り返し進まない
def labelname = "scrapy-down"
pipeline {
agent {
kubernetes {
label labelname
yaml '''
apiVersion: v1
kind: Pod
metadata:
labels:
scrapy: scrapy
spec:
volumes:
- name: data
hostPath:
path: ${env.DATADIR}
containers:
- name: scrapy
image: toru2220/scrapy-chrome
volumeMounts:
- name: data
mountPath: /data
command:
- cat
tty: true
resources:
requests:
memory: "256Mi"
cpu: "128m"
limits:
memory: "512Mi"
cpu: "512m"
'''
}
}
stages {
stage('scrapy-down') {
parallel {
stage('down') {
steps {
container('scrapy'){
sh """
ls -l /data
"""
}
}
}
}
}
}
}
Podの起動結果
# kubectl get pods
NAME READY STATUS RESTARTS AGE
scrapy-down-qh663-7r6zz 1/2 CreateContainerError 0 31s
できない例2(env.DATADIR)
コンテナの起動、処理の完了まで成功するが、volumeのマウントに失敗している
マウントしたいホスト側ボリューム
/mnt/testdir# ls -l
合計 0
-rw-r--r-- 1 root root 0 1月 27 21:57 testfile
def labelname = "scrapy-down"
pipeline {
agent {
kubernetes {
label labelname
yaml '''
apiVersion: v1
kind: Pod
metadata:
labels:
scrapy: scrapy
spec:
volumes:
- name: data
hostPath:
path: env.DATADIR
containers:
- name: scrapy
image: toru2220/scrapy-chrome
volumeMounts:
- name: data
mountPath: /data
command:
- cat
tty: true
resources:
requests:
memory: "256Mi"
cpu: "128m"
limits:
memory: "512Mi"
cpu: "512m"
'''
}
}
stages {
stage('scrapy-down') {
parallel {
stage('down') {
steps {
container('scrapy'){
sh """
ls -l /data
"""
}
}
}
}
}
}
}
Jenkinsログ
Started by user unknown or anonymous
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Still waiting to schedule task
All nodes of label ‘scrapy-down’ are offline
Agent scrapy-down-tdncp-1tv7j is provisioned from template Kubernetes Pod Template
---
apiVersion: "v1"
kind: "Pod"
metadata:
labels:
scrapy: "scrapy"
jenkins: "slave"
jenkins/label: "scrapy-down"
name: "scrapy-down-tdncp-1tv7j"
spec:
containers:
- command:
- "cat"
image: "toru2220/scrapy-chrome"
name: "scrapy"
resources:
limits:
memory: "512Mi"
cpu: "512m"
requests:
memory: "256Mi"
cpu: "128m"
tty: true
volumeMounts:
- mountPath: "/data"
name: "data"
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
image: "jenkins/jnlp-slave:3.35-5-alpine"
name: "jnlp"
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
nodeSelector:
beta.kubernetes.io/os: "linux"
restartPolicy: "Never"
securityContext: {}
volumes:
- hostPath:
path: "DATADIR"
name: "data"
- emptyDir:
medium: ""
name: "workspace-volume"
Running on scrapy-down-tdncp-1tv7j in /home/jenkins/agent/workspace/test
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (scrapy-down)
[Pipeline] parallel
[Pipeline] { (Branch: down)
[Pipeline] stage
[Pipeline] { (down)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ ls -l /data/
total 0
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS
成功例
yamlをgroovyの文字列として定義
kubernetesのyamlへ定義する前にgroovyの文字列置換(replaceAll)で値を変更したのち指定する
(非常に古典的)
def labelname = "scrapy-down"
def kubeyaml = """
apiVersion: v1
kind: Pod
metadata:
labels:
scrapy: scrapy
spec:
volumes:
- name: data
hostPath:
path: DATADIRVOL
containers:
- name: scrapy
image: toru2220/scrapy-chrome
volumeMounts:
- name: data
mountPath: /data
command:
- cat
tty: true
resources:
requests:
memory: 256Mi
cpu: 128m
limits:
memory: 512Mi
cpu: 512m
"""
pipeline {
agent {
kubernetes {
label labelname
yaml kubeyaml.replaceAll(/DATADIRVOL/,DATADIR)
}
}
stages {
stage('scrapy-down') {
parallel {
stage('down') {
steps {
container('scrapy'){
sh """
ls -l /data/
"""
}
}
}
}
}
}
}
Jenkinsログ
Started by user unknown or anonymous
Running in Durability level: MAX_SURVIVABILITY
[Pipeline] Start of Pipeline
[Pipeline] podTemplate
[Pipeline] {
[Pipeline] node
Still waiting to schedule task
‘Jenkins’ doesn’t have label ‘scrapy-down’
Agent scrapy-down-hd708-533lf is provisioned from template Kubernetes Pod Template
---
apiVersion: "v1"
kind: "Pod"
metadata:
labels:
scrapy: "scrapy"
jenkins: "slave"
jenkins/label: "scrapy-down"
name: "scrapy-down-hd708-533lf"
spec:
containers:
- command:
- "cat"
image: "toru2220/scrapy-chrome"
name: "scrapy"
resources:
limits:
memory: "512Mi"
cpu: "512m"
requests:
memory: "256Mi"
cpu: "128m"
tty: true
volumeMounts:
- mountPath: "/data"
name: "data"
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
image: "jenkins/jnlp-slave:3.35-5-alpine"
name: "jnlp"
volumeMounts:
- mountPath: "/home/jenkins/agent"
name: "workspace-volume"
readOnly: false
nodeSelector:
beta.kubernetes.io/os: "linux"
restartPolicy: "Never"
securityContext: {}
volumes:
- hostPath:
path: "/mnt/albion"
name: "data"
- emptyDir:
medium: ""
name: "workspace-volume"
Running on scrapy-down-hd708-533lf in /home/jenkins/agent/workspace/test
[Pipeline] {
[Pipeline] withEnv
[Pipeline] {
[Pipeline] stage
[Pipeline] { (scrapy-down)
[Pipeline] parallel
[Pipeline] { (Branch: down)
[Pipeline] stage
[Pipeline] { (down)
[Pipeline] container
[Pipeline] {
[Pipeline] sh
+ ls -l /data/
total 0
-rw-r--r-- 1 root root 0 1月 27 21:57 testfile
[Pipeline] }
[Pipeline] // container
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // parallel
[Pipeline] }
[Pipeline] // stage
[Pipeline] }
[Pipeline] // withEnv
[Pipeline] }
[Pipeline] // node
[Pipeline] }
[Pipeline] // podTemplate
[Pipeline] End of Pipeline
Finished: SUCCESS