Recently I was working creating a pipeline in gitlab for a CDK stack and I encounter this PythonFunction:
const dbInitScript = new PythonFunction(this, 'DbInitScript', {
entry: 'assets/init-db',
runtime: Runtime.PYTHON_3_11,
handler: 'index.handler',
logRetention: RetentionDays.ONE_WEEK,
timeout: Duration.seconds(10),
vpc: this.coreVpc,
vpcSubnets: { subnetType: SubnetType.PRIVATE_ISOLATED },
bundling: {
bundlingFileAccess: BundlingFileAccess.VOLUME_COPY,
},
});
In the PythonFunction from @aws-cdk/aws-lambda-python-alpha module a lambda is been created and the entry point is in the assets/init-db path, there is a python script to execute some SQL statements when the stack is created and the requeriments.txt in my case with these python modules:
cfnresponse>=1.1.1,<1.99 pymysql>=1.0.2,<1.99 urllib3<2.0.0,>=1.26.0
so when you run cdk synth in the pipeline, cdk see there are dependencies that needs to be installed it will spin up docker in order to create the bundle with the python script and the dependencies. In order this work in a gitlab pipeline the image you are using needs to have docker cli installed and also these variables needs to be present in the gitlab-ci-yml file:
services: - docker:dind variables: DOCKER_TLS_CERTDIR: '' DOCKER_HOST: tcp://docker:2375 DOCKER_DRIVER: overlay
docker in docker (docker:dind) service includes all the docker commands CDK needs to build the docker image. In order both dockers containers can communicate between them, it has to be through tcp, that is why the variable DOCKER_HOST needs to be tcp://docker:2375.
By default docker:dind generates certificates but in our case we need to disable it, that is why the variable DOCKER_TLS_CERTDIR is setup to be empty.
For more information you can see this post and this forum post. This is in order to not have this error when trying to run cdk synth in the pipeline:
👾 build » post-compile » synth:silent | cdk synth -q
ERROR: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?With this in place the pipeline should be able to spin up docker successfully.
Other error I faced while running the unit test in the pipeline was this one:
AWS CDK Gitlab CI - RuntimeError: Bundling did not produce any output. Check that content is written to /asset-output
RetainCoreStack › Snapshot
Bundling did not produce any output. Check that content is written to /asset-output.In order to solve this error in the PythonFunction in the bundling setup, I had to add this property:
bundling: {
bundlingFileAccess: BundlingFileAccess.VOLUME_COPY,
},This is because trying to mount a volume while using docker in docker with a shared socket will try to mount the host’s directory and not the container running the job. Using BundlingFileAccess.VOLUME_COPY will create a volume and copy files instead of trying to mount it.
