PHP Cloud Native Buildpacks Now in the Official Builder

In my previous post, I talked about how to use the PHP Cloud Native buildpacks. It was not super tricky but required some manual work to set up. This is because the PHP CNBs were not, at the time, part of an official builder.

What's a builder? It's basically an image containing a bunch of CNBs, all ready for your use. See this link for more details.

If you are to run `pack suggest-builders`, then you will see the list of official builders. At the time of writing, that is Heroku, Cloud Foundry (bionic) and Cloud Foundry (cflinuxfs3).

$ pack suggest-builders
Suggested builders:
    Heroku:            heroku/buildpacks:18            heroku-18 base image with buildpacks for Ruby, Java, Node.js, Python, Golang, & PHP
    Cloud Foundry:     cloudfoundry/cnb:bionic         Ubuntu bionic base image with buildpacks for Java, NodeJS and Golang
    Cloud Foundry:     cloudfoundry/cnb:cflinuxfs3     cflinuxfs3 base image with buildpacks for Java, NodeJS, Python, Golang, PHP, HTTPD and NGINX

Tip: Learn more about a specific builder with:
    pack inspect-builder [builder image]

Notice how PHP is now included in the Cloud Foundry cflinuxfs3 builder.

So now, instead of all the work I had you doing in my last post, you can just run `pack build --builder "cloudfoundry/cnb:cflinuxfs3"`. The builder will automatically select the PHP CNBs, because it sees you're trying to build a PHP app. The result will be an image you can run with `docker run -it -e PORT=8080 -p 8080:8080 image-name`, just like in my previous post.

That's it! If you'd like to understand the output a bit more, keep reading. Otherwise go and enjoy!

---

If you're curious about the output, the interesting new bit is the DETECTING phase. In the previous post, we manually specified just the PHP CNBs, so that's all you'd see in the output. Now, we see many CNBs running, but only the PHP CNBs will be selected and actually run at build time.

===> DETECTING
[detector] ======== Results ========
[detector] skip: org.cloudfoundry.archiveexpanding@1.0.0-RC03
[detector] pass: org.cloudfoundry.openjdk@1.0.0-RC03
[detector] skip: org.cloudfoundry.buildsystem@1.0.0-RC03
[detector] pass: org.cloudfoundry.jvmapplication@1.0.0-RC03
[detector] pass: org.cloudfoundry.tomcat@1.0.0-RC03
[detector] pass: org.cloudfoundry.springboot@1.0.0-RC03
[detector] pass: org.cloudfoundry.distzip@1.0.0-RC03
[detector] skip: org.cloudfoundry.procfile@1.0.0-RC03
[detector] skip: org.cloudfoundry.azureapplicationinsights@1.0.0-RC03
[detector] skip: org.cloudfoundry.debug@1.0.0-RC03
[detector] skip: org.cloudfoundry.googlestackdriver@1.0.0-RC03
[detector] skip: org.cloudfoundry.jdbc@1.0.0-RC03
[detector] skip: org.cloudfoundry.jmx@1.0.0-RC03
[detector] pass: org.cloudfoundry.springautoreconfiguration@1.0.0-RC03
[detector] Resolving plan... (try #1)
[detector] fail: org.cloudfoundry.jvmapplication@1.0.0-RC03 requires jvm-application
[detector] Resolving plan... (try #2)
[detector] fail: org.cloudfoundry.jvmapplication@1.0.0-RC03 requires openjdk-jre
[detector] Resolving plan... (try #3)
[detector] fail: org.cloudfoundry.jvmapplication@1.0.0-RC03 requires jvm-application
[detector] ======== Output: org.cloudfoundry.yarn@0.0.24 ========
[detector] no "yarn.lock" found at: /workspace/yarn.lock
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.node-engine@0.0.46
[detector] fail: org.cloudfoundry.yarn@0.0.24
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.node-engine@0.0.46
[detector] fail: org.cloudfoundry.npm@0.0.29
[detector] ======== Output: org.cloudfoundry.pipenv@0.0.14 ========
[detector] no Pipfile found
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.python@0.0.20
[detector] skip: org.cloudfoundry.pipenv@0.0.14
[detector] pass: org.cloudfoundry.pip@0.0.20
[detector] Resolving plan... (try #1)
[detector] fail: org.cloudfoundry.pip@0.0.20 requires requirements
[detector] ======== Results ========
[detector] fail: org.cloudfoundry.conda@0.0.13
[detector] ======== Output: org.cloudfoundry.dotnet-core-conf@0.0.20 ========
[detector] *.runtimeconfig.json file not found
[detector] ======== Results ========
[detector] fail: org.cloudfoundry.dotnet-core-conf@0.0.20
[detector] ======== Output: org.cloudfoundry.go-mod@0.0.18 ========
[detector] no "go.mod" found at: /workspace/go.mod
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.go-compiler@0.0.20
[detector] fail: org.cloudfoundry.go-mod@0.0.18
[detector] ======== Output: org.cloudfoundry.dep@0.0.17 ========
[detector] failed detection: no Gopkg.toml found at root level
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.go-compiler@0.0.20
[detector] fail: org.cloudfoundry.dep@0.0.17
[detector] ======== Output: org.cloudfoundry.php-composer@0.0.16 ========
[detector] no "composer.json" found in the following locations: [/workspace/composer.json /workspace/htdocs/composer.json]
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.php-dist@0.0.28
[detector] skip: org.cloudfoundry.php-composer@0.0.16
[detector] pass: org.cloudfoundry.httpd@0.0.18
[detector] pass: org.cloudfoundry.php-web@0.0.18
[detector] Resolving plan... (try #1)
[detector] skip: org.cloudfoundry.httpd@0.0.18 provides unused httpd
[detector] Success! (2)

See how they all fail, until you get down to the PHP CNBs. Then you see Success!.

Then just like the last time, we see the PHP CNBs running during the BUILD phase.

===> BUILDING
[builder]
[builder] PHP Buildpack 0.0.28
[builder]   PHP 7.2.22: Contributing to layer
[builder]     Downloading from https://buildpacks.cloudfoundry.org/dependencies/php/php7-7.2.22-linux-x64-cflinuxfs3-c9315fce.tgz
[builder]     Verifying checksum
[builder]        Expanding to /layers/org.cloudfoundry.php-dist/php-binary
[builder]     Writing PATH to shared
[builder]     Writing MIBDIRS to shared
[builder]     Writing PHP_HOME to shared
[builder]     Writing PHP_EXTENSION_DIR to shared
[builder]     Writing PHP_API to shared
[builder]
[builder] PHP Web Buildpack 0.0.18
[builder]   Configuring PHP Script
[builder]   PHP Web 173b6da1f1b7b7a59af0c2ebcfe1c4ea4b183063a90b92f1f2ae6d8a9ebb0bfd: Contributing to layer
[builder]     Writing PHPRC to shared
[builder]     Writing PHP_INI_SCAN_DIR to shared
[builder] WARNING: `app.php` start script not found. App will not start unless you specify a custom start command.
[builder]   Process types:
[builder]     task: php /workspace/app.php
[builder]     web:  php /workspace/app.php

That's all there is too it.

PHP Cloud Native Buildpacks

At work, I've been helping to rewrite the PHP buildpack as a set of Cloud Native Buildpacks. The PHP CNBs are coming together, current quality is alpha, but I think they're ready enough for people to try them out and report how they work for you. This post has instructions and a demo to use the PHP CNBs.

But first, a slight digression.

A little about the architecture of the PHP CNBs. The previous PHP buildpack has been decomposed into a set of five PHP CNBs, two of which are optional. There are php-cnb, php-composer-cnb, httpd-cnb, nginx-cnb and php-web-cnb.

Here's a rough description of each:

• php-cnb provides PHP binaries, that's it.
• php-composer-cnb provides all the functionality related to Composer. It installs and runs Composer.
• httpd-cnb provides Apache Web Server. It is optional.
• nginx-cnb provides Nginx Web Server. It is optional.
• php-web-cnb ties everything together. It generates the configuration for PHP, PHP-FPM, HTTPD and Nginx. It also contains the logic to generate start commands for various types of PHP apps. It can run PHP cli scripts, PHP's bundled web server, PHP-FPM plus HTTPD and PHP-FPM plus Nginx.

What's also super cool about these CNBs is that some of them like httpd-cnb and nginx-cnb can work all on their own. Want to stand up a proxy or a static site, httpd-cnb or nginx-cnb can be given an httpd.conf or nginx.conf file and they'll run their respective server with that config for you. Similarly, you can mix and match CNBs. Don't want to use the PHP binaries provided by php-cnb, you could substitute another CNB that provides PHP. Pick the parts you like, replace the ones you don't with other things.

On to the show.

If you want to get started there's a little setup that you need to perform.

First,  `git clone` these repos and optionally check out a release branch.

• https://github.com/cloudfoundry/php-cnb
• https://github.com/cloudfoundry/php-composer-cnb
• https://github.com/cloudfoundry/httpd-cnb
• https://github.com/cloudfoundry/nginx-cnb
• https://github.com/cloudfoundry/php-web-cnb

Second, install the latest version of the pack cli. That is v0.2.1 at time of writing.

Third, install Docker if you don't have it already. Make sure it's running too.

Fourth, package up each buildpack that you'd like to use. In each folder that you cloned, you can run `./scripts/package.sh` and it will build the CNB (this requires Golang to be installed). Note the path at which the packaged CNB can be found. You'll need to pass this to the pack cli so it can find the buildpacks. To do that, run `export BUILDPACKS='--buildpack /path/to/buildpack1 --buildpack /path/to/buildpack2 ...'` and paste in the path to each buildpack you packaged. Order is important, use the order listed in the bullet list above.

If you'd like to build them all at once, you can run the following command from the directory where you cloned all of the repos:

for buildpack in ./php-cnb ./php-composer-cnb ./httpd-cnb ./nginx-cnb ./php-web-cnb; do pushd "$buildpack" && ./scripts/package.sh && popd; done | grep "Creating package in" | awk '{printf "--buildpack %s ", $5}'

This will run the package script for each CNB & then print out a list of the locations for each package. Simply copy the output, then run `export BUILDPACKS='paste output from command here'`. We set this as a convenience to make using the pack cli easier and the commands shorter.

At this point, you're all set to build some images. To do this, you run `pack build $BUILDPACKS -p /path/to/php/files`.  This will build an image using the buildpacks that we've specified in our environment variable and it will use the PHP code that you've pointed to with the `-p` argument (you can skip `-p` if the files are in the current directory.

As `pack build` runs, you'll see each build pack run. If there are any errors the buildpack will fail and tell you what happened. If it succeeds, you'll end up with a docker image that you can run using the command `docker run`. For example, if you have a web app, you can `docker run -it -e PORT=8080 -p 8080:8080 `.  The PORT env variable tells the web server which port it should listen to. That should match with the port that you expose using the `-p` flag.

Time for a full example.

Let's say you want to run PHP MyAdmin. The following is a demo of how you could do that. For simplicity sake, it spins up a Percona DB as well. That allows you to have something to connect to within PHP MyAdmin. If you already have a MySQL DB, you can skip that part and point `htdocs/config.inc.php` to your existing server (you could also skip the docker network bits, that just makes it easy to connect to the deployed Percona DB).

Download and run the Gist below. This will set everything up for you.

Here are the highlights:

• Line #10 runs Percona
• Lines #15 - #21 download PHP MyAdmin & configure `htdocs/config.inc.php`. If you want to adjust PHP MyAdmin's config, you can do so at this point.
• Lines #24 - #36 adds a php.ini snippet that enables the PHP extensions needed by PHP MyAdmin
• Line #45 runs `pack build`
• Line #48 runs the image that we build with Docker.

At this point, you can go to http://localhost:8080 in your browser and you should be able to access PHP MyAdmin. Enter `root` and `hacKm3` as the credentials and you can access the database. If you edited the config to point to your own MySQL server, use the credentials for that server instead.

Last notes:

• If you want to adjust the PHP MyAdmin config. Run `docker stop php-myadmin`. Edit `htdocs/config.inc.php` then run lines #45 & #48 again.
• If you want to clean up & remove everything run `docker stop php-myadmin test-db` followed by `docker system prune --volumes`. The latter will clean up a bunch of things for you, be careful when running that if you are running other things with Docker.

Please provide any feedback about the PHP CNBs to the respective CNB Github page. Open an issue with your comments/questions. Thanks & hope you enjoy!

Cloud Native Buildpacks

In the past, I've worked with buildpacks through my time using Cloud Foundry. Cloud Foundry has first class support for buildpacks, which allows you to push code and let the buildpack handle the messy parts of actually running your code. Things like installing a language runtime, installing servers, etc...

Recently the buildpacks world has expanded with the CNCF's acceptance of the Cloud Native Buildpacks project into the CNCF sandbox (sometimes called v3 buildpacks). In addition to an excellent and easily readable spec, this work brings us the `pack` CLI tool, which allows you to run Cloud Native Buildpacks on your local PC and easily deploy the output, which is an OCI image, to Docker or anywhere else you can run an OCI image.

In this post, I'm going to walk through some basics and show you how to get started with `pack`, build some image and run them.

Getting Started

To get started you need to install Docker. The Community Edition works fine. Follow the previous link to get that installed, if you don't have it already.

Then install the `pack` CLI.  You can download `pack` from its Github project here. At the time of writing, I'm using the 0.0.9 release. To download the tar or zip, extract the `pack` binary and put it somewhere on your PATH. On Mac/Linux, `/usr/local/bin` is a good place. Once installed, you should be able to run `pack version` and see `v0.0.9 (git sha: a1a1a0eef63bd09136ab76663bdbc3b0ab3a4931)`.

Hello World

To get a basic app going, we need to do one more thing first. Obtain some buildpacks to use. So run `git clone https://github.com/buildpack/samples`, which is a repo that has a couple very basic sample buildpacks.

Sidebar. At the time of writing, the sample buildpack we're using has an error with it's metadata. You may not need to do this in the future. Edit `samples/hello-world-buildpack/buildpack.toml` and put in the following:

[buildpack]
id = "io.buildpacks.samples.buildpack.hello-world"
version = "0.0.1"
name = "Hello World Buildpack"

[[stacks]]
id = "io.buildpacks.stacks.bionic"

Now that we have buildpacks, we need an app to run. We'll create that now. Run `mkdir hello-world` and then `cd hello-world`. In that folder create `app.sh` and put the following in that file.

#!/bin/bash

while [ 1 -eq 1 ]; do
  echo "Hello World!"
  sleep 5
done

Last step, run `chmod 755 app.sh` to make it executable.

At this point, we now have a buildpack to use and our application code. It's time to run `pack` and make an image.

From our application directory run `pack build --buildpack $(cd ..; pwd)/samples/hello-world-buildpack/ hello-world-app` or replace `$(cd ..; pwd)/samples/hello-world-buildpack/` with the full path to the sample repo you cloned above. This will create an image called `hello-world-app` using the `hello-world-buildpack`, which does nothing (it's a no-op). The output should look something like this.

$ pack build --buildpack $(cd ..; pwd)/samples/hello-world-buildpack/ hello-world-app
Defaulting app directory to current working directory /Users/dmikusa/Downloads/hello-world (use --path to override)
Using default builder image packs/samples:v3alpha2
Pulling builder image packs/samples:v3alpha2 (use --no-pull flag to skip this step)
Selected run image packs/run:v3alpha2 from stack io.buildpacks.stacks.bionic
Pulling run image packs/run:v3alpha2 (use --no-pull flag to skip this step)
Using cache volume pack-cache-153f385b25c48f5d30ee0544d75bee63
===> DETECTING
Using manually-provided group
[detector] 2019/02/17 21:09:40 Trying group of 1...
[detector] 2019/02/17 21:09:41 ======== Results ========
[detector] 2019/02/17 21:09:41 Hello World Buildpack: pass
===> ANALYZING
Reading information from previous image for possible re-use
[analyzer] 2019/02/17 21:09:42 WARNING: image 'hello-world-app' not found or requires authentication to access
[analyzer] 2019/02/17 21:09:42 removing cached layers for buildpack 'config' not in group
===> BUILDING
[builder] ---> Hello World buildpack
[builder]      env_dir: /platform/env
[builder]      plan_path: /tmp/plan.333599924/io.buildpacks.samples.buildpack.hello-world/plan.toml
[builder]      layers_dir: /workspace/io.buildpacks.samples.buildpack.hello-world
[builder] ---> Done
===> EXPORTING
[exporter] 2019/02/17 21:09:48 adding layer 'app' with diffID 'sha256:361cdaf2662ea41f08da0204a4c0393beb629cff85df2b3d650ed7423dc188f2'
[exporter] 2019/02/17 21:09:48 adding layer 'config' with diffID 'sha256:ab046f0bf0b24db6ae8f59e437cc570925e451cb60ae714fcb125ab4095dd9bb'
[exporter] 2019/02/17 21:09:49 adding layer 'launcher' with diffID 'sha256:d77dc7ed6207d6bb9c389aa5f087ea7fffea9238e2de84b03f8b3c1152e1e58f'
[exporter] 2019/02/17 21:09:49 setting metadata label 'io.buildpacks.lifecycle.metadata'
[exporter] 2019/02/17 21:09:49 setting env var 'PACK_LAYERS_DIR=/workspace'
[exporter] 2019/02/17 21:09:49 setting env var 'PACK_APP_DIR=/workspace/app'
[exporter] 2019/02/17 21:09:49 setting entrypoint '/lifecycle/launcher'
[exporter] 2019/02/17 21:09:49 setting empty cmd
[exporter] 2019/02/17 21:09:49 writing image
[exporter] 2019/02/17 21:09:49
[exporter] *** Image: hello-world-app@9b265f861002fa1018d48577fb2f78c4e32b58f72f81c8ef9f6692d2040b4d60
Successfully built image hello-world-app

The interesting bits for now are DETECTING, where the buildpack's detection script runs. This buildpack doesn't do anything but we can see it's marked as "pass" which means the buildpack's build script will get a chance to run. Down below you can see that happening under BUILDING. This again does nothing, but echo a few directories where files reside during build. Legit buildpacks would use detect to determine when they should/shouldn't run and build to install things like runtimes, servers and all the stuff necessary to run your apps.

 The output from above is a image that you can run. If you execute `docker images`, you'll see `hello-world-app` listed.

$ docker images
REPOSITORY                                 TAG                 IMAGE ID            CREATED             SIZE
hello-world-app                            latest              9b265f861002        6 minutes ago       164MB

You can then run it with `docker run -it --name=hello hello-world-app bash app.sh`. The app will run forever printing "Hello World!". Run `docker stop hello` to stop the app.

Hello World++

To spice things up just a little bit and show what it's like to deploy changes to our app, let's edit our `app.sh` script. Set it to this.

#!/bin/bash

while [ 1 -eq 1 ]; do
  if [ "$NAME" == "" ]; then
    echo "Hello World!"
  else
    echo "Hello $NAME!"
  fi
  sleep 5
done

This will allow us to provide a name to print. Run `pack build --buildpack $(cd ..; pwd)/samples/hello-world-buildpack/ hello-world-app` again. This will create a new image with our updated app.

Side note, if you run `docker images` you'll see that the old image is no longer used and can be removed at your leisure.

$ docker images
REPOSITORY                                 TAG                 IMAGE ID            CREATED             SIZE
hello-world-app                            latest              793b5e60b5ad        9 seconds ago       164MB
                                                   9b265f861002        20 minutes ago      164MB

To run the updated app image, you can use the same command `docker run -it --name=hello hello-world-app bash app.sh` and you'll see the same output. However, if you run `docker run -it -e NAME=Daniel --name=hello hello-world-app bash app.sh` you'll see our enhancement.

$ docker run -it -e NAME=Daniel --name=hello hello-world-app bash app.sh
Hello Daniel!

... 

We use Docker's ability to set environment variables to inject some data into our application. More importantly though, you can see that pushing updates and changes is the same process as you used before which makes integrating into build systems and CI/CD systems simple.

Summary

I hope you find getting started is easy. Once you get Docker & pack installed it's one command to stamp out an image using a buildpack and our application. Right now, that buildpack isn't doing anything, so it's not the best demonstration of why you'd want use buildpacks or the full power of them, but I hope this is enough to get you thinking about how this can integrate into your build flows, maybe your CI/CD system and how it can work for you.

My next post will be more practical. It'll dig into some actual buildpacks and show how you can use them to make images for actual applications, and I hope this will better showcase why you would want to use buildpacks.