Friday, September 03, 2021

Recent Happenings

I have over the years done a lot of work with buildpacks. Both on Cloud Foundry and now with Cloud Native Buildpacks. This year I've been fortunate enough to start a job at VMware working on them full time, which I'm pretty excited about.

I am primarily working to develop the Java-related Paketo buildpacks, as well as contribute to the Buildpacks project & to also maintain the Java Cloud Foundry buildpack.

To support this effort, I'm going to try and start writing more about these efforts. What's new, what's cool and more about how to use buildpacks.

To kick this off, I have a post I wrote up for work: Container Images the Easy Way with Cloud Native Buildpacks. If you're new or wondering about buildpacks, this is a good place to start. It talks about how buildpacks can help you and gives some comparison to using Dockerfiles to build your container images, something many people are doing at the moment.

I also had the opportunity to be on and discuss similar topics, so if you'd rather watch a video than read, you can find some of similar info about buildpacks there.

Friday, April 24, 2020

PHP Cloud Native Buildpack Updates

It's been a little while since I've posted an update on the PHP Cloud Native Buildpacks. The good news is that lots of progress has been made. We've basically achieved feature parity with the old PHP buildpack and I believe the PHP CNB's should be working for most apps now!

If you're coming from the old PHP buildpack, there are some differences. This is basically a major version bump, so it was an opportunity to make a few breaking changes that we believe will generally improve the user experience. Check out the migration documentation for details on what's changed.

Here's a quick demo of what you can do.
  1. Start out by cloning the Symfony Demo app.
  2. Add a file to the root called buildpack.yml. In that file, add the following:

      webdirectory: public

    This will tell the buildpack the location of the files that we want it to serve.
  3. Create the folder .php.ini.d and in it put the file symfony.ini. Inside that file, add the line: variables_order = "EGPCS". This tells PHP that we want the $_ENV superglobal. I'm not a Symfony expert, but it seems to be required for Symfony to configure itself from actual environment variables, which we'll be doing.
These are the only changes we need to make for the demo app to work.

Next up, install prerequisites. You only need to do this once.
  1. If you don't have the pack cli install, go do that now.
  2. If you don't have docker installed, do that.
Now, from the root of our application directory we just run pack build -b paketo-buildpacks/php -e APP_ENV=prod symfony-5-demo to build an image.

Breaking it down:
  • pack build is the command to build an image from Cloud Native Buildpacks
  • the -b flag indicates which Cloud Native Buildpacks to use. In this case, we point it to the meta buildpack for PHP. Meta buildpacks are collections of Cloud Native Buildpacks.
  • the -e flag set an environment variable picked up by Symfony during build. It tells it that we're doing a production build.
  • the last bit is the image name we want to create
Lastly, we can run our image with docker run -e PORT=8080 -p 8080:8080 -e APP_ENV=prod symfony-5-demo.

Breaking it down:
  • docker run will run your image
  • -e sets the PORT environment variable which tells the buildpack what port it should have the app listen to.
  • -p tells Docker that we want to expose port 8080. This needs to match the value of PORT so that the port on which the app is listening matches the port on which docker is allowing traffic.
  • the second -e again tells Symfony that we're running in production mode
  • the last bit is the image name to run, this should match the image name we used with pack build.
At this point, you can go to http://localhost:8080/ in your browser and access the Symfony Demo App that's been built with Cloud Native Buildpacks and is running inside your Docker container.

Some final notes:
  • By default, it's going to run with PHP's built-in web server. That's not very "production", so we can add Nginx or Apache Web Server by adding the line webserver: httpd or webserver: nginx to buildpack.yml (add it under the php: block). That's it. No server configuration files required. The PHP Cloud Native Buildpacks handle it all for you.
  • The first time you run pack build, it will be slow. It needs to download resources like the build & run images, plus of course it needs to download PHP itself. After the first time, things will speed up dramatically. The images will exist locally and PHP downloads are smartly cached. With downloads cached, a full build takes about 10 seconds on my laptop.
That's it. Hope you all enjoy. Feel free to raise any feedback on Github.

Sunday, September 08, 2019

PHP Cloud Native Buildpacks Now in the Official Builder

In my previous post, I talked about how to use the PHP Cloud Native buildpacks. It was not super tricky but required some manual work to set up. This is because the PHP CNBs were not, at the time, part of an official builder.

What's a builder? It's basically an image containing a bunch of CNBs, all ready for your use. See this link for more details.

If you are to run `pack suggest-builders`, then you will see the list of official builders. At the time of writing, that is Heroku, Cloud Foundry (bionic) and Cloud Foundry (cflinuxfs3).

$ pack suggest-builders
Suggested builders:
    Heroku:            heroku/buildpacks:18            heroku-18 base image with buildpacks for Ruby, Java, Node.js, Python, Golang, & PHP
    Cloud Foundry:     cloudfoundry/cnb:bionic         Ubuntu bionic base image with buildpacks for Java, NodeJS and Golang
    Cloud Foundry:     cloudfoundry/cnb:cflinuxfs3     cflinuxfs3 base image with buildpacks for Java, NodeJS, Python, Golang, PHP, HTTPD and NGINX

Tip: Learn more about a specific builder with:
    pack inspect-builder [builder image]

Notice how PHP is now included in the Cloud Foundry cflinuxfs3 builder.

So now, instead of all the work I had you doing in my last post, you can just run `pack build --builder "cloudfoundry/cnb:cflinuxfs3"`. The builder will automatically select the PHP CNBs, because it sees you're trying to build a PHP app. The result will be an image you can run with `docker run -it -e PORT=8080 -p 8080:8080 image-name`, just like in my previous post.

That's it! If you'd like to understand the output a bit more, keep reading. Otherwise go and enjoy!

If you're curious about the output, the interesting new bit is the DETECTING phase. In the previous post, we manually specified just the PHP CNBs, so that's all you'd see in the output. Now, we see many CNBs running, but only the PHP CNBs will be selected and actually run at build time.

[detector] ======== Results ========
[detector] skip: org.cloudfoundry.archiveexpanding@1.0.0-RC03
[detector] pass: org.cloudfoundry.openjdk@1.0.0-RC03
[detector] skip: org.cloudfoundry.buildsystem@1.0.0-RC03
[detector] pass: org.cloudfoundry.jvmapplication@1.0.0-RC03
[detector] pass: org.cloudfoundry.tomcat@1.0.0-RC03
[detector] pass: org.cloudfoundry.springboot@1.0.0-RC03
[detector] pass: org.cloudfoundry.distzip@1.0.0-RC03
[detector] skip: org.cloudfoundry.procfile@1.0.0-RC03
[detector] skip: org.cloudfoundry.azureapplicationinsights@1.0.0-RC03
[detector] skip: org.cloudfoundry.debug@1.0.0-RC03
[detector] skip: org.cloudfoundry.googlestackdriver@1.0.0-RC03
[detector] skip: org.cloudfoundry.jdbc@1.0.0-RC03
[detector] skip: org.cloudfoundry.jmx@1.0.0-RC03
[detector] pass: org.cloudfoundry.springautoreconfiguration@1.0.0-RC03
[detector] Resolving plan... (try #1)
[detector] fail: org.cloudfoundry.jvmapplication@1.0.0-RC03 requires jvm-application
[detector] Resolving plan... (try #2)
[detector] fail: org.cloudfoundry.jvmapplication@1.0.0-RC03 requires openjdk-jre
[detector] Resolving plan... (try #3)
[detector] fail: org.cloudfoundry.jvmapplication@1.0.0-RC03 requires jvm-application
[detector] ======== Output: org.cloudfoundry.yarn@0.0.24 ========
[detector] no "yarn.lock" found at: /workspace/yarn.lock
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.node-engine@0.0.46
[detector] fail: org.cloudfoundry.yarn@0.0.24
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.node-engine@0.0.46
[detector] fail: org.cloudfoundry.npm@0.0.29
[detector] ======== Output: org.cloudfoundry.pipenv@0.0.14 ========
[detector] no Pipfile found
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.python@0.0.20
[detector] skip: org.cloudfoundry.pipenv@0.0.14
[detector] pass: org.cloudfoundry.pip@0.0.20
[detector] Resolving plan... (try #1)
[detector] fail: org.cloudfoundry.pip@0.0.20 requires requirements
[detector] ======== Results ========
[detector] fail: org.cloudfoundry.conda@0.0.13
[detector] ======== Output: org.cloudfoundry.dotnet-core-conf@0.0.20 ========
[detector] *.runtimeconfig.json file not found
[detector] ======== Results ========
[detector] fail: org.cloudfoundry.dotnet-core-conf@0.0.20
[detector] ======== Output: org.cloudfoundry.go-mod@0.0.18 ========
[detector] no "go.mod" found at: /workspace/go.mod
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.go-compiler@0.0.20
[detector] fail: org.cloudfoundry.go-mod@0.0.18
[detector] ======== Output: org.cloudfoundry.dep@0.0.17 ========
[detector] failed detection: no Gopkg.toml found at root level
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.go-compiler@0.0.20
[detector] fail: org.cloudfoundry.dep@0.0.17
[detector] ======== Output: org.cloudfoundry.php-composer@0.0.16 ========
[detector] no "composer.json" found in the following locations: [/workspace/composer.json /workspace/htdocs/composer.json]
[detector] ======== Results ========
[detector] pass: org.cloudfoundry.php-dist@0.0.28
[detector] skip: org.cloudfoundry.php-composer@0.0.16
[detector] pass: org.cloudfoundry.httpd@0.0.18
[detector] pass: org.cloudfoundry.php-web@0.0.18
[detector] Resolving plan... (try #1)
[detector] skip: org.cloudfoundry.httpd@0.0.18 provides unused httpd
[detector] Success! (2)

See how they all fail, until you get down to the PHP CNBs. Then you see Success!.

Then just like the last time, we see the PHP CNBs running during the BUILD phase.

[builder] PHP Buildpack 0.0.28
[builder]   PHP 7.2.22: Contributing to layer
[builder]     Downloading from
[builder]     Verifying checksum
[builder]        Expanding to /layers/org.cloudfoundry.php-dist/php-binary
[builder]     Writing PATH to shared
[builder]     Writing MIBDIRS to shared
[builder]     Writing PHP_HOME to shared
[builder]     Writing PHP_EXTENSION_DIR to shared
[builder]     Writing PHP_API to shared
[builder] PHP Web Buildpack 0.0.18
[builder]   Configuring PHP Script
[builder]   PHP Web 173b6da1f1b7b7a59af0c2ebcfe1c4ea4b183063a90b92f1f2ae6d8a9ebb0bfd: Contributing to layer
[builder]     Writing PHPRC to shared
[builder]     Writing PHP_INI_SCAN_DIR to shared
[builder] WARNING: `app.php` start script not found. App will not start unless you specify a custom start command.
[builder]   Process types:
[builder]     task: php /workspace/app.php
[builder]     web:  php /workspace/app.php

That's all there is too it.

Thursday, July 04, 2019

PHP Cloud Native Buildpacks

At work, I've been helping to rewrite the PHP buildpack as a set of Cloud Native Buildpacks. The PHP CNBs are coming together, current quality is alpha, but I think they're ready enough for people to try them out and report how they work for you. This post has instructions and a demo to use the PHP CNBs.

But first, a slight digression.

A little about the architecture of the PHP CNBs. The previous PHP buildpack has been decomposed into a set of five PHP CNBs, two of which are optional. There are php-cnb, php-composer-cnb, httpd-cnb, nginx-cnb and php-web-cnb.

Here's a rough description of each:
  • php-cnb provides PHP binaries, that's it.
  • php-composer-cnb provides all the functionality related to Composer. It installs and runs Composer.
  • httpd-cnb provides Apache Web Server. It is optional.
  • nginx-cnb provides Nginx Web Server. It is optional.
  • php-web-cnb ties everything together. It generates the configuration for PHP, PHP-FPM, HTTPD and Nginx. It also contains the logic to generate start commands for various types of PHP apps. It can run PHP cli scripts, PHP's bundled web server, PHP-FPM plus HTTPD and PHP-FPM plus Nginx.
What's also super cool about these CNBs is that some of them like httpd-cnb and nginx-cnb can work all on their own. Want to stand up a proxy or a static site, httpd-cnb or nginx-cnb can be given an httpd.conf or nginx.conf file and they'll run their respective server with that config for you. Similarly, you can mix and match CNBs. Don't want to use the PHP binaries provided by php-cnb, you could substitute another CNB that provides PHP. Pick the parts you like, replace the ones you don't with other things.

On to the show.

If you want to get started there's a little setup that you need to perform.

First,  `git clone` these repos and optionally check out a release branch.

Second, install the latest version of the pack cli. That is v0.2.1 at time of writing.

Third, install Docker if you don't have it already. Make sure it's running too.

Fourth, package up each buildpack that you'd like to use. In each folder that you cloned, you can run `./scripts/` and it will build the CNB (this requires Golang to be installed). Note the path at which the packaged CNB can be found. You'll need to pass this to the pack cli so it can find the buildpacks. To do that, run `export BUILDPACKS='--buildpack /path/to/buildpack1 --buildpack /path/to/buildpack2 ...'` and paste in the path to each buildpack you packaged. Order is important, use the order listed in the bullet list above.

If you'd like to build them all at once, you can run the following command from the directory where you cloned all of the repos:

for buildpack in ./php-cnb ./php-composer-cnb ./httpd-cnb ./nginx-cnb ./php-web-cnb; do pushd "$buildpack" && ./scripts/ && popd; done | grep "Creating package in" | awk '{printf "--buildpack %s ", $5}'

This will run the package script for each CNB & then print out a list of the locations for each package. Simply copy the output, then run `export BUILDPACKS='paste output from command here'`. We set this as a convenience to make using the pack cli easier and the commands shorter.

At this point, you're all set to build some images. To do this, you run `pack build $BUILDPACKS -p /path/to/php/files`.  This will build an image using the buildpacks that we've specified in our environment variable and it will use the PHP code that you've pointed to with the `-p` argument (you can skip `-p` if the files are in the current directory.

As `pack build` runs, you'll see each build pack run. If there are any errors the buildpack will fail and tell you what happened. If it succeeds, you'll end up with a docker image that you can run using the command `docker run`. For example, if you have a web app, you can `docker run -it -e PORT=8080 -p 8080:8080 `.  The PORT env variable tells the web server which port it should listen to. That should match with the port that you expose using the `-p` flag.

Time for a full example.

Let's say you want to run PHP MyAdmin. The following is a demo of how you could do that. For simplicity sake, it spins up a Percona DB as well. That allows you to have something to connect to within PHP MyAdmin. If you already have a MySQL DB, you can skip that part and point `htdocs/` to your existing server (you could also skip the docker network bits, that just makes it easy to connect to the deployed Percona DB).

Download and run the Gist below. This will set everything up for you.

Here are the highlights:
  • Line #10 runs Percona
  • Lines #15 - #21 download PHP MyAdmin & configure `htdocs/`. If you want to adjust PHP MyAdmin's config, you can do so at this point.
  • Lines #24 - #36 adds a php.ini snippet that enables the PHP extensions needed by PHP MyAdmin
  • Line #45 runs `pack build`
  • Line #48 runs the image that we build with Docker.
At this point, you can go to http://localhost:8080 in your browser and you should be able to access PHP MyAdmin. Enter `root` and `hacKm3` as the credentials and you can access the database. If you edited the config to point to your own MySQL server, use the credentials for that server instead.

Last notes:
  • If you want to adjust the PHP MyAdmin config. Run `docker stop php-myadmin`. Edit `htdocs/` then run lines #45 & #48 again.
  • If you want to clean up & remove everything run `docker stop php-myadmin test-db` followed by `docker system prune --volumes`. The latter will clean up a bunch of things for you, be careful when running that if you are running other things with Docker.
Please provide any feedback about the PHP CNBs to the respective CNB Github page. Open an issue with your comments/questions. Thanks & hope you enjoy!