Debugging Java Applications on CloudFoundry

In many cases, it’s possible to run applications locally and when doing so it’s easy enough to debug them using Eclipse or your favorite Java IDE. In some cases, perhaps due to required third party services or maybe just because you want to see how it runs in a different environment, you want to run the application in CloudFoundry.

Because of the way that CloudFoundry deploys your applications and isolates them, it’s not possible to just connect to your application with the remote Java debugger. Instead you need to flip things around and instruct the application to connect to you.

Here are the instructions for setting up remote debugging when using bosh-lite or a CloudFoundry installation that can connect directly to your PC or laptop.

1. Open Eclipse and your project.
2. Right click on your project, go to Debug as and pick Debug Configurations.
3. Create a new Remote Java Application.
4. Make sure your project is selected, pick Standard (Socket Listen) from the Connection Type drop down and set a port (make sure this is open if you’re running a firewall).
5. Press the Debug button.

The debugger should now be running. If you switch to the Debug perspective, you should see your application listed in the Debug panel and it should say “Waiting for vm to connect at port ”.

The next step is to push your application to CloudFoundry and instruct it to connect to the debugger running on your local machine. Here are the steps to accomplish that (note I’m using the Spring Music application for this demo).

1. Edit your manifest.yml file. Set the instances count to 1. If you were to set this greater than one, you’d end up with multiple applications trying to connect to your debugger.
2. Also in manifest.yml, add the env section and setup a variable called JAVA_OPTS. In that, you’ll want to add the remote debugger configuration which looks like this -agentlib:jdwp=transport=dt_socket,address=<your-ip>:<your-port>.
3. Save the manifest.yml file.
4. Run cf push.

When that completes, your application has started successfully and you see that the application is now connected to the debugger running in your IDE you are all set. From here, you can add breakpoints and interrogate the application just as you would if it were running locally.

If this does not work, try running cf logs --recent to investigate what happened when the application started. If there was a connection problem, you’ll see it in the output.

Firewalls & NAT

Previously I mentioned that these instructions would only work if the remote debugger could connect directly to your PC or laptop. If you’re like most people and behind a corporate or personal firewall and NAT, debugging from a public cloud offering like PWS is not going to work. It simply won’t be able to connect to your machine. Fortunately, there are a some approaches to working around this.

Port Forwarding

One options is port forwarding. If you’re behind a home router, this may be an option for you. Most home routers (enterprise ones do too, but your admin probably won’t want to do this) support the option for forward a request to the router on a designated port to a machine on the local network. As you can probably guess, if we set up port forwarding on our home router for the debug port, we can then redirect those requests to our PC or laptop.

The basic steps for this would look like this.

1. Pick an incoming port and configure the router to forward this to your PC or laptop. Your router may support forwarding this to a different port on your local machine. For simplicity, I’d suggest using the same port for both.
2. Start the debugger and listen on the port that the router is going to forward requests to on your machine.
3. Edit your manifest.yml file. Set JAVA_OPTS to -agentlib:jdwp=transport=dt_socket,address=<your-ip>:<your-port>.
4. Run cf push.

If port forwarding works correctly, you should see the same results as above with the remote application connecting to your IDE’s debugger. If not, you may need to look at the application logs or even your router’s logs to troubleshoot further.

SSH Tunnel

Another option, if you have a publicly accessible server somewhere, is to tunnel the connection to your machine over SSH. With this option, you simply connect to your public machine with SSH and setup a reverse tunnel. The application will connect to the tunnel, which is listening on your public machine and it’s traffic will be forwarded to your local machine over SSH.

The basic steps for this would look like this.

1. Obtain a public server.
2. Install SSHD. Edit /etc/ssh/sshd_config, add or set GatewayPorts to yes. Restart SSHD.
3. On your local machine run ssh -f -N -T -R 0.0.0.0:<public-port>:127.0.0.1:<debugger-port> <user>@<public-server-ip> (Windows users can use cygwin or possibly Putty, although the command will be different). This will instruct SSH to connect to the remote host, setup a reverse tunnel and go into the background. The reverse tunnel will listen on your public server on the port you specify (i.e. public-port) and forward traffic to the debugger port on your local machine. You can use different port numbers, but it is easiest if you just use the same port.
4. Start the debugger and listen on the same port (i.e. debugger-port) that you used in the SSH command.
5. Edit your manifest.yml file. Set JAVA_OPTS to -agentlib:jdwp=transport=dt_socket,address=<your-ip>:<your-port>.
6. Run cf push.

If the SSH reverse tunnel works correctly, you should see the same results as above. If not, you’ll need to check the application logs or public server logs for more details.

Concerns / Risks

When using an SSH tunnel in this manner, there are a few things to keep in mind.

1. By routing the debugger through an SSH tunnel, you’re going to add latency to the connection, especially if it goes over the public Internet. This latency will be noticed through the debugger and can slow down the startup of an application. To compensate for this, you may need to increase the application startup timeout with the -t argument to cf push.
2. If there is a firewall on your public server, it may may block the connections from your application to the public server. In other words, it will likely prevent the tunnel from listening on the . You’ll need to adjust your firewall rules to allow this.
3. You must set the GatewayPorts setting in your sshd_config file. This enables remote hosts (i.e. your application’s debugger) to connect to the <public-ip>:<public-port>. Without this, SSHD will only bind to the localhost.