Developing in the Cloud: The No-Bullshit Microservices Dev Guide (Part 4)

Welcome to the brave new world of developing in the cloud! You’ve heard all about the technical ins and outs of setting up a cloud-based microservices development environment and have decided to give it a whirl. Now you need to stock your toolbox with solutions that let you understand what your code is doing, so you know when something’s going wrong and can get accurate feedback as to what it is and how to fix it.

There are three general approaches to tooling up for cloud development. The first is to monitor the dev environment as you would production, using logs, APMs, and repurposing tools that are optimized to provide high-level feedback in production. The second approach is to use versions of the traditional debuggers you’ve used on your laptop which have been retrofitted to the cloud. The final option is to discover a new generation of debugging tools that are custom-designed to provide observability for cloud development.

Let’s have a better look at what each category of tools offers.

Production solutions

Traditional production tools allow you to see what your code is doing during the development process. Just as you would use them in production. The tools are optimized to provide high-level, low-fidelity feedback.

Because production tools are essential, many types of solutions are available, with several options to choose from for each type. Tools include:

• APMs, such as AppDynamics or DataDog, as well as open-source options like Prometheus
• Error tracking software like Sentry and Bugsnag
• Log aggregation tools like Logz.io, SumoLogic, and the open-source Elasticsearch and Jaeger
• Dedicated tools for microservices, including HoneyComb and Instana, as well as serverless tools, such as Epsagon and Lumigo

Advantages

The major advantages of production tools focus on the central role they play in monitoring. The tools are widely available; most devs with any ops background are familiar with them and know how to use them. Production monitoring, tracking, and logging solutions are mature and reliable. Most are also relatively technology-agnostic, and you can use them in a variety of settings.

Disadvantages

Using production tools to monitor and debug microservices in dev, however, is far from ideal. You cannot use them during the early stages of development. And once you can, data is collected at low fidelity. So while the solutions can be relied on to alert you that something is wrong, they provide little direction as to just what that “something” is and how you can fix it.

Production tools generally provide a predefined, limited set of data.  Each time a dev needs to change the gathered data to get more information, it means writing additional code and redeploying the app. This can make debugging a long, painstaking, and painful process.

For devs that are not already familiar with them, learning to use production tools entails a significant learning curve. Since each tool only focuses on one aspect of production, devs require multiple solutions to monitor various aspects of the application. This duplication increases the burden on devs, consumes more resources, adds complexity to the debugging process, and drives up costs.

Debugger solutions

The second option for debugging microservices in the cloud is to utilize traditional debuggers which have been retrofitted for use in the cloud by tools such as Telepresence, Squash, and Draft. Beyond that standard functionality, each of these three solutions has its strengths and weaknesses.

Telepresence

Telepresence serves as a VPN for Kubernetes by making a laptop a network entity within the cluster. In effect, it allows you to execute your application locally, debug it on your machine and monitor it like a local application, while it’s acting as if it were running in the cloud. Beyond the obvious advantages of using a tool with which devs are familiar and comfortable, it also enables the use of valuable admin tools such as Postman, DataGrip, and others to access resources within Kubernetes.

On the downside, Telepresence works only with Kubernetes (obviously). It requires the reconfiguration of elements in the cloud deployment, including redirecting services to the dev’s machine. Thus, taking down from the cluster any deployment that is running locally, and making cloud resources available for local use.

Also, using a regular debugger to break into a microservices environment can easily disrupt it. Finally, using Telepresence to debug multiple microservices at the same time can be a cumbersome and error-prone process.

Squash

This is a debugger for microservices that utilizes remote debugging engines to provide a unified cross-microservice debugger experience. Plugins are available for several IDEs, which can be used to debug multiple processes within a microservices environment.

Alas, Squash also has some disadvantages. It currently supports only two languages, Java and Go. Plus, it requires a complex remote debugging configuration and cannot debug multiple replicas of the same deployment. Perhaps most discouraging is the fact that using a regular debugger in a microservices environment can easily break the system in unexpected ways. So you cannot always get an accurate read on what is actually occurring. Nonetheless, Squash is still a huge step up in the right direction for microservices debugging.

Draft

A third debugger solution is a promising tool that Microsoft is building to streamline the end-to-end development flow in Kubernetes. As we mentioned in Chasing Clouds, the previous installment of this series, it focuses on deploying code changes from a dev’s laptop to a cluster, as well as from a Git repo to a cluster through CI.

In this awesome video from a KubeCon SA 2018 session, Michelle Noorali demos using Draft to debug applications remotely. Unfortunately, docs on how we can do it ourselves are still missing. 🙁

When Draft does mature a bit, it should be an interesting tool for debugging Kubernetes, which we expect to have many of the same properties as Squash.

Non-breaking breakpoint solutions

Non-breaking breakpoint solutions, such as Stackdriver and Rookout, are designed as all-purpose debugging tools that enable devs to see what their code is doing, just as they would in their own IDE. The tools are optimized to provide high-fidelity data wherever your code is running, even in the cloud.

Among the main advantages of this new class of tools is that they do not break the environment or impact performance in any way. Dedicated web-based IDEs, installed and set up via SDKs, makes them easy to use. No redeployments are necessary, and they can simultaneously debug multiple microservices as well as multiple replicas of the same microservice. Non-breaking breakpoint solutions offer devs unparalleled visibility into interactions between microservices, as well as within individual microservices. They can be used in production as well as in development and staging.

By providing full debugging and data collection capabilities in production, non-breaking breakpoint solutions add clear and significant value to the dev toolkit. However, as a wholly new type of tool, they also require devs to adjust their techniques and learn to debug with a new tool that behaves differently than breakpoints that break.

While Stackdriver and Rookout utilize similar approaches, the solutions differ in some significant ways:

Stackdriver’s orientation is solely toward the Google Cloud — in fact, it is available free of cost to Google Cloud users. Rookout, on the other hand, is cloud-agnostic and can also be used on-premises. Rookout provides additional features for collaboration between devs and for sending data to third-party targets such as log aggregation and APMs. Check back here for more details about how Rookout and Stackdriver stack up in a future blog post.

It’s been quite a journey!

With this final post, our No-Bullshit Microservices Dev Guide draws to a close. Congratulations on making it through!

We hope it has helped you assess when local is the best option for developing your microservices and which tools can be helpful. When you decide to aim for the cloud, what to consider when provisioning your environment; and finally, the best tools for debugging your microservices in the cloud.

Still got questions? Bring them on! Microservices are a whole new development approach. Add in the cloud, and we are all, to some extent, on a steep learning curve. We’re happy to help as well as to hear your tips, suggestions, and feedback.