Rookout is now a part of the Dynatrace family!

Table of Contents
Get the latest news

The Bare Necessities of Live Application Debugging

Oded Keret | VP of Product

6 minutes

Table of Contents

As R&D managers, we want to make sure our developers are happy and efficient. We want to ensure that they have the bare necessities they require to develop the best applications, whether it be the right desktops, the right headset, the right brand of coffee, and as many cookies as our budget can afford.

In some areas we take their personal preferences and their professional recommendations into account when applicable, such as when choosing a coding language, an IDE, an open source framework, or a cloud vendor. In other areas we choose for them, like when security and compliance are on the line, when their preferred choice is beyond our budget, or when we have our own professional considerations that don’t fully align with theirs and even conflicts with their preference. Our developers don’t like being tracked on the progress of their tasks, yet we choose a tool and a process for managing releases and tracking tasks.

Tool choice considerations

Some tool selections feel like a bonus we can give our developers. When using their favorite IDE, for example, developers are happier and more efficient. Other tools are crucial to the efficiency of the team, to the quality of the product, to the success of the business. Our developers wouldn’t necessarily choose to use an exception catching tool, yet we know that in order to spot and resolve issues quickly, we should have one in place.

Another common consideration in tool selection is whether it feels necessary or not. When considering some tools, we wonder whether they will actually be used by our team or if they really only sound nice in theory. We try to imagine how our team will adopt the tool, what may prevent them from using it, and how often it will get used. It is not uncommon that we are so used to doing things the long and difficult way, that we can’t imagine changing our workflow.

When considering a Live Debugging solution, it is sometimes unclear, at first glance, as to which category it belongs. Sure enough, if your team struggles with solving production issues on a daily basis, or if there are certain technologies and deployments you just can’t reach with your local debugger, well, adding a Live Debugger into your tech stack is a no brainer. In other cases, at first it may seem that the choice isn’t as clear cut. Let’s examine the reasons and rationalization that impact us when we find ourselves hesitating on whether or not to introduce Live Debugging into our developer’s toolbox.

An urgent need of a sense of urgency

Take a minute and ask yourself the following questions (and try to answer them honestly 😉 ). How many customer issues does your team handle? How long, on average, does it take your team to handle each issue? And where is the majority of that time spent?

Some of us have very clear answers. We are required to report these numbers as KPIs of our R&D organization. We understand every support ticket is a potential for an unsatisfied customer and that every minute of downtime can immediately be translated into a loss of business transactions. We measure our response to incidents and present our progress on a weekly or monthly basis.

Some of us, however, don’t think this is an urgent issue. We think the number of escalated cases is small, and that our team is “probably good enough at dealing with them, otherwise we would have heard about it”.

In my experience, if that is your response, you likely belong to one of two groups. Either your team and your product are indeed that good, and if that’s the case I really have nothing to offer you. But the far more likely case is that things are so bad you don’t even have the time to measure how bad they are. And if that’s the case, you may want to free up some time for you and your team by adopting a technology and a skill that will help you solve these incidents much, much faster.

New things seem hard before you try them

Whenever a new tool is introduced, some friction is to be expected. This is usually based on our own experience as developers or as managers who have already tried to introduce some tools. Tools require an installation, some training, and worst of all  – the ability to adapt and change the way you and your team do things. That last part about adapting to change is especially hard to handle when working with developers, who are very sensitive due to their already complex and challenging work environment.

And as we’ve hinted earlier in this article, developers are specifically sensitive to having tools “dictated” to them by management or by the company.

Keeping that in mind, that’s exactly why we designed Rookout to be a natural fit into the developer’s workflow. Well, also because we’re a tool that’s designed by developers, for developers. It may sound cliche, but it’s true. With an IDE-like user interface and a simple, one-time installation, we made sure that it’s very easy for developers to introduce Rookout into their debugging flow. And by integrating Rookout with popular observability and collaboration tools such as Jira, Slack, Datadog, and Elasticsearch, we ensure that the change to the way they work is minimal.

By making Rookout developer-friendly and easy to learn, and by ensuring that it saves them precious time with every debug session, we are also able to give them the feeling that tool was made for them, rather than a “mandatory” tool forced upon them by corporate.

The fallacy of Not Invented Here

A very common response of a software engineer (ourselves included!) to seeing a new dev tool is to challenge its developers and to believe that we could have developed it better ourselves. This response becomes more severe in cases where we’ve actually tried implementing something similar ourselves or if we think we have an alternative, tailored solution for a neighboring solution.

Accepting a commercial tool would mean that the local solution we built (or are planning to build) isn’t good enough and that we are not good enough as engineers.

Even worse – it would imply that the time and effort that we have already invested in developing it have been for nothing. Or that the engineers who have been dedicated to developing and maintaining the homegrown solution will now have no work to do.

As experienced R&D managers, we know the origins of this fallacy, which mixes “Not Invented Here” with a healthy portion of “Buy Versus Build” bias. We know that when applicable, paying for a specialized tool will free our engineers to work on the actual product. We know that a solution with an entire company behind it will have a much richer set of features and a more robust, secure and scalable architecture. And we know that the “sunken investment bias” is, as it is named – a bias.

Where live application debugging fits

Introducing live debugging, and specifically Rookout, into your development and debugging workflow will keep your team effective and satisfied. By allowing you to solve customer issues faster, it will help your company meet its business goals, and will free your developers to develop new features and provide more value to your customers.

Rookout was designed in a way that will integrate seamlessly with how your developers work today, and its ease of use and installation will make a build vs buy decision very easy to make. In conclusion, no matter which tool you use, Live Debugging is a necessity, not a bonus. And Rookout lets you implement live debugging quickly and easily, in a way that will keep your developers happy.

Rookout Sandbox

No registration needed

Play Now

Table of Contents
Get the latest news

Developer Tools for Kubernetes in 2021: Helm, Kustomize, and Skaffold (Part 1)

Liran Haimovitch | Co-Founder & CTO

7 minutes

Table of Contents

The last few years has seen an avalanche of tools to enable easier software development on Kubernetes. While some tools grow and adapt, others get left behind. What’s a better way to open 2021 than with an up-to-date review of the current options?

In this 5-part blog series, I’ll go over the various developer tools for Kubernetes, their function within the development workflow, and important updates for each tool. For this post, I’ll focus on tools used to define our Kubernetes applications – Helm, Kustomize, and Skaffold.

Kubernetes Manifests (YAMLs)

If you are new to Kubernetes, I recommend this introduction, which I found quite in-depth. For our purposes, the most important thing for you to know is that Kubernetes has a Declarative approach for orchestrating applications. You write your application’s desired state in a configuration file(s), known as “manifest,” in a YAML format and send it over to Kubernetes to make it happen.

Helm for Kubernetes

Helm is the reigning king, the de-facto standard for packaging, sharing, and deploying k8s applications. Think of Helm as a package manager – it allows you to group multiple YAML configuration snippets into a logical package known as a “chart” in a consistent and structured way.

As a package author, creating a Helm chart provides you with significant benefits:

  1. You can make your application configurable at deploy time using custom arguments. To do this, you use templates.
  2. You can publish your application into a private or public repository in a versioned and trackable way.
  3. You can make your application dependent on other Helm charts.
  4. Overall, Helm provides you with strong encapsulation mechanics to ensure your application deploys as intended.

As a package user, installing a Helm chart provides you even more:

  1. You gain access to an extensive repository of publicly available charts and any private charts made available by your organization.
  2. You can customize applications based on the settings provided by the package author.
  3. You have a versioned source of truth from where to deploy packages.
  4. You don’t have to worry about the intricacies of the Kubernetes specification language.
  5. Helm supports Atomic operations to minimize risks to your applications and clusters.
  6. Overall, you get an easy-to-understand and ready-to-use application package.

    What’s new with Helm?

    In late 2019 we saw the release of Helm v3, with the removal of the cluster-side component (Tiller) and a slew of other features. At the end of 2020, Helm v2 has been deprecated, and most public charts have been migrated to the v3 format. If you are still using Helm v2, make sure to allocate time to upgrade your near-term roadmap (while you are at it, check out this plugin that may help).

    Over the last few releases, the Helm team has been busy addressing one of the biggest complaints by Helm users – the difficulty of working with YAML templates. Helm now includes a powerful linting command that should be your new go-to when debugging issues with those pesky YAMLs.

    Another neat feature added to Helm is the post-render feature, allowing you to customize Helm charts with tools such as Kustomize. For example, this blog post shows how to use post-render to add Rookout to an existing Helm chart.

    Last but not least, 2020 also saw the move to a more decentralized way of managing charts and the launch of two central repositories for sharing them:

    1. ArtifactHub by CNCF
    2. ChartCenter by JFrog (check out this webinar by Josh, a member of the Rookout team, to learn more about it)

    Kustomize for Kubernetes

    Helm’s biggest drawback is that customization of applications is limited to pre-existing configuration options. Not only that, the chart author has to implement those customization options in the somewhat cumbersome manner of templating. Well, that’s where Kustomize comes in.

    Kustomize allows you to build a Kubernetes application as a series of layers and patches, empowering limitless customizations. Kustomize uses a YAML-based Kubernetes-aware patch format to add/remove/update any part of the application manifest. Kustomize became an integral part of kubectl on version 1.14, and all you have to do to invoke it is to execute kubectl -k. The docs are a bit sparse but check out the official examples page and this blog post to get some sense of things.

    Kustomize is a powerful tool that empowers you to modify Kubernetes applications in any way you want to. Unfortunately, this means the learning curve can be quite steep. Arbitrary customization also means the potential for misconfiguring an app increases. One of Kustomize’s advanced use-cases uses Helm’s post-render feature to patch an existing Helm chart without requiring forking, hopefully making version upgrades seamless.

    What’s new with Kustomize?

    Many tools within the Kubernetes ecosystem have embedded Kustomize to add its functionality to theirs. The most prominent examples are orchestration and continuous deployment tools such as  ArgoCD, Flux, and Kubestack. If you search for Kustomize-like patch functionality, check out your CD tool, and you just may find it there.

    If, like me, you are not familiar with the Kubernetes build process, you might not know that the maintainers had frozen the Kustomize version embedded in kubectl at 2.0.3 during the initial integration. Besides making the docs even more confusing, this means the version inside kubectl is missing a slew of enhancements that made their way in over the last two years. The team is making significant progress on reintegration, and hopefully they will resolve it soon. In the meantime, consider using Kustomize as a standalone CLI tool if you need the latest version.

    Skaffold for Kubernetes

    Skaffold takes a bit of a different approach, following the DevOps best practice of keeping the development environments and workflows consistent across the SDLC. Skaffold builds and deploys Kubernetes applications for dev workflows and continuous integration (CI) and continuous deployment (CD).

    For the building, Skaffold can utilize Dockerfiles, Buildpacks, Bazel, and even custom scripts (more on building container images in an upcoming blog post!). For deploying, Skaffold contains its limited templating engine and can invoke kubectl, Helm, or Kustomize.

    Skaffold has three main modes of operation:

    1. skaffold dev – this will run Skaffold in a watch, build, deploy loop. In this mode, you can edit your source files locally, and Skaffold will deploy them to a cluster of your choice. Skaffold supports port forwarding and log tailing to enable a smoother development experience when working in this mode.
    2. skaffold build – this will run Skaffold to build your artifacts once and push them to a repository of your choice.
    3. skaffold deploy – this will deploy your built application into a cluster of your choice, potentially utilizing Helm or Kustomize to do so. If you want to build and deploy using a single command, you can use skaffold run.

      What’s new with Skaffold?

      In 2020, the Skaffold team focused on making the project easier to adapt to various workflows and more interoperable with other tools. Some of the improvements included more flexible integrations with CI/CD and GitOps and better support for Python and Java.

      Additionally, Skaffold now has a new operating model in beta – skaffold debug. In this mode, Skaffold attempts to configure the application runtime for remote debugging automatically. While this is a neat feature, using traditional debuggers in a microservices environment is tricky at best, especially when working with remote clusters. If you are having trouble with that, we highly recommend you check out non-breaking debuggers such as Rookout.

      Summary: Our Recommendations

      When it comes to packaging, deploying, and sharing the notorious Kubernetes application manifests, we have seen significant market consolidation. The CNCF is now the home of the leading tools with Helm, an official CNCF project, and Kustomize integrated into kubectl and many other tools. Like in many other parts of the Kubernetes ecosystem, the tools here have significantly matured, and each has a clear purpose.

      • Helm: Use Helm to package, share, and install well-defined Kubernetes applications.
      • Kustomize: Use Kustomize to modify existing Kubernetes applications using patches.
      • Skaffold: Skaffold is a useful tool (and popular!), but configuring Kubernetes applications is not its primary purpose.

      I hope you found this short guide useful – please don’t hesitate to reach out to share your feedback and let me know which other topics you would like me to cover.

      Check out the full Tools For Kubernetes Series:

      Part 2: Skaffold, Tilt & Garden

      Part 3: Lens, VSCode, IntelliJ & Gitpod 

      Part 4: Docker, BuildKit, Buildpacks, Jib & Kaniko

      Part 5: Development Machines

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      Relationship Goals: Ignite Your Product Manager-Dev Relationship

      Noa Goldman | Senior Product Manager

      9 minutes

      Table of Contents

      The R&D team at every company is made up of a variety of personalities- developers, tech leads, team leads, you name it. The Product team is also quite a diverse group and is usually made up of creative UI/UX designers and product managers. When you throw them together, you get a meaningful relationship that creates magical features for top-notch products.

      When I first joined Rookout’s product management team, I told myself that the job and working alongside developers would be as easy as breathing. As it were, I had been a developer previously before I decided to look for a career change. I found myself continuously reminding myself: “You used to be a developer, you’re a developer at heart, you know what developers want”. Sounds like a recipe for success, right?

      Well, I quickly learned that having been a certain kind of developer at a certain company doesn’t mean that I know and understand all developers. Developers, much like snowflakes or McNuggets, come in different shapes and sizes. Understanding this was the easy part. The challenge, I soon came to understand, is to help your team understand it.

      The relationship between developers and product managers has been challenging since the beginning of time. There is a continuous dissonance- developers want to reduce tech debt and use cool technologies, while product managers want data and “boring” features that help sell the product. Relationships are hard and take work, patience, and a whole lot of compromise – and none more so than the relationship between developers and product managers.

      Managing a product for developers, made by developers, can be even more challenging. Let me take you on a quick relationship therapy session and give you my tips on how to keep the love going.

      Understanding Goes Both Ways

      When pitching a new feature design, the discussion and debate over it is sure to be kindled quickly. The most important thing to do when this happens is simple: listen to each other.

      •  Each developer on the team has at least two opinions and is sure that this truth is absolute. As a product manager, you have to listen carefully to understand where developers are coming from.
      • Product managers’ motives are not always clear to the R&D team, but it’s their job to make sure the developers understand the reasons behind these ideas. And truly, this goes both ways.

      When working on a technical product, it’s important to remember that not all developers are the same. Some of them work in huge monoliths and some with microservices. Some devs are tech-savvy, living and breathing code, and some are familiar with just one language. It’s important to understand the type of developers you’re working with versus the type of developers the product is meant for.

      The relationship can be damaged when product managers and developers don’t agree, or decide on a feature without any explanation for why. It’s critical to make yourself understood, no matter which side you stand on.

      That being said, remember that, unfortunately, you’re not always right. Understanding each other has to be followed with an open mind, and both sides have to accept the fact that sometimes someone else is right. You’re there to succeed together, and it doesn’t matter at the end of the day whose idea specifically was chosen. It sounds a little bit like a Sesame Street episode, but what can I say? Sesame Street is always right!

      Communication is key

      Simply saying “this is my decision” doesn’t do much for, well, most people in general.

      Keeping the team engaged by introducing them to the user’s story – such as sharing quotes from meetings with them – helps them to understand the other side’s pains and needs. Presenting real conversations and telling the story with the user’s words creates a new dimension and depth to the story that can not be done in any other way.

      Make sure to back your arguments with data. Show the users’ behavior and use tracking tools to display real sessions and share screen recordings from meetings with customers. Try to use numbers and facts. Ultimately- keep your team engaged. Spice it up! The more they know, the better they’ll be able to perform and work alongside you.

      It goes the same for all developers in the team. Simply saying “it will take way too long to develop”, or “it’s impossible” won’t do much to help product managers understand. Even if you are 100% certain that a specific feature is wrong to develop for technical reasons, the product manager doesn’t always understand that. Oftentimes, they require a slow and detailed explanation. Relationships take patience and ultimately, it will help the team deliver faster.

      Pick your battles

      Developing a product is much like arguing over what to order for dinner or whose turn it is to do the dishes.
      When doing so, it is important to remember: you don’t always have to win the argument. Sometimes, it’s perfectly fine and actually recommended, to take the back seat and let someone else decide which route to follow.

      Let’s take a look at a classic example that occurs, no matter if you are a product manager or a developer.  A new feature is coming up and the team needs to choose the right color tone for a very small icon on your ‘About Us’ page. It’s important, right? Definitely. It is so important that the room erupts as everyone voices their opinions, and it seems like anyone has its own five opinions. Everyone is, of course, 100% sure that they are right.

      Well, you can sit there and argue for hours, fight, and make sure that your specific idea will be the one chosen. Or, you can decide that there are more important things than winning the fight, and simply move on. Choosing to avoid this battle will actually help create a better relationship and help to gain each other’s trust.

      And on that note, as with any relationship, it’s critical to earn your team’s trust and to earn yours. It’s not only easier but necessary, to pick your battles and fight for what’s really important. So leave certain decisions to your partners. This way, when something is really important, it will be quick and easy to get your way. It is a give and take relationship, so make sure you know what are the truly important decisions that are worth fighting for. Anything else should be discussed and decided together as a team.

      Make decisions together, as a team

      Being a team player is not just something to write on your resume. It’s key to working together to create great products.

      It might sound like a bad cliche, but it’s a true one: It’s not a race, it’s a marathon. You will continue to release features, fix bugs, and design together for a long time. When the decision is made by only one person, even if it’s the best thing to do if the whole team is not up for it the marathon will be harder to complete.

      It’s important to remember that whether you are a developer or a product manager, you are a part of a team. Therefore, the team should be synced, aligned, and all should trust the final decision. Moving forward with the next steps and decisions will be easier if your partners are working arm in arm with you.

      Sometimes, someone else knows better

      Product managers have to know their users best. They need to be able to be familiar with their pain points and struggles and ultimately understand best what they need.

      In every product, some decisions are made purely on a tech basis. As a product manager for a technical product, it’s essential to be familiar with the technology, to know how developers work, what they do every day, and the meaning of using different languages, frameworks, and terms.

      Sometimes, the fact of the matter is that you’re not the most technical person – or most learned person – in the room. This even more so when you are managing a product and your persona is a developer.

      When the team decides to support a new language or add a new feature about a trendy, technical topic, it’s ok – and even recommended – to approach the dev team and say “your knowledge is way deeper than mine, can you explain it to me?”. It’s good to do your own research, but it’s also good to go and ask someone who knows better.

      Much like if you had to cook an omelet with Gordon Ramsay around, you would definitely ask for advice, even if you are familiar with how to properly cook eggs. Same with developers- even if you know the language or the tech, it’s worth asking for deeper insights where you can.

      And devs, we all know that product managers can oftentimes ask too many questions and usually at the worst timing. Sometimes, the questions don’t make the most sense. And yet, it’s important to take the time and help the product manager on the team to make sure every last detail is understood when it comes to new features. Don’t be the Gordon Ramsay of your team. Help the product managers understand how to make the tastiest omelets!

      Finally

      Well, for better or for worse, product managers and developers are in this relationship together. There’s no point in fighting over who’s right or who’s stronger. The journey is done together, hand in hand, and as long as the team is going in the right direction, it really doesn’t matter.

      Working together will not only create a positive workspace but lead to better relationships and improve your coffee break experience, which we all know is what truly matters 😉 But really, it will also lead to a significant improvement in both developer and product workflows. By establishing good and strong relationships within your team, you’ll be able to increase velocity. This means that you’ll be able to deliver faster and fix bugs quicker, and thus help to guide your company in the right direction.

      Let’s be honest. At the end of the day, it’s all about speed and quality. And by keeping the love not only alive but thriving, between devs and product, you’ll make sure that your company is experiencing faster quality feature delivery than ever before- and you’ll guarantee a perfect journey together to a better product. Trust me, as someone who’s been there, it’ll be worth it.

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      Technology Choices During Hyper Growth

      Oded Keret | VP of Product

      5 minutes

      Table of Contents

      Previously in our hypergrowth series, we spoke about the business and cultural aspects of hypergrowth. We highlighted guidelines such as focusing on sales enablement and empowering your employees as a means to ensure that your company culture can be sustained as your business grows by 40% year over year.

       

      If you’re the one in charge of the software engineering organization, your responsibility is twofold. On the one hand, you’ll need to figure out how these challenges apply to your research and development teams, as well as the tools and infrastructure they use to perform their job. On the other hand, as the business grows, the technology and infrastructure will need to grow with it. You will be tempted to meet these growing demands by hiring even more engineers (hint: this isn’t the answer), which will bring with it its own set of challenges.

       

      As we dive into the second part of our hypergrowth series, we will analyze the impact of this growth on your organization and provide guidelines that will allow you to let your company keep growing.

       

      Excited About Growth! - Oprah Winfrey "You Get a Car" | Make a Meme

      The bare necessities of scaling

      In my experience, the first (and most obvious) impact of hypergrowth on the engineering team will be the need to scale. As more customers use your applications and services, the load on your infrastructure increases.

       

      If you did not have a cloud strategy in place from day one, your teams will scramble as you attempt to scale your infrastructure up and out by deploying more of it in hybrid and public cloud deployments, rather than purchasing additional physical hardware. If you had the very lucky foresight to go cloud first, you’ll still struggle to balance the cost of additional hardware with the risk of service outages and frustrated customers, which will lead to loss of business.

      In extreme cases, such as those presented by Netflix and Youtube during the Covid-19 outbreak, you may even intentionally slow down your business growth to avoid losing your business altogether.

       

      Beyond cloud scaling, other measures can be taken to ensure your teams maintain the product quality and the customer satisfaction it promises. Most importantly, ensure that your development and operations teams have the best logging, monitoring, and debugging tools available to make sure they can quickly solve any problem that arises as your organization shifts to the cloud.

      If your team has not already done so, this is also the time to shift-left your performance testing. Get ahead of the game and find and fix performance bottlenecks early on to make sure they don’t stop your teams from further scaling up the capacity of your applications.

       

      As with your cloud infrastructure choices, tool selection in this domain should also take into account that continued growth will impact the cost and usage of the chosen testing and operations tools as well. Opt to go after SaaS based tools to ensure that your teams stay productive at scale.

      CTOs — Scalability is not a side project | by Thomas Thimothée | Kerala Ventures | Medium

      The catch 22 of hiring for growth

      As the challenges of cloud scaling and quality grow, the velocity of your development teams may suffer. As your business continues to grow, your software engineers may find themselves spending most of their time solving customer issues or adjusting the architecture of their applications to meet the new cloud needs, rather than developing new functionality that will let your business grow even further.

       

      The natural solution to this problem might appear to be quite straightforward: hire more engineers. Not so fast, though. Hiring software engineers is expensive and time consuming. The culture you have worked so hard to entail in your teams will be much harder to maintain as the team grows and becomes global. You will find yourself investing in tools and methodologies that allow your growing teams to collaborate, and continue to write code efficiently in an ever-growing complexity.

       

      Keep in mind that the engineers you hire will require additional training and onboarding time as they attempt to learn the existing code base and understand the expected behavior of the application. Your attempts at increasing velocity may get thwarted by the need to develop training materials, invest in documentation of your existing code base, and by code review sessions in which your experienced engineers help the newly hired engineers find their footing.

       

      To mitigate these challenges, look for tools and methods that will let you increase velocity and maintain product quality without depending on additional hiring. If you haven’t adopted an Agile, DevOops, and Automation approach yet, this is the time to go there. Look for SaaS based CI/CD, source control management, and test automation tools. Such tools can be force multipliers that make your development team faster and stronger than the sum of its engineers.

      Why Should YOU Be At Conference?: School Library Media Specialist Edition | NJ Library Association Conference

       

      Another approach to increasing velocity and quality without additional hiring is to select the correct development frameworks and cloud architectures. Look for software languages and frameworks that promise understandability across a growing, global team, as well as a good fit for cloud and auto-scaling architectures.

      Without locking yourself to a specific cloud vendor, consider mixing the benefits of a microservices or serverless architecture with the ease and simplicity of commonly used IaaS and PaaS solutions.

      Let your Data grow with you

      Make sure the technologies you select can handle an ever-increasing demand for data. Give your engineers the tools they need to fetch data to support their own DevOps flows, as well as business data that will provide actionable insight to the product, sales, and marketing areas of your company. Choosing tools that allow easy collection, scalable storage, and smart analysis of the collected data could be the choice that will allow your company to maintain its hypergrowth state.

       

      We hope this writeup gave you some insight into the challenges your software engineering organization may face as your company goes through a hyper growth cycle. Some hard choices need to be made, and we hope the guidelines we suggested will help you choose the correct path. If we missed any tools or technological considerations on this journey, be sure to drop us a line and let us know!

      And sign up to Rookout (or something) 😉

       

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      A Pivotal Paradox: 6 Lessons Learned Managing A Fully Remote Team

      Dudi Cohen | VP R&D

      10 minutes

      Table of Contents

      A mere few months ago the majority of the world was forced to change drastically, including the move into a ‘fully remote’ mode of office work. As reality was bearing down upon us, tech managers and CEOs everywhere were huddled together trying to figure out how to not only make it work, but work well. They had to figure out the best plan of action- ensuring their employees were up for the task and that they had everything required to work from home for a seemingly indefinite amount of time. Alas, this is a difficult task as each employee is different and each of their needs varies from the other.

       

      The changes to becoming ‘fully remote’ that companies are still currently dealing with have affected each employee differently, especially in terms of their individual velocity. And of course, as is the way of the world, a dev’s individual velocity ultimately affects the team’s velocity. When looking at individual employees and their impact on the team’s velocity, there is one type of employee that stands out in particular: the pivotal employee.

       

      Lesson 1: Providing the basics

      When we were told at Rookout that we were to begin WFH (work from home), our first reaction was to make sure that our productivity wouldn’t take a hit. Our goal was to make sure that our team’s velocity wouldn’t slow down.

       

      Every company has had to prepare itself somehow. Some companies have previously practiced WFH methods, while other companies had to start from scratch. Throughout the process, we noticed that companies who previously didn’t allow or practice WFH for their employees had the hardest time acclimating – to the point that they were starting from the most basic step of having to buy laptops for their employees. Yet, these companies weren’t the only ones scrambling to figure it out. Even companies that had allowed their employees to WFH a day or two per week had to adapt and help their employees- such as helping them to create a home office by buying them office chairs, keyboards, or mouses (for example, companies such as Shopify and Google did just this).

       

      Regardless of where your starting position was at the beginning of these turbulent times (unless you’re a company like Gitlab or ArsTechnica), you probably handled it iteratively, much like Maslow’s Hierarchy of Needs, in what I have personally dubbed as ‘Maslow’s Hierarchy of WFH’:

      On the lowest rung is your employees’ Physiological Needs. You must make sure that they have all their physiological office needs met, even while not in the office. These are aspects such as having a working laptop, fast internet that won’t kick them off Zoom calls, and coffee that will keep them alert enough to work properly.

       

      The next rung on the pyramid is your team’s Safety Needs, such as having the correct VPN.

       

      Right after, their Social Belonging needs to be taken care of. This can be done in a variety of ways, such as with online happy hour zoom calls or zoom coffee breaks, for instance.

       

      Then their Self-Esteem needs have to be met. This can be done by ensuring each team member gets the proper recognition for their work, even if you’re not physically around to make sure they get it.

       

      Finally, you reach Self-Actualization- like writing this blog post on working from home.
      As you move up Maslow’s pyramid, you keep assessing how good your team’s velocity is. Throughout each level, you look around and think of how you and your team coped with this sudden and major change to becoming ‘fully remote’?

      Lesson 2: Managing to manage remotely

      As VP of R&D at Rookout, managing my dev team during these distributed times was definitely a challenge. Everybody was working from home and scattered throughout different cities. As everyone was getting settled into the new fully remote routine, we underwent our own journey through Maslow’s Hierarchy of WFH.

      Managing remotely is difficult, especially if you’re not used to it. When your job is to write code, it is mainly just you and your laptop. Sometimes, however, it is necessary to sync with your teammates and ask them questions, as well as of course syncing with your manager to understand what your tasks and goals are.

       

      When you are managing your team, you need to make sure that no one falls by the wayside. At Rookout we believe in minimum status meetings and minimum time wasters. However, when you’re not in the office, there’s no more being able to casually ask your dev, “how’s it going with your project?” while chit-chatting in the coffee corner, or being able to walk around to your team members and make yourself available for queries.

       

      Some of your team will proactively pull on your coat by Slacking you or ask for a Zoom session. However, a more significant part of the team will be those devs that you’ll need to Slack, call, poke, beep, fax, or really do almost anything to make them talk to you (even if they desperately need your help). Basically, this means that while you’re dealing with changing to a work from home workflow, you’ll also have to learn how to fit your managing style differently to every employee.

       

      Lesson 3: Coping with the highs and the lows

      The most frequently asked question that managers get asked by their own managers and colleagues in the last few months is “How is your team doing?” and not “How is that magic micro-service doing?” or “Where did you get that Zoom background?”. They’re simply trying to find out how your team is coping with their new distributed routine (or, to be exact, how you’re coping with managing them).

      About 10 years ago, I was talking to a friend who had just had his first baby. I asked him how things were going with the new baby and he replied with the best answer I have ever heard.
      “Having a kid changes your peaks. In the past I had highs and lows- sometimes I was happy and sometimes sad. With a kid, though, everything is to the extreme – your lows are feeling like the world is crashing down and your highs are like you’ve conquered the world.” For the past three years – since my daughter was born – I relate to this answer every day.

       

      Nowadays, when people ask me “How’s your team doing?” I tell them that becoming a ‘fully remote’ company really tests your team in an extreme way. All the peaks change. You know that team member that you have difficulty keeping track of in normal times? Well, good luck doing that when he is WFH and has such bad Wifi to the point that you can’t even see him on video. But hey, at least you’re not interrupting him and he’ll get some work done…right? And what about the team member that needed your help all the time and wasn’t able to move forward without it? Well, let’s be honest. He’ll probably just live his best life watching Netflix until he manages to catch you in between your other Zoom sessions.

      Lesson 4: Finding your team’s equilibrium

      So, real talk, how is my team doing? Did the team’s velocity change? Well, as Thanos says, “Everything is balanced as it should be”. In an almost karmically balanced way that none of us could have foreseen, some of my team have reduced their velocity, while other team members’ velocity has skyrocketed. However, the most interesting phenomenon that arose from this scenario is the pivotal devs.

      You know that one dev that knows everything? I don’t mean that they are the expert in everything, but rather that they know something about everything. This is the dev who knows all about the data flow in your system, why that piece of crap legacy code wasn’t deleted yet, about that feature that nobody uses, how to use the CLI to everything and, if needed, they will even configure your Emacs and VIM (although they don’t use them).

       

      This is the dev that everybody turns to when they have a question. They are the Google personified of your company. Even if it will take your team 10 minutes to find an answer, they will prefer to ask this dev instead. They are the oracle, the chosen one. They are your pivot. Really, you’ll sometimes even see a line forming near their desk.

       

      How much work, though, can the pivot truly accomplish in the office? How is their velocity? Well, frankly, it’s terrible. Everybody asks them questions all the time, making it so that they switch contexts like crazy. But what happens when everyone is out of the office? The pivot is at home and it isn’t as easy to disturb them as it used to be.

       

      Pivots flourish in these distributed times. They now have full focus and don’t need to switch contexts non-stop. Unfortunately, this increase in productivity means that the others might start to stumble blindly in search for answers or in search of the pivot. All of a sudden you’ll see a drop in the velocity of everybody else because the answers are simply more difficult to find when the pivot isn’t around to give them to you.

       

      Lesson 5: It’s time for independence

      The velocity drop around the pivot happened because some of the team members got too used to not being independent. Having the pivot around and available reduced the need to find the answers for themselves. Now is the time to start developing that much-needed independence.

       

      In order to reduce the impact on velocity, you must first make the pivot more available to your team and set your expectations with them. But, you must make sure that the pivot’s answers aren’t “fish”, but are rather “fishing rods”. His answers should be links to documentation, links to tutorials, or explanations as to how he acquired his data. Now is the time to teach your team how to fish. Don’t feed them. Let them feed themselves. Otherwise, when the next time the pivot isn’t available they will starve. See the following (very scientific and accurate, I’ll have you know) graph I doodled with the amazing Excalidraw:

      But how do you teach your devs to fish? Apart from trying to have your pivot show them where to get all the answers, of course. Well, it all boils down to ensuring that they have the right tools that help them properly understand their code. One of the major contributors to your team’s reliance upon the pivot dev is their lack of understanding of the code and the software that they’re working on. By giving them the ability to deep dive into the code and get the data they need to figure out what is happening in it, you’ll ensure a rise in their independence. So don’t worry, you and your pivot can sit back and relax at home in your pajamas, without having to worry that your devs aren’t able to work properly.

      Lesson 6: Reap the fruits

      Making everybody independent will increase your team’s productivity and velocity. After becoming ‘fully remote’ I doubt you will ever go back to what it used to be. There are multiple indications in the industry that the way we work will change in the future. After you’ve seen what your pivotal people can achieve when your team is more independent, you will keep making sure that they have the right conditions, as well as the right tools, to flourish. And after you’ve seen what your team can achieve when they are focused on being independent, you will never allow any dependence to develop again. Either way, whether your company returns to business as usual or changes its day-to-day, you’ll come back stronger than ever.

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      Supporting The Fight Against COVID-19

      Shahar Fogel | CEO

      1 minute

      Table of Contents

      We need all hands on deck to help find solutions to the global crisis. In light of this, we are giving away Rookout’s platform to healthcare organizations for free through the end of the year.

      This is not the time for clever marketing ploys. We see the value we can offer, and we want to offer it. Technologists around the world are working together, now more distributed than ever, to find ways they can help. We can enable engineers to debug their applications remotely, on the fly, and significantly reduce time to get their products working.

      Software development teams using Rookout have seen the time it takes to build applications and fix broken applications reduced from hours to a few seconds. The idea is we can help minimize the chore aspects of debugging remotely and free up vital R&D resources. Time is of the essence, and we can help them get it.

      Rookout is HIPAA compliant and fully committed to the security of private information.

      If you’re a healthcare professional, please share this information with your team to help ensure the offer gets to the right people. This is a tough time for many, but I am confident we will get through this better, together.

      Stay safe, stay healthy
      — Shahar and the Rookout Team

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      Boosting Development Velocity: Adapting Software Architecture to Changing Team Dynamics

      Dudi Cohen | VP R&D

      8 minutes

      Table of Contents

      Software developers and software architects have, for a very long time, stood on opposite sides of the “whose is better” competition. They have completely different beliefs, with each vowing that theirs is the correct one. Some swear by Java as the holy grail of backend; some worship Go as the right solution to all your backend problems. Yet, the truth is that determining the ultimate answer is far from straightforward. The choice of architecture depends on various factors, including toolsets and company-specific needs. However, even with careful planning, what happens when external factors affect your decision and you’re not prepared? 

      In the software world, this level of unpreparedness can come at a high cost to the velocity and productivity of your devs. Fortunately, at Rookout, we come prepared. Our unique developer tool gives an unparalleled ability to gain understandability into your code, which allows for an easy move between architectures. Unfortunately, when looking at the second part of the equation, there’s no such magical solution for the developers who work with those architectures.

       While the transition between architectures may be easy, the transition from their regular work environments – like working with a monolith architecture together- to a distributed work environment can take its toll on your team. The change to a distributed team from a monolith team can be difficult. 

      That’s why we will explore the concept of development velocity and its intrinsic connection to software architecture in this article. We’ll also delve into the challenges faced by developers when transitioning between different architectural paradigms and examine the critical role of team structure and management in optimizing productivity. So grab that cup of coffee, sit back, and let’s uncover the key considerations for team leaders aiming to boost development velocity amidst changing dynamics.

      Optimizing Development Velocity: Where To Begin

      Many guidelines and articles compare the two architectures, some better than others. From my experience working in a software development company, I’ve noticed that one of the most important factors to consider is how your team is structured and managed. While coffee breaks and happy hours aid company success, ultimately, the importance lies in how the company is built. Transitions between software architectures can pose challenges for developers, affecting their velocity and overall productivity. 

      Let’s explore the structural path to maximizing productivity and maintaining a cohesive team.

      1. Small, collaborative teams: If you have a very small team that works together, communicates with each other properly, and you’re not planning on increasing your team’s size – then perhaps a monolithic approach could be a good choice. Letting them build something together makes sense: the team cooperates well, and they know and understand each other’s code. 
      2. Distributed teams: If your team is distributed, no one really talks to each other, and everyone practices from different time zones. In this case, you should avoid a monolith, as the limited communication will likely cause it to become a modern-day Tower of Babel.
      3. Siloed R&D teams: Is your R&D structured into multiple siloed teams? Every team is the best in one area, and you just need minimal interaction between them? Go for building multiple microservices, but know that it is crucial to establish a consensus on the appropriate interface to ensure seamless coordination.

      In some organizations, you can see the opposite happening: the management and architects decide on the software architecture, which changes the team’s structure (or the team’s architecture). However, and this may come as a shock to many, I feel that most of the time, your team and its characteristics aren’t as dynamically changeable as your software architecture. 

      Force Majeure and Architecture

      Unexpected circumstances can affect your architecture, and usually at the most inopportune moments. These circumstances can be internal or external. As a manager or architect, you need to consider them and try not to let them cloud your judgment. Here are some common examples you might find yourself navigating:

      1. Trendy tech: Your manager (or sometimes your own team) read or heard from someone that “microservices is on the list of Top Trends of 2020”. When this happens, the natural response for human nature is to reach for the new and the shiny. I recommend you try not to succumb to it. While the new and the shiny may look flashy and cool and great for marketing, it isn’t always what is best for your company’s success. With that said, sometimes it can be a good solution to test drive and experiment with a small part of your software and understand whether that “trend” is good for you. Experimenting with a small part can satisfy your manager and won’t demotivate your “trendy” team.
      2. Transition challenges: You want to make a change but find that your team has been working with microservices all their lives. Suddenly, talking about a monolith and looking at one seems very scary. As often is the case, making significant changes like this can make even the merest of mortals think twice before embarking upon such a journey. There are many ways to transition into changes, though; not all are as anxiety-inducing. The process can be successful, especially if you do it gradually and educate your team thoroughly. 

      So, what happens when you encounter “Force Majeure”? And no, your manager telling you that you “must use microservices because everybody is doing it” is not “Force Majeure”. “Force Majeure” can be a sudden change to your team due to massive layoffs, for example, or maybe even an M&A that tripled your team. 

      Another example is what we saw with the COVID-19 pandemic. While the pandemic itself obviously didn’t suddenly change your architecture, the unprecedented circumstances that surrounded it caused the majority of people to suddenly find their team distributed and working from home. If, from the get-go, your team was tight, small, and very communicative – this allowed you to have a big ole monolith. Suddenly, the edge you had that enabled your monolith has dissipated.

      Maintaining optimal development velocity amidst change

      Software companies, ourselves at Rookout included, are lucky with the opportunity they have to work remotely. Companies and R&D teams who practice WFH (working from home) regularly were able to adapt quickly to situations that arose (such as COVID-19), albeit with less of a reason to take as many coffee breaks. Companies whose team’s architecture was distributed and working all the time remotely found that they didn’t actually need to adapt and were continuing “business as usual”.

      If this change doesn’t come naturally to your team, in order to adapt, you first have to ask yourself, “Do I need to adapt?” and then, “How is your team’s architecture compared to your software architecture?”.

      If, in the first place, your team has been working on completely different microservices, in which every service was the responsibility of one team member, then anyway, you are a distributed team working on a distributed architecture. 

      Simply draw a diagram of how connected your team is in order to figure it out. A common team working on a monolith would look like this:

      They communicate all the time, work on the same code, might depend on each other to deploy, and have a lot of inside jokes.

      Break them apart without breaking apart the software, and you’ll get what looks like this:

      Communication here is more complicated. This team is still working on the same code, but now the communication overhead between them will hurt their development velocity. This team must adapt and compensate for the new situation they find themselves in.

      When looking at a team working on microservices, it is as if the change is almost seamless. 

      They go from this:

      Into this:

      Everything was distributed in the first place, and while you might need to make sure your interfaces are tight, you probably won’t notice any significant overhead.

      If the shoe fits

      The main issue here isn’t about which architecture is better. Each has its pros and cons, but, as we’ve seen, one of them fits better if you are fully remote and distributed. Whatever “Force Majeure” you’re currently facing – if you are – won’t be around forever. As such, I wouldn’t recommend you start doing anything crazy like fitting your architecture for apocalyptic scenarios. On the other hand, you need to take a step back, look at your team and your architecture, and understand how a sudden change in your team’s day-to-day work affects your R&D velocity regarding your own software architecture. 

      Focusing on changes and adaptation

      When adapting to your new situation, you have to focus. You can’t do everything all at once to cope with current challenges; rather, you must focus on the biggest gaps and address those first. You can’t ignore communication gaps between your team members, even if they are siloed naturally and working on different projects, because communication IS key. 

      Sometimes, though, it is less critical than other gaps arising from different factors. Your software architecture is only one of the various factors to consider. In some companies, the biggest challenge may be making sure everyone has a laptop. In others, it can be ensuring everyone has proper remote access to their servers – whether they need full access and start using a VPN or only to debug remotely using Rookout. The factors are many, but ultimately we need to remember that what’s important in these unexpected circumstances is not whose architecture is better but this: maximum development velocity.

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      Code Deployment Bottlenecks And How To Tame Them

      Liran Haimovitch | Co-Founder & CTO

      3 minutes

      Table of Contents

      If you take a long hard look at the DevOps movement, you will find it actually divides neatly into two sub-movements. The bigger and often noisier of the two is about technology, advocating for the latest and the greatest tools and techniques, be they Cloud, CI/CD, Serverless, or Containers. The smaller sister, however, is much different, stemming from Management Theorem and focusing more on processes.

      Nothing says Management Theorem as much as ‘The Goal’ by Eliyahu M. Goldratt, that pesky little book ‘The Pheonix Project’ by Gene Kim, Kevin Behr, and George Spafford, was based on, which teaches us all about the Theory of Constraints and Bottlenecks. Today, I wanted to share with you my thoughts on the biggest bottleneck our customers and prospects are struggling with: the dreaded deployment.

      How do you know Deployment is the Bottleneck?

      To be blunt, deploying new code is always the bottleneck. Be it to development, staging, or production environments, you can (almost) never deploy fast enough. But you might want better proof than a comic strip (though personally, xkcd is more than enough for me ).

      Let’s start by checking where work is piling up. Open your issue tracking system (Jira, Trello, or something else owned by Atlassian) and take a look – how many of your tickets are pending deployment?

      Next, take a look at your team – engineers, product managers, support, and customer success. What are they worried about? What are they waiting for? If they are anything like the ones I’m working with, they are waiting for the next deployment to fix a bug or get a piece of data for them.

      How do you increase capacity at the Bottleneck?

      The obvious answer is to increase capacity within the bottlenecks. Some of the tested and tried approaches we have used for optimizing deployments are:

      1. Building a (better) CI/CD pipeline, which ultimately makes deployments cheaper and faster, thus requiring less time and hand-holding.
      2. Design your applications along the Twelve Factors, especially Disposability, making deployments and restarts faster and easier across environments.
      3. Adopting various safety nets such as monitoring tools and advanced deployment and rollback procedures such as Canary Releases, which increases your risk tolerance to bad deployments.

      Unfortunately, we’ve noticed that many of these action items are easier said than done. After all, there is a reason deployments have become a bottleneck, and if it was that easy to fix engineering leaders throughout the world would have squashed it long ago.

      If you are building a new software product, you should definitely go for it. But for most existing (and profitable!) software products out there, this journey is likely to take years, cost decades of human time, and may not even net a positive ROI.

      “You are reading this book for two reasons. First, you are a programmer. Second, you want to be a better programmer. Good. We need better programmers.”

      ― Robert C. Martin, Clean Code: A Handbook of Agile Software Craftsmanship

      Riding the Bottleneck

      At its core, the Theory of Constraints doesn’t focus so much on increasing capacity at the bottleneck, but rather maximizing the value you are getting out of the bottleneck. So how do you go about it? What we’ve learned from our experience is the following:

      1. Make sure you are always deploying the most important task. Have the second important task waiting for its turn. Avoid switching halfway due to shifting priorities.
      2. Make sure whatever you are deploying is truly ready for it – tested, documented, and meets all your prerequisites. Avoid switching halfway due to missing pieces.

      Avoid unnecessary deployments by using tools such as WalkMe for in-app assistance, LaunchDarkly for managing features, and Rookout for understanding your code and adding logs and metrics on the fly.

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      3 Cool Tech Tapas We Found At Cisco Live Europe

      Liran Haimovitch | Co-Founder & CTO

      3 minutes

      Table of Contents

      I love traveling to Barcelona for conferences. Between Gaudi’s architecture and tech architectures, there’s really not much more I can ask for.

      Walking into Cisco Live!, the first thing I encountered was the huge indoor fountain called the “Rain Wall”, which was built to raise awareness and celebrate Cisco’s Corporate Social Responsibility program.

      Once I was finally able to tear my eyes away from that spectacle, I spent a lot of time going over the fine mix of booths, folks and a great deal of tapas I consumed every day (yum! I regret nothing!). Looking back at the experience, I realize there were a couple of notable trends worth mentioning. Some of them are the rising stars we’ve become accustomed to discussing, and some just a mere taste of what is to come.

      Get Your Head Out of the Clouds

      Stepping into the conference was quite a change of scenery for me. As someone who has, over the last few years, predominantly dealt with cloud technologies, I was now suddenly seeing actual hardware. As you can imagine, I was not so sure-footed at first, as I felt like I had stepped into some kind of alternate reality where people didn’t like chocolate and actually liked mornings. A significant portion of the conference was about the cloud, including hybrid cloud, but at the end of the day, the heart of the conference was hardware and networking-focused.

      As a SaaS-based company, it seems to us that everything is happening in the cloud. In the real world though, interest and investment in enterprise data-centers are still going strong.

      Everyone’s a Dev

      Most of the network engineers I met at the conference are well acquainted with the world of software engineering. While they don’t define themselves (yet) as software engineers, the majority of them do write code as an integral part of their jobs. This is further amplified inside the conference itself by the huge DevNet zone focused on training and tooling CCIE’s (Cisco Certified Internetwork Expert) on software development. Being excited to be part of DevNet ourselves, we posted this little Rookout tutorial online.

      The Focus on Observability

      Last, but definitely not least, AppDynamics hit it out of the park with the launch of two new features in the realm of observability.

      The first was focused on full-stack observability through the integration between AppDynamics and the Cisco Workload Manager. This integration allows obtaining full-stack visibility from the hardware and the network, all the way up to app performance. The second was focused on business observability, seen through AppDynamics’s Business iQ in which they measure your software’s performance in dollar figures. Business and dev are becoming intertwined in a way that affects both sides’ metrics for the very first time, as I mentioned in my Predictions for 2020.

      Our clients have long been experiencing this exact notion – that there’s a lot of potential combining code-level data extraction with new observability stacks. Understanding the code’s behavior rapidly, combined with observability and business metrics seems to be the cocktail du jour.

      Sometimes It IS All Fun and Games

      My experience at Cisco Live Barcelona really emphasized the importance of incorporating all aspects of technology. There’s really no limit to where we can reach, whether it’s by entering your talk by flying in on a jetpack or making it rain inside in the name of social responsibility. We need to aim as far and as high as we can and once we reach those heights: keep going.

      I definitely now need a few weeks to rest my feet, but I’m looking forward to meeting old and new friends at Cisco Live Las Vegas in a couple of months!

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      Why We’ve Decided To Make An Ungated Rookout Sandbox

      Oded Keret | VP of Product

      5 minutes

      Table of Contents

      A long time ago (in April 2019), in a galaxy far far away (in Tel Aviv, Planet Earth) we launched a self-serve debugging option. Users could now sign up for Rookout without having to contact us first. We invested significant resources in online promotion and — just like any startup at our stage — we hoped to soon see legions of users signing up and using the tool to fetch data directly from their code. However, like many other startups in that initial stage, we soon discovered that many of the people who signed up had quickly “bounced”, never to be heard from again.

      Welcome to the Rookout Sandbox. Get a quick taste of what Rookout can do.

      We had several theories as to why folks leave without as much as installing our SDK. Our suspicion was that perhaps, for some developers, installing an SDK isn’t as simple or straightforward as we had initially assumed. We figured developers would not easily trust a startup with what appears as having access to their code, even though we absolutely do not. Some members of the team believed the setup was more complex than we anticipated, while others suspected our product still had some quality issues that prevented a smooth setup experience.

      User test it: do, or do not. There is no try

      Naturally, we performed some user testing. We also reached out to some recent signups by email and chat and collected data using in-app surveys. We learned that some of the assumptions above were correct and we also gained a couple more insights.

      Perhaps the most interesting thing we learned was that some people who signed up for our online offering don’t really understand what they had signed up for. Although they visited our web pages and watched an online video of a demo, some still thought they were about to install an IDE plugin, or that they were just about to use another web-based IDE. Some even believed they had signed up for an online coding school.

      At this point, we were stumped. We worked so hard to provide a smooth, friendly onboarding experience, we fixed so many bugs and glitches, invested in the design for a brand new Welcome Wizard, and built a bunch of “workarounds” for tricky setup steps like setting up the Rookout SDK and fetching the developer’s code from their favorite git repo. And now, after months of hard work and frustration, we learn that some people don’t even know what they’re expected to get.

      Saved by the Vader: “I find your lack of faith disturbing”

      Happily, Liran, Rookout’s CTO, came up with a great metaphor. Like many things in our day-to-day conversations, it had to do with video games and Star Wars. He mentioned that in Star Wars: Battlefront, you get to play as Darth Vader during the tutorial phase. Other than being an epic experience that makes any mere mortal feel like the dark lord himself, it also has an important message: this is what you’ll be able to do when you make it through the game. It provides the motivation to work hard and sets expectations as to what you can achieve. What if we had a similar experience for new Rookout signups, just before we ask them to “work hard” at setting up and learning how to use a new tool? That’s when we came up with the “Rookout Sandbox” experience.

      The original quest log.
      Step 1 – set a breakpoint.
      Step 2 – configure integrations.
      Step 3 – take over the galaxy.

      We set out to provide the ability to get a true, end-to-end taste of what Rookout can do for a developer, without all the hassle of installing, fetching code, and learning how to use a new tool. Just set a non-breaking breakpoint, trigger your app, and instantly get the data you need. Something so simple and straightforward, baby Yoda can do it without waking up from his (or her?) nap. Inspired by the gaming metaphor we took things a step further by adding a “quest log”: a set of hints, leading the user to check out some advanced Rookout abilities such as conditional breakpoints, rate limiting, log line formatting, and more.

      Never tell me the odds

      We launched the Sandbox experience a few months ago, and the initial feedback was highly positive. Users told us they were quickly able to understand what the product does. Some users even used the sandbox to demonstrate Rookout to their friends, allowing us to reach even further into the developer community and into the very center of our target audience.

      Another significant change was when we made the Sandbox experience *ungated* – that is, users can simply enter the sandbox with a click of a button, without needing to sign up or provide us with their name and email address. When we made this change, weekly Sandbox usage increased by 300%.

      We made the sandbox experience ungated in mid-November. Can you spot the change?

      Sandbox: a new hope

      The sandbox experience has allowed us to overcome one of the biggest challenges for any newly-launched product: quickly demonstrating the value of the product with as little friction as possible. Of course, things are still not perfect. We keep iterating on our onboarding experience, collecting data and feedback from our users as we try and refine our setup flow. This time, however, we are certain that new users understand what they should expect to gain, and why they should try and gain it.

      We hope you take a chance to try out our sandbox for yourself. You should be able to complete it in a couple of minutes and if for some reason you can’t — do let us know what stopped you! If nothing else works, we may end up adding Darth Vader and lightsabers. Or maybe baby Yoda. This is the way.

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      Debugging in Production: How To Stop Fearing The Inevitable

      Maor Rudick

      5 minutes

      Table of Contents

      You’ve been staring at your screen for hours, trying to check why a certain bug is occurring only for end-users in your production environment. You’ve tried a multitude of approaches: adding log lines in all kinds of indicative places, logging potentially relevant variables to get an indication of the state in which the bug occurs, and the list just goes on. Sounds familiar?

      So what happens when you’re stuck? Do you reach for that last holy grail of debugging in production and hope for the best? This type of debugging is what most developers hope to never need to do, yet often have to. The ability to debug in production is a valuable tool as it eradicates the necessity of reproducing issues and rather allows the developer to find the issue and fix it while the system is still running.

       “Each new user of a new system uncovers a new class of bugs.” – Brian W. Kernighan

      Debugging modern infrastructures and its challenges

      Many common debugging methods are quite difficult, can be complex, and are often inefficient and not cost-effective. While debugging, it is often time-consuming to try and understand what exactly is happening and how to get the necessary data to fix it.

      The rise of modern infrastructures, such as serverless and microservices, took away the visibility that used to exist into software. Visibility became limited at scale, and slower response times arose when it came to understanding and debugging production environments. When comparing these modern infrastructures with the older monoliths, some key differences between them are apparent. The main one is the fact that monoliths are simpler structures.

      Debugging approaches: Modern infrastructures vs. monoliths

      When comparing debugging monolithic infrastructures with debugging microservices, there is a clear difference. Debugging monolithic applications is easier than debugging microservices, due to the fact that monoliths are a single system with a single codebase. Most significantly, due to it being a single entity, logging is kept to a single location instead of being divided up as is done with microservices. Microservices, in comparison, are multiple independent and maintainable services, which due to their quantity, make debugging more complicated.

      Serverless architectures are similar to microservices, yet are even more complex, which complicates even further. Serverless applications are broken down into smaller pieces than microservices and are also fully managed by the cloud provider. Tracing is a crucial component of debugging serverless applications, as it aids in analyzing the whole picture. Yet as the serverless architectures become more complex, tracing can be difficult to do, often resulting in a waste of time and resources.

      Debugging in production – the plot thickens

      Production debugging is even more challenging than traditional debugging. So why do we even need to debug in production? Debugging in production allows one to find the issue and fix it while the program is still running. Production debugging also eliminates the need to reproduce issues. Reproducing problems is similar to looking for a needle in a haystack- sometimes it works, often it doesn’t. Eradicating the need to attempt reproducing problems is significant as it can save precious time and resources.

      Debugging in production does carry certain dangers. There are three main dangers involved in debugging apps in a production environment. The first danger is the fact that production debugging can disrupt the current users of the running application. Another danger lies in slowing down the performance of the application or even crashing the app altogether, which would also greatly disrupt users. Last but not least, it might necessitate restarting the application in order to debug, which can also involve the first two dangers – since the app would need to be stopped and be unavailable to current users.

      Debugging techniques: how to debug in production

      So how do you dip your toes in the deep waters of production debugging? There are a few ways you can go about it, some more code-friendly than others. One way is to simply ignore the issue until you can’t possibly ignore it any longer. Another way is to test absolutely everything you possibly can, in the hopes that you’ll leave no room for doubt when something goes wrong. A third path you could take would be to write as many logs as possible, as well as monitor everything as much as possible.

      Of course, you could also, as Mark Zuckerberg so aptly put it, “move fast and break things”. You increase the speed of software updates that take care of whatever went wrong, without actually investing the resources necessary to debug well. Last- and definitely not least- is the newest approach that focuses on decoupling the data layer from the applications and lends visibility into production, which can be achieved on demand.

      The most common method out of these options is logging. As you debug, you go through all the log files in order to find the data you need to understand where the bug is and what happened at the time of the error. This is not only time-consuming, as you will have to have to access and sift through many log files, but it’s also often necessary to write additional logs, and then redeploy and restart your application, just to get additional data.

      The optimal debugging tool would be on par with the fifth and newest method. It would have all the full stack trace and variable data that you’d need to debug your app while eliminating the need to restart the application in order to debug it. It would also not disrupt the app’s users to debug an issue, nor would it slow down the app’s performance to gather the required debug data.

      Production debugging: looking to the future

      The future of production debugging lies in being able to reach the necessary data, whenever and wherever it’s needed. In order to successfully debug in production a method of debugging is needed that won’t break anything, slow down any applications, or cause the users any ruckus. One such way is by setting breakpoints that don’t stop the application, but rather let it continue to run while enabling visibility into the running code and letting the developers collect the data they need. No more breaking things- just more data, more simply.

      Is your elusive production bug still getting you down? Hopefully, this article gave you a better perspective into production debugging and it’ll be less of a scary journey for you moving forward. If you need someone to hold your hand while you proceed on your debugging path, let us know and we’ll do our best virtually.

      Rookout Sandbox

      No registration needed

      Play Now

      Table of Contents
      Get the latest news

      Monitoring Systems Of Engagement: Riding The Waves Into The Future Of Software

      Liran Haimovitch | Co-Founder & CTO

      3 minutes

      Table of Contents

      If you’re building software, it is very likely that you are familiar with Conway’s Law. It is the single most important rule for software development. Employing this law will facilitate your success. Failing to abide by it, on the other hand, will guarantee your failure. According to M. Conway:

      “…organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.”
      — M. Conway

      I’ve witnessed this first hand in many places throughout my career, among them command and control services, Big Data processing systems, and even security components embedded into the very cars you are now driving. And yet, nowhere has this been more apparent to me than in software monitoring.

      First Wave: the birth of IT departments

      In the early days, just as businesses were adopting computers, they began setting up IT departments. These departments were in charge of digitalizing the organization’s internal processes. This included providing employees with computers and the network to connect them, setting up server rooms, and deploying software such as Microsoft Exchange and Oracle ERP. Today, this arrangement is often referred to as the system of record.

      Just as Conway’s law predicted, this is precisely what they monitored. They made sure the computers and servers were alive, the network was connecting everything, and that hardware utilization for the servers (CPU, memory, storage) was within bounds.

      Second Wave: moving to the cloud

      The ground started shifting as infrastructure began moving out of organizations with the help of hosting companies, cloud computing, and now PaaS and CaaS platforms. An ever-growing number of utility services were being consumed as SaaS, reducing the number of services IT had to deploy and operate.

      At the same time, IT was being shifted from the back office to the front, and business management was expecting it to be the very focus of the customer’s journey. Mobile apps became the norm through the use of in-house developers, as well as outsourcing companies, flashy marketing websites, online purchasing, and loyalty programs.

      Here, again, a new generation of software monitoring solutions arose: Cloud Monitoring tools. These tools are in fact a drop-in replacement for data-center monitoring tools and Application Performance Monitoring (APM). APM tools provide quality of service metrics such as uptime and latency, thus allowing IT to detect and remediate service disruption.

      Third Wave: monitoring beyond IT

      Today, we have passed the tipping point. Most IT and business leaders want to unload their system of record to SaaS providers as much as possible. At the same time, digital presence is becoming a key differentiating factor for all enterprises, even in traditionally offline markets. All these new services are being deployed to the cloud, with little to no “classic” IT involvement.

      With all this occurring, where is software monitoring to go? If our business leaders are measuring us by how we engage the brand’s customers and the business value we are generating, then that is how we must measure our software. In fact, many forward-looking digital companies such as Facebook and Netflix have been monitoring themselves using metrics such as ‘likes’ and ‘hours watched’ for a long time now.

      What’s next?

      IT organizations are freeing themselves from the shackles of traditional IT, focusing more and more on an agile, DevOps, and product-centric approach for delivering unique software.

      By monitoring software through the lens of business values and metrics, we have a much better perspective of how we are doing. As the CTO of a DevOps-oriented startup, I’ve learned that by doing so we make better decisions, allocate resources where they will truly make an impact, and are able to show our superiors and subordinates why it matters.

      You know what your software is set out to achieve. Define it. Measure it. Track it. Optimize it.

      Rookout Sandbox

      No registration needed

      Play Now