Rookout Transforms Flytrex's Ability To Debug In Production

Download PDF

Flytrex also encountered a unique challenge in being regulated by the FAA. FAA regulations stipulate that companies are unable to deploy to production without approval, as the production system is regulated by the FAA; as such, changing to a version that hasn’t yet been tested, verified, and certified by the FAA is not allowed. Therefore,a search for a solution had to be done that would allow them to find the source of the bug quickly, assess the situation, and be able to rectify it or inform the FAA about it as quickly as possible.

Rookout In-Flight: An example

There’s no better way to illustrate how a live debugging tool works than when your code is running. Or in Flytrex’s case, when a drone is in flight.

Rookout gave us complete confidence in being able to handle any bugs we were faced with, especially in prod. This confidence was due to our ability to gain a full understanding of the bug we were encountering. We therefore knew how to work around it, as we understood the issue completely, and can conclude whether it was safe or not safe to fly in that particular situation, even if the actual fix came a few days later. We were able to quickly understand the source of the bug, thanks to Rookout

During an internal test, with their drone up in the air, an issue with session counting was found. This issue took place due to a surprising race condition, which is very difficult to catch. The bug itself wasn’t even in the code, but rather in a library that was being employed. Rookout was used to pinpoint the bug and, while the drone was in flight and their people were in the field, were able to provide a workaround and ultimately a solution.

When in a test, while our people are in the field, having a certain component or scenario not working is quite uncomfortable, to say the least. Rookout gave us the confidence that we needed in knowing that we understand what’s happening in our systems, that it’s something we can and will fix. The certainty that we know how to solve whatever issue we’re facing helps us both internally, and with the FAA.