January 23, 2024
Dealing with bugs that are surfaced by your error monitoring and observability tools is a crucial aspect of software development. The Goast team has built a platform that can understand and diagnose those issues as they are discovered in real time. This is an important first step in issue handling as accurate categorization and prioritization of critical vs. ignorable bugs can save development teams a ton of time when tackling exceptions and crashes. Let’s jump into how Goast’s newest release of intelligent issue categorization gives users insight to how the platform understands & resolves bugs and also drives the company’s pricing model.
Generative AI is the backbone of the Goast platform, enabling it to comprehend entire codebases with a deep understanding of its nuances and purpose. This understanding is pivotal in accurately categorizing issues that arise. Without an understanding of the application’s codebase, it’s impossible to know if an issue is related to the code, a third party dependency, client input, or something else entirely.
Goast stands out by categorizing issues into solvable and unsolvable categories. Its proprietary orchestration engine takes in stack traces and crash data, and then determines the exact nature of an issue. This precision ensures that developers focus only on what truly needs their attention.
Let’s take a look at some common examples of exceptions that we’ve seen and the generated explanations of why each tag was added.
Issues are tagged as “Solvable'' when they can be resolved by making a change to the codebase. When this isn’t apparent, an issue may be tagged with other categories as needed.
In some cases, you might see an issue tagged with “Needs Info.” This is typically because Goast believes it needs more context in order to solve the issue. Since the first pass of issue categorization happens before the engine gathers relevant context from the codebase (we’ll talk more about this later), sometimes Goast just needs to look into the code to correctly handle the issue. In other cases, it may recognize that an issue could be solved in multiple ways, in which case it needs to refer to the codebase or get additional context from the user.
Many exceptions that are caught by error monitoring services are expected, transient, or just plain noise. Below are a couple examples of when the platform recognizes that an issue can be ignored.
Finally, if the source maps are not available or … Goast will tag issues as “Obfuscated.” Just like an engineer, Goast struggles to understand exceptions when symbols are missing or minified.
Most dev teams deal with all sorts of issues - some are noise, expected exceptions, client errors, or third party issues. On the other hand, some issues may be critical and call for an immediate hotfix. With this in mind, the Goast Team has decided to offer a pay-per-fix pricing model. That way our customers are only paying for fixes that provide value to their team. Issues that are categorized as ignorable or are unsolvable by Goast won’t cost customers a dime. We think this is an important distinction of the service when compared to competitors and will continue to ensure customer satisfaction in the future. This flexible payment model is only possible due to our intelligent categorization technology.
Currently, the Goast Team is working on additional categorization features that will improve functionality and give users more insight into the thought process of Goast’s categorization algorithm. Introducing multiple passes with various amounts of context, feedback loops, and automatic reinforcement learning will enable us to more quickly and accurately categorize our user’s issues.
Strack traces and error data can embed different amounts of information. Some stack traces you can look at and clearly understand if they are solvable or ignorable. These we can quickly and efficiently categorize at the beginning of the process. Some issues need more investigation. The engine might need to look at the code where the exception is thrown or areas around that code to determine if the error is something that is directly surfaced to a customer in an intentional way, or something that should be handled.
Bread crumbs and other usage data leading up to the crash can also sometimes be invaluable for understanding the state of the system or inputs that might contribute to the problem. And more general metadata such as number of occurrences or number of affected users can be used to understand the severity. These are all areas we are focusing on to improve the categorization we provide to our customers.
With automatic reinforcement learning we can continuously improve the results of categorization for our users. These improvements include a mechanism that enables the engine to recategorize issues after it tries to solve them, automatically incorporating reinforcement learning. That way when the platform categorizes an issue as solvable but fails to resolve it, or you have a unique error that is ignorable, this information feeds back into the engine.
Then when that particular issue is solved, either by regenerating a solution with additional context or by another engineer pushing a fix, the system will understand how to categorize and resolve similar issues in the future.
Goast.ai represents a significant leap forward in automating issue categorization and resolution in software development. Its use of Generative AI ensures that developers can focus on what they do best, coding and creating, while Goast handles the rest.
