A Developer’s Take On Solving Hard-to-Replicate Performance Problems In-Production

Causal is the business and financial planning platform that allows users to build models effortlessly, connect them directly to their data, and share them with interactive dashboards and visuals. Teams rely on them to build complex models with real business impact, so the UX needs to be fast and reliable, and for the team to guarantee that, they need detailed visibility into the performance of their application.

Initially, they used Sentry for error monitoring, but since performance monitoring is included by default, the team went all-in and saw how out-of-the-box solutions to common problems freed up developer time while customizable instrumentation helped solve performance issues unique to their business.

If you’ve ever worked on a frontend app you’ve probably heard ‘our users are saying it’s slow, we don’t really know where that’s happening or if anyone’s reporting it’. We need to be able to see these things before a user reports them. Andrew Churchill, Software Engineer, Causal.

Read on to learn how Andrew’s team:

• Improved the performance of their service,

• Proactively addresses performance issues over time and,

• Uses distributed tracing to find the root cause of performance issues from the frontend to the backend, with two recent examples

Complex formulas need custom solutions

On the frontend, Causal liked Sentry’s out-of-the-box Redux, Postgres, and React error and performance monitoring support; and set up custom dashboards to monitor all transactions for real-time performance insights.

On the backend, they can effectively monitor their middleware, and the Sentry Express SDK gives them deeper insights into individual transactions when they need it. For example, the team recently built an entirely new system that changed how they approached permissions. They needed to get a sense of how it was performing since it’s used in almost every request on the backend and could easily become a bottleneck for subsequent requests.

Besides single event investigations, Sentry is also very useful to get a general overview of how our app is performing from dev to QA to production and we use dashboards to make sure that we’re always up to date on how performance is looking across the frontend, the backend, and our evaluation service.

Solving real-time performance issues

Users rely on Causal’s formula editor to calculate different revenue variables, which often involves formulas with complex values. By combining automatic and custom instrumentation around react components, the team is able to see how fast or slow that editor is performing and investigate any bottlenecks. A recent example of this shows how Sentry helped them identify a drop in performance with a particular lifecycle method being called, dive deeper into what happened, and resolve it:

On the backend, Andrew’s team recently hit a snag, where a user’s model was taking longer than expected to evaluate and they couldn’t easily replicate the state of the data locally without considerable effort. With Sentry they were able to identify the slow transaction and see which phase of the dependency graph generation was causing the lag:

Sentry is particularly useful for us to understand problems that are hard to replicate locally.

Causal needed a performance monitoring tool that could automate certain instrumentation processes while leaving room to build custom instrumentation as they added new products. Going all-in with Sentry, they’ve seen how built-in solutions to common problems can free up developer time, while customizable instrumentation helps them take on performance issues unique to their business.

As we update our own features, we expand our use of Sentry’s custom implementations and get lots of value in both understanding specific performance problems and getting a general sense of how our app is performing.

Causal will soon giving users the ability to evaluate models with 1b+ cells in just seconds, so keep an eye out if you’re a fan. And if you’d like to join Andrew's awesome team, they’re hiring across the front-and backend, check out their job listings for more.

If you'd like to learn more about they're getting the most out of Sentry, check out our full conversation with Andrew here.