The Monitor — Andy Tuba, Senior Software Developer at Reddit

For the sixth edition of The Monitor we spoke to Andy Tuba, a Senior Software Engineer at Reddit. Reddit is a site that needs no introduction, but we’re gonna write one anyway because otherwise this section would just be blank. They bill themselves as the front page of the internet, and considering they’re the 8th most popular website in the world, that isn’t just marketing pablum. Some people spend their entire work day on Reddit, instead of the site for whatever less popular company they work for.

(Editor’s note: This video literally contains more tuba than any film — long or short — since Mr. Holland’s Opus.)

I’ve been in an employee at Reddit for going on two years now, but I’ve been a contibutor to the Reddit ecosystem for going on seven years. There are tons of third party extensions for Reddit: there’s mobile apps, there’s desktop browser extensions, there’s stylesheets; many, many ways to alter and expand upon Reddit’s own site. I’ve personally been a long-time contributor to the Reddit enhancement suite, which is what eventually led to me working here.

My goal as an engineer is to listen to the users, take a close look at how they’re using Reddit, learn more about what they want out of the site, and give it to them. I want it to be easy for users to find the communities they’re interested in, and then chat, comment, and enjoy all the cat pictures and discussions they like.

Deploying Code

When you want to deploy some code at Reddit to make improvements for those users, there’s a similar process across all our teams. After your code is reviewed, you go to the Slack channel that represents your team and you ask Harold the Robot, “Could I please have the conch?”

Harold might say, “You have to wait in line, but when it’s getting to be your turn I’ll tell you.”

Then once your turn arrives, he’ll say, “It’s your turn, you have the conch, you may deploy.”

Once you log into your service health dashboards, your error monitoring dashboards, and your server error logs, it’s time to roll out. We have a tool called the Rolling Pin that compiles and pushes to all the app servers for your service.

Monitoring That Code

Of course, I don’t want there to be bugs when I push code out, but there likely are going to be bugs. It’s pretty unavoidable. Two key ways I keep an eye on my code is:

• Self-monitoring

• Sentry

What I mean by self-monitoring is that as an engineer, and really just as a professional, there are days where you’ll feel sleep or cranky and really just not want to deal with anyone. When that happens to me, I dig into what’s causing it: did I not eat a good lunch? Did I not get enough sleep last night?

I ask myself if there’s something I can go do to improve how I’m feeling: take a walk around the block, find a snack, do a little meditation.

I set reminders for myself to go eat lunch by 1:30, to start wrapping up and packing to go home at 6. It does no one any good to push yourself beyond reason, that’s a great way to make mistakes and write a lot of bugs.

And with Sentry, what I really appreciate is that it’s collecting related errors into a single issue. On a site as big as Reddit we have a lot of errors coming, but usually they have the same root cause or maybe 50 same root causes. Those 50 different root causes might lead to 50,000,000 items, but when I log in to my dashboard and see there’s only 50 items to deal with that each happened a million times, it’s so much easier than trawling through every single bug report.

We also use Sentry’s Slack integration so that we’re pinged to the right channel whenever a new issue pops up. That’s a lot better than staring at a dashboard all day.

This is just part of what Andy had to say. Go here to watch our video interview and read the rest of his post.