Operations observability on top of your data stack
Meet Avenue, the cutting-edge platform for operationally complex companies that brings your data and operations stack to life.
Today, I had the opportunity to engage in a conversation with Justin Bleuel, a member of the Avenue team. As one of the early users of their service, I was pleased to discuss their innovative product offering.
Justin's professional journey began at Uber, where he was a product manager for internal tooling. This experience exposed him to the robust operational framework required to support a rapidly growing business. Uber, like all cutting-edge tech companies, spent a lot of resources to collect good data and store it in a data warehouse. As an internal tools PM, Justin's mandate was to help the operations team take action on this data. I was particularly interested to learn how their tech team effectively managed diverse operational metrics and swiftly translated them into actionable insights.
While at Uber, Justin worked closely with the operations team,who were the driving force in each city's success. The General Managers (GMs) oversaw their respective cities' profitability and overall marketplace health. Maintaining a healthy marketplace encompassed hundreds of important metrics and moments where things could go wrong, such as vehicle inspections, accurate insurance documentation, surge pricing, supply and demand balance, driver onboarding, and consistent driver ratings. Justin was primarily involved in activating Uber’s plethora of business data and building in-house tools to gather data on the driver and restaurant experience and marketplace performance.
The operations team at Uber had an array of internal tools that were vital to company operations. Еhe GMs were able to make business decisions and detect issues in their markets based on these tools. One notable example was their internal dashboards, including Heaven (marketplace monitoring), Terra (Uber Eats marketplace monitoring), Wok (restaurant performance), Summary, and Leaderboard, which provided insights into market performance.
It is worth mentioning that Avenue has completed the prestigious Y Combinator (YC) program and has recently announced a seed funding round led by Accel.
The purpose of this newsletter is to delve into various approaches to data activation. Avenue is vital in this domain, providing essential components for effective data utilization.
Can you walk us through your background and talk about what led you to start with Avenue?
Yeah, definitely. I'm Justin, co-founder and CEO of Avenue. Avenue is a command center for operations teams, and we'll talk a lot about operational analytics today.
In undergrad, I studied computer science and math and then worked in product management at Uber and Uber Eats. At Uber, I saw firsthand what operations teams at high-growth companies look like. I co-founded Avenue with a friend of mine from high school, Jeff Barg. In school, we built iOS apps together. We were one of the first apps released when the iPad app store launched. We even reached a few hundred thousand downloads. We've been tinkering and working together for a while, and Avenue is our first proper venture together.
There are a bunch of people who have left Uber in the past two or three years to start their own companies. Was Avenue inspired by any internal tool?
Yes, Uber had a lot of internal tools and needs that could’ve been solved with Avenue. My first experience with operations teams was at Uber; they were the folks who would roll up into the GM of each city. Uber had a city model. The GMs (general managers) were responsible for the P&L of each town, and they were responsible for the health of their marketplace. Unfortunately, marketplace health isn't a single metric – it’s a rollup of several composite parts: are vehicles being inspected? Are the driver's insurance documents accurate? Are we surging too frequently? Are we oversupplied or undersupplied? How can we onboard more? Are drivers getting consistently rated low? How's the support volume, etc.
So ops teams were responsible for everything underneath that marketplace health metric. And they had a ton of tools internally that would help them. We had internal dashboards called Summary, which would show us the performance of each market. And it was widespread that the ops team would effectively open them, refresh them, and check these dashboards throughout the day to see if there were issues. On the Uber Eats team side, we built some proactive tools, one of which was called Terra, which would tell us if we were surging in a market, a weather issue, or a performance-related event that might cause a demand spike. Also, when I worked on Uber Eats, our team released an alerting tool build for restaurant operators on the iOS app store.,
I was also responsible for helping to drive restaurant reliability. Things like is the restaurant not accepting orders? Are menu hours inaccurate? Is the order cancellation rate above average?
Every operations team at a tech company has some version of what we had at Uber. And every version gets rebuilt ten times over and deprecated at some point.
Working alongside engineering teams, I saw that they had tools like Datadog that gave them peace of mind that their system was reliable, it had high uptime, and they were responsible for maintaining it. But operations teams are similarly building these systems. But they have no tool to say, “What is the reliability of our restaurant's online ratio system?” for example. They would have dashboards to show performance. They would have playbooks or SOPs, like Google Docs, that would tell them they help answer the questions on what we should do when performance starts to drop. With Avenue, we want to bring engineering practices ensuring reliability and uptime to the operations teams in a single tool.
Going through your blog on the website, I noticed many case studies related to brick-and-mortar grocery chains, which look pretty similar to Uber Eats business from an operations standpoint. I assume that this segment was one of your first customers. At what point of their growth and with what complexity of operations should companies consider turning to Avenue?
Avenue is right for any company with a data warehouse, whether a Series A company or enterprise size. We serve both. In terms of the end user of Avenue, as a company increases in size, we serve more specific teams, so for a Series A, our end user tends to be the COO or Head of Operations, whereas, at an enterprise company, Avenue is brought on to solve a particular use case. For example, a publicly traded food delivery company uses us to manage restaurant menu quality.
Over the last decade, we have witnessed the growth of various Ops roles – marketing ops people, product ops people, etc. And I think the reason is that with the availability of so much data. Managers require operational hyperpower to synthesize this data and deliver insights from it to drive business. Can you share a bit of how you see this operational environment in your customer's organizations?
Regardless of vertical or industry, ops teams are all solving similar issues. If you have that ops word appended to your title, you're responsible for standing up a process, running that system, and building intuition around where things get stuck.
And so that's what our tool solves well, at least when I see people use our tool. We don't sell specifically to sales ops teams, and we don't sell to Salesforce admins, but a lot of companies who set up Snowflake and load Salesforce data into Snowflake will use Avenue to monitor. We target and market to them in a more narrow go-to-market. But at the company stage we've been at, we've just expanded into those types of use cases as companies have used us for it.
And that's taking me to another question you mentioned. This case arose when the company set up Snowflake – they needed to figure out what was happening with their operating systems. But how well do you think this ideal customer should be familiar with SQL, for example?
Avenue is built for operators, but the most effective teams have at least one teammate who is familiar with SQL. We’ve worked to make the tool user-friendly for non-SQL users. We have integrations with DBT and Metabase, so you can import your data views directly from your existing BI tools. We have an AI chatbot to help with SQL creation and a visual query editor. Our GPT-powered chatbot, Copilot, can help you write and edit SQL in Avenue. The visual query editor allows users who don’t know SQL to build monitors. We’ve also got a superb customer success team who helps customers with implementation.
How to balance the efficiency of alerts and frequency?
If you're setting up a monitor, we have two types. One is notification only. I want a pulse about this metric or to see if this is the expected rate. And then there's another one that creates incidents and requires a response. When I see this, I need to coordinate with another team. I should escalate it to another team and send an email to an external vendor. That is the playbook that you can build out. Our most active customers have upwards of 400 monitors and over 100 teammates in the tool. Avenue has become the internal QA system for many of our customers. For others, it’s used purely for edge cases. Fintech companies, in particular, benefit from Avenue notifications when monitoring for Fraud. One customer, a neo-bank, has recouped over 3 million in fraud with Avenue alerts.
It reflects the evolution of Avenue product offerings that we have because when I signed up a year or so ago, I was activated with the ability to push the alerts from Data Warehouse to Slack, and that's what I needed. I need to write SQL, and I need to set up cron and publish the Slack channel. And what I see now is that you guys have nailed this problem of what is behind alerting. Can you speak about evolution?
We had an idea from my time at Uber that ops teams build systems like engineering teams do. Still, engineering teams can speak to the reliability and uptime of their system in a way that ops seem can't.
Back to the Uber example: How are we reacting if a restaurant has had five canceled orders in a row? They would be like, we check this dashboard, and then we reach out to this team, there's a playbook in a Google Doc.
It's another Confluence page that exists in the ether. So that was a concept in our mind, like Datadog/PagerDuty for ops teams. But the entry point was around alerting because we thought that would give us the most sense of, hey, is something important enough to grab your attention? And kind of the root of the problem before Playbooking and how do we solve it is what are the types of things that we want to make sure that we see when it comes up? We spoke to many companies, and their resolution for identifying problems was to have weekly business reviews or daily meetings around this dashboard or even a company like Flexport. They described this process as dashboard sweeps. They had a formal dashboard sweep process where they would scan a bunch of dashboards, and then they had a messy escalation system for when we see that there's this issue, who do we report it to? So that was how we had the initial problem, which was still around what we're building.
Assuming the data warehouse provides a new data layer and we will be an application layer on top of that shared database, what types of alerts would you want to see? And then we followed the thread of companies saying, okay, I know the alert comes through, but I want to ensure it's auto-assigned to Danny. And if Danny doesn't respond, it escalates to me. So now we'll build escalation paths. We'll build assignments. We'll build other functionality. That's how our product had snowballed forward, and now, on sales calls, we're much more opinionated.
There are pillars to this problem. The first is just running the business and seeing where things fall through the cracks. Once you have enough alerts, you will be like, Hey, I don't just care that I get alerts about it. I care that we solve the problem when it comes up. And then, in the third stage, you have a system to flag my team. I have a system that my team closes the loop and resolves it I care about. How long are they taking? Who's refixing? How frequently is this coming up? What is my capacity on my operations team?
The support team has Intercom, Zendesk, where they get a unit of work, it can get triaged and resolved, and you have a sense of capacity. How many support tickets get solved, how long does it take, knowledge base, et cetera? Engineers have Linear and Jira where they can get a unit of. We close the loop.
Planned work as a project or a bug can be flown in, and they can triage it. It can get assigned. You have a sense of capacity and performance. What is the engineering team capable of doing? But your operations team needs a system of record for issues, how they triage them, and how it gets resolved. So that's that third pillar of audit trail time to resolution.
I like your point about this Flexport dashboard sweeps experience. The reason Avenue exists today is the false promises of the BI system because the tool promises actions. Still, they only added some heavy human layer labor. BI systems need more narrative. And your North Star brings a narrative to this static data.
Yeah. It's like activating it. We've always played with what that word is. Sometimes, we say operationalize because it's like ops teams, but it is like activating it and making it useful like you make this investment. Consolidate all of your data in one place, but if you want to take action or do something with it.
Traditional BI tools help point out retroactively that your team missed their SLAs. At the end of the week's business review, the metric is red, and the script tends to go, "The metric is red; here's what we plan to do next week to hopefully move that metric from red to green". With Avenue, you're able to detect drops before they impact metrics.
Also, if we've been drawing this parallel between the engineering and operational teams, data contracts have been a hot topic in engineering communities. How can that be used in the operational ward? Do we need some data contracts?
Yeah, we work with a mortgage service provider, and how they think about delinquency or escrow or things going stale or the service they're providing their quality control system, will be distinct from similar providers.
And that's the promise that they do the same for all these fast delivery or grocery companies. They think about metrics like promise time (SLA’s). How quickly are we getting to the customer? They chop up their system in terms of picking and packing and handoff and delivery.
So those are similar, but the way that data is loaded, the thing they're optimizing, the piece they care about will be distinct. So it is part of onboarding Avenue, or part of teams that find success using Avenue that they need to see it as cementing their system in place. It's like turning on the faucet, and we'll tune it. You have to start tracking things to improve the ideas we will improve. And that's a little about the playing of, right now, those teams don't have explicit SLAs.
Avenue, the first time you can say, how long does it take us if a restaurant cancels to turn it off the platform?
What's your suggestion to, like, the operation team in 2023? For startups that are thinking of setting up their ops team or for some mature organization that wants to reconfigure somehow what they have today. What's your main advice on how to make it a modern operational stack?
Yeah. In 2021, everything was headcount oriented. We saw problems; we would grow the team to identify them. Companies whose bottom lines were dependent on delivering within a matter of minutes (whether that’s groceries, car rides, medication) enlisted “live ops” teams to monitor their real-time marketplace. The thought was – to throw more people at the problem. Refresh dashboards. Don’t let any balls get dropped. That’ll solve the problem.
We used to sell Avenue like being a sidekick or another team member who could help support it. Now that people are more, there's more scrutiny on the teams, what they're doing, what value they're driving, and their performance. Both optimistic and negative, too. On some operations teams, they like to do a lot of the work, but they sometimes don't get recognition for it, and like my recommendation, you need it the same way.
But, like your support team, you can take a step back and consider how many tickets come up. Who's resolved and identified them, and how long do they take? Do we need to improve our knowledge base? You can make incremental improvements and optimize their work because you have a baseline. Same with the engineering team. Do we have projects? What is the scope? How many? What's our capacity? What's our team capable of? What does next year's product roadmap look like regarding points and scope, and do we have enough engineers to do that? You can do all that planning and backward-looking like performance understanding but like operations teams without a tool like Avenue.
You need a baseline to start with. How many issues have they looked at? How long is it taking? Who's solving them? Some customers use the metrics out of Avenue to hand off to the product team and say, Hey, our ops team is supporting underwriting. And right now, we do, we solve, this comes up 20 times a week. It takes us 10, two hours to solve each time. So it's 40 ops hours of work. Should we prioritize this on the product roadmap and build it into the product? Or is it something that Ops should keep holding onto? And to close the loop there, it was tough as a product manager at Uber to know if an ops was complaining about a problem. Like just complaining about the deal with this frustrating restaurant, can you build it into the product and be like, is that valuable? Is that a real problem in Uber's operations, or are you in the Netherlands with one restaurant partner? That's an annoying problem.
And so Avenue gets, allows you to quantify a lot of the types of problems that come up in a way where this receipt error happened many times over this month. And it's like this—investment to solve it.
How do you view your close competitors?
Our biggest competitors right now are internal tooling teams. We have edges with companies like Zapier and Retool.