Success criteria > Requirements
"Everything Starts Out Looking Like a Toy" #102
Hi, I’m Greg 👋! I’ve written more than 100 essays on product, data, and process where I noodle on an idea or a future product direction, often data that exists between systems.
We don’t talk enough about system “handshakes”, the expectations for workflow, and the jobs we expect data to do. Read more: What is Data Operations?
This week’s toy: an AR view that resets your room.
With this, you could map a room and try on new versions of it. The possibilities are … endless. Edition 102 of this newsletter is here - it’s July 18, 2022.
The Big Idea
A short long-form essay about data things
⚙️ Success criteria > Requirements
Product Requirements documents (see this one) represent a yellow brick road outcome for the product development process. Read one in isolation and it looks like a blueprint for success. Understanding the true evolution of a PRD is a bit trickier when you encounter the “messy middle” of product development.
Here’s how it might work in practice. At the beginning of the process, things look rosy. You define the feature, describe it with the optimistic prose of what it might look like, and think deeply about the problem you’re trying to solve. While you consider the person’s motivations, you think about the challenge you’re overcoming, any assumptions you’re making, things you call out as impossible, and the acceptance criteria required by the QA team to call this feature done.
There’s a problem with designing a feature before you know the real constraints. Despite your best estimate, you don’t really know how long it will take to get to your ideal state. When you start documenting the work for a feature and you haven’t sized it yet with your technical team, it could be small, or quite large. XKCD describes the product manager’s quandary well:
Improving the Odds to Success
What’s a product manager to do when wanting to tilt the scales and get to a better outcome more effectively?
Think about these two things for a moment. They are both similar, yet one is bounded. In software, unlimited things are bad. Setting up boundaries is a success criteria for avoiding unlimited things (how meta).
Requirements: Describing what is required to deliver a software outcome
Success criteria: Specifying the critical path to take to achieve success
In other words, the requirements that make a perfect feature might conflict with other unknown or known conditions in the environment that will prevent the requirements from being fulfilled. Success criteria, on the other hand, are a binary condition. You either succeed or you don’t.
Building solid success criteria
Product requirements are necessary but not sufficient for defining success. Defining effective success criteria starts with a test:
Is it a true/false test? If yes, you have the potential for fulfilling the success criteria. If it’s not true/false, write it again until it becomes a true-false question.
If other people on your team were looking at this item and it was completed, would they consider it successful as well? This is a test to check for alignment in declaring victory. If you’re the only one who thinks this is a success, you might be alone on an island.
Be conservative. In this case, it’s more important to be able to declare success incrementally than it is to go for a “big bang” and fail. Unless the team disagrees, as in step 2. Getting alignment on the granularity of success is critical to making progress.
Success Criteria > Requirements
There are a few reasons that success criteria are superior to requirements. They have some of the same structure and are more flexible.
We expect to change success criteria when we encounter a roadblock. That doesn’t mean that we throw away our original requirement. Instead, it means we examine the environment to find a new way to get to the goal. Since the true path to a feature or a product is rarely linear or straightforward, taking a mindset of incremental improvement really helps.
Success criteria can be understood easily by anyone on the team. Requirements can be opaque, especially when describing new things. Delivering a true-false condition any member of the Quality Assurance team can test or any member of the executive team is able to validate goes a long way toward describing when you are done.
When you focus on writing success criteria, you also commit to updating the success criteria when the conditions of the feature or product change. Decisions often get made in a vacuum and the success criteria for a feature need to change → see last week’s essay on a remote decision log for a few tips on improving that process.
What’s the takeaway? Spend more time than you do today on establishing success criteria for your features during the design and development process. As things change, you’ll need to update success criteria too, so build in the expectation of change as a success criteria for your success criteria 😃.
Links for Reading and Sharing
These are links that caught my 👀
1/ Saas Pricing Examples - It’s not as easy as you might think to set up a pricing page. There are different features, plans, and strategies to getting someone not only to sign up and commit to a particular plan. Vitaly Friedman lays out these details in Designing a Perfect Pricing Page UX, and shares some clever ideas to progressively display information, highlight key features (but not all of them) and generally improve that grid with three options you probably have on your website.
2/ Defining “non-technical” - Most people in tech companies draw a line between folks who have “technical” skills and those who deal in non-technical items. I’ve never liked this divide because I’ve found that many talented people have skills that fall along a technical continuum rather than some clear binary of “technical skills.” This is a good summary of how to assess technical skills and how to get better.
3/ Should your line charts ever be vertical? - I’ve thought a lot about graphs, but haven’t considered the challenge of whether a line chart should be vertical. Most of the ones you see are horizontal. Nick Desbarats makes a compelling case that some charts should be vertical in Are Vertical Line Charts Ever a Good Idea?, especially this one:
What to do next
Hit reply if you’ve got links to share, data stories, or want to say hello.
I’m grateful you read this far. Thank you. If you found this useful, consider sharing with a friend.
Want more essays? Read on Data Operations or other writings at gregmeyer.com.
The next big thing always starts out being dismissed as a “toy.” - Chris Dixon