Embrace the stacking workflow for better collaboration at scale 💥

In this post, I would like to write about my recent discovery on stacking workflow. I found it a game changer for my personal and corporate developer experience at scale. Unfortunately, it has not been popularized in our developer community yet. Hopefully, this post contribute to that mission. Stay tuned and read on guys 😊

Traditionally, we have been using the branching workflow for our development process. It is a great way to manage our codebase and collaborate with our team. However, it has some limitations when it comes to scaling our development process. Imagine we have a team of 10 developers working on the same codebase, you will end up with a lot of issues such as:

  • Merge conflicts
  • Rebasing frequently to keep up with the latest changes from the main / master branch
  • CI/CD pipeline is not stable as they are testing against stale codebase
  • Long waiting time / blocking the developer flow for the code review process across timezones
  • Difficulty in managing the release timeline
  • Rollback mechanism is not seamless
  • etc.

Traditional workflow

The above diagram is a good representation of the traditional workflow. As we can see, the more developers we have, the longer time it takes for the develop / preview / ship process (credit to Vercel for the terminology).

To overcome these limitations, we can embrace the stacking workflow which is a more efficient way to manage our codebase and collaborate with our team at scale.

Stacked workflow promotes the practice of breaking up large chunks of work into a series of logically smaller yet dependent PRs to ease the review process while keeping developers unblocked and productive.

Stacked workflow

As we can see, the stacked workflow is more efficient as it reduces the review time and the number of PRs that need to be reviewed. While waiting for the previous PRs to be reviewed / merged / deployed, the developer can continue to work on the next PR which reduces the overall time it takes to get the work done. Let's take a look at the following scenario:

Let's say we have server and UI code are located in the same repository. We need to implement a new feature that requires both new API endpoints and UI changes to the codebase. We can break it down into two logically chunks of work:

  • New API endpoints to the server
  • New UI changes to the frontend (this can be web, ios and android)

Initially, BE and FE developers can work on their respective PRs without being blocked by each other. Until the point of integration, where the BE's work needs to merge into the main branch, deployed and potentially hidden behind a feature flag, then the FE developer can continue to work on the next PR. We can avoid this blocking time entirely by stacking FE's changes on top of BE's changes. This way, the entire feature can be tested end-to-end before it gets deployed to production. It's a win-win situation for everyone. Better yet, we can do this process iteratively, as things requires more changes, let's say BE's update or refactoring, we can just stack another round of changes on top of it.

There are several tools that can help us achieve this workflow:

These tools are built by battled tested teams based on their own experience and best practices originated from Meta and Google.

My personal interpretation of the stacking workflow tools is that it helps us to manage all branches as a graph of PRs. We can manipulate the graph as we see fit. It gives us a ton of flexibility, yet it comes with great responsibility. We need to be disciplined and follow the best practices to make it work. Otherwise, we will end up with a mess and a lot of frustration. I picked up graphite as a learning tool to understand how it works and how it can be used in our daily workflow under this repository graphite-pro. I have tried to use it with git at first without graphite's powerful CLI to understand the concept and how it works under the hood. Then I tried mix and match both git and graphite to see how it works in practice. Eventually, I felt comfortable with it and stick to only using graphite, yet git knowledge is still important as it is necessary for troubleshooting when things go wrong.

Last but not least, even thought stacking workflow is a powerful technique that can help us collaborate with our team at scale, it does not complete the big picture. We still need a handful of practices / services to make it work at scale which I won't cover in this post. Yet, I would highly encourage you to give the following videos a watch:

Credibility:

  • Gergely Orosz for all the great diagrams of visualizing the workflow
  • Stacking.dev for the great introduction of stacking workflow

There you have it guys, I hope this article is useful and explain well how stacking workflow can help us collaborate with our team at scale. It might take sometimes to get used to it, yet it is a powerful technique that can help us streamline our development process and improve our developer experience at scale. Together, we make the world a better place through quality software 💞

❤️❤️❤️ Be well, Be happy ❤️❤️❤️