A Good Process is a Competitive Advantage

Earlier this year, I joined a struggling specialist finance startup. Their software is essentially a 3-way marketplace and was put together by a London-based contracting company over the past 9 years. While product-market fit has been achieved, the company is experiencing significant growing pains because of the poor design of the software and a leadership group that periodically unloads their anxiety and stress on to those around them.

The crown jewel of the software is the calculation engine which governs how the marketplace works. But, despite its critical importance to the business, it is inaccurate (in some cases wildly so), risky to change, fragile, has no tests, and is a major source of frustration for all parties on the marketplace, the business' owners, the sales team, the customer success team, and the contractors. I have no doubt in my mind that revenue has been lost because of it.

Despite it being top of my list of things to fix, I've had to carefully navigate a situation with too many contractual obligations to meet that were undersold by an organisation that had, up to this point, been too sales-led.

Recently, however, a major feature was built that surfaced numerous issues with the calculation engine, and these needed to be resolved in order to satisfy one of those contracts. Given the risks associated with making these changes were largely of the contractors' own doing, I decided it was right to make the changes myself. I'm also of the opinion that core working knowledge of a critical component should be in-house, as that lack of internal knowledge is itself a business risk, so the situation represented an opportunity to do something about it.

But the founder of the contracting company contacted me to warn me of the difficulties I'd face. He told me that, historically, he and his team had run into these problems:

1. The requirements for calculation changes were never agreed upon upfront
2. It was difficult to find information relating to previous changes, as the context was spread across tickets, Slack, email, and spreadsheets
3. Getting testing time from stakeholders was difficult as their time is unpredictable from week-to-week
4. No stakeholder had ever taken ownership of said changes, and that getting sign off was always difficult
5. Releasing calculation changes had always been fraught with problems, both in terms of the quality of the changes and the strain on the relationship between the two companies

His stance was that these problems had persisted for 9 years which was proof enough that reasonable improvements to the process were unlikely. I disagreed.

At this point, I'd been at the company for 6 months. While there is a Head of Technology, who on paper is my boss, the startup's internal issues almost entirely consume his time, and so I've taken control of the contractors, and I now largely control the technical direction of the company too. Being so hands-on has allowed me to quickly build up context of past problems, including having some awareness of what previous calculation changes were made, and how the decisions to change them came to be.

One thing that immediately stood out to me, following the conversation with the founder of the contracting company, was his insistence that changes should be fully agreed upon upfront. The calculation engine is complex. There are numerous finance types all tangled together, which means there is context buried in the code that stakeholders aren't privy to. The complexity makes it unreasonable to expect that changes could be described in full on a ticket without some back-and-forth.

The second thing I noticed is the release process was causing havoc with all of this. It was overly complex, slow, error-prone, releases would happen out of hours, and when I joined we had more test environments than engineers (yes, really; I don't understand it either). The process was rigid and led to infrequent releases with lots of work bundled together. What no one had recognised was the tension between releasing major new features to meet deadlines and the time it takes to make calculation changes.

When calculation changes were ongoing alongside other work, these two outcomes kept occurring:

1. A feature would be released, to meet a deadline, at the expense of incomplete calculation changes or rushed testing
2. A feature would be delayed so that calculation changes could be finished

These would be followed by rounds of finger-pointing, hotfixes, or both.

Armed with the information above, I spent 30 minutes drafting a document with a process similar to the following:

1. Calculation changes are requested by the business and prioritised by the product/tech team.
2. Ideally, changes should be captured upfront. But it's more reasonable to suggest that in-person conversations, and those over Slack, will capture most of the context and need to be distilled later on prior to sign off.
3. The tech team makes the changes and pushes them to a preview environment. This allows the tech team to continue to release other work unimpeded, which prevents testing from being rushed and allows stakeholders to test when it's convenient for them to do so.
4. All calculation changes should be written down in this document.
5. At least one stakeholder must test those changes and acknowledge they have done so.
6. The tech team may ask for an additional round of testing if bringing the preview work into the main codebase is deemed risky enough.
7. The business signs the changes off in this document.
8. Once signed off, the tech team works to release the changes.

If you've been following along, then you'll have an important question: if the release process was so awful, how could changes be deployed to a preview environment? The answer is that I'd been asking the team to streamline the pipeline, which had gradually happened over the previous 3 months. Is it perfect? Far from it. But we are now able to release during the day, and applying a label to a pull request deploys that branch to a preview environment.

We've now followed this process for several rounds of changes, and it's working well enough that I've been tasked with addressing more of the longstanding calculation issues.

Hopefully I've given enough information to convey the problem and the solution well enough. The important thing for me though is I can't help but wonder how much business has been lost from almost a decade of handling things inadequately.

Good processes don't magically create themselves, and so the default position is to not have them. Given that, it makes sense to me that having a good process is a competitive advantage.