Article text:

Taking baby steps helps us go faster.

Much has been written about this topic, but it comes up so often in pairing that I feel it’s worth repeating.

I’ll illustrate why with an example from a different domain: recording music. As an amateur guitar player, I attempt to make recorded music. Typically, what I do is throw together a skeleton for a song — the basic structure, the chord progressions, melody, and so on — using a single sequenced instrument, like a nice synth patch. That might take me an afternoon for a 5-minute piece of music.

Then I start working out guitar parts — if it’s going to be that style of arrangement — and begin recording them (musos usually call this “tracking”.)

Take a fiddly guitar solo, for example; a 16-bar solo might last 30 seconds at ~120 beats per minute. Easy, you might think to record it in one take. Well, not so much. I’m trying to get the best take possible because it’s metal and standards are high.

I might record the whole solo as one take, but it will take me several takes to get one I’m happy with. And even then, I might really like the performance on take #3 in the first 4 bars, and really like the last 4 bars of take #6, and be happy with the middle 8 from take #1. I can edit them together, it’s a doddle these days, to make one “super take” that’s a keeper.

Every take costs time: at least 30 seconds if I let my audio workstation software loop over those 16 bars writing a new take each time.

To get the takes I’m happy with, it cost me 6 x 30 seconds (3 minutes).

Now, imagine I recorded those takes in 4-bar sections. Each take would last 7.5 seconds. To get the first 4 bars so I’m happy with them, I would need 3 x 7.5 seconds (22.5 seconds). To get the last 4 bars, 6 x 7.5 seconds (45 seconds), and to get the middle 8, just 15 seconds.

So, recording it in 4 bar sections would cost me 1m 22.5 seconds.

Of course, there would be a bit of an overhead to doing smaller takes, but what I tend to find is that — overall — I get the performances I want sooner if I bite off smaller chunks.

A performance purist, of course, would insist that I record the whole thing in one take for every guitar part. And that’s essentially what playing live is. But playing live comes with its own overhead: rehearsal time. When I’m recording takes of guitar parts, I’m essentially also rehearsing them.

The line between rehearsal and performance has been blurred by modern digital recording technology. Having a multitrack studio in my home that I can spend as much time recording in as I want means that I don’t need to be rehearsed to within an inch of my life like we had to be back in the old days when studio time cost real money.

Indeed, the lines between composing, rehearsing, performing, and recording have been completely blurred. And this is much the same as in programming today.

Remember when compilers took ages? Some of us will even remember when compilers ran on big central computers, and you might have to wait 15–30 minutes to find out if your code was syntactically correct (let alone if it worked.)

Those bad old days go some way to explaining the need for much up-front effort in “getting it right”, and fuelled the artificial divide between “designing” and “coding” and “testing” that sadly persists in dev culture today.

The reality now is that I don’t have to go to some computer lab somewhere to book time on a central mainframe, any more than I have to go to a recording studio to book time with their sound engineer. I have unfettered access to the tools, and it costs me very little. So I can experiment. And that’s what programming (and recording music) essentially is, when all’s said and done: an experiment.

Everything we do is an experiment. And experiments can go wrong, so we may have to run them again. And again. And again. Until we get a result we’re happy with.

So biting off small chunks is vital if we’re to make an experimental approach — an iterative approach — work. Because bigger chunks mean longer cycles and longer cycles mean we either have to settle for less — okay, the first four bars aren’t that great, but it’s the least bad take of the 6 we had time for — or we have to spend more time to get enough iterations (movie directors call it “coverage”) to better ensure that we end up with enough of the good stuff.

This is why live performances generally don’t sound as polished as studio performances, and why software built in big chunks tends to take longer and/or not be as good.

In guitar, the more complex and challenging the music, the smaller the steps we should take. I could probably record a blues-rock number in much bigger takes because there’s less to get wrong. Likewise in software, the more there is that can go wrong, the better it is to take baby steps.

It’s a basic probability, really. Guessing a 4-digit number is an order of magnitude easier if we guess one digit at a time.

As much as I’d like to say this was done by a bot I made that pulls the article text, I (human) just manually copy/pasted it here.

As I close off this post, it is worth pointing out a weakness in the hostage taker’s position and motives. In all cases I’ve seen, the individual in question would prefer to maintain their position in the company — they don’t want to be let go, which is one of the reasons they are fighting so hard. That itself is leverage — it means the threat of quitting, which is their main bargaining chip, is a relatively hollow one. It also means a manager who is strong enough can slowly chip away at parts of the tyrant’s empire, giving small pieces to other teams, with no piece large enough to trigger a full-scale revolt. Finally, what is left ends up being so small that if the “brilliant ass-hole” decides to leave, the pain of the loss is minimized.

There are two things required for this to happen: a manager with a strong enough stomach to do what is necessary, and the willingness on the part of the company to take the risk. Either way there must be someone who cares more about the company’s long-term success than their own career, who is unwilling to let the problem become some future schmoe’s headache. And such people are few and far between.

¹ The real names and locations withheld.

Article text:

The phrase “no brilliant ass-holes” gets a lot of lip-service around software companies, but the degree to which it is truly believed is only revealed during pivotal moments when a difficult decision has to be made. If you’re lucky, you’ll have avoided such dilemmas for most of your career, but you are unlikely to avoid them forever. Eventually some high-paid troublemaker will be in a position of influence over your team or your company, and you will have to decide how to address the acrimony that they breed. The challenge is exacerbated when the person in question has control of a critical code-base or is an irreplaceable resource: in other words, when they are holding a “hostage”. In such situations, most people — and most companies — will chose to turn a blind eye; they’ll try to put off the inevitable confrontation, hoping it will resolve itself.

The goal of this post is to outline the consequences of not taking action, or of delaying it, by relating one such experience from my own career, at a time when I was leading a multi-platform software team at B2C company. Like with technical debt, the pain of the eventual confrontation was compounded the longer the it was put off, until in this case the entire company ended up being held hostage. My example — one of many — should serve as a warning against what happens when a company tries to appease a hostage-taker.

As a technical lead, my own philosophy has always been to spread code ownership around the team as much as possible, to whichever team members feel they can handle it — regardless of tenure or seniority. This is the approach I was using at the aforementioned B2C company as well, and it was proving effective on all but one of the platforms. In this latter case, a long-tenured developer — who we’ll call “Martin”¹ — had taken full ownership of all architecture decisions and development on the platform, and maintained that control with an iron grip.

When I arrived at the company, the rest of his team were only being given menial or tangential tasks. He never allowed any of them higher-level decision making responsibilities on critical aspects of the code, and this was hindering their technical growth. Martin had also instituted a number of unusual (read: sub-optimal) design decisions, which he continued to defend, and which made understanding the code difficult. Obscurity served as an effective defence mechanism — anyone who wanted more ownership of the code had to go through him. This meant he would know what was happening and either shut it down or find a way to redirect it.

I spotted this toxic pattern fairly quickly, and had hoped to address it, but there was a problem. Despite being an individual contributor and reporting to me, Martin had tremendous clout and influence throughout the company. Not only had he been there since its inception, he was also the only one who could work with the critical platform, and all feature and bug requests had to go through him. To his credit, he was fairly responsive. When bugs showed up, at any hour of the day, he would drop what he was doing and push a fix. This established his credibility and was likely one of the reasons he was given the amount of leeway he had.

During one-on-ones I heard from his co-workers that they, unsurprisingly, didn’t like working with Martin. One or two of the juniors respected his intelligence and influence, but all of them expressed a feeling of living under his shadow, since they were not being given responsibilities that matched their skills. I tried to convince Martin to spread ownership around the team. Though he seemed to agree in principle, he never carried through. He asserted that only he knew how to manage the code-base — which was true, given its inscrutability — and he consistently pushed back on any architectural improvements from other team members that would make the code more accessible.

I’ll admit I was naive at the time. As a young, idealist manager who believed in fully developing my reports’ skills (that’s why I had gone into management in the first place), I didn’t yet realize the political consequences of confronting toxic team members like Martin. The back-and-forth between us escalated over time until it turned into all-out war. HR had to be brought in. The head of my own department — my boss — didn’t want to believe that Martin had to be dealt with quickly, because he didn’t fancy dealing with the technical repercussions of getting on his bad side.

This fear, coupled with Martin’s willingness and ability to address code problems quickly, became the carrot and stick that shifted upper management against my own recommendations. Martin also played dirty. He began to undermine my reputation at every opportunity, complained, gossiped and even lied, and ultimately, through his influence on HR and the execs, he scuttled my career as a lead at that company. By coincidence, around the same time the company lost a large chunk of its business, and so I and the majority of the engineering department were let go anyways. Martin, of course, was retained.

Although I soon moved on to a better position, I continued to keep in touch with former coworkers at the company, including his new manager who was a long-time friend of mine, and so I managed to learn the coda to this story. Martin continued to exercise control over the platform, until eventually the rest of his platform team unilaterally quit, citing Martin’s oppressive tactics as the explicit cause. Now the company was in even more of a bind than before. They couldn’t let Martin go, since he was the only remaining developer on that critical platform. When they tried to hire new team members, Martin — who was expected to be included in the hiring process — sabotaged the interviews, demanding interviewees answer ridiculous questions, and rejecting them based on spurious criteria.

One candidate confronted him on the absurdity of his approach to code design during their interview. Since he had never been forced to accept feedback from anyone on his design choices, it is likely he genuinely believed that his approach was the right one, and so when candidates mentioned best practices that didn’t align with his “expert beginner” philosophies, he rejected them. Either way, the hostage situation was now complete — he had become a bus-factor of 1.

In the end, management’s attempt to appease Martin and get on his good side, driven by fear of him quitting, didn’t soften his hard-line stance — it only validated it. I had originally blamed myself for not finding a better solution to the conflict between him and his team. But now I realized that Martin never wanted to play nice. He was playing a game of power, and didn’t intend to lose or to forget what game he was playing. He cared more about winning than management cared about the long-term success of the engineering department.

Martin also suspected that he wasn’t going to be fired by the weaklings in the upper echelons, including the CEO, so he knew he had all the leverage. When the rest of the team quit, they lost their only opportunity to balance the scales. The more they waited, the harder the decision got, until it was nearly impossible to address, and they had to accept an uneasy truce working under the same roof as the tyrant. The last piece of news I heard about this situation was that the engineering team had been downsized even further, and Martin and his manager were the only two remaining engineers on the product. Martin had won.

I have since seen this same pattern play out in another company as well; it begins with fearful management, unwilling to confront a difficult developer who refuses to collaborate, who doesn’t want to lose power, and so maintains a stranglehold on a critical piece of infrastructure until it is too late. The execs end up having to concede to all his demands. Their short-sightedness mistakenly glorifies and provides justification for a philosophy of appeasement; their desire to procrastinate on a difficult confrontation until some unspecified later time (perhaps when it becomes some other manager’s problem), ends up poisoning the rest of the team who then leave for greener pastures, thus solidifying the hostage-taker’s position. When said hostage-taker amasses enough money and experience to leave — which they undoubtedly will — the company is left up a certain creek without a paddle. But why should they expect such a person to do differently, to show gratitude for all the concessions the company has made over the years, or sympathy for the bind they are leaving the company in?

My guess is, when the time comes for Martin to leave, he will start making increasingly more untenable demands from his superiors, and when the execs inevitably push back, he will use that as moral justification for quitting and leaving them high and dry. Not that the moral justification is even necessary— it would merely be a way of enhancing his own self-perception as the party on the side of right.

How do such hostage situations take root in companies? Why do managers and even execs embark on a strategy of appeasement, sacrificing long-term security for short-term ease? In my estimation, managers hesitate to address such problems because they know they will end up “heroically” putting their career on the line when it seems no one else will. Why help a company that is ready to throw them under the bus for making tough choices? It is a case of the tragedy of the commons. Even if some brave soul tried to address the hostage-taker and managed, kamikaze-like, to take them out, their own job and credibility would likely be ruined with it. So most prefer to juggle the hot potato until they can pass it to the next person, and make it their problem.

Here’s the link to the article.