Engineering Management - Process
This is the second part of a series of notes regarding my thoughts during my time in engineering management at Facebook. Read the intro here.
Let process be implemented by those who practice it
Here, I use process to describe both process in the sense of "the steps an individual or group follows in order to perform a function" as well as general systems of organization.
As a company scales, it becomes necessary to add or codify process. The key tradeoff under consideration is always whether the drag imposed by the process is worth the gains in efficiency or effectiveness.
It is difficult to assess this tradeoff since so many factors are involved, so here is one principle which may be orthogonally helpful: only allow processes to be implemented which are specifically desired and put into place by those who will be directly involved in using it. Often, managers and executives suggest process because it will help them better command, control, coordinate, or communicate. New process should never be implemented in the service of these goals, because its benefit is illusory and often highly overestimated: managers can perceive its benefits to them, but because they are at least one (or more) levels removed from the direct operation, they do not perceive its true costs.
On the other hand, individuals doing hands-on work (e.g. engineers) can easily recognize when it is appropriate to add elements of organization or process, as they have a more direct ability to see how the benefits would outweigh the costs. That is the only time when a new system of organization or process should be added.
Managers will need to suppress their natural fear that things are too chaotic or out of control due to the lack of visibility into details. They should focus on leading by setting accurate and informative context and goals, facillitating the natural organization that results from collaboration between technology workers (who are often not the type prone to falling into anarchy anyhow).
A process created and implemented by the individual contributors themselves is much more tuned to optimize the real work being done. A process designed by managers at best approximates the actual flow of work that it needs to govern, optimize, or codify. This is the source of many clueless and inefficient processes.
People feel greater ownership over process they create themselves, and are thus more empowered to adjust it in the future as circumstances inevitably evolve rather than allow it to calcify. Process imposed externally ("from above") is more difficult to break down and tends to be unnecessarily enshrined, thus producing extra organizational inertia.
On coordination and communication:
What about coordination and communication? Don't managers need to do that? The answer is yes; it's one of their vital functions. However, process created to serve this should be process practiced amongst managers, and not imposed on individual contributors.
For example, the broadcasting of team status and project updates is a useful function to help inform others in the department or company of what's going on. This is a process that can be done entirely by managers, and does not need the direct participation of individuals on their teams, i.e. managers can compose these updates and circulate them but should not be recursively requiring them of their team members, since the manager can (and should) be aware of team operations as a result of working with them.
The key is that here, we have a process that benefits managers wherein the cost is also borne by them, thus allowing them to optimize the process for their own mutual benefits. It doesn't (and shouldn't) need to involve anyone else.
Regarding subtle dangers of descriptive process:
Prescriptive process says "Here are the steps you must take in order to do X." Descriptive process says, "Let's write down the steps we're taking today when we do X." Descriptive process is, at first, a common and harmless-seeming suggestion but leads inexorably to the same unnecessary process requirements and calcified systems of organization, and therefore should be avoided with the same vigor:
First, the harmless suggestion to write down "the steps we go through to do X" is taken up (where X might be "launch a product" or "take an idea from conception to delivery" or "request a new desk"), and the process is described in a document and posted onto (perhaps) an intranet page.
Then one day, a new person asks, "How do we do X?" In reply, instead of an old-timer explaining to them informally how they individually might go about doing X, they are referred to the document. After all, it's easier than explaining verbally. In a fast-growing organization, new people join rapidly; soon, this new person becomes an old-timer. Another new person happens to ask them, and they in turn are referred to the document. This cycle repeats merely a couple of times, and the document itself becomes viewed as the authority on how something is done.
The original ad-hoc process (which was adaptable, organic, and worked well because its informality allowed people to perceive it as flexible) has been replaced in the minds of most of the current people (newer people always outnumber old-timers in a hyper-growth company) by the document, which is now viewed as a overly-precise specification of Steps You Take to Do X. Subtly, operating flexibility is reduced and through the fault of no one, descriptive process has become prescriptive. This is sometimes worse, because deliberately prescriptive process usually exists under the aegis of a specific person with authority, who can be personally appealed to. A calcified originally-just-descriptive process exists under the aegis of an independent document, to which people acribe an ineffable authority akin to law, i.e. one greater than any one individual and thus even more difficult to overturn or innovate against when the need arises.
At various times I've been involved in the due diligence process of potential acquisition targets. One of the most surprising findings was that in many cases, a company that was smaller than us had more formal processes, and this was cited by its own people as a major factor as to why they moved more slowly than we did, executed on fewer ideas, and ultimately ended up in a weaker market position than we were (which was sometimes why we were considering the acqusition).
At Facebook, there was a cultural resistance to process, to the point where the pattern around introducing process typically went "new process is reluctantly introduced only right before the point where things tip into chaos." Push this point as far as humanly possible, and then some, because what you receive in return is high organizational speed. If your organization has less process than another one of equivalent size, you will innovate and execute faster, taking ideas from conception to market more rapidly. Managers may need to psychologically contend with more chaos than they are comfortable with, but there is a huge difference between chaos that makes one uncomfortable and chaos that actually threatens the business. Stepping as close to the latter as possible confers one of the greatest advantages in the technology business: execution speed.
Process typically builds up at a regular and roughly constant rate. Shaping this rate is therefore key to long-term efficiency. If your company has a certain amount of process at size X and it's less than other companies of size X, you're faster, and when you're much much larger you'll have less comparative bureaucracy, and the same multipliers will apply: doing things twice as fast now while you're small helps you get things done in two weeks while your competitor needs four weeks, but once you're large you'll be able to do something in two years while your competitor takes another two to catch up. Two additional years might just mean the end of them.