sommai - Fotolia
There's a meme floating around the DevOps landscape. It's called Conway's Law, published by Melvin Conway in 1968. It goes like this: "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure."
In other words, a big, bloated and rigid organization will design big, bloated and rigid software.
History has shown Conway to be right. That's the bad news.
The good news is we've learned from our mistakes. Today, we understand that decentralization is the key to designing a better system. Companies made up of small, self-directed teams are more productive than large, bureaucratic and command-based organizations. A new corporate communication organizational structure allows for greater collaboration and exposure for good ideas, and it ultimately enables DevOps.
Makes sense. It's the Agile and DevOps way of life.
But when I do the math, something weird sticks out. Granted, small teams work best. But if you look at the major players in DevOps and software development -- Google, Microsoft, Oracle, Amazon and Facebook -- you find organizations with significant headcounts. The teams might be smaller, but these companies still have a lot of employees.
What gives? How do these companies do DevOps? You still have managers, you still have contributors, and you still have office politics and departmental agendas.
So, what's different now?
In order to answer this question, I need to share a lesson I learned from a preschool teacher:
Any typical classroom problem can be broken down into three elements: the student, the teacher and the physical classroom. Correcting the problem is going to require changing one of those three elements. I have a choice: Change the teacher, change the student or change the physical classroom. Given my options, I am going to choose to change the physical classroom every time. It's the easiest thing to do.
In other words, the easiest way to change an organization is to change the physical infrastructure in which members act. Remember, Conway said an enterprise designs a system that's an exact copy of its corporate communication organizational structure. Changing an organization's communication structure changes the organization and changes the system the organization designs.
Because their physical communication structure supports and fosters decentralization, these big companies are able to decentralize and still maintain cohesiveness as a single organization.
The old way of communicating
Back around 1970, typical corporate communication organizational structures were opaque, bureaucratic and command-based. The physical mediums for the exchange of information were paper and voice. If a company had a product to release or a problem to solve, bosses at the top got together in a meeting to hash things out. Individual bosses talked one-on-one on the phone.
If your company had telephonic conferencing capability, bosses from different geographical locations shared a common phone call. Maybe some interoffice memos were exchanged. Bosses talked to bosses to determine the big picture. Then, they segmented work and distributed it to subordinate bosses for implementation. These subordinate bosses in turn delegated work to their subordinate bosses, or should the boss be a line manager, to contributors within his or her workgroup.
As authoritarian and edict-driven as the process seemed, it made sense. The communication structure in force at the time allowed few other options. Again, the primary instruments of communication were paper and voice.
A voice-based corporate communication organizational structure is synchronous. You have to be in the meeting to get the information available. Before voicemail, you had to answer the phone and be on a call. A communication structure based on paper allows for asynchronicity. Still, you have to be able to get to the paper in order to get the necessary information.
Remember, these were the days when system specifications lived in reams of paper organized in binders that sat on bookshelves. There was no such thing as full-text search. If you didn't know about the binder, or couldn't find it, you were sunk.
Typically, the group creating the information was the entity able to provide access to information. Hence, the dynamic of "manager as gatekeeper," as represented by the following:
Need schematics for component Y? Go talk to Marvin. He manages the group making the component. He'll be able to introduce you to someone who can help you out.
If Marvin didn't think you should have access to the schematic, you'd have your boss negotiate access to the information with Marvin. That's how the typical interface between subsystems worked: discovery and negotiation.
The world goes digital
Then, along comes magnetic storage, email and the internet. The world goes digital. The corporate communication organizational structure changes. Information distribution goes from being closed to open. Magnetic storage and standard documentation formats allow for text search and retrieval. Email allows conversations to become asynchronous, yet timely. The internet allows for easy, standardized network connectivity. Programming becomes distributed. The physical communication structure goes from being opaque and proprietary to open and standardized. No longer do you need Marvin to get the spec. Today, you do a search on GitHub or the company wiki.
This openness created a new dynamic: bottom-up contribution managed by an authority based on acceptance.
Remember, back in the days of voice- and paper-based communication, work was done by management decree. The bosses set the priority, determined the means of implementation and managed production. It had to be that way by virtue of the primitive physical communication structure. If Pam, a contributor way down in the org chart, had an idea for a better way to do things, the interface in play made it hard for that better idea to emerge.
First, for Pam to get started, she needed to have access to all the information required to formulate a sound idea. Then, after Pam created the idea, she had to convince her boss the idea was worth spending time and political capital to move it up the command chain. The whole process was laborious. The social and technical interfaces between groups were unique and hard to use. The communication structure of the time squashed many a good idea.
Today, all Pam needs to do in order to come up with a better idea is to download the project from GitHub -- private or public repos -- and figure out how it all works. If she gets jammed up, she can email the project contributors directly. Pam opens a feature branch, implements her idea, provides proof that her implementation is actually an improvement -- documentation and tests -- and then creates a pull request.
The project's maintainers review Pam's work in the pull request. If there's a problem, they comment. If all is well, the pull request is accepted and the code is merged into a deployment branch. In this scenario, the project maintainers are the authority. The maintainers express authority by way of acceptance. Nobody told Pam what to do. She figured it out for herself by doing her own discovery and using her creativity against the information gathered. The GitHub interface between Pam and the project is open, standardized and clear.
In terms of corporate communication organizational structure, GitHub is and was a game changer. GitHub is the change to the physical classroom described by the preschool teacher above. It changes the actors' behavior.
Is Agile still relevant?
The key to cracking Agile, even in the age of modern software development, is getting back to basics. Knowing what you want to get out of this transformation will make journey easier. Agile, after all, is about learning from mistakes for the sake of the customer.
The team is the product
When I first heard about Conway's Law, I was reminded of the book, Software for Your Head, written by Jim and Michele McCarthy. The basic premise of the book is team equals product.
Jim McCarthy wrote, "You can find every attribute of the product in the team and every attribute of the team in the product."
According to the McCarthys, a product is a direct reflection of the people making it. Again, Jim McCarthy wrote, "For example, if the software is slow, the team is loaded with bureaucracy and can't make decisions."
When I think about Conway and the McCarthys, I've come to understand that Conway did a good job of describing the relationship between organization structure and system design in terms of the constraints inherent in communication structures. But his paper doesn't go into human interaction dynamics present in a collaborative activity.
The McCarthys pick up where Conway left off. Just as GitHub is like a playground and provides a standard set of rules by which any interested party can collaborate with others to improve a product at the artifact level, the McCarthys provide a way for small, transient and self-directed teams to work together with greater efficiency in the cognitive-emotional domain. The McCarthys understand the impact of emotions on thinking. The result of their thinking is The Core Protocols. It's worth the read.
Human interaction and the new corporate communication organizational structure
I am a big fan of The Core Protocols, particularly in the light of the growing prevalence of automation in DevOps and the enterprise. More automation means less human interaction, and I can't help but wonder if the skills and sensitivities required for effective human collaboration will diminish due of lack of exercise.
Those of us writing DevOps automation code are accustomed to negotiating software, not people. The total control we experience writing an automation script doesn't map well onto the skills required for effective human interaction. Many would rather just hit the figurative mute button, rather than work to achieve full communication with another human being.
Just has GitHub provides a communication structure for the effective decentralization of physical software production within large organizations, The Core Protocols might very well be the foundation for the next step in effective human collaboration: efficiently combining thinking and feeling within a corporate communication organizational structure that's worth being copied.