Today I’d like to share with you some perspectives on how software is made. Specifically I’d like to share with you my view on one of the better ways to think about the roles and responsibilities regarding the members on a software team with in terms of their role as generalists and specialists.
Before we begin I would like to remind everyone of the major activity categories of a software development initiative. First we began with a vision which is what we imagine the world should look like once the solution is built and deployed. The vision is the center of the universe in software development as it guides all other activities.
Following visioning activities come design activities which may also include the creation of prototype solutions in order to better the vet specifics of certain design ideas. Following design of course we move into development phases, testing phases, deployment phases and then the operations and maintenance phases of the software solution.
While this may sound like I am describing a waterfall type methodology, in actuality what I am attempting to depict is particular categories of activities that are necessary to create a software system. Regardless of the planning, scheduling or project management methodology used, the fact is that all this work must be done in some form in order to increase the likelihood that a successful solution will be created and delivered to the customer.
In consideration of the different skills, knowledge and experience required for each of the different activities of a software development project one could presume that different skill sets would be required in order to complete each of these activities in the best possible manner.
I’d like to introduce my descriptions of two terms that are commonly used in many subject areas and domains. First, the generalist. The generalist is someone who has worked in many different roles on various software projects. The generalist has experiences with different methodologies, tools and technologies, business domains as well as problem and solution domains. To me the generalist is an experienced practitioner with 7 or more years of hands on experience in 3 or more specialty areas. For example, a well-rounded generalist may have worked as a project manager, a business analyst, a systems engineer, an application developer or an enterprise architect at different points in his or her career.
The experiences of different contexts, vocabulary, tools and methods enable the generalist to have a broad perspective and a repository of knowledge and experiences that are best employed in particular strategic software project activities.
Next I would like to describe the specialist. In the context of software projects the specialist has deep knowledge and skills in a particular technical domain and/or toolset. For example a software engineer who specializes in front end web application development is deeply experienced in tools and technologies such as HTML, CSS, JavaScript and perhaps an assortment of popular frameworks such as angular or react. Another type of specialist is a database administrator or DBA. The DBA knows about how to install, configure, tune and troubleshoot database systems like MySQL, Oracle or SQL Server.
Key reasons to draw distinctions between the generalist and the specialist is because the thinking styles associated with each role are applied in optimal ways to particular activities. These thinking styles are primarily influenced by the direct experiences had by the individuals in their career so far.
For example of how these thinking styles are applied consider in the visioning phase of a project which ordinarily is at earlier phases. The generalists have their broader experience to offer to the designers of the system in order to help the design team produce design artifacts that take into consideration technical and structural factors that will be necessary to leverage in order to increase the likelihood that the design can be realized as close to the specification as possible. In order words, the generalist can help keep the design technically achievable.
In these early phases of a software project, it is important to suspend technology or solution favoritism in order for solutions that are a better overall and strategic fit and make optimal use of resources are offered to the system designers. The primary reason that generalists are better suited to this role is that with a broader experience they are less likely to have solution bias that would affect their contributions to the design phase. For software projects in many cases the generalist in this role is called solution architect.
Once the visioning and initial design work is beginning to take shape key decisions will need to be made about technology tools, non-functional and operational requirements. Many of these decisions take input from traditional organizational standards and/or strategic direction from an enterprise architecture team. In some agile methodologies, specifically scrum, these activities can occur in an architecture sprint or sometimes called a sprint called “sprint zero”.
During the architecture sprint (or “sprint zero”) the specialists begin their critical role. The role of the specialist during the architecture sprint is to sanity check the decisions that the solution architects are proposing for the software design. For example, the solution architect may believe that using the standard library functions from C++ are a good solution choice based on their understanding of the vision and the requirements thus far. Unless the solution architect has a lengthy history of C++ development experience, which may or may not be the case, it is important to vet solution recommendations with specialists in the area to ensure that any assumptions made are either reinforced or challenged with other alternatives by the specialist.
One of the key things to remember with any project is that there are almost always time constraints, budget constraints and functionality requirements that end up as opposing forces. A primary responsibility of the solution architect is to ensure that solution options are chosen that take all three of these key project objectives into consideration at all times.
While a specialist in a particular tool or technology is likely to have greater knowledge and skill with their specific tools, in most cases their opinions on broader topics will tend to be colored by their preference and familiarity with those tools. This brings me to the key point in drawing a distinction between specialists and generalists on a software project. If specialists are utilized in solution architect rolls there is a high risk that solutions will be recommended that may not be the best choices for long term lifecycle management of the solution. Additionally, system integration choices that supply overall system constraints may not have opportunity for consideration.
A recent example of an experience where a specialist made a recommendation was a project I was working on where we had and extract, transfer and load (ETL) solution to design and develop. Resources on the project were primarily web developers who used PHP primarily in their work. The specialist resources were empowered to select the technology for the ETL software component. Interestingly enough and without surprise they chose PHP.
While it may be technically possible to create working software in almost any modern language the question becomes this: is this the best possible solution for this kind of problem given our requirements and constraints? The unfortunate outcome of the selection of an improper tool resulted in wasted effort and inability to meet a delivery promised to the customer. The work that had been completed in the improper tool was scrapped. In an environment where resources were limited, which is pretty much every environment, this decision by a specialist resulted in waste that could have been avoided.
In looking over activities and roles in a software team one could surmise that there are strategic activities and tactical activities. The strategic activities are best completed by resources with more experience and less of a bias with regard to tools, technologies and methods. Tactical activities including specific expert advisement as well as the creation and development of software objects in conformance with the design and specification are better completed by specialists who have experience with the tools. This places the individuals with the proper thinking styles in points of highest potential contribution to the project.
While the examples discussed in this article focus on the technical aspects of software solution design and development the roles of specialist and generalist apply universally to other domains. One example would be the difference between what a chief financial officer is concerned with versus what an accountant is concerned with. While there is certainly overlap in domain and particular activities, the perspectives necessarily differ based on the demands of the activities that must be completed and the knowledge, skills and information necessary to complete them.
While much of this information I’ve shared today seems like common sense, I have found in my experiences that it seems to be forgotten more frequently then I would expect. I hope this perspective is helpful to you in planning and executing software projects in your organization in the future.