So, you’re the newly minted leader of a group of software developers. Are things really going well or have you merely ‘assumed’ control?
Have You “Assumed” Control
So, you’re now the manager / director / vice president of engineering responsible for a software development team. Are you in control or do you merely assume so? How much do you know about your process, resources, etc.?
It’s 8PM, Do You Know Where Your Source Code Is?
Do you know where your source code is? Without source there is no product. No worries you say, “it’s all in source control.” Cool. Where? Who can access it? Is your IP kept separate from the open source and third party bits you use? Do you have appropriate access control if you have dual company relationships? How much can be accessed by non-developers / contractors? Have your development and IT department worked to establish appropriate backup behaviors to prevent you from capturing third-party source into your corporate backup system in a way that will not withstand an audit? There are mainstream third-party libraries which upon termination of use require removal of all copies. Failure get you an additional year of license payments.
What’s up Doc?
How long would it take to put together a top-to-bottom explanation of any given project in your organization portfolio? Not to have someone explain it all, but simply to arrange a soup-to-nuts set of documents arranged in some kind of self-guided tree that anyone could follow should everyone in the development group be waylaid by aliens. Follow-on, how much critical information is stored only as tribal knowledge, squirreled away in the brains of your staff? How many of your processes require human intervention from a special person to work at all? Does the documentation you do have match the process you use? Does each piece of documentation have an owner?
The Open Source of Our Disk Content
Do you use open source components? In this day and age, who doesn’t? How long would it take you to inventory your open source use on a per product release basis? Do you have an engineer who works with legal to ensure that the open source you use have licenses that harmonize? Do you have an approval process for using open source? Does you appropriately fulfill the licensing requirements? If you modify the source, do you return it to the author? Do you make the source available on your web site (on a per product / release basis)? On our site, why? In a word “abandonware.” Is this really necessary? Well, there’s open source poisoning. And let’s not forget FSF vs. Cisco. Are your developers using code they found on the web and not worrying because about it because there is no stated license? You do remember that that material is protected under copyright law under the Berne Convention (additional exposition here).
The Very (Threat) Model of a Modern Software General
Does your product ever have interact with anything else? Of course it does. Do you have a threat model for it? At every abstraction level? Do your developers know how to read and create a threat model? A good threat model will help drive your security position papers. Security should be a tangible thing and not simply a warm fuzzy. Make your security staff’s job easier. This also reduces the time required to respond to identified security threats in the wild.
Old Isn’t Gold
There’s a phrase that show up a lot in ads for stock services. You know the one. “Past performance is not a guarantee of future results.” This is absolutely true in software. Whether you’re referring to tools, processes or people, the way things have been done is in no way guaranteed to be best practice, or even supported, in the future. This mirage of stability and security may take the form of build systems without build masters, tool chains no longer available or supported, or any tools that new hires have never heard of. Software development is not a capitalized asset, but rather on ongoing expense. We expect that pens today will serve us well tomorrow, but this is definitely not the case with software tools. Strangely, many companies expect that people will want (and in fact demand) their new software products because of the added functionality. Why would the software tools used to product said software be any different. Additionally, there is the issue of support from the vendor and in some cases the very existence of vendor. It’s akin to acting offended when the doctor you had as a child retires.
Pay no Attention to the Man Behind the Curtain
Do you have projects which are the equivalent of the mystery meat you were served in school for lunch? You know the ones. Old projects or products kept around because there’s one really important customer who still uses it. The kind of projects that rely of AS400‘s or AIX cross-compilers to work. The ones where if one particular individual left the company, you’d end up having to pay them as a contractor just to do the build. This is the hardware / process analog to knowing where your software is. If you have these, they will require special attention and not in a good way.
See No Evil or Monkey See, Monkey Do?
If your software development organization doesn’t believe that there is any reason to review the full offering every two years or never looks outside the organization when setting future directions, you have a big problem. The superstars who brought the organization their last breakthrough are only as good as the last thing they learned. As was heard at GE, “Woe unto ye corporate superstars who have not this day performed a miracle, for you shall be known as bums.” If you never look outside, one day you will discover that you have not been surpassed, you have been rendered irrelevant.
Can’t Touch This
If you’re not organization won’t even consider touching sections of the code base because there might be testing impact (scheduling or resources) or the “you can only change the code if a failure has been identified via some test,” you have severe issues. There are advances in every aspect of computing every day. From the processors, to the compilers, debuggers, static and dynamic analysis tools, to the very algorithms, someone is working to make the software development process easier, faster, safer, and less fattening. To declare a piece or vast chunk of the code base untouchable says that you don’t understand it and are afraid that merely looking under the hood will loose all kinds of unspeakable horror; or that you hold the belief that the code in question is the realm of genius and mere mortals may not intrude.
Back to the Future
In any sufficiently complex system, there is a not insignificant amount of time between design and production. When instantiated in hardware, the lifetime is further extended. Issue of support will demand that less than state-of-the-art tools be expected and managed. If your product has a lifetime of ten years and you build it with tools, techniques and targets from ten years ago, where do you think you’ll be by the time you end-of-life the product? Where will the people who can work problems be? If you’re lucky your organization will have an Office of the CTO. For those of you who just did a head tilt, the mandate of the CTO’s office to keep abreast of futures. So even if you can’t use the bleeding edge tech today, you can be aligned for when you can. These are people you can go to with the “what if” questions. They provide options and evaluations. Let’s be real, you probably have your product developers working on existing products. You also probably have a very short design phase for new features and products. It’s amazing how many products have suffered from just getting something working and calling it good. Wouldn’t you rather have something better than the top result on Google driving your product design?
Is This the Way to the Vice-President’s Office?
In any healthy organization, you can expect that your developers will want to advance. Do you have a clear, straight-forward and consistently-applied technical career ladder? It should cover both the qualifications, duties and expectations of each rung. Is it up-to-date? Do the engineers understand it? Are there soft aspects? Is it actually used? This is a hearts and minds thing. It speaks to the relationship between staff and management. Some people will reach a particular level and be completely content. Others will want to run the ladder. Having a documented path, gives everyone a basis for conversation and a way to set goals.
On the Bench or On the Beach?
So the organization you’ve assumed has the people, skills and processes to handle all your current needs. You’re golden, right? Now ask yourself how deep your bench is. Do you have an inventory of your teams skills? Is it up-to-date? How fast can you assemble an inventory? Look at it.
Are you tracking the current software industry trends in terms of desired skills, programming languages and tools? Why? Look at your bench. See the rookie end? How do you think they select what they study in school? The smart ones are being guided into the high-demand skill areas by the time they’re in their second year of school. Where is all this drive coming from? The FANG companies. They’re the cool kids. They run at scale. They have the money to explore. They’re the companies you’re competing with for new graduates. Save money and see where they’ve been, what’s worked, what hasn’t and where they’re signalling the future is. Remember, you can tell the pioneers by the arrows in their backs.
Did you notice how the TIOBE and Stack Overflow surveys differ on programming languages? They’re using two different metrics. TIOBE sample commits to open source, Stack Overflow surveys active developers.
Now look at the middle of your bench. It’ll be the biggest section. They’re your bread and butter. Are their skills fresh? This speaks to the career ladder. If you’re using a ladder with out-of-date qualifications and expectations, where does that put you? Do you have mechanisms in place to keep your teams fresh date current?
The veteran end of your bench represent your heavy hitters. These individuals have depth or breadth of experience in domain or technology. If you’re extremely lucky you’ll have a few who have both. There may be a tendency to let them get a pass when it comes to their staying current. Don’t ever give them a pass. With great experience comes great expectations. It’s far too easy to simply trust the word of experience without making sure that they aren’t simply recycling knowledge they gained decades ago. Remember that your organization is competing in a global arena. These are the people who should be spending a large amount of their time exploring both the state-of-the-art and the competitions; and also digesting and disbursing that knowledge within the organization. If you’ve got insular vets setting direction, you’ve got a recipe for failure. Maybe not today, but rest assured, the piper always gets paid. Choice of processor, operating system, development language, communications protocol, user interface all come with cost not only today, but downstream. Finally, are your veterans creating information silos in order to ensure their position or do they make sure that knowledge is spread as far as possible within the organization? The former creates an environment of dependency, the latter spurs innovation. The rookies and mid-career developers must never believe that the views of the veterans are unassailable.
View to a Skill
We’ve explored the experience axis of the bench, but what about the skills axis?
If you speak with your development staff about a new technology / methodology / language and you hear, “that’s cool, but here we …,” it’s time to get those wagons out of the circle. This is sure sign that developers are either unwilling to work on their skills or have been beaten down by management in the past. The former is a one-on-one issue, the latter is a systemic problem.
Back to your skills inventory. Does your organization have the right skill sets to accomplish your mandate? If not, can you get there from here? If you can’t what will you do? If you can, how will you engage and implement? Do you have individuals with skills who aren’t being engaged where they could have a positive impact? It shouldn’t matter their tenure. Do you have team members with skills your teams need? How can you uplift the teams with their knowledge? What if they don’t have skill in the area of teaching others? Your organization should have an active mentoring program in place. If you can mentor one person, you can explain to someone who can train others. Your mentors can be brought into group teaching as domain experts. Your team members with teaching experience already know how to extract information from various sources. To them domain experts are a source capable of answering questions.
The Grass is Always Greener
What is the flow of the development staff? Do developers see sustaining and current projects as a dead end? This may be indicative of the shiny ball syndrome. If developers are climbing over one another to get to the greenfield project is it because there’s the perception that only the shiny and new get resources and the attention of management? Does every greenfield project manifest the same patterns of behavior as the sustaining and current only with newer tools? Are greenfield projects used to reward performers rather than being staffed based on background? When a developer moves to a new project, is sufficient knowledge transfer done to ensure no loss of continuity? This information is particularly indicative of the nature of the organization and is a way that management communicates the value and priority of various projects.
Princely Behavior
Is your organization a minefield of fiefdoms and politics? Allocate half your time to dealing with it. Few things will run your development teams into the ground faster than time lost for engineers trying to please two masters. Many people would like to believe that software development is a democratic process. That’s a very naive view. If that were the case, we’d vote on every decision and everyone would have equal weight to their opinion.
And, Action
Keep in mind the words of Brian Kernighan and P. J. Plauger in their book The Elements of Programming, “Write and test a big program in small pieces.” Make your plan with modest, attainable, measurable goals. It’s no different than creating a software project plan. As you proceed, beware individuals with an all-or-nothing attitude. No large undertaking will cover all the bases regardless of planning. Don’t let the desire for a mythical über-solution stand in the way of a good partial solution. Solve problems in the priority order of the problems, not the perceived importance of individuals or groups.
Homework
- Dark Ages II: When The Digital Data Die
- Working Effectively with Legacy Code
- Threat Modeling: Designing for Security
- The Prince
- Principles: Life and Work
- Failure is Not an Option
Image credit: Four Worlds. Creative Commons.
Charles Wilson's Blog 