Author: [email protected]

Why Engineers Hate your “Boiler Plate” Job Descriptions?

Why Engineers Hate your “Boiler Plate” Job Descriptions?

How to Attract the most relevant applicants with great job postings

One of the main problems in SaaS/Software engineering hiring is the way job descriptions are written. While I knew this for some time (read years) The problem is, I was too lazy to change anything about it! That is until recently, one of the candidates I was interviewing for an Engineering Manager Role said this in our introductory call,

Me: Hope you’ve had a discussion with Ms.ABC (our HR) regarding the Roles and Responsibilities. If there are questions on it I can answer them, or we can get into the agenda.

Candidate: Yes I had a discussion with Ms. ABC. But, quite frankly it was your boilerplate JD. I’d actually want to understand what exactly I’d be doing. What will I be in charge of? What will I move?

Needless to say, I spent the next ~30 minutes walking through the current team structure, where he’d come in, what will he own, what the growth trajectory looks like etc. Ultimately, we did 2 more calls before both of us were satisfied that there are mutual synergies and went ahead. It made me reevaluate all of our Job Descriptions over the weekend and rewrote almost half of them to include factual details on projects, outcomes expected, tools available, glimpse of growth among other things.

After this, I asked the HR to send this “revised” JD to the candidates once again.

 And the result was visible from Monday!

Either candidates that the HR thought super suited started dropping voluntarily from the process or candidates started expressing interest, doing more research on our stack, infra, product proposition and competitor benchmarking, before the call. Some even did a cold reach-out on Linkedin.

So, I wanted to share the small titbit here.

Why General Descriptions Don’t work? 

Most Job descriptions barely resemble “specifications” at all, but feel more like generic stubs. Sort of like the equivalent of shopping for a car with as much details as “red and goes fast” or “black and built to last.”

 It leaves too much open to the imagination for it to be a successful criterion to enable fitment. With criteria as broad as this you’ll end up spending an inordinate amount of time executing the search, since so many things appear to be a match. For me, Red and goes fast is always a 1971 Ford Mustang, for you it could be a 1998 Ferrari 365.

The truth is that statements like the above — or its equivalent in engineering hiring — “Get me a backed dev with OOPS in Python/Java/Go with 5 years experience” — guarantee a similarly frustrating shopping experience. You’ve made it needlessly difficult for yourself and your HR/TA team to identify the specific talent you want. In this trite example you’ve indicated that you’re looking for a mid-level engineer that knows OOPS, but that basically includes everyone that ever graduated with a CS degree in the past 5 (to 10?) years. Surprisingly, many job “specifications” we see contain rarely any more info. These are the “Boiler Plate” Job Descriptions.

How did we find ourselves in this mess?

I understand why hiring managers do this. Sometimes they’re not exactly sure what they want — after all, it takes real time and effort to work out the specific vision for the role. But instead of acknowledging this and then solving the real problem (their own laziness), they delegate the JD writing to their Team/HR and it turns into generic tech JD. But the hiring manager is unfazed — “I’ll know the right candidate when I see them,” they say. Really? It could be true in some instances. Sometimes, we start with a Backend developer, then we come across a candidate with experience in building a full pipeline or a payment system. Then we expand the role to cover wider scope and evaluate against it. But generally,  How will they know the right candidate if they can’t write down specifically what the candidate looks like?

Another reason for generic job specs is because a hiring manager is recruiting in a talent-constrained market (sound familiar?), and it feels like a smart move to cast as wide of a net as possible. Theoretically, one should be able to get more candidates into the top of the funnel this way, right?

Perhaps. Theoretically. But in my experience, this approach usually, and utterly, backfires. Here’s why:

Boiler Plate Job Descriptionsaren’t designed to appeal to any engineer in particular

In the current job market, engineers are faced with a wide variety of options from some amazing established companies and lots of seemingly “sexy” startups. (after “the great resignation”)

You need to take your opportunityto stand out! The more specific you are about the challenges a specific engineer will get to work in a specific role, the more traction you’ll get with (the right) candidates.

The idea is to make an engineer excited when you describe what they will “Get to do” in the first 12-15 months in the role. 

Be very specific, 

  • Talk about the product/modules they will own/drive/be part of, 
  • Talk about the outcomes and metrics they will own and drive,
  • Talk about the toolchains & frameworks they’ll use (or get to choose), 
  • Talk about What’s hard/challenging about the role, How are they a great fit. 
  • The more details you can squeeze into the spec to help them visualize their role and the projects they’ll be working on the better.

Boiler Plate Job Descriptions don’t arm others to help you

Another bad thing about generic JD is that they don’t help others help you. Think of how much reach could you get by using everyone in your network as a recruiter. But in today’s scenario, “everyone else” is already asking them if they know any Python Engineers or React Developers or Go Engineers. Why would they help you? Because you’ve taken the time to get specific about what you want? Maybe. 

Take a look at the following Job Description. Anyone in software engineering who had some deployment, infra-planning & communication seems to be a candidate for the role. 

I recently got a request from a founder friend of mine to refer a Sr.Tech Lead/Engineering Manager for an early-stage startup. When I saw the JD, it was so generic it did not even have the primary stack on it. Assume sharing it from your handle. I politely declined to share it and asked for some more information and said will come back once he shares. (I believe he is very busy and hence hasn’t come back) 

The bottomline is, Make them want to help you — give them a JD that’s so amazing, well-written, specific, (even entertaining)  — that they can’t help but pass it on, post it, tell their friends about it, etc. If you make it stand out — you’ll get more attention from the folks that can help because you’ll arm them with something interesting & effective that they can use to reach out to their network.

Boiler Plate Job Descriptions don’t enable you to know what success looks like

This is a very simple point — see above — if you can’t explain what the ideal candidate looks like, how will you know when you’ve found them? The JD shared by my friend looks as vague as this.

Typical Boiler Plate JD

Actually, his company was looking for a guy who could not just do Infrastructure Architecture. They wanted someone who has architected/built a cloud native SaaS application. His team has built the application and has no idea how to convert it to truly cloud native format to scale without breaking the bank!

The main idea here is about not being willing to settle for less. I realise the market is tough right now, and maybe you’ll need to make compromises. But do you want to start at the wrong end of the pool? When you go out the door with a generic description, you preemptively give up the battle. If you need to settle — fine! — but know exactly what points you’re compromising on.

The larger problem is that if you don’t know what the best candidate really looks like then the other people involved in making a decision likely don’t know what s/he looks like either. A well-defined, specific description of the role enables everyone involved in the interviewing and hiring process to be on the same proverbial page.

Now go get started!

Fixing your job descriptions will take some work(as I found out). To get to specifics, you’ll need to dig in and make additional efforts. You might also need to do some retraining in your organization and teach others these principles, too.

But when you get through the hard work, your postings will turn into valuable weapons that will,

 a.) appeal to the engineers you want to reach,

 b.) enable others to help you expand your outreach, and 

c.) get your hiring team on the same page to quickly come to the right decision.

If you’re curious to see what roles we currently hire for, we have a lot of openings in Product, Design and Engineering:

It ranges from Kubernetes Architect, React Native Mobile Dev, Sr.Backend Dev, Tech Lead-Mobile Technologies, Engineering Managers, Associate Product managers, Product Managers etc.

More details can be had at https://angel.co/company/itilite-1/jobs or you can ping me 🙂  

Do you really need a Product Manager for a successful Product?

Do you really need a Product Manager for a successful Product?

This post is a summary of a series of “Mentoring” and “Advisory”  calls I did with some early stage startups, over the past 6 months. Most of the time, one of the founder ideates, one builds/leads the build. But, they want to go fast and think they need a Product Manager. Unfortunately, most of them don’t need a Product Manager. If you are at a similar juncture, read on to find out more.. 

The title is a controversial question, I know! 

The State of Product Management:

Off lately, Product Managers have to wear too many hats, leaving the role vague and blurring the boundaries of their area of responsibility. This ultimately leads to diminishing the value of the product manager’s core functions. Product Management is a strategic, cross-functional, front-line role that brings great value to the product and business.

But, it commonly gets abused by many fast-paced organisations expecting product managers to fill in the gaps in various disciplines. This may be process, pricing, unit-economics, partnerships, product-marketing to name a few. They can definitely do that due to their broad professional background.

Admittedly, product managers do have a broad background, otherwise they would have a hard time to be able to effectively collaborate with the stakeholders, lead the product and make the informed decisions. But this definitely should not end up with the product managers becoming de-facto “deciders” or “doers” originally intended to be done by other roles in other functions.

How do you decide if you need a Product Manager or Not?

Like any problem, there are two approaches, if an intellectual debate is more to your taste, continue reading on. If it is more of a rational “doer” approach, head straight down to it. 

Intellectual Approach

Ask yourselves some questions:

If you are a founder or a  leader or a decision maker,  before hiring a Product Manager, question yourself as to your expectations from the product manager. 

Think hard on what you want them to do:

  1. What do you want your new product manager to change/fix in your organization? What is it that you are unable to do?
  2. Do you not already have the in-house expertise that would help you address the current issues?

If you are still unsure about whether or not you need a product manager “in the house”, 

I recommend that you go through this checklist and answer Yes/No to each of its questions:

  1. Do you have a vision for your product? Do you believe it is aligned with the market needs?
  2. Are you sure you are building the right product — the one that delivers value to your target audience?
  3. Do you have a direction for your product? A long-term and a short-term roadmap?
  4. Till now, have you been able to execute your roadmap without major distractions?
  5. Are you capable of maintaining the strategic focus across all levels of the organization?
  6. Do you know your competitors and what they have on the game? Proposition, not features.
  7. Do you have an established feedback loop with your clients? (Not the feature request types)
  8. Do you mostly base your decisions on evidence/data?
  9. Do you find it easy to say “No” to various stakeholders from various functions while hearing their “suggestions” and “inputs” and explain them why what they think is not the “most” right thing?

If you answered “No” to more than 4 questions, you probably need a Product Manager, No doubt in that. 

But the reality is, that hiring a highly capable Product Manager won’t magically change the DNA of your organisation. I have seen multiple orgs regress into a worser situation than before. Because, the person responsibl has delegated the product decisions to that Product manager with a shiny belt, without enabling/empowering him/her. 

The result 

Rational Approach

If you’re a CEO, founder, or senior leader considering hiring a PM, check this list and see if you need one. Lets play a guess and eliminate game. 

If you can see your organisation is reflected in this article, don’t bother hiring a PM — save some money and hire a cheaper role. You would also spare a PM some misery.

Don’t bother hiring a PM…

If you have a fixed idea of what to build

You already know what you want to build, you just need somebody to build it. You’ve hired some engineers. You need somebody to gather the requirements from you and the team, and maybe manage the back-and-forth of different requirements from many stakeholders. This person then passes the requirements along to the engineers and makes sure they deliver on time.

You need a Project Manager, not a Product Manager.

If your Sales team or clients are dictating what to build

You have a handful of big clients and you’re ready to bend over backwards to deliver what they need, including building custom features. Your Sales team knows best what to build, surely, as they’re the ones talking to the customers all the time. Now it’s just a matter of writing the stories and prioritising them.

You need a Delivery Manager, not a Product Manager

If they won’t have access to your customers

You have some very-important-people as customers and their time is precious. You don’t want the new person you just hired to talk to them directly — may be they will say something untoward?

I don’t know what you need, but you certainly don’t need a Product Manager. 

If you’re not ready to delegate authority

You know that product managers should be given a problem to solve, not a feature to build. Heck, you were probably a Product person yourself, who has now set up your own startup. You have the vision and the strategy and you know exactly how to get there…

What’s left for the Product Managers to do, then? Maybe hire an Engineering Manager or a Tech Lead?

If you see technology as a support function

An easy way to assess this: How much of your company budget is dedicated for product/technology/innovation? If you’re not willing to invest significant resources to staff the product/technology team properly, they’ll be left firefighting all year long. 

Don’t hire a Product Manager — yet. Assess how you see technology plays a role in your company’s vision. Set aside a proper budget, hire a strong CTO or CPO, and let them build their team. Only do that if you’re willing to listen to them though — or don’t bother doing it at all.

In Sum and summary, Hire a Product Manager only if you believe you can delegate authority, and can come to a rational decision based on data. If not, hire a Project Manager, Engineering Manager or any of the other roles.

A Tech Lead writing code is a disservice to the company.

A Tech Lead writing code is a disservice to the company.

You have been coding your whole life or at least most of your professional life. Recently, you have been promoted/designated or as a Lead Engineer or a Tech Lead. Does anything change for you?

Should you stop coding?

People generally say, hell no!

Hell No!

And why should you now?

  • You like it; 
  • You enjoy it and probably 
  • really good at it too. 

But then you start leading a team, which means that everything should change or at the minimum, something should change Or shouldn’t?

It’s an eternal question for every engineering manager. I have tried to answer it all along my career and 

The hardest thing is to understand that you are not “just a” developer anymore.

I know, the above statement is controversial with multiple of my readers.

Most of you are now in a role with,

  • different responsibilities, 
  • different daily schedules, and 
  • tasks that involve different mental processes.

And you are most likely trying to combine two things at this time.

  • You’re trying to be a good developer (that you used to be).
  • You’re also trying to act as a “Coordinator” “Communicator” and also “Manage” things

I know, your designation/title says Tech “Lead” or “Lead” Engineer and not Engineering Manager.  But, in most organisations, a TL is looked upon as an EM in waiting (For more insight on career tracks for a TL – Check out my previous article on Engineering Leadership on Startups )

And working two jobs may often lead to early burning out and, frankly, not being any good in either of them.

I will take two very probable examples here.

Case 1: You are a Lead and you want to own a particular piece of code rewrite, which is giving a lot of concurrency nightmares to both the product support and your on-call teams. Most design/debug and development tasks require high concentration and focus, which contradicts the very nature of the team leader’s work. Multiple planned meetings, calls, messages — a lead needs to be on alert. It’s tough to consider all the edge cases when your slack/hangout/teams is buzzing all the time.

Another essential element is most of these buzzing & pinging can be controlled if your team is good at Asynchronous Communication. (I will write more about it in a future article)

Case 2: You are a Lead, driving a new subscription module for your latest product. There are simply so many stakeholders, your PM, Payments team, external partner/vendor, Infra team etc. It will be hard to be prepared to answer your teammates’ or vendors or customers’ questions if all you can think about is the efficiency of that function you just wrote.

Time Share:

Another thing is that spending a lot of time on development gives you little time to do your actual job as a lead/manager. And your job is managing other people. Though you will probably make time for your primary duties — assigning tasks, making estimations, validating designs, communicating with the stakeholder — you will miss out on all the other “noncrucial” parts of your job.

You can get so invested in a feature that you will miss some critical signs of your employees becoming demotivated to do their work, tired, or less happy. And, as you are busy, you become less innovative. Who will come up with a new architecture for the 3-year-old service? Most certainly not you, since you are too deep in the code.

Anti-Growth

Minefields you’d inadvertently trigger are either the Martyr Effect or the Hero Syndrome (Think Bruce Wayne or Tony Stark). On the first & second one, You’d always take the toughest part of the code or the most interesting part of the code, respectively. Either way, you’d be creating a team who’d be ill-prepared to take up challenges on their own or ill-equipped.

But what if I do not want to give up coding?

It may so happen that plain management isn’t your path. So, from here on, you may not enjoy what the role has to offer. Being a Team Lead/EM is all about people; being a Principal Engineer (or staff engineer or architect) is about code. If programming is critical for you and brings you more joy, you may be more suited for a Technical Leader role. So choose wisely.

But as a Team Lead, you are still most welcome to join code reviews and help your teammates with challenging coding problems if you want to.

Consult. Guide. Assist. Communicate

This is going to be your Motto.

And of course, You should definitely continue to code in other “Non-paying” parts of your job. Start automating your units, create boiler plates, write smaller, niche, critical elements of your system.

How to select SSO Standard for your SaaS Application.

How to select SSO Standard for your SaaS Application.

For anyone developing any application on the cloud, the major concern is always how is security implemented. Typically, you start with an authentication system viz. Usernames & Passwords. As your application grows in size of use cases and adoption, you’ll soon find a necessity to improve your security posture, these could range from MFA, Federated Identity management and finally authorisation. You now have customers who ask if you can support their AD authorisation or OneLogin or Okta etc. 

This is when you’ll think about implementing a Single-Sign-On. But, the choice of how to keep data and identities secure begins much earlier for software architects and developers: selecting the standard that should be used to keep federated identities safe. This will involve two things, architecting an authorisation system – could be a separate service or bound with your application – this choice is critical to how you can grow as an organisation. 

Architecture Choice:

If you choose to integrate it with your main product and 2 months later your board directs you to develop a new offering, you’ll end up doing it all over again. On the contrary, if you’re not going to pivot to any new business line, the additional time you will incur in building an external “Accounts service” will be a tax on the GTM. 

Standards Choice:

IT Administrators and Security Architects must first choose the protocol or framework to use to maintain federated identity, or the mechanism of connecting a person’s electronic identity and attributes, safe while designing a plan to keep data and identities secure.

A Single Sign-On (SSO) account has the advantage of allowing employees to log in once to an application or network and not have to log in to several apps or networks during the workday. While this is beneficial to employees in terms of increasing productivity by eliminating the need to remember several passwords, it is also beneficial to IT and Security functions. The Identity and Access Management (IAM) platform responsible for maintaining employees’ credentials can assist make it more manageable by registering fewer passwords in the system.

It is, however, not an easy choice. Security Assertion Markup Language (SAML), OpenID, and open authorization are the leading candidates in the federation process (OAuth). Let’s take a closer look at these technologies and determine when SAML, OAuth, and OpenID should be used.

What is Single Sign-On (SSO)?

SSO (Single Sign-On) is an authentication method that allows apps to validate users by using other trustworthy apps. Single sign-on allows a user to use a single ID and password to log into several applications.

SSO is an important part of an Identity and Access Management (IAM) platform for managing access. User identity verification is crucial for establishing what permissions a user will have.

SSO Standards

  • SAML

SAML is a protocol that allows an Identity Provider (IdP) to send a user’s credentials to a service provider for authentication and authorization. SAML allows for Single Sign-On (SSO) and streamlines password management. It is beneficial to businesses because employees are using an increasing number of applications to complete their tasks.

Keeping track of passwords for hundreds of programs used by hundreds, if not thousands, of employees can be difficult. SAML comes to the rescue by providing a single sign-on standard for businesses.

  • OAuth 

OAuth 2.0 is a secure authorization standard. It allows secure delegated access by providing third-party services with access tokens rather than exposing user credentials. It does not, however, authenticate; it just authorizes.

You’ve probably used OAuth 2.0 if you’ve ever signed up for a new app and consented to allow it automatically source fresh contacts from Facebook or your phone contacts. This standard ensures that delegated access is secure. This means that a program can operate on behalf of a user and access resources from a server without the user needing to provide their credentials. This is accomplished by allowing the Identity Provider (IdP) to issue tokens to third-party apps with the user’s permission.

  • OpenID

The OpenID Connect (OIDC) standard is used for authentication. OIDC is used by identity providers (those who generate and administer identities) so that users can log in with their IdP first and then access applications without having to re-enter their credentials.

This authentication option is recognizable if you’ve used your Google account to sign in to apps like YouTube or Facebook to log into an online shopping cart. Organizations use OpenID Connect to authenticate users, and it is an open standard. This is used by IdPs so that users can sign in to the IdP and then use their sign-in information to access other websites and apps without having to log in or disclose their sign-in information.

SAML VS OAuth VS OpenID

OAuth 2.0 is a framework for regulating authorization to a protected resource, such as a program or a set of files, whereas OpenID Connect and SAML are both federated authentication industry standards. As a result, OAuth 2.0 is used in quite different situations than the other two protocols, and it can be used in conjunction with either OpenID Connect or SAML.

OpenID Connect is based on the OAuth 2.0 protocol and uses an ID token, which is a JSON Web Token (JWT) that standardizes areas where OAuth 2.0 provides for flexibility, such as scopes and endpoint discovery. It depends on user authentication and is often used to make user logins easier on consumer websites and mobile apps.

Unlike JWT, SAML does not rely on OAuth and instead relies on a message exchange to authenticate in the XML SAML format. It’s more commonly used in enterprise settings to allow users to log in to several applications with a single password.

Final Thoughts

As technology advances and systems become more interconnected, federated identification becomes increasingly useful since it is more convenient for users. It saves them time by reducing the number of accounts and passwords they have to remember, but it raises some security concerns.

SAML has one feature that OAuth2 lacks: the SAML token contains the user identity information (because of signing). With OAuth2, you don’t get that out of the box, and instead, the Resource Server needs to make an additional round trip to validate the token with the Authorization Server.

On the other hand, with OAuth2 you can invalidate an access token on the Authorization Server, and disable it from further access to the Resource Server.

SAML provides a simpler and more standardized solution which covers all of our current and projected needs at ITILITE and avoids the use of workarounds for interoperability with native applications.

What Does It Take To Become a “Senior” Software Engineer.

What Does It Take To Become a “Senior” Software Engineer.

This article is a result of a discussion with one of our ” Ninja-neer”. He was interested in “Delivering Business Value” but not interested to take up People Management or other responsibilities. Do I have any pointers for people like him? Of course. So, we started discussing on ways he can contribute at a different level. In the end we talked for about 90+ minutes. This is an extract & summary of that discussion.

In the late 2000s, it was a trend for companies to hire developers based on the programming language they had experience with, frameworks, tech stack, and such. (I still remember the disappointing gaze I got when I told the interviewer that I have only worked with CVS and Mercurial and not in SVN, which the team I was interviewing for was using)

It is preferable to hire engineers skilled with a particular stack, it is not crucial. After all, great software developers should be able to learn and ramp up quickly with the massive knowledge available on the internet.

With that being the absolute baseline, companies started to value developers with great complimentary soft skills, as their technical expertise is now baseline to work in the industry, setting the bar even higher for people starting a career right out of college.

The Three Fundamental Traits

After almost 12 years of managing/leading Software Engineering teams as a Technical Lead, PM, Engineering Manager, Director etc.,  I have observed the skills that tech organisations generally value the most. I believe I have identified a pattern that generally falls into three different categories:

1. Technical expertise and craftsmanship

Understanding the fundamental concepts of computing is the baseline to becoming a software engineer. Even though this looks like common sense, this science is vast and is continuously evolving. Gone are the days when knowing some data structures, array transformations and basic algorithms will get you over the ledge. Also, the organisational/product context is very important as well. For example, my peers at Paypal prided in getting sub 500ms latency for all the “processes” they wrote, while my colleagues in Hinduja Tech focussed on ensuring “zero-packet loss” from the telematics devices.

It really boils down to what is your company’s key priorities are. It can be quicker release cycles/velocity, resilience/ fault-tolerance or efficient memory management. Whatever it is, you need to first understand the “value” and then follow it in your implementations.

2. Scope and autonomy

We do not live in a world where working alone and implementing specifications from LLD/UML diagrams is sufficient anymore. For that matter, in the last 18+ years, I have met exactly two people who were able to pull it off and one was a 62-year-old Ingres developer, who was single-handedly managing the 40year old databases of PA. Those who know how to navigate complexity requiring minimum supervision are now extremely valuable professionals. Actively communicating and ensuring alignment is more important in these times of high-velocity organisations.

3. Communication and influence

Even though nobody expects you to be a skilled public speaker, we are long past the era where programmers were introverts that spoke an unintelligible alien language. Knowing how to work with people and interact with non-technical partners is a valued skill in the market.

I had a very first-hand experience of why clarity in communication is so important as you grow up the ladder in your tech org. 

I was (hastily) called to a meeting, where my boss (VP) was explaining to our CEO, of why we should not be building the next generation of BRTS for Congo, Senegal and Ghanna on Desfire EV1. The primary concern was around security and privacy. There were major concerns around its security and was exposed just before the London Olympics. (It had taken me 3 meetings over 2 weeks to convince my Boss to go with EV2) I still do not know why he thought I might be able to do it in 10 minutes and that too in front of the CEO!! But the important thing was, My boss was willing to give me a chance to try it and in the process, he was giving me visibility to the inner workings on the 11th floor (C-Suite).   

How do I convince my CEO to opt for a solution with almost 20% additional initial cost? 

Is it with NXPs’ security from relay attack or with 16KB vs 4KB of usable memory or something else? Then it struck me if the topline is something my CEO was interested in he could definitely understand the bottom line!  I fumbled something around potential “Revenue loss” and did a whiteboard tabulation of some numbers. (Desfire

Surprisingly, my CEO got it despite my ramblings and corrected my statement, It is a potential Revenue Leakage, not a loss!

That one meeting changed almost every communication I did after it.

Becoming a Senior IC – Sr.Engineer, Staff Engineer, Principal Engineer, Architect (or any of the other dozen designations)

Again, this article is aimed to give some clarity on what are the options for rising up the ranks as an Indivudual Contributor. The Tech Ladder in most Startups are similar with slight deviations. After you become a Sr. Engineer you have 2 tracks – Technical Excellence leading to Staff Engineer and Principal Engineer. The other track involves People and Budget management leading to Tech Lead, Engineering Manager and Director, VP etc.

 I’d like to help you understand important aspects that can place you at higher levels based upon the expectations set on each level of proficiency grouped into the following tiers: Beginner, Intermediate and Advanced.

ExpertiseScope & AutonomyInfluence
BeginnerLearningFeature & GuidedCollobrators
IntermediateProficencyProduct, Performance & TacticalTeam(s) Wide
AdvancedExpertiseDomain, Industry & StrategicTeams & Function wide

This grid above depicts the career trajectory of software engineers in a super-simplified way. It is generally more complex than that, but it still serves as a good guideline to identify career points of inflection. Note that I did not use the so popular “associate,” “mid-level,” and “senior” on the different levels. This is more of a grouping of related circles of roles.

Starting a Career as a Software Engineer

As a beginner in this new and adventurous area, there are lots of low-hanging fruits you can learn from. In fact, learning should be your focus. You should acquire as much knowledge as you can from being exposed to a variety of problems.

Up until a point, you will work on “very specific” problems or small features or bug fixes until you ramp up and have a good understanding of the lay of the land (product or system) you are helping to develop.

You will pair with more experienced engineers and learn from code reviews and feedback from your partners. Engineers at this level spend a reasonable amount of time learning until they get proficient with tools and acquire more domain knowledge. You’ll learn a lot of Tricks and Tools from your senior peers and you will also find the kind of problems you’re proficient in solving. Which will result in similar problems, fixes or features you’re assigned with.

The trick is to emrace the “Streotype” and make it your “Niche”, while also diversifing enough to get a hang of other things and continue to learn.

Working as a Proficient Developer

At this point, you would have gotten your hands dirty for a few years and developed mastery of computer science including algorithm design, data structures, design patterns, and the tools and frameworks you work with. You have very deep experience with at least one or two part of the technology stack you work with.

It is now taken for granted you will be able to deliver complex pieces of software with very little supervision. In fact, there is also the expectation you can help less experienced engineers to grow and guide them to execute the tactical plan you created. You help people to review their code as well as solve problems and develop new features.

One thing to remember is, “Code Review is a Bidirectional Learning exercise” – The proficient ones understand/learn new approaches from the beginner, the beginner

It is the time in your career that you start getting the opportunity to lead small projects and time-bound initiatives and likely start to get more exposure to cross-functional partners and some non-technical stakeholders. Most software engineers stay at this level for many years.

The Non-Comissioned Officers are called as the backbone of an Army. Similarly, Sr.Developer is the backbone of any product/Engineering Team. As this is the most visible and “on-the-ground” leadership.

The Making of a Senior Software Engineer

At this level, Coding in general starts to become less important as you are now a visible voice for your team and across the organization. You now understand how to make difficult trade-offs in the architectural level of your application across the entire domain.

As a domain expert, you own a substantial part of your company’s codebase, supervise its evolution and work from other engineers, as well as advise other teams on how to better approach or integrate with your services and applications.

As an advisor, your contribution is clear and visible across multiple teams. You are highly influential and your advice is constantly sought from other engineers and cross-functional partners.

This is the inflection point where you start considering a transition to leadership roles. It usually takes some years to land at this level. The next step for you is growing the impact of your work across teams, organizations, companies, and industry-wide.

Even though colleges prepare you to develop software, as you grow in your career that skill starts to become less important and other soft skills turn out to be more relevant. I hope i was able to nake justification to the topic of growing as an indivudual contributor and make higher impact and inspire you to reflect on your own trajectory and how to proceed with the next steps.

The 5 ways to Fail as Engineering Managers in Startups

The 5 ways to Fail as Engineering Managers in Startups

This article is a compilation of multiple years of my experience being an Engineering Manager and subsequently running the Tech Org and managing multiple Engineering Managers. I have tried to summarise and condense them.

Having a good manager can make you feel supported, can boost your career growth (and sometimes personal), and help make your team and company a happy place. On the other hand, having a bad manager can make your work-life miserable and could hinder your growth and drain you.

Engineering Managers have a huge impact on their team’s, morale, outcomes, timelines and most importantly the professional growth and help them carve a career path. But, you may have seen, heard or felt that some or most Engineering Managers are anything but the above description, right? Do you want to know the root cause of the problem?

It is the practice of making a high-performing Individual Contributor/Engineer the Tech Lead and thence to an EM!

Trust me when I say this, I have seen it multiple times. I have seen many good Engineers burn out as soon as they have people management responsibilities. An Engineer may be okay to mentor some junior devs and help them get the right design etc. But, S/he needs to have a people-first mindset to become an Effective Engineering Manager (or any of the myriad titles with the job function).

The 5 ways to Fail as an Engineering Manager!

So, assume an Engineer is looking to move into Engineering Management, the following are the pitfalls S/he should be aware of as these are the most common ways EMs fail.

1, Too much Solutioning, not enough listening.

Interestingly, this can happen both when you’re not confident as a leader and when you are too confident. We tend to focus on solutions too much instead of supporting/empowering others or listening for more context. Sometimes people only need someone to vent to and are not looking for solutions immediately. Even when they are, we can act as coaches and guide them to the solutions, helping them grow in the process so that next time they will be able to solve on their own. Even when they need an immediate solution, we might fail to get the whole context by not authentically listening to them.

Such leaders usually jump to solutions right after hearing about an issue, and even when they ask for more details and input, they are not listening authentically. They might get impatient when the discussion drags on.

There are two critical Skills to practice to overcome this pitfall. Effective Listening and asking more Leading Questions.

2. The silver bullet or the Golden Rule Fallacy

We might not be very conscious about it, but we all have a natural, default style when it comes to management. This is sculpted by our general personality, our experiences, our bosses and how they treated us and things we’ve learned along the way. As managers, we unconsciously rely on this style, and without guidance, we tend to use that style with every direct report. Even when it becomes conscious, we justify this with ‘this is who I am’ and sometimes even with core values and our self-image

Don’t “Treat others as you wish to be treated”

The above statement could be borderline Blasphemy to many people in many aspects (including cultural or religious). How could it be untrue? If most major religions/cultures preach it?

The reason for this paradox is simple, we all assume we want to be treated fair. But, fairness to me may be unfair to you and the other way around.

For example, I tend to react very well to negative stimuli, i.e: critical remarks. I use them to better myself and continuously improvise (most times, at least) whereas some other person may feel it draining, for them, the Positive Reinforcement techniques may work well.

While having strong core values is vital to being a successful leader, using a single management style just simply won’t work with all your Team Members over the years. Doing this way WILL HARM some of the Team Members (and of course hinder your performance as an engineering manager, too). The Golden Rule managers often talk about the one true way to do things. They get overprotective/defensive about their style as they face more and more challenges. They often see the failure to be with the team members who don’t respond well to their style instead of adapting to theirs. This is especially important as more and more of you’re team members tend to be Millenials.

The most obvious display of the Golden-Rule/One-Trick Managers is hiring #minimes. They hire a team full of similar styled team members. You may have recognised certain trends over the years, a very hands-on manager will not only hire, but also treasure a very hands-on problem solver by empowering them. On the other hand, a Process-oriented manager will hire their lieutenants to be fully process-driven ones.

The problem with the first example is, you’ll have an army of Debuggers, Fixers and Solvers but very few(if any) to think & execute in a scalable & sustainable way.

The problem with the second example is, you’ll have an entire team quoting the “Rule-book” to each other in no time and meanwhile, the company may be bleeding.

There is only one approach here, As managers, it’s our core job to form a good working relationship with our team members. This will require us to adapt our style or adopt new styles.

3, Low self-confidence

Yes. I meant it. I have known multiple awesome engineers in my career who started having Low Self-Confidence after they became managers.

Honestly, I have gone through it myself at various points, before climbing up the rope. The reason is also quite obvious, when I was an IC/Sr.Dev I know what was the outcome and what was the timeline and quality of deliverables was something I prided in. So, nothing was ever out of control for me (except maybe twice).

And if something exceptional happened, I can “Report” it and either get the “Scope” or “Timeline” modified and my self-worth was left unchanged. Now, as the first-time manager, I realised that I am that “Exception Handler”. Sure, I can go to my Delivery Director or Group Program Manager etc, but I am supposed to be the first line of defence from exceptions affecting the Business! This is the no: 1 cause for low self-confidence.

But, it is by no means the only one. The second most cause according to me is delayed feedback and low observability of Business Value delivered. It’s usually really hard to see our work’s positive effect, feedback loops are just too long, and cause-and-effect relations aren’t always easy to see or quantify.

People with low self-confidence usually have a hard time saying “I don’t know”, which is essential as an engineering manager. We cling to the thought that we have to know answers to everything that comes up; otherwise, we’re just not good enough.

I’ve seen some insecure managers trying to do team members’ jobs. They do this not because they don’t trust their team, but they need something they’re proficient with to feel more secure and confident. Another way for such managers to feel that they are still worth something is to be too nitpicky, for example, in code reviews or simply when giving feedback.

Among many things, all this can lead to the engineers feeling that their engineering manager is competing against them in a way. This is THE worst feeling you can give to your team and a sure way to fail as a Manager.

Be the force multiplier to the team, not another grunt.

There are proven ways to come out of this zone. Discuss with your peers (other EMs/TLs) and your Manager, open up your insecurities & fears. You will realise that this is much more common and also learn from them.

4, People or Business Attitude (as opposed to People for Business)

There are two most common styles of management, Too much Business Focus and Too Much People Focus.

There are Managers and Leaders who are only into Business and view upon their team as only “Resources”. They are driven by Goals, deadlines, KPIs and Metrics. They seemingly don’t care about their team’s wellbeing.

There are Leaders who are only into the people part of their team. They create a virtual haven for their team. They shield and protect their teams from the other parts of the company and the world at large. They seemingly don’t care about business outcomes or performance that much.

Needless to say, these two styles are diametrically opposed.

Fortunately, I have seen and worked with organisations with both styles of Management and Managers. (And believe I did pick some elements from both). In the StartUp eco-system, Boot-Strapped and bootstrap influenced organisations tend to be slightly tilted toward the People First management philosophy. And generally, organisations that are VC funded and with an aggressive growth appetite will be tilted toward the Business First management philosophy.

But universally, all leaders I have worked with accept/agree that the key to success is balancing the two approaches.

Too much focus on the business in a leader will sometimes result in that leader prioritizing short-term wins over long-term ones. Such managers will talk a lot about holding people accountable. They are generally okay in abandoning the team members who don’t fall in-line in terms of Team Commitments and Performance, instead of coaching or aligning. The cost here could be enormous. People will get burnt out quick and will leave, The company culture suffers.

Too much focus on people without the consideration that your team is responsible for the company successful can be even worse. Such leaders will position themselves as the “Gatekeepers of hell” with their team. They will defend their team no matter what and will view every discussion/motion as “the Battle of Thermopylae”!

In the end, is is not as much as balancing these views. Its actually building synergies between these two seemingly conflicting ideals. You as a leader and manager will have to find ways for your teams to grow and be successful in tandem with the business goals.

5 – Not Delegating Enough.

The most common mistakes for leaders and managers are usually focused around delegation; either a manager is delegating too much or not enough. This is especially common for an Engineering Manager. Most Engineering Managers think of themselves as a “Specalist” Engineer than a Manager, especially applicable to an EM at the early part of his career. Any manager who fails to delegate will become overloaded and fail to move the business forward. A manager who over delegates with no explanation as to why could lose the respect of their team. The key rules to live by as a manager when it comes to delegating are:

  • Only ask someone to do something you would be happy to do yourself if you had the time
  • Only delegate a task to someone who is happy to take on the task
  • Only delegate to someone is capable of completing it to a level you would be happy with yourself or can get there with quick review comments

The trick is to know when to Cascade, Delegate & Escalate!

Concluding Remark

Obviously, this list is not-exhaustive and there are other significant issues causing to failures of Engineering Managers. But, this is a Ranked list from my personal experience.

I was extremely fortunate to work with some of the best leaders and managers and each one of them has shaped my skill, style and everything in between. While mentioning my “managers”, “My Teams” over the last 8-10 years have played an equally important role in this transformation.

Also, If you’re looking forward to learn how can you be a manager/leader your team will not run away, check out this short course by Laurie Ruttimann – https://www.linkedin.com/learning/be-the-manager-people-won-t-leave/be-someone-people-trust-no-matter-what

Business Value Delivery by Engineering Teams in StartUps – Part 2

Business Value Delivery by Engineering Teams in StartUps – Part 2

In this multi-part post, I will try to articulate my view on the importance of business value and its delivery by engineering teams. This is the second part, where I will share my perspective on the “How of it”.

Part 2: The How of it – Define, Visualise, Prioritise, Develop, Deliver & Measure.

The PMI Model of Delivering Business Value.

1). Define Systems Development Strategy 

The first thing a “Tech” Founder need to do is define the Systems Development Strategy. At a very high level, the systems development strategy should detail the state of the current/planned systems, the high-level business strategy for the next 2-5 years and maps out a plan to get there. An engineering leader will drive the creation and implementation of the development strategy to ensure the business can meet their current and future needs. Working closely with architects and technical leads, the engineering leader can formulate a solid development strategy.

The development strategy should detail the core architecture direction and technologies for the systems, including high-level plans for delivery. The development strategy is the crux of all efforts to deliver business value. Without a firm foundation of proper system architecture and technology, the business will have a difficult time delivering the value they need to survive. 

If you’re an Engineering Leader who joined the startup after the MVP is created, it is your responsibility to understand the business strategy and formulate the Development Strategy as early as possible.

If your startup doesn’t have a solid development strategy or similar document, the following is a great place to start:

  • Gather business needs: Gather high-level business needs/strategy to cater for the now and future (2-5 years) horizon. Not a deep dive, but deep enough to judge existing systems and measure other options. (Question like How many new new users will be added month-on-month, what is the order of magnitude of transactions we plan to rake, is it thousands or millions or tens of millions – Each will point you in a different direction on the system design)
  • Review of existing systems: Analysis of current systems around fit for purpose and whether it can be maintained and extended to meet the future needs uncovered in the above. (The MVP may seem to work fine and it will be tempting to build “On-Top” of it with a plethora of “Features”, resist the urge and pressure, if applicable)
  • Technologies / Architecture: Based on the review of the first two bullet points, you may recommend a strategic direction. The decision here could range from rebuilding the entire system with a new solution, to replacing components of the system with off the shelf/Open-source components. Alternatively, you may find the existing system is a strong foundation which needs modernising or scaling. In which case, the development strategy document would detail a range of architectural and technologies for future development. 

The above is a good starting point and will allow the business to get started on implementing the development strategy. You may do it even before starting with the Startup and make it a Pre-Joining exercise with the Founders and Senior folks. At the end of this exercise, you will have performed an extensive analysis of the current systems and have a strategic direction for the systems.

2). Help the Business Define Requirements

It is essential to understand what needs to be delivered before you can go ahead and deliver the next Amazon or Airbnb. It has been my experience that on occasion, the business will need some “External Inputs” to finalise what is required. 

When the business has a lot of ideas for improvements, they can sometimes get muddled together and lost. To counter this, we at ITILITE do a “Quarter Theme“. Before ITILITE I worked with Zarget, where we had a similar “Themed Quarterly Roadmaps” as well. This “Theming” helps in prioritising the focus areas. More on that in the next section

After which, we can visualise the entire scope of these ideas using User Story Maps. User story maps are visual representations of functionality requirements where all the requirements documented using a system of cards. It becomes a more straightforward (not easy) task to slice and dice these requirements using a story map to cull anything that is not critical to the business. 

For the remaining requirements, we need to gather a little more information to progress to the next step, for each requirement we need to capture:

  • Description: High-level description of the change. Not a HLD/LLD but enough to provide a high-level order of magnitude estimate.
  • Business benefits: Here we are looking to understand what benefits we can expect from the business change. 
  • High-Level estimate: Order of magnitude level estimate, lots of refinement to still take place, however, gives us a good idea around sizing.
  • Business SME and Sponsor: Details of people we can go to get more information.

The detail we capture for each of these changes is small, the reason being these items are a wish list only and not confirmed, so we do not want to waste more time on these then we need (lean thinking). While it is the domain of product managers and business analysts to flesh out business requirements and benefit statements, the engineering leader also plays an essential part in this process. Engineering leaders can use their experience to provide the high-level estimates for development, or indeed recommend ways to implement the requirement without the need to write additional code. 

Another area where engineering leaders should influence is ensuring and non-functional (technical strategy items or technical debt) is included for development prioritisation. These technical plumbing is not attractive to the business but could be critical for the business to achieve their long term goals. Engineering leaders are the people that need to fight to ensure they are on the table.

Also, while you analyse requirements, where possible, try to group requirements where they affect the same code or system module. Grouping requirements will assist us in prioritisation, sequencing and hence the Go-To-Market, which is a key parameter for the business. The last thing we need to do is storing these requirements in our product backlog, to be reviewed and prioritised by the business in our next step.

3) Visualise the work and prioritise

In our third step, we are getting closer to the business deciding on their valuable items. Taking our list of requirements from our product backlog, we now present these to the business to discuss and rank in order of importance.

As discussed above, there will always be x+n “Projects” in the asks. Where “x” is the number of features you can effectively deliver in the timeline. And all “Projects” will look like they are P0 to solve.

If Everything Is a Priority, Then Nothing Is!

Well, the quote wasn’t from Morpheus, I just liked that Meme (it is debated to be in between Yuri Van Der Sluis and Garr Reynolds)

Having an extensive list of items to visualise enables the business to understand that we cannot have everything, and need to select the items that will make the most significant difference to their business (i.e., highest business value). 

This is again, not because of intent, but because of trying to do “Too-Many” things and “Too-Soon”. Independently, all of the asks may sound truly important. Every Leader/Function within your Start-Up will come with several competing “Projects”. The Finance Team may want that flashy invoicing module or an ERP integration with your suppliers/customers, the Customer Success would want that Advanced Analytics platform integrated, The Support Teams may want that long-standing “Quirks” on the product ironed out. Left to Engineering, this is a sure recipe for disaster. This is where a Strong Product Leadership helps!

A business analyst or product manager typically runs these planning and prioritisation meetings. However, the engineering leader also has a place at the table to provide insight and assistance to the businesses decision-making process. Who from the business should attend these meetings? It is essential to gather a broad cross-section of business stakeholders for every Department or Function that uses the Product in question. We don’t want one department having too much influence that may not be of benefit to the business. 

The meeting could have the following Agenda:

  • Review items: The group will discuss each item in the (curated) product backlog in an open and honest discussion. 
  • Accept or reject: The item will be approved for development or rejected. Rejected items will have their requestor notified, to ensure they are in the loop. 
  • Ranking: Approved items get added to the backlog in priority order. 

At ITILITE, We have the backlog/Thematic items in a Google Sheet, which is distributed at least a week before the meeting to ensure the Leaders have enough time to review, ask any questions before the meeting to ensure a smooth meeting. During the meeting, we view the sheet, top to bottom taking notes where required.

At the end of this meeting, we have something special. We have a prioritised list of business value and Key Outcomes.  

The prioritisation meeting can be held quarterly or monthly, depending on the speed of change in your Start Up. We do it on a quarterly cycle to meet with the Leadership, so as not to overload these folks from actually getting their work done. 

If the business has urgent changes which require attention, an emergency prioritisation session can be called-in where a meeting can occur to review and approve changes to the delivery schedule. Alignment should happen outside on one-on-ones and this meeting is a platform for other leaders to either ascent or dissent on the re-prioritisation.

4) Schedule and communicate delivery

In the fourth step, we now have a list that ranks all the business requirements in priority order. We now have confidence that the business indeed wants these work items completed and the order they prefer. The engineering team can now spend time working out how to deliver these items. Remembering from step two, we gathered very high-level requirements, (so not to waste time before they were endorsed), we now need to finish fleshing these items out enough to commence delivery. 

There are a few mechanisms we can use to gather the information we need to get going, and the main one I like is the feature or project kickoff & inceptions. The kickoff is a process where we get the delivery team together to discuss the work that needs to be delivered. Inceptions can run anywhere up to a few weeks for big projects; it depends on how much time you allow here. During our inception, the delivery team all get on the same page with the requirements in question and can ask questions of each other or the sponsor to get all the information they need. 

Technical delivery decisions can also be made, including creating simple prototypes to test out delivery options. Once the inception is complete, the delivery team have all their information, more confident delivery estimates are possible, sprint planning can take place, and the overall delivery schedule is known.

From here, the final step is the communication of the delivery schedule to all relevant stakeholders. Ensuring people can ask questions or point out any problems they see with this schedule. 

5) Deliver value often get feedback

The final step here is to get the job done. The best way to deliver software is in small chunks completed during our sprints (typically two-week blocks). Sprints are the quickest way to deliver business value, allowing the business to gradually use this value much quicker than waiting for a monolithic release to occur. 

At the end of each sprint, the team should be running product demo events. A product showcase allows the product and engineering teams to show off their excellent work to the business, who have an opportunity to provide their feedback on the product. This can start even before the first “releasable” product is out. It can start with mockups, design and prototypes. Then it will progress to V0, V1 and so forth. This feedback loop is another mechanism to ensure we are hitting the mark in terms of the delivery of business value.

Conclusion

I hope i was able to do justice to the process in this article. The key to delivering business value is having close relationships with the stakeholders, ensuring that they are involved in each step of the process. The business stakeholders are the only folks that can define business value. However, it is the role of engineering leaders to ensure proper technical oversight takes place to ensure the timely delivery of business value. 

Business Value delivery by Engineering Teams in StartUps – Part 1

Business Value delivery by Engineering Teams in StartUps – Part 1

In this multi-part post, I will try to articulate my view on the importance of business value and its delivery by engineering teams. While most of this is written from the view of a StartUp, some elements of an established organisation are also used.

Part 1: Defining Business Value & Role of Leadership in it.

Business value is a concept that can mean multiple things to multiple people and the tricky part is all of them could potentially be valid. A product manager may value a long list of features that his/her customers have demanded for months. Another Product Manager working with internal teams to improve efficiency (revenue) will value the enhancements the accounting or support team was after. While the support manager may value a more stable product to keep the customers, s/he deals with happy. 

Business value & impacts are a difficult thing to define and deliver, while it is even more difficult to measure.A collaborative effort is required to define and deliver business value, with consideration needed to ensure all voices are heard.

While most of what I will be covering in this article is typically the purview of product management, I have learned that engineering leaders have a critical role to play in this space. (Will write more on that in the next part.)

Engineering leaders bring product development experience and technical expertise to the table to provide a crucial element to the delivery of business value which I will try and explain in this article.

What is Business Value?

I would define “Business value” as any improvements to systems, processes or people that augment the products or the ability to deliver products or services to the customers, thereby increasing the revenue or experience or both. No two companies will have the same definition of business value. Forget two different companies, a company in its 5th year will have a very different perception of value to its first year. This is due to their products and customers being different and requiring different elements to add value. One company may find value in the ability to build out its new product offering quickly. While another may find value in responding to customer support requests in a timely manner. 

Due to the rapid changes around us, the things that businesses value changes often. Companies often face new challenges that require a quick response.

Be agile, be nimble” is the key phrase.

These challenges can come in the form of new product features released by competitors, or a specific feature request by a key customer, or changes in the market that render the current product/feature obsolete. Business needs or desires, therefore change just as quickly as any of these external changes.

You have probably worked for a company that comes to the engineering team with new requirements, seemingly daily?

It is not because they cannot make up their mind; It is in response to the changing business needs. This changing goalpost is one of the main reasons that Agile development practices have taken precedence from more traditional waterfall methodologies for software development. 

Velocity is everything, a report by McKinsey on how Developer Velocity fuels Business Performance will give more insights on this. A snapshot from the report is below.

Why is business value important? 

Reacting to change and delivering business value with haste is a crucial area of importance for modern businesses. All companies exist for a purpose. The majority of companies exist to return a profit for their owners (individuals or shareholders), while some companies exist to provide a social service. The critical thing to note is that they all exist to fulfil a specific purpose which guides their definition of business value. 

No matter the company large or small, if they stop innovating, and their products or services stop being relevant to society at large and market in particular, that company will whiter and eventually die.

Kodak is a prime example of this occurring in recent history. In today’s world, IT, whether it be hardware or software, is the largest driver of business value. It is therefore critical that the software engineering teams keep delivering the things that the business need to fuel their innovation.

We, as engineers, are not employed to just build that shiney app in the latest technologies, but to deliver our contribution in support of the business purpose (If not drive it!)

The importance of Engineering Leadership in Delivering Business Value

An engineering leader is, of course, a People Leader, and s/he is also responsible for the Execution, both technology and delivery of the engineering team. However there is a third dimension which often goes unrecognised, is that great engineering executives must also be great Business Leaders; they help drive alignment with other leadership/executives and shape the strategy and direction of the business itself.

It is this underutilised/forgotten element which I will try to detail here.

A People Leader & an Execution Champion:

Engineering leadership is often naively thought of as being simply a great Architect or Engineer or a Manager. But most of you already know it’s more than that. Team leadership will involve some combination of team building, culture, leadership development, and performance management.

For detailed coverage on Engineering Leadership – Please checkout my Previous Post

Most of this responsibilities will be bang in the middle of the comfort-zone of a rising Engineering Leader.  But one of the hardest things for most engineering leaders as we scale is, to continue having an accurate forecast of when products and features will be delivered – what the business always asks for.

That is partially because this bleeds into the third, and the least recognised dimension of engineering leadership.

The Missing Sauce: A Business Strategist

Engineering leadership isn’t just about delivering products faster, or making engineers more productive. It’s about guiding the team in the same direction as the business, about continuously improving, and it’s about being the voice of engineering as a part of the decision-making process of the executive team. Of course, these are all dependent on our ability to understand the work our engineering teams are doing and how it aligns to business goals.

The third dimension – Business Alignment – is often overlooked or made difficult by other executives, but is absolutely necessary for the management of a successful engineering org. This is the strategic practice of engineering management, and all operational decisions depend on it. Business alignment means ensuring your organization is focused on the right projects that align with the business’s goals. 

The Product org can detail/design and Engineering org can build as many features as they can agree on, but what/how does it matter, if they do not align with the business objectives or goals? Business alignment involves the right allocation of resources that supports business objectives, and helping to drive those business decisions of which projects are strategically important. (at itilite, this is always the First Principle)

How do we deliver business value?

So how do we actually deliver business value? Business value isn’t created by a soloist delivering a virtuoso performance, but a collaboration of the business, product, engineering and customer success teams working together to realise a shared vision. 

Below are the five ways this can take place; together, these provide a roadmap for delivering business value;

  • Define systems development strategy
  • Help business define requirements 
  • Visualise the work and prioritise
  • Schedule and communicate delivery
  • Deliver value often and get feedback

I will try to articulate through each of these one at a time and dig into a little more detail in Part 2 of this article.

Engineering Leadership in Start-Ups: Engineering Manager, Director, VP of Engineering.

Engineering Leadership in Start-Ups: Engineering Manager, Director, VP of Engineering.

This post is partly the result of my discussions with our People practice leader and talent acquisition executive. ITILITE is at a phase of growth, where are looking for more engineering & product management bandwidth. And I had to think hard to write the various Job-Descriptions. So, I have tried to generalise it using my experiences from the last 2-3 stints. In case you’re interested to explore an Engineering Management role with ITILITE, please get in touch with me or write to careers{at}itilite.com

Engineering Leadership

As apps are becoming increasingly omnipresent and in most cases, there is a startup behind them. Engineers make up to 70% of a tech startup’s workforce, there is an increasing need for managers who look after those developers. As a result, there is a rise in the number of engineering managers in recent years. Engineering managers are responsible for delivery teams that develop these “Apps”. The following is a very generalised version of what you could do in these roles and a possible career progression.

Engineer to Tech Lead/Lead Developer

The first step in your journey from an Individual Contributor(IC) to a management role. This could be a mix of people management, delivery management, process management etc, depending on the context of your organisation. In most organisations, it is a “technical mentorship” role with some aspects of people management, quality and delivery ownership.

Most Tech Leads are natural technical leaders. They are great engineers on their own, they were well respected by the engineers around them, they worked reasonably well with the team, they understood how the product/module was designed, built and shipped, they had a decent sense for making the right kinds of product tradeoffs and they were willing to do just enough project management and people development to keep the team/project humming along. 

In this role,

  • Most TLs would retain some independent deliverables in addition to anchoring and owning the deliveries of their team.
  • Most of the team still works on the same module/feature or sub-system
  • They do code & design reviews, suggest changes and have the final say for their modules.
  • Together with the Product Managers, they “own” the feature/module.

We at itilite, call them Engineering Owners, much like Product Owners

Tech Lead to Engineering Managers

The next step in the Engineering Manager. In this role, you will be “Managing” a collection of inter-related modules/projects. In this role, the focus on timely delivery, people management and quality are higher than technical design & architecture. But, you are very much an Engineer and may be required to occasionally write quick hacks, frameworks for your developers to build atop.

The main difference is you will be responsible for the delivery of multiple projects in a related area. You will be expected to optimise the resources (Devs, Testers, etc.) available with you to maximise the outputs of your group, across multiple projects/modules

In this role, you’d be

  • Expected to actively engage with the Product Management teams to define what needs to be built
  • Defining how you will measure the outcomes of what your team is building and quantify the outcomes with metrics
  • Ensuring quality, getting stakeholder alignment and signoffs
  • Macromanage the overall deliverables of your group

The Pivot – Tech, Product, Solution Architect

The next step in your career gives you two options. One with people management, P&L accountability and other a purely technical role. If you’re planning for a pureplay technical role, some organisations have Staff Engineer, Principal Engineer etc. In essence, they are mostly a combination of Tech Lead+Architect type roles. Depending on your seniority/tenure and organisational context, you may be reporting to an Engineering/Delivery Manager, Director/VP or the CTO. In this rolw,

  • You will work closely with Engineering Managers, Quality Assurance leads/managers and Product Owners to design the system architecture, define the performance baselines
  • You will work with Tech Leads and Sr.Devs to drive the performance, redundancy, scalability among other stuff.
  • You will be called into discussions/decide when the team can’t reach consensus on engineering choices

Engineering Manager to Director of Engineering

A Director of Engineering role is completely different. You now have multiple leads+managers, likely multiple projects within a general focus area of the organisation. This will mean there will be way more individual deliverables and project milestones than you can track in detail on a regular basis. Now you have to manage both people and projects “from the outside” rather than “from the inside”. You’ll likely start appreciating the metrics and dashboards, as they will help you in tracking those multiple projects and deadlines, schedules, overruns etc.

You have to make sure that your managers and leads are managing their resources appropriately and support them in their effort rather than managing individual contributors and projects directly.

Lots of great technical leaders have difficulty making this transition.

While being an engineering lead/manager is certainly managing, it’s type of managing from “within the project” is much easier than “managing from outside the project” and as a director, you almost always have to manage multiple people and projects “from the outside”.

Also, as a director, you will be responsible for a number of aspects of the culture, such us

  • What kind of people are you hiring, setting responsibilities and workload expectations,
  • What is the team(s) doing for fun, how do they interact with other functions
  • What kinds of performance is rewarded/encouraged vs punished/discouraged.

Now, moving to some serious responsibilities, you may be the first major line of responsibility for what to do when things does not work,

  • an employee not working out,
  • a project falling behind,
  • a project not meeting it’s objectives,
  • hiring not happening in time, etc…

While most of these things are the direct responsibility of the engineering manager, the engineering manager is usually not left to face these issues alone, they work on it with the director and the director is expected to guide the process to the right decision/outcome.

I’ve seen people who were great technical leaders and good engineering managers who did not enjoy being a director at all (or weren’t as good at it) because it was a whole different type of managing bordering the administration.

Director to Vice-President

The VP of Engineering is the executive responsible for all of engineering. Development, Quality, DevOps and partly to Security and Product Management as well. While both the engineering manager and director of engineering have managers who themselves have likely been engineering managers and directors before, the VP may work for the CEO (in an early stage Startup or a smaller company) who has never been a VP of Engineering before.

A large company may have multiple levels of VPs, but in most cases, you work for someone who hasn’t been a VP of Engineering or doesn’t actually know how to do your job. This means, there simply is no first-hand experience from your Manager, that you can rely on to solve your problem. The first time you step into that role and realize that, it’s a sobering thought. You’re a pretty much on your own to figure things out. Not only are you completely responsible for everything that happens in the engineering organization, but when things aren’t going right, there’s pretty much no help from anywhere else. You and your team have to figure it out by yourselves. Many successful VPs eventually come to like this autonomy, but it can be a big adjustment when moving from director to VP.

At the director level, you can always go to your VP for help and consulting on difficult issues and they can and should help you a lot. At the VP level, you may consult with the executive team or the CEO on some big decisions, but you’re more likely talking to them about larger tradeoffs that affect other parts of the company, not how you solve issues within your team.

As a VP, you are primarily responsible for setting up processes and procedures for your organization to make it productive:

  • Team/Project tools such as bug system, project tracking, source code management, versioning, build system, etc.
  • Defining/improving processes to track, monitor and report on projects.
  • Defining processes to deal with projects that run into trouble.
  • Hiring: How you hire? What kind of people do you hire? how do you maintain the quality of new hire?
  • Firing: When someone isn’t working out, how do you fix it: reassignment, training, performance plan, transfer, firing?
  • Training: How does your team get the training they might need, it could be hard-skills, soft-skills or managerial
  • Rewards: How do you reward your top individual contributors and for your top managers?

You may be part of the Leadership “Council” or participate regularly in business discussions that may or may not concern your department directly. In a startup, you are often “the” technical representative on exec staff. You help craft the strategy of the business. You are relied upon for technical direction of the company (sometimes with the help of a CTO).

As a VP, you are expected to understand many important aspects of other departments, what is important to other departments and how your department serves or interacts with or depends upon other departments. Two classic example might be,

  • Sales depending upon certain product features/capabilities being delivered in a given timeframe to be able to convert a prospect.
  • Customer success depending upon certain product fixes being delivered in a given timeframe.

As a VP, you will participate in the setting of these timeframes and balancing these against all the other things your department is being tasked to do.

As you can see, Engineering Management/Leadership is a very interesting career option. We have multiple opening across Product and Engineering functions at ITILITE. Please see if any of these roles interest you.

Building a Log-Management & Analytics Solution for Your StartUp

Building a Log-Management & Analytics Solution for Your StartUp

Building a Log-Management & Analytics Solution for Your StartUp

Background:

As described in an earlier post, I run the Engineering at an early stage #traveltech #startup called Itilite. So, one of my responsibility is to architect, build and manage the cloud infrastructure for the company. Even though I have had designed/built and maintained the cloud infrastructure in my previous roles, this one was really challenging and interesting. Due in part to the fact, that the organisation is a high growth #traveltech startup and hence,

  1. The architecture landscape is still evolving,
  2. Performance criteria for the previous month look like the minimum acceptable criteria the next
  3. The sheer volume of user-growth, growth of traffic-per-user
  4. Addition of partner inventories which increases the capacity by an order of magnitude

And several others. Somewhere down the lane, after the infrastructure, code-pipeline and CI is set-up, you reach a point where managing (read: trigger intervention, analysis, storage, archival, retention) logs across several set of infrastructure clusters like development/testing, staging and production becomes a bit of an overkill.

Enter Log Management & Analytics

Having worked up from a simple tail/multitail to Graylog-aggregation of 18 server logs, including App-servers, Database servers, API-endpoints and everything in between. But, as my honoured colleague (former) Mr.Naveen Venkat (CPO of Zarget) used to mention in my days with Zarget, There are no “Go-To” persons in a start-up. You “Go-Figure” yourself!

There is definitely no “One size fits all” solution and especially, in a Start-up environment, you are always running behind Features, Timelines or Customers (scope, timeline, or cost in conventional PMI model).

So, After some due research to account for the recent advances in Logstash and Beats. I narrowed down on the possible contenders that can power our little log management system. They are,

  1. ELK Stack  — Build it from scratch, but have flexibility.
  2. Graylog  — Out of the box functionality, but you may have to tune up individual components to suit your needs.
  3. Fluentd — Entirely new log-management paradigm, interesting and we explored it a bit.

(I did not consider anything exotic or involves us paying (in future) anything more than what we pay for it in first year. So, some great tools like splunk, nagios, logpacker, logrythm were not considered)

Evaluation Process:

I wrote an Ansible script to create a replica environment and pull in the necessary configurations. And used previously written load-test job to simulate a typical work hour. This configuration was used for each of the frameworks/tools considered.

I started experimenting with Graylog, due to familiarity with the tool. Configured it the best way, I felt appropriate at that point in time.

Slight setback:

However, the collector I had used (Sidecar with Filebeat) had a major problem in sending files over 255KB and the interval was less than 5 secs. And the packets that are to be sent to the Elasticsearch never made it. And the pile-up caused a major issue for application stability.

One of the main use-case for us is to ingest XML/JSON data from multiple sources. (We run a polynomial regression across multiple sources, and use the nth derivatives to do further business operations). Our architecture had accounted for several things, but by design, we used to hit momentary peaks in CPU utilisation for the “Merges”. And all of these were “NICE” loads.

When the daily logs you need to export is in upwards of 5GB for an app (JSON logs), add multiple APIs and some micro-services application logs, web-server, load-balancers, CI (Jenkins), database-query-log, bin-log, redis and … yes, you get the point?

(())Upon further investigation, The sidecar collector was actually not the culprit. Our architecture had accounted for several things, but by design, we used to hit momentary peaks in CPU utilisation for the “Merges”. And all of these were “NICE” loads! (in our defence) 

So, once the CPU hit 100% mark, sidecar started behaving very differently. But, ultimately fixed it with a patched version of sidecar and actually shifting to NXLog.

Experiment with the ELK is a different beast in itself, as provisioning and configuring took a lot more time than I was comfortable with. So, switched to AWS “Packaged Service” . We deployed the ES domain in AWS, fired up a couple of Kibana and Logstash instances and connected them (after what appeard to be forever), it was a charm. Was able to get all information required in Kibana. One down-side is that you need to plan the Elastic Search indices according to how your log sources will grow. For us, it was impractical.

Fluentd was an excellent platform for normalising your logs, but then it also depended on Kibana/ES for the ultimate analysis frontend.

So, finally we settled down to good old Graylog.

Advantages of Graylog

 The tool perfectly fit into our workflow and evolving environment:

  1. Graylog is a free & open-source software. — So we wont have pay now or in future.
  2. Its trigger actions and notifications are a good compliment to Graylog monitoring, just a bit deeper!
  3. With error stack traces received from Graylog, engineers understand the context of any issue in the source code. This saves time and efforts for debugging/troubleshooting and bug fixing.
  4. The tool has a powerful search syntax, so it is easy to find exactly what you are looking for, even if you have terabytes of log data. The search queries could be saved. For really complex scenarios, you could write an ElasticSearch query and save it in the dashboard as a function.
  5. Graylog offers an archiving functionality, so everything older than 30 days could be stored on slow storage and re-imported into Graylog when such a need appears (for example, when the dev team need to investigate a certain event from the past).
  6. Java, Python & Ruby applications could be easily connected with Graylog as there is an out-of-box library for this.

#logmanagement #analytics #startup #hustle #opensource #graylog #elk

Bitnami