Today we'll discuss a question I've heard a few times recently, “How can leaders measure if their expensive technology team is effective / efficient?

To some people, that's a fair question. Try to imagine you have a group of a few hundred software engineers working for you. One of your directs says that she requires another fifty engineers to continue to achieve her goals.

Are they doing their job well? How do you know? What if they're wasting time? What if they're ineffective? Should you just take their word for it that they're making good use of their existing team?

Now going back to the question, there are plenty of people (particularly individual contributors), who don't find it fair to ask if a team is effective.

They view the question demonstrates a lack of trust in their motivation or competence. They think attempts to measure their work output are ineffectual at best.

I'm going to walk through a few assumptions on the topic, and some of my own conclusions.

Let's talk about Creation vs Modification

If you want someone to build you a logo for your company, you can find hundreds of people willing to do it for $5+ dollars. They'll build you a perfectly fine logo. It won't be unique, but it will have all the image sizes you need, a social media kit, and so on.

It is cheap because the work product is easy to create. There are software kits online to build decent logos yourself, and there are websites which will even create perfectly fine logos for free.

However, these logos will be variations of existing logos. The logo you get will be customized to your company, but it won't be significantly different from other logos. It will be a modified logo of a previously created logo.

If you want a unique logo design, the pricing is massively different. The price will increase from $5 a logo to thousands of dollars. Why? Because to apply creativity to a problem takes time, brainpower, multiple attempts, and generally a more experienced and skilled worker.

That's the cost difference between an act of modification versus an act of creation.

What work can you measure?

If you're doing the same work repeatedly, it's likely measurable. In other words, you can tell if you did it faster or slower than expected. As a side note, when you do the same work repeatedly, it's usually cheap as well. When you can have an instruction book with the exact steps to follow, it becomes easy to find cheaper labor.

You can measure this repetitious work because you know how long it will take, you know the inputs, you know the outputs.

This is also true for very familiar work. If you are building a fourth login page for your website, you might know how long it will take. You'll have already built the same feature, you'll already know the pitfalls. It is an act of modification at this point, not creation.

Hopefully you won't be doing a lot of familiar work, because it indicates that you're wasting a lot of time working on things you've done before.

What about measuring acts of actual creation?

Building new technology, or creating new media, is making something new which hasn't existed before. It's the most expensive and most valuable activity to take. It's the difference between building the 79th version of your CD player, and building the first MP3 player.

How long would it take your company to design the 79th version of their CD player? If you've updated your CD player 78 times already, you probably have a good idea. Assuming the changes are incremental at each step.

How long will it take you to invent the first MP3 player? Hard to guess, right? The idea of being able to measure creation feels silly.

Before we go too far along this path, I want to say that there's value in being able to swag / estimate work. There's just a large difference between a rough guess, and a measurable work product.

I would guess that we could build a prototype in 4-6 months.” is very different from saying, “This will take 4 months.” One is setting expectations, one is setting a deadline.

My conclusion:

  • If technology workers are doing measurable work, they're doing easy work. They're doing repetitious work. This work isn't usually the most valuable work, and you should minimize it when possible.
  • If technology workers are adding huge value, such as building new things their company has never built (and perhaps the world hasn't built), then their work is hard to measure. They can swag, but not accurately predict their work.

But I haven't designed the first MP3 player.

Most of our work won't be as dramatic as designing the first MP3 player.

However, creative work is common for technology teams. Any time you need to explore, test, invent, or experiment, you are likely in creation mode.

"Build a chat feature for our mobile app, so that customers can request help."

These days, this doesn't sound like a groundbreaking invention. Plenty of apps have chat features. But if your company hasn't built this feature before, there is a massive amount of creation which needs to take place.

  1. Are you going to build a chat platform, or integrate an existing one?
  2. How will customers authenticate?
  3. Our support team will need notifications for when customers are requesting help. Need to somehow seamlessly merge that with the calls coming in.
  4. We'll need long term records of the chats, but we need to be clear with customers of our storage policies. Need to get legal involved.
  5. And so on.

What sounds like a plug and play assignment is rife with invention, testing, experiments, and the amount of time it will take is hard to guess.