Closing the Knowledge Gap

If they were handing out Lamborghini’s for free, let’s say the Aventador, would you take one? I know that for myself, this would be a bit of a poser. I’ve seen enough lotteries where you could win dreamcars, but in the back of my head I would always admit that I would probably put it up for sale. Let’s be really honest here; I definitely would like to take it for a drive, but I don’t have any sports-driving skills, it’s way to big for most parking spots, and having to cross the 5 speedbumps in my neighbourhood just to get home would completely trash the spoilers. But the real long-term killer is the required budget for upkeep; maintenance costs alone would kill any chance of a decent holiday for the family. Even if we stretch reality a bit further and include free spare parts, you need specialized tools and knowledge, and you need the room to house it properly. In the Netherlands that last bit can be a real killer.

That all said, if I’m planning to do a lot of miles, nobody can deny that a Lamborghini can really take care of that! But the thing that really jumps into my mind about them is mr Clarkson of Top Gear fame, who enthasiastically explains how you can actually switch off most “protection” to get the best performance, and end up with a car that feels like it is actively trying to kill you. A bit dramatized, I admit, but he was definitely on edge about his personal safety in that car, while pushing it to its limits.

Now read on as I try to revert back to software and applications, while maintaining relevance to the Lamborghini introduction.

The lure of going full-service

Software is difficult to compare to cars, because the construction of every single car costs money, even when it is fully designed. With applications, you just produce a copy at virtually no additional cost. However, setting aside the costs of fixing any design or build flaws however, they both have costs associated with using and operating it, which is what I wanted to discuss. If you are following the Way of the Microservice, you will inevitably find all sorts of tools and languages on your path, free to use, but for which you don’t have the necessary skills. The obvious Aventador here is Apache Kafka: a piece of messaging middleware with impressive capabilities and performance, but at the same time so many configuration settings that, if your don’t know what you’re doing, your tweaking can just as easily hurt performance as improve it. Comparable situations come up if you try to go for frameworks like the Akka toolkit, Node.JS, and Spring, or programming languages such as Go, Kotlin, and Rust. You’re likely to have consultants running around advocating their personal favorites, or YUDders (Young Upcoming Developers) itching to try their hand at some sexy new technology.

For any company with a sizeable IT department taking care of operations, you will quickly run into all kinds of obstacles, and most of them having to do with the long-term view. Don’t forget, we come from an era where IT used to be classed with office-equipment; something you need, but want to optimize for as low as possible costs. So IT has been punished for anything that could be traced back to a non-working application, either because the programmers left a bug, the server “went down for a pastime”, the network lost connectivity, you name it. And what did we do about it? (Mind you, this is not purely IT’s fault, we hardly had a choice!) We went for bomb-proofing everything. Nothing changes without lots of people getting a say on it, standardizing on as few possible variations as possible, and externalizing risks. Especially the last one led to a huge market in IT services, where we took the hit of high costs, in exchange for being able to depend on specialists, and the point the finger elsewhere if something failed. It also allowed us to rely on external knowledge; we don’t need to know everything, we can just call in the experts.

So what do we do in this world of self-service Cloud providers? We go for the full-service deals: let the provider work out how to keep our infrastructure running, and reduce our knowledge gap to what their pre-digested services exposes to us. Again reducing choice and externalizing risks. We could even go with hybrid solutions, getting the software from vendor A, and running it in Cloud B. Say a BMW M5 in regular maintenance with a (brand-neutral) garage from a well-known chain. It’s still way more expensive than getting that free Aventador, but at least I don’t have to figure out how to care for it.

Meanwhile, back at the office

So the business got fed up with waiting for a full-service deal from IT, got into contact with their favorite vendor, and received a Kafka-based solution anyway. Hold-on, is that realistic? Yup. It happens. So do we need to roll over and just take it? Give in to any demand for hot new technology and let ‘them’ experience the pain we wanted to shield them from? Is the only alternative really “the SaaS-way or the highway”? What we need to realize is that the world has changed in more than one way. Not only do we have a new economy that favors agile players, we’re also seeing IT finally doing to itself what it has been doing to others, namely automate the hell out of the boring bits of work. The consequence is that what remains is not just ‘not boring’, it is damn-right challenging! We are confronted with a world where (eventually) practically all IT jobs will tax you to your limits, and that sentence does not continue with “before you can rest easy”. If we want to keep up, we need to acknowledge that we need to keep learning, experimenting, and trying out things nobody did before.

Hold on, that doesn’t preclude me from trying to make the IT infrastructure as safe as possible, right? Agility and automation requires standards to be able move quickly. How can I keep up if all my tools and platforms keep changing? Is this a Catch-22? I need a stable base to allow me to be agile, but being agile I want to be able to change my base as well. So what is happening is that the neatly layered structure of Business and IT is dissolving, and DevOps, no BusDevOps should be the goal. And if we accept that as a premise, we need to allow way more flexibility than we’re used to in IT, because transferring ownership to the teams should also transfer responsibility. There can still be a common base, but that base is just as much a moving target, due to the teams’ constantly changing requirements, as the business as a whole. Flexibility and agility makes IT a much more knowledge-intensive field of study. So you need to be able to say “Hell, yeah! I’ll take that Aventador!” Not because it is cool, but you want to see how far it can take you on your road. That requires study, but you should be used to that, and take it in stride. And, not burdened with a long-term Return-on-Investment, you can start small, just a few high-profile trips, before you start to find the best place for it, which might be in the garage, or even on display in the visitor area. And you will have learned a lot while working with it!

This entry was posted in Microservices, Software Architecture, Software Engineering, Tools. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s