Isolating complexity

Friday before I left work, I felt like I'd done very little valuable that day. I can console myself with the knowledge that I did my best, and that I'll pick things up another day. I helped those I work with, progressed some tasks, but ultimately spent a lot of the day thinking and planning, hitting roadblocks, and feeling a little useless.

Working on a greenfield effort, where some of my suggestions were deemed not possible; I've aimed to take a machete to the jungles of complexity in the software we work on. I am striving for a few things, which to my view are basic. Whilst a violent analogy this doesn't come from a place of malice. Perhaps despair, awe, futility. In writing things down, I hope to find a path.

A Distant Spark of Light

I often do not live very much in the present when I'm upset. Without an immediate, often physical reason for discomfort, I resign myself to working towards a future-perfect, or even improved scenario.

It's hard to imagine now, but I used to really like going out and running. I Had a simple method, I'd set a destination up in my head, I'd leave the house, warm up, and then run to the next landmark until I reached my destination.

By doing a lot of small things, you'd be surprised the distance you can cover.

For me, the distant light right now is that we have less complexity, and I want to get us to a place of even less. Where complexity must live it will be isolated.

Why I Want These Things

I've had these things for years; they've benefitted me, they have given benefit to those I've worked for. Sadly, much of my efforts to spread these have been met with derision, pomp and puff.

People who do not need nice things, are remarkably good at watching as others around them struggle. They are remarkably good at making themselves struggle. People are remarkably good at knowing better, and picking petty flaws in the language, rather than the ideas. Efforts to adjust to this take opportunity for another attempt.

I'm not perfect. While efforts against my advice have floundered, I've a track record of remaining relatively technically care-free, coming in and fixing things after things go wrong. I've simple rules, I understand them, and I try my very best to explain them.

I wasn't a passenger; I did offer to help. Hubris and condescension are not my intent, but it was what I've received from people, no-doubt more competent, no-doubt more intelligent, no-doubt better perhaps, viewed through their own lenses.

Simple Goals

I know that true simplicity will likely lead to unhappiness in the team. If I take an inch each day, or week, or month, I can make this happen.

• Smaller, atomic data-structures
• Trade-offs for speed to incur storage cost
• Rejection of AIO framework solutions
• Natural keys where possible
• Progressive enhancement
• Handing non-engineering tasks back to the business
• Pushing parts of the system out to shared libraries
• Deferring flourishes

The Hold-Up

I've already had to move on several of these. Pushing uploads out of the core-presentation foundation seemed unfamiliar to everyone but me.

Even if you run Instagram, even if uploads are on the critical path to how a user interacts with a system, coupling them to the presentation is a recipe for disaster. My attempts to explain what seemed obvious to me had failed. I'll try again here.

Software Should Be

• simple
• small
• predictable
• focused
• have measurable usage & impact

I do not believe the above should be controversial, if applied pragmatically. Uploads and storage of complex data are tasks which take more time than storage of simple and normalised value chains.

Towards A De-Coupled Future

De-coupling means a simple value-chain can complete faster, which means you may serve more people. Perhaps some of that time, even if you don't need it today, could be put towards a rainy-day fund for when you do need a little extra

If you won't let me bank that time now, you won't have it later!

De-coupling also makes it easier to change where you store and retrieve data. I won't cover it too much here in the interests of brevity, but if the only way to load and save data is via file, or network, you're waiting for resources you cannot fully, or easily control.

True Story

Other than partitioning, less time tied up transmitting things through the main application, and predictable storage, what benefit(s) does your plan have?

I only wish that were not something said to me. I might be paraphrasing, but those were the accepted points. This was then supported by the CTO about tested, existing code, which had already been written.

More upsetting is that the focus was not on logical flaws, or something difficult to reason about.

People insisted they understood, but seemed more worried about shielding my ego, ensuring feelings were not hurt, inventing stories not on a backlog or part of any discussion or future. I am more concerned with the waste; than the problems they admitted they could see.

What also upset me was that non-reliance on a single database, via an application, traded for predictable storage paths, fan-out computing was reportedly understood, but not accepted as a benefit.

Something else which upset me was that apparently... blog-posts are a metric of software quality now; as-are AIO frameworks, popularity, feelings, deliberately rushing towards something you admit is likely going to be a source of problems.

The last thing that upsets me is that I am willing to be wrong and have explained to me why I am wrong so that I do not repeat mistakes. Not receiving this makes these situations 100% waste.

Dealing with Not Achieving the Basic

I've dealt with this my entire software career. I don't have a formal background; although I love math, believe in the empiricist scientific method, I'm more of a whizz with the basics than an aficionado of the frontier and fine cuisine.

I know this has been as-much of a benefit as a drawback. Those at the top, only see things from a niche perspective. I'm incredibly good at working with older people, people with dyslexia, autism; I've had a lot of success with ESL and non-native English speakers.

I Also take notes on situations. That is apparently odd. I find it makes discussions about improvement easier to ground, the empirically measurable.

• initial intent
• execution
• objections
• understood flaws
• outcomes
• my behaviour

The last might seem odd, but I accept fully that I can be at fault. I respect others that do this. It's not encouraged nearly enough, and is incredibly difficult, especially when you don't have pressure to change externally to do the right thing.

Why Continue?

When our great ancestors first crawled, slithered, writhed out of the ooze, their life was objectively much worse than mine.

• We've no evidence of them having tools
• There were very little protections
• Most died out, producing something mildly better or worse
• They could not understand for a very long time

I'm okay with things not being perfect. In-fact if they got perfect today, I'd find new things to want tomorrow. I encourage this type of optimistic nihilism. They who constantly strives towards progress, not for some vain continuation of the species, but to move forwards, not feel trapped, in despair, hopeless or helpless.