For the last 8 months, I’ve been working on a distributed vector database designed to run on commodity hardware—old laptops, standard PCs, you name it. Going in, I thought I’d get maybe 1,000 to 2,000 vector ingestions per second, and about the same for reads.
To my absolute surprise, I hit around 10k max ingestions and 8k reads when handling 64-dimensional vectors. Even when handling heavy 1536-dimensional vectors (the standard size for modern LLM embeddings), it still pushed around 2k writes and 2k reads on a dusty Intel i3 7th Gen U-series processor. Keep in mind, this was happening while 2 worker nodes, 3 placement drivers, an API gateway, and an auth module were all running simultaneously on that exact same machine.
I know this isn’t a textbook, perfectly isolated distributed systems benchmark, but that’s entirely beside the point. The point is: look at how incredibly capable modern hardware actually is.
So the next logical question is: if the hardware is this powerful, why does everything feel so incredibly slow and laggy?
It’s a valid question. But the hard truth is that it’s not the hardware failing us. It’s the software.
Just look around. If you spin up a React project with a single component that says “Hello World,” it’s somehow pushing 300KBs. If you try to make a completely blank Android app using React Native, you’re looking at a 12MB payload. We literally went to the moon with 4KB of RAM, and now a desktop Calculator app takes 5MB just to boot in a best-case scenario.
We treat our devices like infinite magic boxes. An old, discarded smartphone sitting in a drawer right now has more raw compute, memory, and networking capability than the servers that ran the early internet. Yet, if you try to open a modern web dashboard on that same phone, it will choke and stutter.
For a long time, the industry assumed hardware would just keep getting faster indefinitely. The mindset was that it didn’t matter if the software got heavy and slow—we just had to move fast, ship features, and get users hooked. Sadly, this is no longer true.
When Moore’s Law began to hit its physical limits, hardware manufacturers pivoted. Instead of giving us massive leaps in single-core speed, they started giving us more cores—2, 4, 6, 8, and now even 16 cores on standard consumer machines. But here is the tragedy: a massive chunk of modern software completely ignores this. Because of how we build things today, most application logic remains stubbornly single-threaded. We are running software that only knows how to use one lane on an eight-lane highway, and we wonder why there’s a traffic jam.
The Rise of the Garbage Collector
In the first three or four decades of computer science, compiled languages dominated. The assumption was simple: if we write code in a human-readable format, it must be compiled down to machine language so we can actually see where the bottlenecks are, what instructions the CPU is executing, and how we can improve them. It gave us predictability.
But compiling took time, and the languages were unforgiving. You had to manage your own memory, which felt a lot like walking around with a gun pointed at your foot, just waiting for the trigger to slip.
Garbage-collected languages have been around since the dawn of computer science, but their mainstream appeal was undeniable: don’t manage your memory. Let the garbage collector handle allocation and release so you can focus entirely on business logic. It’s a brilliant idea in theory, but automating memory management turns out to be incredibly hard. We’ve been at it for decades, and we are still figuring it out. Java made massive strides in this regard, but it brought its own massive overheads.
Then came the “revolutionary” idea: what if we make dynamic languages where you don’t even have to declare types? The runtime will just figure it out on the fly and execute instructions line by line.
In the 1990s and 2000s, people thought computation was going to get so cheap that efficiency wouldn’t matter. The attitude shifted to: it doesn’t matter what the code is written in, just write it and move on. Business shouldn’t stop. This attitude gave rise to Node, Ruby on Rails, Python, and the rest. People flocked to these ecosystems because you can move incredibly fast. But the hidden tax is that their execution is fundamentally slower, and their architectures (like Python’s GIL or Node’s single-threaded event loop) actively fight against the multi-core hardware reality we live in.
Just for perspective, I ran an HTTP server benchmark across different languages on that same low-power i3 7th Gen U-series chip.
- C++: 20,000 requests per second.
- Node.js: 12,000 req/sec.
- Python: 6,500 req/sec.
- Ruby: 4,000 req/sec.
Now, you’ve probably heard the saying, “A bad workman blames his tools.” That benchmark isn’t a perfectly optimized, world-class representation of Node’s absolute theoretical limit, nor am I a C++ master. But the baseline capability of native, compiled code is just fundamentally higher.
We’ve traded mechanical sympathy for developer ergonomics. And the craziest part? We didn’t even get better ergonomics. We just got node_modules folders the size of black holes and dependency chains so deep that nobody actually knows what code is executing on their machine anymore.
The Cultural Rot of Corporate Engineering
The technological evolution is one thing, but the path we have taken as an industry culture is what really hurts. People no longer care about software. They aren’t here for the love of the game. They saw that tech pays well, thought “I’m smart, I should go into software and earn easy money,” and they did.
There’s nothing inherently wrong with wanting to get paid, but somewhere along the line, these people became the majority. They are the loudest voices in the room now, and they have little to no regard for the craft. They care about results, and only results.
If a task takes 5 seconds to execute, and with just a little bit of thought they could optimize it to 1 second, they won’t do it unless a higher-up explicitly pressures them for optimizations. These people are perfect corporate players. They understood the corporate game, and in the process, they murdered software engineering.
Who am I to judge? Sometimes I feel like I am one of them. But I am actively trying to change that. I am trying to respect the craft. The more time I spend with computers, the more I love programming. It is basically the power of a god—the ability to create worlds from pure logic.
The “Clean Code” Disease
Currently, our industry is plagued by a disease called “Clean Code.” There is an obsession that code must be perfectly beautiful, hyper-readable, and deeply abstracted.
Why?
I want you to try a small experiment. Pick any trivial problem. Solve it using strict “Clean Code” principles. Abstract it, make it beautiful, go all in, make it perfect.
Then, in a second attempt, start with a blank slate. Try logically solving the problem using common sense. Account for function call overhead, memory allocation, loop complexity, and data locality. Strip away the layers and just simplify the raw logic.
I guarantee that the second version will be twice as fast as the “perfect” code of your dreams.
To put that in perspective, look at historical hardware trends. When Moore’s Law was in full effect, hardware performance roughly doubled every two years. If your heavy, beautifully abstracted code runs 16x slower than a common-sense approach (which is highly common in modern web backends), you have mathematically erased 4 generations of hardware progress. Simply wanting to make your code “look good” in an editor just cost you 8 years of hardware development.
I refuse to do that.
Modern software developers like to pretend they are artists, that the text on the screen is the art, and it must be beautiful. No. The art is the logic you come up with. The art is the ingenuity of your brain devising a seemingly simple architecture for a problem that has three layers of hidden complexity.
Instead of building lean, fast interfaces—like a snappy terminal application where every keystroke is instantly registered—we build bloated web apps that have to serialize JSON, send it over HTTP, deserialize it, parse megabytes of JavaScript, build a virtual DOM, diff it, and paint pixels just to toggle a single checkbox.
The true appreciation of this art is the feedback from the user. It’s the joy they feel when they use the software. It is not your team lead leaving a comment on your code-writing style in a PR. An artist who looks for shortcuts and self-pleasure at the expense of the quality of the final piece is not an artist. In my eyes, it’s a sin, yet today it is somehow accepted and glorified.
You only truly feel the magic of the job when your code finishes a task in 1 second that a customer expected to wait minutes for. In an era where a single click on a browser can take 5 seconds just to load simple data, a genuinely fast application feels like witchcraft.
Developers today don’t even want to learn the fundamentals. Pick a random dev and ask them what is faster: $O(n)$ or $O(n \log n)$. They will instantly say $O(n)$. That is partially correct, but the true answer is that $O(n)$ scales faster; it is not inherently faster in a vacuum. There are massive constants and hardware realities affecting how fast code executes. Many times, you will use a seemingly “worse” time complexity because you know the input is small enough that the constant overhead of the “better” algorithm will destroy performance. If this weren’t true, we would just run Radix sort (which is $O(n)$) all day long and never bother with Merge, Quick, or Insertion sort at all.
To get back on track, we don’t need drastic, industry-breaking changes. We just need to be conscious of the code we write. Stop programming blindly and defensively.
AI, Slop, and a Glimmer of Hope
I am actually getting a bit optimistic lately, mostly due to the rise of languages like Rust, Zig, and Go. People are finally talking about performance again, and I believe we are looking forward to better days in software quality.
My only worry is that we are going to blindly follow AI and ignore the actual code being written. AI is a great tool, but it is trained on massive codebases of average code. Therefore, it will output the logic of an average programmer. And as we established, the average programmer does not care about the quality of the software or the craft; they are mostly there for the money.
AI coding assistants don’t natively want to write highly optimized, data-oriented code. They write the most statistically probable code, which means they confidently spit out the exact same abstracted, object-heavy boilerplate that caused this mess in the first place. If we fall for the marketing hype of the AI labs and let it write all our abstractions, waiting 10 seconds for a button click will be considered normal again.
Not to mention the absolute slop mega-corporations are pushing right now. Windows has become an absolute show (I use Linux, by the way). Look at the murder of GitHub. I love GitHub. I visit it daily, and it aches my heart to see its uptime hovering around 85% over the last 90 days, plagued by hideous bugs. And this is just the tip of the iceberg. Software everywhere is degrading because companies are prioritizing shipping useless new features nobody wants instead of improving robustness and quality.
If you want to understand what we’ve lost and how to get it back, I highly recommend watching Casey Muratori and Jonathan Blow speak on this topic.
Please, let’s try to make better software. Respect the craft.