(Michael Chinen)

I’ve been re-watching AMC’s epic series Halt and Catch Fire. As a historical drama covering computer and internet developments and an seizing opportunity with ambition in the 70s to the 90s, it was nostalgic and motivating to watch when it came out in 2014. It never really got that popular. It was sort of scoped like Mad Men, covering several years per season, with a much smaller budget, and there are some holes in the writing, including a few two-dimensional characters. Still, it fills a unique niche and has a solid fan base. It’s one of a few series I re-watch occasionally.

HACF breaks the main roles up into hardware/systems engineers, software ‘creative’ engineers, investors, and vision/product people that work together or against each other as leaders or founders of a company. When I watch shows with plausible tech or science experts, it’s fun to see how expertise is communicated to general audience. I’m not alone – there are many that track the technobable in Star Trek (e.g. inverse tachyon beam) or noted how Emmet Brown endearingly lost his science credit when he called it a ‘jigawatt’. There is also the non-sarcastic appreciation of fictitious displays of tech expertise and the realism of the futuristic interfaces in movies like Hackers. One thing I liked about HACF is despite using a decent amount of technobable, it plausibly captures the approach and spirit of hacking and coding, like reverse engineering a memoryless chip by rigging up a hex LED system to read out the values for each of the 65536 inputs to a ROM.

The expertise of the coders are demonstrated mainly by others admiring the structural complexity of their code as objects of beauty. This is something that feels extra nostalgic now. Some examples from real life: Donald Knuth wrote the Art of Computer Programming. The book I learned 3D programming from was called The Black Art of Macintosh Game Programming (there were many like this in the 90’s/early 00’s, and this was the mac version of the popular Andre LaMothe books). In 2008 when I joined Google Summer Of Code, they gave everyone a free O’Reilly book called Beautiful Code, which covered a lot of real world algorithms and problems that had ‘beauty’ in their code solutions.

I think this code mysticism was already a fading trend, but LLMs with their code generation have made coding seem a bit less magical because now I didn’t write it, or at least, I didn’t have to write it. Maybe getting rid of the woo is a good thing; maybe there’s a lot of ego behind appreciating code this way. But re-watching HACF made me think about some of the nice parts of banging your head against a wall and waking up with the solution that is just right for your constraints.

It’s also possible I’m not representative of others here. I should definitely ask some younger startup employees and grad students how they feel about coding. From my observation, people still appreciate good, elegant, clean, efficient code, and attribute expertise to the people that can produce this regularly and well. But it feels like the legendary 10x hackerman is slowly being made more approachable with ‘average’ coders being able to write better and faster by employing an LLM to fill in their gaps.

But the point I’m making is not about the hackerman rockstar, but about the worship of the code. It seems like a special kind of appreciation that is reserved for the arts. In most other science domains, expertise and feelings of admiration were generally attributed to the innovator, not the invention. Code takes on a life of it’s own. But now that something else is getting close to be able to create nice code, maybe that kills a part of the appreciation of the art. Here is a hot take that is certainly problematic: Once something is able to be mass produced by sufficiently skilled artisans, it is not artistically interesting to make more of that thing. The thing becomes a craft – it can still be difficult for you or me to produce from scratch, but with the right tools and knowledge, a large enough percentage of the population can do it.

LLMs are not quite there yet for coding tasks that are beyond leetcode. If you ask it to do complex in-domain things related to speech/music DSP (e.g. please synthesize a flute with ASDR envelope) or ML, it gets the outline, but fails on the details. To me, the leetcode problems are more difficult, but the LLM has seen enough of them to be able to have inductive bias. It’s also only able to do the local code suggestions at the single functional or class level – it starts to fail once you need a nice structure (e.g. using polymorphism gracefully in game entity objects/maps). The latter point – structural complexity of class/API/function is still out of the LLMs grasp. It’s closer to the aesthetic of (physical) architecture that combines form and function with consistency. Maybe it’s difficult due to context window restrictions, but I won’t be surprised if it requires something extra to be able to learn this particular aesthetic.

There is also a kind of beauty in low level, highly constrained programming and system design, such as embedded systems and low resource or highly reliable systems. This is an information theoretic-type of beauty that can be seen through an objective lens, with things like Kolmogorov complexity and reducing a solution to the smallest amount of object code. This is the type that is demonstrated a lot in the first season of HACF. I don’t think the LLMs are great at this either, since correctness and meeting precise resource requirements are one thing it struggles with. Since this can be made objective, it’s probably easier to approach for the LLMs, though I’m not sure how many people are focusing on this right now. For the past few years it’s been the case that more data trumps all, but I could see these kinds of things (and other things that require correctness and precision like math proofs) require some special focus.

Perhaps part of the fading mysticism that I perceive is that these two areas of code appreciation are at the opposite extremes and more of the actual problems and engineering are in the middle, at least for my career path. Less people need to roll out their entire stack from scratch on their own, and this is a good thing. The beauty still exists and is even more complex and interesting if you zoom out of an individual’s work to the team or organization. But the object is now missing that 70’s-90’s American individualism aspect that has been culturally ingrained in my generation. Again, maybe that is a good change. Maybe the mystery and appreciation of a relatively new frontier was part of a generation’s collective motivation for diving so deep into the matrix. I don’t know. I’m curious if this is just my own feeling or others notice this too. It’s entirely possible that this is just a way of coping with ‘losing to the machines’, or at least, losing ground to the machines. To be clear, I am fascinated by the research and application progress of ML and will continue to be. But I think it’s fine to be nostalgic about things too.