There is an idea which recurs in many forms in the tech industry, which I summarise as: “where we’re going, we don’t need people”.
It is the idea that things would run better if we could remove people from the process.
That human foibles and frailty are an impediment to some objective like efficiency or productivity.
Here are some examples of how this idea appears in specific tech projects:
Cryptocurrencies aimed to remove the need for trust from the financial system.
If transactions could take place with no trust between parties, commerce could be entirely anonymous.
Trust in human relationships is replaced by verifiable, mathematically provable trust in the system itself.
This is an extreme version of cash transactions.
David Graeber described in Debt: the first 5,000 years how debt relationships need trust, but trade/barter transactions would take place between people who could not trust each other.
For example, barter could be a ritual exchange between people groups who traveled and did not usually encounter each other.
Cash can enable trust-less transactions within a legal system that enforces its role as legal tender.
Crypto aims to invent an entirely governance-independent system of tender, and suggests this would be better than a system that relies even on trust in government.
This is the ideology of technique: we should replace trust in humans with trust in automation.
Large language models and AI in general promise the ability to replace humans in contexts like therapy, companionship, even the kind of questions and conversations you might rely on a trusted friend for.
Often this is a well-intentioned appeal to scale: how can we possibly afford so much human time and attention that’s required for everyone’s needs?
Teachers, for example, can’t possibly be expected to attend to the needs of so many students, but AI can be fully attentive to every student.
This is a way to sidestep the problem: that societies undervalue education, companionship, the caring professions of social reproduction.
Society refuses to fund one teacher per five students, preferring instead one teacher per fifty students, augmented with AI to increase their productivity.
This is the ideology of scale: we should demand the maximum output with as few humans as possible.
The metaverse and other digital world concepts propose to replace physical interactions with digitally mediated experiences.
Often this comes hand-in-hand with the idea that these experiences will be completely personalised and self-centred, tailored to each individual’s proclivities and weaknesses.
This reached its height during the worst of the COVID pandemic, with Facebook’s renaming to Meta and Apple’s launch of consumer VR wearables.
The message was that we would all spend our days in virtual meetings and social events, every interaction with another mediated through the cloud.
The vision never really manifested, but it has continued simmering in the background as tech companies continue to work on improved devices, now further integrated with LLMs.
This is the ideology of transcendence: that intelligence should be liberated from physical limits of any kind.
Venture capital demands returns on capital invested, and recent iterations of this culture since the 2000s have focused on speed of growth.
The firm Andreesen Horowitz summarises Uber’s end-run arund laws like so: “Uber won its battles, city after city, because they became essential to people so quickly that their social license with everyday citizens overcame the political and regulatory pressures to stop it.” (emphasis mine)
This world view sees society itself as a barrier to individual achievement.
This is straightforwardly libertarian, but combined with the economic imperative to win at all costs to achieve the greatest return, it acquires an urgency that makes it actively rapacious.
This is the ideology of speed: we must not wait for deliberative processes that consider humans’ needs.
Fernando Borretti, who writes more eloquently than me, describes something similar in a recent post:
We see this with ordinary consumer products and services, where often the advertising seems premised on you being some kind of stylite who lives in a little human terrarium, probably in a building called something like “The Soho”, food arrives through a hole in the wall, and you have this sovereign, solipsistic existence that leaves no physical trace in the world.
I am forever trying to explain, to people I know who aren’t in tech, why I am such a big fan of the Oxide Computer Company.
I keep talking about Oxide because the way they work, and the way they present their work, really resonates with me.
This post is meant to summarise those ways.
What Oxide does:
Oxide sells computers for use in data centres. These are large, powerful, expensive machines used by organisations that have a serious need to process data, and do it on infrastructure they control.
They design and produce a huge vertical slice of the whole computer stack in-house. If you’re familiar with traditional desktop computers, you might say they design the motherboard and case, among other bits. They don’t make CPUs or memory chips, but they design all the parts that connect the CPU to storage, memory, power, networking etc.
Their philosophy is to provide value through this vertical integration: to get more efficiency from parts and systems designed to work together, to better support their customers because most problems can be solved in-house rather than waiting on vendors, to offer an Apple-like experience to everyone from the data centre operator all the way up to the software developers.
Working in public:
Oxide produces a lot of material that they share publicly, to such a great extent that it’s actually possible and valuable to learn a lot from following them.
They podcast regularly with the core team and other employees, friends and acquaintances, and tech industry figures. This ranges from deep dives into their own tech, bugs they’ve fixed, to tech current affairs, to just having fun and talking about baseball.
They make some of their internal documents public, especially when they’re of interest to the broader tech world, or when they want to talk about something on a podcast and provide some internal material as supplemental reading.
They open-source a large amount of the software that runs their stack, and consequently end up having issue tracker and pull request discussions in public also.
Commitment to open-source:
Oxide publishes a lot of their work under permissive licenses. This ranges from helper libraries and experimental side-projects to the core services that run their entire stack, to contributing to their dependencies like IllumOS. They can do this because they’re not selling you just the software, they’re selling you an extremely valuable and powerful box full of CPUs that is animated by the software.
Open source is a classic way to “work in public”, and being able to see issue and pull request discussions from the team is interesting in its own right, and supports other things they publish. Being able to link to issues from a podcast or blog post makes the entire body of work richer.
Their commitment to making the software available to their customers is another way they live out their values. They’re saying that, as an Oxide customer, you get to know exactly what’s running in your own data centre.
A writing and reading culture:
Oxide talk proudly about writing a lot. They also publish a small but significant portion of what they write. Their public RFDs are really interesting to read, whether they’re generally applicable, or deeply specific to their own work.
They have a very literate attitude, one that seems to come all the way from the top down. They read a lot, and discuss books they’re read, and recommend books. They seem to have an interest in history - their own history and what they can learn from it, the history of software and tech, of places.
Their hiring process reflects this, by asking for long-form written answers as well as real work samples. This is an occasion where you can observe them living their culture out into all their practises in a way that’s unusual and refreshing.
A serious approach to work:
Oxide’s work is serious business, at a fairly low level in the computing stack. There’s some amount of prestige associated with that, some amount of envy from me, a mere web developer. It means they have the obligation to get things right, to not accept “good enough”, to root-cause their issues instead of work around them. That rigor isn’t economically viable for all businesses, but it’s critical for Oxide. This makes their transparency all the more interesting; it feels like outsiders rarely get a glimpse into this world.
They want to take responsibility and ownership. They often refer to being able to provide “Oxide value” by owning as much of their software and hardware as possible: instead of passing the buck to a vendor when there’s an issue, they can actually say the buck stops with them. Their product strategy is built around this.
They put their values first, and try to tie other practises to those values. They say that writing down commitments publicly is an important part of accountability, which I heartily agree with. It’s easy to be cynical about this, and to criticise when they maybe don’t meet these values (commonly seen regarding their hiring). But I still find it so worthwhile that they can be seen trying.
For most of my career I’ve been fairly independent, working in a loosely-structured research organisation or at my own startup.
I haven’t usually taken the few mentorship opportunities that were available to me.
Being able to listen to, read and appreciate all Oxide’s output feels like a window into a culture driven by opinionated veterans.
A little bit of what I’ve missed out on, taking the path that I have.
Much of what Oxide does wouldn’t make sense for a less well-resourced company.
Or in another industry, or for a team with different personalities.
But it’s inspiring to see that someone is out there doing it.
Karl Popper described science - I think convincingly - like this:
A theory cannot be proven true, only disproven by experiment.
Therefore, falsifiability is a critical feature of scientific theories. A theory needs to make predictions that could be proven false by observation.
It follows that when testing a theory, focus on how you might prove it wrong.
If you fail to disprove it, then you might tentatively accept the theory as the best explanation we have so far!
And that acceptance will lead to more theories which we can then try to falsify.
He described this process as one of alternating cycles of “conjecture” (coming up with a theory) and “refutation” (attempting to falsify or criticise the theory).
I was introduced to this idea by David Deutsch’s book The Beginning of Infinity, where he uses the phrase “conjecture and criticism”, which alliterates nicely.
I prefer the more colloquial “guess and check”.
I think this cycle also describes common practises involved in developing software.
Writing the source code of, say, a sorting function is a conjecture: “I guess this source code correctly implements the quicksort algorithm”.
Writing a test case is one way of “refuting”, “criticising”, or checking your conjectural source code. We usually supply “happy path” examples which, while not constituting “proof”, do provide some evidence. If these attempts to falsify your conjecture fail, you might be happy to proceed, depending on how complicated the original code is.
Property-based testing is a way to automatically generate a lot of criticism, provided you have a way of judging the output. The properties themselves are further conjectures, for example of the form “I guess this source code checks that all elements of a list are smaller than their successor”. We depend on these properties being easier to state in code than the function they’re checking!
Proofs and model checking approach the problem from a different direction. Additional conjectures such as “this proof correctly models the problem of sorting” or “this model accurately reflects the system I am building”, combined with “this source code corresponds accurately to this proof statement or model component” can be subject to their own criticism. A proof checker, itself correctly implemented, supplies criticism of a proof text.
I’ve described writing both the original code, and the tests or properties or proofs, as conjectural. In my view these approaches to the same underlying “explanation”, from different angles, buttress each other - acting as potential refutations of each other’s conjecture.
Different modes of expression - source code, examples, properties, proofs - give our minds different affordances to express and explain ourselves. Where these different expressions intersect and either agree or disagree, we get to understand the mistakes we made in each expression.
It’s by challenging ourselves that we can be more confident that our theories, our guesses, about what our code is doing. As Popper translated Xenophanes:
As for certain truth, no one has known it … and even if by chance he were to utter the perfect truth, he would himself not know it; for all is but a woven web of guesses.
A Pattern Language (APL) is one of my favourite books.
When I first read it, I was surpriseds that it explained a bunch of things that I already found beautiful in buildings.
It’s also an early example of a hypermedia text which can be read non-linearly, like an archipelago of islands connected by bridges.
But one of its most grandiose pretensions, which I really enjoy, is that it takes seriously that architecture is part of society.
How seriously does it take this?
Here’s the first pattern in the book:
1. Independent Regions**
Metropolitan regions will not come to balance until each one is small and autonomous enough to be an independent sphere of culture. … Therefore: wherever possible, work toward the evolution of independent regions in the worId; each with a population between 2 and 10 million; each with its own natural and geographic boundaries; each with its own economy; each one autonomous and self-governing; each with a seat in a world government, without the intervening power of larger states or countries.
(The ** after the pattern name indicates the authors believe they have discovered a truly fundamental and necessary pattern.)
Want good buildings? Start by reorganising the geopolitical framework of the entire world. Maybe it’s no wonder the pattern language proved impractical.
The book proceeds downards in scale from there, through the layout of settlements within a region and their relationship to the countryside (3. City Country Fingers), the arrangement of neighbourhoods (8. Mosaic of Subcultures) and their connections (16. Web of Public Transportation), streets (49. Looped Local Roads), blocks (38. Row Houses), buildings (107. Wings of Light), rooms (142. Sequence of Sitting Spaces), all the way on down to the details of the windowsills. Pattern 240. Half-Inch Trim** (another double star), lays out the problem thus:
Totalitarian, machine buildings do not require trim because they are precise enough to do without. But they buy their precision at a dreadful price: by killing the possibility of freedom in the building plan.
Along the way, the book detours into aged care (40. Old People Everywhere), industrial relations (80. Self-Governing Workshops And Offices), public service (149. Reception Welcomes You), religion (66. Holy Ground) and nightlife (63. Dancing In The Street). It addresses construction (218. Wall Membranes) and wildlife (74. Animals).
It’s an impossibly broad scope, and I love it.
It acknowledges and confronts the interconnectedness of building - how even the smallest project will be influenced by these larger scales.
I wish APL’s imitators had tried to mimic even a fraction of this ambition.
To software:
The core idea of the pattern language which can be used to generate infinitely many works that share common structural qualities was adopted in the software world, to the extent that the most famous of the book’s authors, Christopher Alexander, was invited to speak at the OOPSLA conference in 1996 to a crowd of object-oriented programmers.
The idea of design patterns has inspired many programming works.
The most famous is probably the “gang of four” patterns book from 1994.
Another is Kent Beck’s “Implementation Patterns” from 2007.
Yet another (rarely discussed as an entry in the “pattern” genre), is the original Domain-Driven Design “blue book” by Eric Evans in 2004.
The book explicitly models its chapters after APL’s pattern structure, with a problem statement and solution.
But no work in software that I’m aware of tries to imitate the sheer dynamic range of APL.
The GOF book and Implementation Patterns are concerned with the smallest details of code (functions and classes) within a single process.
Microsoft or AWS’s “cloud design patterns” resources address the integration of multiple processes.
The DDD book looks at the question of desinging an entire software system. It crosses over into the “rest of the business” with ideas like the universal language, and its discussions of how to work with experts to understand a domain. But it also focuses quite deep down into actual code and syntax.
I’m not aware of a software book about organisational structure (e.g. Team Topologies) that is pattern-oriented.
No single pattern language could encompass the massive variety of software development - and despite APL’s grandeur, the authors don’t pretend it is the only pattern language that should exist for architecture.
But fundamentally I believe that the economic environment in which code is written dramatically affects the right low-level syntax that should evolve, so explicitly weaving thogether those highest and lowest levels would be fascinating.
Just one example: Alex Russell’s struggle against careless organisational slop in his Reckoning series really reminds me of Alexander et al’s fight with bureaucracy in The Oregon Experiment.
A pattern language for web development should address how to generate healthy relationships between the users, the development team, management, and other stakeholders, as well as how not to get Alex to write an entire case study about you.
In software, we get plenty of advice about how to make beautiful windowsills.
But we don’t realise how connected that is to the whole world order.
We talk about “abstractions” a lot in software development.
Abstraction can sometimes mean “a way to hide details that aren’t relevant to helping me achieve my goals”.
This is what we mean by “abstracting away” an implementation.
Sometimes, per Dijkstra, an abstraction might mean “not to be vague, but to create a new semantic level in which one can be absolutely precise”.
This tantalising idea feels like doing real intellectual work, the kind involved in discovering things like twos-complement integers or Prolog.
Sometimes (and this is what I want to write about), abstraction means “an entity which isn’t tangible, but is nevertheless real and significant”.
This is the “abstract noun” sense of the word.
This is the sense David Deutsch used in a chapter of The Beginning of Infinity titled “The Reality of Abstractions”.
He writes:
In regard to what is or is not real, this leads to the requirement that, if an entity is referred to by our best explanation in the relevant field, we must regard it as really existing.
He intends here to counter the reductionist idea in science that the most low-level or fundamental phenomena (like quantum mechanics) are the only things that are “real” or significant.
He gives this example, comparing “high-level” explanations with a hypothetical reductionist explanation:
Consider one particular copper atom at the tip of the nose of the statue of Sir Winston Churchill that stands in Parliament Square in London. Let me try to explain why that copper atom is there. It is because Churchill served as prime minister in the House of Commons nearby; and because his ideas and leadership contributed to the Allied victory in the Second World War; and because it is customary to honour such people by putting up statues of them; and because bronze, a traditional material for such statues, contains copper, and so on. Thus we explain a low-level physical observation - the presence of a copper atom at a particular location - through extremely high-level theories about emergent phenomena such as ideas, leadership, war and tradition.
Presumably a reductive ‘theory of everything’ would in principle make a low-level prediction of the probability that such a statue will exist, given the condition of (say) the solar system at some earlier date. … But such descriptions and predictions would explain nothing. They would merely describe the trajectory that each copper atom followed from the copper mine, through the smelter and the sculptor’s studio and so on … In fact such a prediction would have to refer to atoms all over the planet, engaged in the complex motion we call the Second World War, among other things.
This might seem obvious? The reductionist approach to science isn’t one we encounter in our daily lives, and addressing it isn’t usually necessary.
But I keep thinking about the idea that “if an entity is referred to by our best explanation … we must regard it as really existing” and how it crops up when doing software design.
I don’t yet know exactly how to turn this vague idea into something graspable. It just keeps feeling relevant.
This often comes up when trying to understand a new part of our business domain at work.
A concept that initially sounded like a simple relationship between two entities turns out to have a whole life of its own, full of complexity we previously elided.
Two concepts that initially seemed similar turn out to have different underlying needs, and diverge.
Database tables need careful thought to walk the line between “what do we know for sure?” and “what do we need to leave room for in the future?”
It also influences how I think about so-called “natural keys” in databases.
Natiral keys encourage thinking of a database row as transparent, nothing more than a reflection of some external, more fundamental reality.
On the contrary, oir “best explanation” of software behaviour often included the existence (or absence) of a specific row in a database.
It deserves its own identity.
In some education about data-oriented design, a “data reductionism” can arise which sounds a lot like this:
Data is all we have. Data is what we need to transform in order to create a user experience. Data is what we load when we open a document. Data is the graphics on the screen, the pulses from the buttons on your gamepad, the cause of your speakers producing waves in the air … Abstraction heavy paradigms try to pretend the computer and its data do not exist at every turn, abstracting away the idea that there are bytes, or CPU pipelines, or other hardware features, and instead bringing the model of the problem into the program.
It’s necessary to climb down off the top of the object-oriented ladder of abstraction sometimes.
But I think the right outcome is to realise that what is needed is not no abstractions, but different ones.
Abstract notions like “quaternion” still exist, even if you are handling a packed table of floats instead of a tangled graph of pointers.
As the problem of performance moved from minimising CPU cycles to maximising cache utilisation, the correct abstraction to address a concrete need may have changed.
But retreating to “it’s all bytes!” is as insufficient as saying “what a peculiar arrangement of copper atoms you have there”.
*I use Fastmail’s “snooze” instead of making a task somewhere else.
Fernando wonders why so few apps have inboxes that work with this algorithm:
This is the utility of inbox zero: it has no false negatives! If the inbox is empty, I know that all of my correspondence has been handled. If the inbox is non-empty, I know there is work to do.
I have been finding that an RSS reader is a great way to treat the web like an inbox.
Prompted by the Australian government tinkering with zany news media regulations a few years ago, I wrote about how RSS is a great way to consume the news (or, it would be, if news organisations wanted it to be):
Instead of being at the mercy of “the algorithm” implemented by various social media sites, I get to act out the “inbox algorithm”:
Pick the site I’m in the mood for
Skim the headlines
If an article is short and I want to read it immediately, read it
If I can’t read it immediately, send it to Raindrop to read later
Mark all articles in the site as read
Newsblur, which I have been using for many years, makes this more or less the default experience.
Though, I understand some RSS apps are more focused on a “feed”-like experience.
Hearing this is what made me realise that Newsblur was letting me treat the web like an inbox, not like a feed.
One commenter on Hacker News describes the reason for feeds as:
the whole problem with chronological feeds, including RSS - chronological feeds incentivises spam-posting, posters compete on quantity to get attention. That’s one of the main reasons fb and other sites implemented algorithmic feeds in the first place
That is a problem with chronological feeds that mix content from all sources into a single view.
But any good RSS app will break down posts by source.
When confronted with a high-volume source with hundreds of unread posts, I used to skim a few posts at a time then just mark the whole site as read.
No point lying to myself about my ability to get through it all.
Other people use rating and intelligence systems to curate their own RSS feeds.
Newsblur has this kind of functionality built in, but I haven’t tried to use it.
These days, I just don’t tend to subscribe to high-volume sites via RSS (aggregators, whether user-contributed or traditional news organisations).