Exploding the idea of a database

#

In a business application, the database is just about the most important and valuable component. It’s where your customers’ data is kept, and your customers’ data is your biggest asset and liability.

In a typical tiered architecture, the database sits at the bottom of everything. It is on the critical path of almost any interaction a user has with the system. Making it fast makes the system fast. Making it reliable makes the system reliable.

Conversely, when there’s a problem with your app, the database is probably in the middle of it. Doing anything of value requires wrangling these highly complex servers, usually via SQL. Of course databases would be a target for developers to try to improve things, as they are a huge part of our daily work.

It feels like there’s a lot going on right now in the database/data storage world.

There are tons of new types of database which tradeoff points in the design space:

For a long time I’ve been unhappy that SQL is the default interface to the database we use at work (MySQL). It’s a language designed for humans to write, not for machine-to-machine communication.

I recently learned about SQLite’s virtual machine: basically, a SQL query is “compiled” into a program in a bytecode language which includes both low-level primitives like loading and storing to registers, as well as high-level operations like creating a btree.

(Interestingly, limbo tests their bytecode generation against SQLite’s as they are trying to be SQLite compatible.)

This made me wonder if SQL could be replaced by a bytecode interface for database clients. Developers wouldn’t want to write bytecode directly, but if we’re already using ORMs to construct SQL expressions, which get parsed by the database back into bytecode anyway, why not skip out the middle-man and have the ORM product bytecode?

Why bother? I dunno, it just feels better than treating a business analytics language from the 70s as a compilation target. It’d be nice if I could bulk insert 1000 rows without having to prepare a statement containing 1000*num_columns placeholder question marks first.

For the forseeable future, we’re sticking with SQL. But I can dream.


Peace is the absence of fear

#

Ursula Franklin said “peace is not the absence of war, but the absence of fear”. The context, from an interview about the use of force in the pursuit of peace, is:

Peace, for me, is a consequence of justice. For that reason, peace is defined, not as the absence of war, but as the absence of fear. And for that reason I am convinced that means that are unjust or violent cannot, even for the best of reasons, provide peace.

https://peacemanagine.org/archive/v10n6p23.htm

This is a high bar. No enforced ceasefire, no polite society with deadly force in its grin, no economic end of history where war is merely too expensive to capital. Instead, justice is present, fear is absent, war naturally evaporated.

Identifying fear with the cause of conflict reframes war as an emotional response - not as a reasonable, rational, regrettable but sadly inevitable way humans tend to behave.

The Religious Society of Friends has since its founding been known for its peace testimony. It sits alongside several other denominations and movements known as “peace churches” which place particular emphasis on nonviolence and pacifism.

These churches generally concur that violence on behalf of nations and their governments is contrary to Christian morality, but agree that the teachings of Jesus were to explain the principles of the Kingdom of God rather than and contrasted with the ways of any earthly government.

https://en.wikipedia.org/wiki/Peace_churches

An example of this emphasis is in the Quaker approach to aid and development. From a Quaker Service Australia report on a project in Cambodia:

KCD has demonstrated how language skills training can promote inter-ethnic peace in other locations. A shared language is a critical prerequisite for promoting understanding between these diverse ethnic communities … This project provides ethnic Vietnamese children with the language skills they need to overcome many of their current disadvantages and provides opportunities for children from different ethnic groups to work and play together. … With the start of Khmer language classes, most Vietnamese children are learning Khmer for the first time.

https://www.qsa.org.au/wp-content/uploads/2023/09/QSA-Notes-June-2023-Peacebuilding-in-Cambodia.pdf

A focus on material needs is important, but a distinctly Quaker-flavored form of community development is to focus on the language barrier that allows fear and estrangement to survive.


The internet is a series of cables encircling the world

#

Sometimes, I get to tell someone who didn’t know that the internet depends critically on vast networks of subsea cables which connect all the earth’s continents.

https://www.submarinecablemap.com/

A world map with the landmasses in grey and coloured lines traversing the oceans connecting them all

This usually happens with people who don’t work in tech, but even those who are professional web developers learn this at some point. People are usually surprised, as if they had thought about it, the idea that the internet happens “in the cloud” often leads them to the impression that most of the internet’s communication happens “through the air” like radio.

I myself was surprised, when I looked into it, that we’ve been doing this since the 1850s:

https://en.wikipedia.org/wiki/Submarine_communications_cable#First_commercial_cables

Recently, I’ve started following an industry blog which tracks developments in subsea cables. This short recent post stood out to me as a very neat example that demonstrates how the internet gets built. Here’s most of this very short post, and one of the maps:

Microsoft has applied for several maritime usage licenses to do ship surveys of proposed routes for new subsea cables connecting Ireland to the UK. Its SOBR2 cable will connect Ireland to Wales as opposed to the usual Cornwall landings. It will apparently land at Malahide Beach and/or Portmarnock on the Irish side.

It is not clear to me whether the intent is to branch the cable or whether there are two landing options. Details are sparse on the cable itself. My educated guess would be a 96 pair unrepeatered cable because it minimizes capex and maximizes bandwidth punch with such systems easily handling a couple petabits per second.

The site survey will focus on the top three meters of the sea floor. It will take samples to ascertain the texture and composition of material with an eye towards a deep burial of the cable itself. The samples will help determine not only burial depth but also how well armoured the cable will be. The Irish Sea is notorious for fibre cuts due to fishing. The Sea is heavily trafficked.

https://subseacables.blogspot.com/2025/01/microsoft-planning-its-first-irish-sea.html

A survey map of the sea between Ireland and Wales, showing a red line connecting the two shores

This terse summary refers to so many things we don’t usually consider when we wonder how our internet traffic gets to servers in America or Europe:

Last year, I was lucky enough to visit Cornwall. I learned that some of the earliest transantlantic cables landed there - being, of course, one of the westernmost points of the English isles.

https://en.wikipedia.org/wiki/PK_Porthcurno

A photo of a seaside town of white-walled buildings nestled above a still blue cove. A cobbled path on a grassy hill winds down from the viewer towards the town. A stony pier juts out into the bay. Behind the town, more steep coastline and hills sweep into the distance.
Sennen Cove, near Land's End. This is not where the cables landed, but it's a nice photo.

This is behind the reference in the post above to the “usual Cornwall landings”.

My work in tech doesn’t usually need me to care about infrastructural details like this. But I really enjoy knowing a bit about how the world around me fits together, what it is made of.

The title of this post is a reference to “the internet is a series of tubes”.


A statement of intent for affordable housing in 2025

#

A big part of my life in the past year has been working towards creating more affordable housing in Sydney. As a layperson with no position of power, this isn’t an easy thing to just do. However, I find it such a worthwhile challenge that I’m going to keep doing it for as long as I can.

Andrew Burges Architects entry in NSW's Pattern Book Design Competition 2024

Housing in Australia

Australia faces a housing affordability crisis, especially in urban areas. There are many reasons for this, but I think the most pernicious is that Australians have largely chosen to treat housing as an investment. This implies that house prices must increase. Governments since the 90s have encouraged this with policy settings such as the combination of negative gearing and capital gains tax discount.

Our public and social housing sector has withered. It is seeing new investment under the current Labor government, but it will take a long time for it to recover.

As a software developer with a relatively high income, I am not eligible for public housing. That’s probably as it should be! But I don’t want my only other option to be playing the housing-as-investment game of snakes and property ladders.

I don’t want to participate in a system which I see as poorly-designed to suit the human needs of security, community, sustainability and affordability. The examples are rife, but here’s just one study which describes the shortcomings of the developer-led approach to housing.

An improvement: rental co-operatives

I’ve come to believe that the best way to deliver on the needs of the middle of the housing market is via European-style limited-equity co-operatives. Under this model of long-term secure rental:

(I’m leaving out a lot of detail here.)

In some European cities, this democratic not-for-profit housing sector provides as much as 20% of dwellings. This is not without significant government support, of course… but our current housing system also benefits hugely from government support!

There is growing awareness of and interest in this model in Australia. Have a look at:

While I don’t think a co-operative sector is a panacea for the many issues we have, I strongly believe it would be a huge improvement. At its best, a co-operative housing sector can deliver quality affordable housing to people on a range of incomes, not only those who are relatively well-off.

Sydney Cohousing

At the start of 2024, trying to work out how to go about creating a La Borda-style development in Sydney, I found and joined Sydney Cohousing.

We are a small group of Sydneysiders who want to promote the ideas of the co-housing movement in Sydney. We want more opportunities to live in places where we know our neighbours, where people of diverse backgrounds, of all ages, of different levels of income can live side-by-side.

Our main project right now is creating a co-operative development of affordable co-housing in urban Sydney. It’s going to be a big challenge; we are blazing a trail that is new to Australia. All these separate elements exist here already, but not all in a single project.

In November, I was elected to be the secretary and president of Sydney Cohousing. (No congratulations are necessary; we’re a small association, and the six of us on the committee pretty much share the work.)

In 2025, I hope that we can make significant progress on our goals:

I think Australians deserve better than what’s on the market right now, and I’m so keen to be part of making that happen.

La Borda residents meeting in the building's common area, Architects Journal 2020