From the Desk of
ZCS floral monogram

Information Expands in a Repeating Pattern

I show you a picture of two people and tell you they are parents. Think for a moment how you would explain the effort it would take to answer the question “are they your parents?” It’s probably not super important information in your daily life now, but the identity of your parents was once so vital that your brain forever retains well-worn grooves to retrieve it precisely, instantaneously, and effortlessly.

There’s no effort, you just know.

The names of your parents is a corpus of information. It’s fixed, it’s finite, it’s interrelated— so in this case the parents that I have are not changing,1 I’m not getting any new parents,2 and my parents are related through their common relationship to me.

Lots of corpuses are simply too small and/or important to threaten being forgotten— the date of your birth, the names of your siblings,3 common spelling rules like i before e, except after c— memorialization outside of your head is unnecessary to the point of irony.4 Of course not everything that starts small stays small forever.

Imagine you’re a kid whose name begins with the letter Z, it’s natural that your knowledge of the alphabet might begin with the letter Z, as well. And it doesn’t stop there, you add in the vowels, and the common consonants, eventually make it to the letter Q, and even as the letters feel like something you just know… think for a moment now about how you would explain what happens in your head when I ask you which letter is before the letter G in the alphabet?

Chances are good you needed to sing the alphabet song to yourself mentally, and that’s what it means to use a list to organize an expanding corpus of information. Sometimes lists are compressed with a pneumonic like Roy G Biv, sometimes lists are compressed with rules like 32 NFL teams break into 2 conferences each with 4 divisions comprised of 4 teams, sometimes they’re arrayed visually in our memories like using a map of the United States to remember the names of all 50 states.

Lists are the general shape of corpuses that can be divided into a logical structure it’s easy to remember (like the NFL teams), and they formed the original structure of the Internet with directories like Yahoo where you clicked into nested sets of lists until you found individual web sites. Eventually, if a corpus keeps growing, lists crack and fail when nesting gets too deep or when boundaries become too difficult to guess.

And surprisingly it doesn’t take a ton of growth for lists to breakdown— take for example the 12 apostles of Jesus— and when they do, lucky as we are to be living in the 21st century, we’re going running for search tools.

When you know something exists but you don’t know exactly what it is, search works great because you can write a query and you can linearly evaluate the results until one matches your expectation. You can search for “Italian” on DoorDash, search for upcoming birthdays on your calendar, search for the “answer to the ultimate question of life the universe and everything” on Google and find a widget that curtly explains the cultural significance of Douglas Adams.5

Corpuses of information start6 small enough to know, they grow for a while fitting into a list, eventually (and unpredictably!) they benefit from search tools, until —if a corpus expands long enough— the only reasonable way to organize it is with a stream.

Maybe you know of stream structures as “algorithmic feeds”. They’re what happens when the size and heterogeneity of a corpus has grown so large that the probability of your being able to imagine and explain— as a search query— the member of the corpus that best fits your needs has shrunk below the ability of a machine learning model to guess what suits you. After all, the model has seen every single thing ever made and you— well, you’re still young, there’s plenty more to see and do!

Streams seem new and confined to Internet Scale applications— maybe you’re imagining TikTok, YouTube, Spotify, Facebook, Threads7— but if you think from the perspective of students all across the globe and throughout time, education is a stream where the student doesn’t know what they don’t know and puts trust in teachers, institutions, curriculums to guide their development.

On the subject of education, let’s turn to a practical example of the way corpuses expand from know to list to search to stream— the music industry.

It’s indisputable that in 1877 Thomas Edison had the world’s greatest record collection. That is of course because he invented recorded music that year; his collection was both the world’s largest (one), the most complete (all), and included the very first record (“Mary Had a Little Lamb”). Asking him for the name of every recorded song was, for a time, as onerous as asking you the names of your parents.

Like a few of Thomas Edison’s inventions, recorded music proved to be very, very popular. The corpus of recorded music grew and grew until more than a century later I was a kid in the 1990s buying CDs by flipping through lists of albums organized by genre into alphabetized bins delineated with white plastic tabs for each artist. Richard Branson’s great contribution to music was the development of the Mega Store where you could reasonably expect to find the CD you wanted listed among those in stock any time you went into Virgin Music.

iTunes introduced search to recorded music, and as much as it felt like wow this changes everything by dint of delivering us from flipping through those dusty bins of CDs, the transition from selling whole albums stuffed with filler tracks to just the hit songs at 99￠ a pop but a huge sad kink in the music industry’s revenue curve.

Popular though the idea that the music industry peaked before the transition from CDs to downloads is, the streaming era of music has created so much demand for music that the industry recorded record revenue from recorded music last year. It turns out being able to connect people to music they love without requiring them to know what to search for is a really big deal.

Sometimes corpuses grow so slowly that it seems like not a lot is changing, but when that growth happens continuously over years, decades, and centuries —in the case of recorded music— the impact is surprisingly straightforward to measure.

Thomas Edison invented recorded music— and amazingly, a bunch of other world changing things which should contribute to how small this next number reads to you!— and his wealth is estimated to be about $200 million (inflation adjusted to 2025 dollars) at the time of his death. Richard Branson has led dozens of world famous Virgin-brand businesses in the decades since he made his fame with the Virgin Megastores— again think BIG numbers— and Forbes estimates his wealth today at $2 billion. Finally we get to Daniel Ek8 and his Forbes estimated wealth of $9 billion. It’s not a clean 10X growth between eras, and it leaves out the search era of music which is harder to fit into this example of the pattern, but it’s striking to see the scale of each era reflected in the value created by these businessmen.

This essay has been percolating in my mind —this pattern has been a go to talk track of mine— since 2021. It occurred to me in the course of imagining how software distribution can evolve beyond the App Store model pioneered by Apple in 2008.

There’s just amazingly little new software adoption, despite computing’s near total domination of our waking hours. Why becomes obvious when you observe the steps involved and their orthogonality to the skills needed to produce software. Distribution is the lion’s share of the work involved in commercial software, and it’s dizzying to me to imagine the chilling effect explicit and implicit understandings of that dynamic have on the development of new tools.

Today, LLMs make writing software a lot easier. In just the early innings of this revolution thus far, anecdotally in my life it feels like the number of people who are using LLMs to write code now stands at roughly the proportion of people who did not code at all before. It’s a flipping enormous increase that is hard to analogize, and the degree to which it’s just getting started cannot be over emphasized.

Inevitably there’s going to be A LOT MORE software in the future than the past or present. LLMs have blown up the cost of entry into software development with the force and thoroughness of a nuclear bomb. So far, people are tinkering and testing, and I can assuredly assert they will not be satisfied to stop there. They’re going to start publishing …if only we give them a place to share their wares.

It’s hard to imagine the scale of new software we’re on the cusp of getting for a few reasons—

The antiquated legal regimes that bind the music publishing industry create essentially insurmountable barriers to the commercial viability of remixed music, and that’s limited the impact of streaming on that medium.

Quote tweets exploded in popularity on Twitter9 because the base tweet offers an intellectual scaffolding for new ideas to expand on the existing discourse. TikTok masterfully extended remixes into their format by letting video creators lift audio from other videos to combine with new content.

Critically, software is far more combinatorial than any other medium of information. A streaming software distribution system unlocks combining tools at the function layer, offering a menu of features that’s geometrically larger than that which is commonly exposed as an API. This will have profound implications for productivity far beyond huge efficiency improvements— we should expect that re-composable functional software will unlock radically new ways of thinking about the commercial viability of software tools— and the variety of new ways of creating and using software are certain to surprise and delight even the most jaded among us.

Consider the 2022 meme that roared through the halls of conservative thought10 on Twitter: The Current Thing11. Small people were very grumpy some folks were using emojis in their Twitter names to express support for the fundamental cause of self-determination, in that case specifically it was yellow and blue shapes for the colors of the Ukrainian flag. Other people suffixed their Twitter names with “e/acc” to express their frustration with the lack of death and dismemberment happening alongside contemporary industrial activity, themselves essentially identifying as emotional luddites wanting to return to a Victorian Era level of ignorance to the precautions that can be taken to safely and reliably get things made.

Absurd though it is on its face, believe me when I tell you I am convinced our near term future of digital environments is one where our computers will intuitively understand which types of virtue peacocking we align with so that it can filter out the rest. With a function that can determine the alignment of a social media account, and a function that can find a social media account for a person, and a function that can find the person behind a piece of content, it’s straightforward to imagine how a speculative computer could combine these functions to filter your feeds into something less rage inducing.

And though it wasn’t a huge burden for me to delete my X dot com account once it was clear that the site would be mismanaged to amplify hateful ideologies, I understand that the interaction between my value system and the allocation of my attention is complex and personal12. I certainly would have preferred carrying on as normal because my computer was natively filtering away content from hateful people, producing a stream of posts prepared for me constrained by the things I care about —versus the goals platform owners may have for the propagation of various dubious and long-ago disproven perspectives.

For social media, the future looks like delivering on the promise of the work of thoughtful pioneers like Tracy Chou and Molly White, with federated rafts of shared values that are enriched by the valuable work of the intrepid among us with the fortitude (and often morbid curiosity) to invest attention in the observation of the abhorrent. Like a Block Party for the entire web that could, for instance, filter out opinions on nutritional supplements from the perspectives of cryptography currency grifters. And be used for silly yet valuable things like filtering out Yelp reviews of restaurants in my town from people who are visiting from simpler places.13

Streaming software distribution unlocks the power of our computers to develop an increasingly sophisticated understanding of how to serve our needs as individuals, perhaps finally delivering on the promise of personal computers that are not merely person-sized14 but are personalized to support the work each of us contribute to the unfolding corpus that is the history of humankind.

KNOW → LIST → SEARCH → STREAM

Footnotes

  1. Frustrating as that often can be to contemplate ↩

  2. Observing the popularity of religion informs how desirous such a thing could be… ↩

  3. or if you’re Chinese and of a certain age, perhaps the names of your cousins ↩

  4. and expressed stupidity, good god I just get entranced by the people who get information like this tattooed on their bodies like come on it's tattooed on your brain is it not ↩

  5. which is better than Claude Sonnet 3.7 which simply responds “42.” :eye-roll: ↩

  6. Everything starts the same. Prior to the beginning of me, the list of my parents was empty. Then, for a while, the list of my friends (excluding my parents) was empty. And, though some may not believe this, for a while, the list of my enemies was empty. The destiny of a corpus may be determined or dynamic, but the starting point is axiomatic. ↩

  7. explicitly excluding the “algorithmic” feed of X dot com here because it is not tuned to deliver what you most want but rather what most serves the dispicable political goals of its ownership ↩

  8. founder and CEO of Spotify ↩

  9. before its untimely demise in 2022 ↩

  10. such that it is lol, truly the description is apt for the gargantuan conservation of thinking invested into it ↩

  11. In sum, it was a backlash of the incurious against the invested, driven by the former’s Big Feelings of jealousy towards the latter for, ya know, caring about things. It was typified by reactionary opposition of the former group to the latter group’s demonstrating support for Ukraine, fueled by the Russian puppets who have wholly co-opted control of the American Right’s political zeitgeist. ↩

  12. though I would certainly be remiss to fail to invite you to believe your attention is valuable and you stand to gain from considering where you invest it— and unambiguously: do not spend time where Nazis are welcome ↩

  13. I understand you never wait in line for food in Tuscaloosa, I do not want your opinion of a restaurant here factored into recommendations for my dinners ↩

  14. versus room-sized, as were the previous generation of computers from the 1970s ↩