Part of “Letters from the Lab”, a series of informal essays on my research written for patrons. You can also listen to this essay (24 minutes).

The boundary of a year is arbitrary, sure. But that doesn’t stop rituals from yielding wisdom and warmth. I begin each year by reflecting on what I’ve learned in the previous one. In the spirit of last year’s essay, I’d like to share a few lessons which others might find meaningful.

Suffering and creative work

Writing a book is a horrible, exhausting struggle, like a long bout with some painful illness. One would never undertake such a thing if one were not driven on by some demon whom one can neither resist nor understand.

—George Orwell, Why I Write

When you’re thinking about something that you don’t understand, you have a terrible, uncomfortable feeling called confusion. It’s a very difficult and unhappy business. And so most of the time you’re rather unhappy, actually, with this confusion. You can’t penetrate this thing. Now, is the confusion’s because we’re all some kind of apes that are kind of stupid working against this, trying to figure out how to put the two sticks together to reach the banana and we can’t quite make it, the idea? And I get this feeling all the time that I’m an ape trying to put two sticks together, so I always feel stupid. Once in a while, though, the sticks go together on me and I reach the banana.

—Richard Feynman, 1963 interview

Writers and artists of all kinds share a pervasive trope with scientists: creative work is immensely satisfying, but the moment to moment experience of producing it can be extraordinarily unpleasant. You’re lost in a blizzard, searching for something just outside your grasp, constantly feeling stupid and inadequate. The day ends—and look at the scraps you have to show for it. Days turn into weeks; little of the work seems usable. Yet something tantalizes. So you chain yourself to the desk. You press on against the freezing wind. And when you finally reach the finish line, a transcendent creative joy redeems all that suffering.

I don’t think it has to be this way.

Until this year, I hadn’t taken seriously the possibility of escaping what Orwell called a “horrible, exhausting struggle”. The trope is too ubiquitous. I’m sure some creatives have already engineered their getaway, but at least among my writer and researcher friends, conversation regularly turns to commiseration over this kind of pervasive pain. I think we take the suffering for granted much too readily. I think we can develop a relationship to creative work in which the doing in each moment is joyful, irrespective of the outcome.

I don’t want to get your hopes up. This isn’t going to be a complete guide to that kind of enlightenment. I certainly don’t have it all figured out myself. But at least in my experience, the high order bit has been to stop taking this suffering as a given, and to interrogate it instead.

Why is the feeling of confusion uncomfortable? Why specifically do I feel pain when I spend all day poking at a problem without feeling any real progress? What am I worried would happen if I just let myself fall into the problem’s contours, to let it take as long as it takes, whether or not I find an answer? For me, when I dig far enough, the answers usually bottom out in social anxieties of various kinds. Fear of not achieving “enough”, of appearing incapable, foolish, slow, unoriginal; these feelings arise in turn from deeper fears of not “belonging”, not being accepted by others.

You may have different answers. Some friends have found that their creative suffering stems from a sort of self-flagellation: they’ve convinced themselves that creative success is just a matter of will, and so if they fail to make progress, that’s a sign of their own essential weakness as an individual. In this scheme, creative troubles produce self-loathing rather than compassion and curiosity. I have another friend whose creative suffering stems in part from a hyperactive fear of running out of money, following some family financial traumas.

At least for me, and for the friends I’ve discussed, these fears arise from deeply internalized—but false—beliefs. It’s worth identifying them and rooting them out. I can’t prescribe a solution here, just the journey. It is possible to rewire your emotional system so that the creative experience does not cause suffering. Consult your local therapist, meditation guide, executive coach, psychedelic dealer, etc, or several in combination.

Abating creative suffering has been one part of the solution for me; the second is its mirror: cultivating creative joy. It’s possible to draw much more satisfaction from the moment-to-moment experiences of creative work. I found this tough to do initially because I’d trained myself to draw satisfaction in my work from outputs, achievements, and others’ approval. In that scheme, when I face a setback, or I spend a day exploring without making apparent progress, that means postponing gratification. Writing workshops, art classes, and research memoirs ritualistically emphasize the benefits of focusing on “process over product.” I’d only understood that in a purely practical sense: when you focus on product in creative work, you’ll often produce worse work. But the adage also applies to the emotional experience. The outcome can’t be the daily reward. It’s too far away, too uncertain. That’s a brittle source of satisfaction. The process has to feel rewarding, too.

Happily, I’ve found that the moment-to-moment experience of my work is quite rewarding once I get out of my own way: the pleasure of following a trail of curiosity; the serendipity of making little connections; the surprise of noticing I’m confused in an unexpected way; the satisfaction of choosing a better word to sharpen my understanding. It’s surprisingly engrossing to watch the gears of my own mind turn.

Advice about “productivity” and “motivation” often rests on an adversarial foundation. The notional goal is to focus better, work harder, achieve more. And the obstacle is you. Your will is just too weak. If you were only more organized, less lazy—you’d reach the stars! Remove distractions; bind yourself to the mast; keep up your streak; set measurable goals; up and to the right. Such techniques tend to assume that the work is a bitter pill to be swallowed, that your impulses are distortions to be overridden. The focus is on enduring the present in service of some distant future. I’m not going to pretend these techniques aren’t useful. We really do have monkey minds which benefit from taming. We really do inappropriately discount long-term rewards. But the adversarial framing is missing something important. Too often “productivity” and “motivation” techniques treat a symptom without addressing its cause. I don’t want to train myself to shout over my impulses, especially when my work depends on following creative instinct. I’d rather cultivate a healthier relationship to the sensations of doing the work, in the moment, so that my impulses will naturally serve me well.

Don’t let me give you the wrong impression. My work is not some kind of daily enlightened bliss. But I’ve made surprisingly rapid progress here this year, and I sense that much more is possible. The trope of the anguished creative seems to be a tragic, self-induced mirage. This illusion can be overcome, and that tractability may be the most important thing I learned this year.

Cultivating a better “tools for thought” scene

One key reason I work independently is that my work, “tools for thought”, doesn’t really have a natural home in an academic field or industry niche. But tools for thought do have a scene, of sorts.

The good news here is that the scene looks enormously better now than it did five years ago. I see much more ambient interest in novel and enabling user interfaces now than I’ve ever seen before. We have more part-time tinkerers; and even more importantly, we have a larger stable of serious people publishing exemplary work.

But it’s still an awfully anemic scene. There are many cheerleaders but very few full-timers producing very few powerful ideas. My instinct is that there are many more transformative ideas hiding just out of reach. It’s hard not to feel that we could move much faster, do much more.

So what’s holding us back? Here are a few bottlenecks I can see.

Money: the most obvious constraint. Few people have the funding needed to support this kind of work even temporarily, much less sustainably. What models might apply?

• Projects like Jupyter and Scratch sustain themselves with grants from philanthropic foundations and corporations, but it seems to me that these grants apply mostly to projects which have already spent years traversing the difficult stages of creative conception.
• In theory, national grant-making agencies should be well-suited to funding those exploratory early stages, but in practice, ambitious systems work is discouraged in the corresponding academic field (human-computer interaction).
• I’ve had some modest success with crowdfunding, but I worry it may not replicate well: others who have attempted this path seem not to have had as much luck, at least so far. I’ll give an update on my crowdfunding experiment later in this essay.
• Small grants from low-overhead programs like Emergent Ventures are a wonderful development, but I fear it would be difficult to cobble together enough of these to make substantive progress on a research direction. And then what? Perhaps small grants like these could bridge people to larger philanthropic funding models. More experimentation here would be good.
• Though modern venture funding is generally a poor fit (a startup’s fundamental drive is growth), perhaps a less steroidal business model would work? Mathematica and VisiCalc and Photoshop are all success stories here, in various senses. The main concern I have is: how much ongoing fundamental exploration can you get done once you turn these things into a business? This might be a way of capturing value from an initial discovery phase, but it’s not obviously a good way of funding an ongoing effort of invention. Mathematica has fared moderately well in this respect, but its success seems quite rare.
• I feel better about the approach Ink & Switch is bravely attempting with Muse. Theirs is a translational model in which spin-out projects might fund a separate “parent” lab’s open-ended explorations. This method is too early to judge, but I’m excited to learn from their experiences.

The intense quantity of money suffusing the tech sector slices both ways. If you can make significant progress on a “tools for thought” research project, you can probably also earn many hundreds of thousands of dollars per year as an employee, or raise a seed round on excellent terms. “Vanilla” tech aside, there are AI and crypto gold rushes going on, and those fields have plenty of interesting problems; why not participate? On the other hand, this cash-flush environment means that a huge number of people can readily accumulate the capital to work on whatever they’d like. There could be so many more “gentle(wo)man scholars” coming out of the tech industry. Of course this is not an ideal choice for funding research, but I do think it’s one of the more promising paths in the short term. I’d like to encourage more people to consider this option. If you’re excited about pivoting into original research, I encourage you to take the rivers of cash seriously. Do the projections yourself. The details vary enormously with your circumstances and lifestyle, but you may be able to buy your creative freedom for a few years’ labor.

Skills. We’re overweight on engineering and underweight on design and theory-building.

One problem seems to be that if you’re an engineer, you can actually build and iterate on a system yourself. Your system may not contain any interesting interface ideas, but you can certainly make things! By contrast, if you’re a designer or synthesist without engineering skills, you can sketch novel concepts for the future of computing. The trouble is that you can’t iterate on an interactive system very far without actually interacting with it. Design and theory-building are necessary but not sufficient.

So to make progress in this space, we need either individual engineer-designer-theorist hybrids, or teams of people with complementary skills. Both situations are somewhat rare. It’s tough for a single person to build strong skills in both engineering and design, because they’re both deep fields, and design in particular usually requires apprenticeship. Teams are rare because finding someone willing to work on weird, unprofitable research projects is hard enough; finding two people willing to simultaneously work on the same one involves multiplying low probabilities.

That said, I believe it’s possible to jump-start more engineer–designer dyads. My sense is that there are many engineers who are very interested in this problem space, but who harbor no delusions about doing the design or theory work themselves. If we could arrange small grants for thoughtful designers, perhaps they could develop concepts far enough that we could matchmake an eager technologist to partner with them for prototyping and iteration.

Part of the trouble here is cultural. Representative framings like “augmenting human cognition” easily attract many engineers but sound awfully Spockian to many designers I know. The situation improves if we start talking about transformative environments for creativity or expression or consciousness. Another problem, as Joe Edelman has pointed out to me, is that these various augmentations are most often discussed in terms of the relationship between an individual and their computer, rather than the relationships between people, which might involve computers. Individualistic framings often resonate poorly with design’s collectivist cultural leanings. But there’s plenty of opportunity in augmenting collective intelligence and creativity. Projects along those lines may attract a broader coalition.

Development. There aren’t enough pathways for legitimate peripheral participation, mentorship, and skill-building. This is most obviously true and perhaps most pressing for people just beginning their journey, but I feel it myself too. I think the best way to work on this problem is by funding grad-student-like apprenticeships, but these will only make sense once we improve the broader funding sustainability problems I’ve described above. Until then, we’ll be widening the top of a pipeline with gaping holes in its middle.

Teamwork. We lack exemplars and models for doing this kind of work in teams—it’s mostly loners. That limits our reach, but it also excludes the substantial majority of people who strongly prefer to work in teams, or as part of a larger institution. Ink & Switch is the best modern team exemplar we’ve got, and the scene would benefit from a written discussion of its operating model. That said, I notice that they’re now mostly focused on technical infrastructure for interfaces, rather than the interfaces themselves. I suspect the latter requires different models. I’m not sure what the right next step is here. Grants for more dyads, as I described in the section on skills, seem like a good place to start.

Campus. We have no faculty lounge or Hamming-style lunch tables, no workshops. No regular context for spontaneous deep discussion of ongoing work, at both the project level and the field level. Most of us don’t even have a consistent context for design crits. Twitter is a poor substitute. There are some enjoyable podcasts and show-and-tell series, but these venues are focused on sharing ideas, rather than criticizing and generating them. A literal campus may not be the right solution, but it’s at least evocative of what we’re missing: an environment which supports collective honing and pollination. With a relatively small amount of funding, I think a good place to start would be a small in-person summit focused on depth. More outlandishly, I’d love to find a physical space in San Francisco to host regular events and co-working sessions.

Accelerating research with execution-oriented teammates

Last year I wrote about an important practical problem in developing new tools for thought: we can only really understand novel interface ideas by using them in authentic contexts, but real-world software systems take a lot of work to build. Worse: because it’s quite expensive to switch back and forth between “research mindset” and “engineering mindset”, a single researcher will tend to see only the forest or the trees, for weeks or months at a time. This makes it tough for one person to build momentum while iterating on research systems.

So this year, with some extra funding generously provided by a few kind donors, I experimented with hiring interns and contractors to help me with execution-oriented work. Some of these collaborations are still ongoing, but I’d like to share a few early insights about how team structures in tools for thought might work best.

The biggest problem with research systems is that uncertainties mount quickly and intensely—much more so than for normal software development work. In a more traditional product development setting, you often understand the limitations and opportunities of your system well enough to curate a robust roadmap of features to build, experiments to run, problems to fix. The team will need guidance and feedback as they’re building new functionality, sure. But at least for some projects, you can specify the work well enough up front that teams can move for weeks at a time without blocking on feedback.

Experimental software development doesn’t really work like this. Sometimes you’ll understand some sub-project well enough that you can specify a few weeks of execution work, but there’s no guarantee that the “incoming” rate of well-defined projects will match the “outgoing” rate of projects completed. In practice, I found that I could often specify only a couple days worth of effort in a particular direction—and sometimes that articulation would require half a day or more of preparation! Figuring out what to do is often the most time-intensive step of these projects. In many cases, I could only see what to do next after the last small increment was completed. Tight feedback loops like these meant that I’d often end up blocking others. And because I hate being the blocker in team situations, I’d often switch away from my ostensible focus to help plan others’ projects a few steps further.

One solution would be to delegate not only execution but also pieces of the core creative problem-solving work to teammates. I’d love that, of course, but now we’re talking about a very different job description, and a much rarer candidate. With a lot of mentorship and attention, I’m confident that more people could develop these skills. But that’s a long road, and much more demanding than hiring a mainstream programmer to help me execute. This path would be more like mentoring a graduate student. We should expect it to take several years, and it’s hard to see how to make that plausible economically.

An alternative I’m trying right now is to hire contractors intermittently and for shorter periods, whenever some specific work is well-specified enough for a bit of execution-oriented attention. This introduces transactional overhead and loss of contextual continuity, but at least it focuses the effort where it’s probably highest-leverage. Fixed-scope engagements minimize the priority inversions which can occur when teammates’ work queues dry up.

I like a metaphor Adam Wiggins has used to describe a related model. Film production typically begins with a “pre-production” phase involving just a few creative staff, figuring out what the movie actually is. Then when you think you’ve got enough unknowns pinned down, you hire hundreds of people for a “production” phase and make it happen. Months later, almost everyone parts ways until the next project gets under way.

All the way at the other end of the spectrum, I’ve had wonderful experiences in creative partnerships with peers capable of outstanding autonomous work. Those relationships are precious and rare. Such people are not often hirable, especially with my limited means; they’re more likely to be partnerships than employees. I’d love to tackle a collaboration like this in 2022, but I recognize that it probably wouldn’t solve the problem I’d originally described—that is, accelerating the implementation of these experimental systems. These sorts of creative partnerships are great because they transform my conception of the problem space. They don’t (in my experience) necessarily mean moving more quickly.

There’s a substantial chasm between these two points on the employment spectrum: hourly contractors, and deep full-time partners. I’d probably make a lot more progress if I could more effectively leverage ongoing full-time staff, but that’s a puzzle which remains to be solved. Indeed, it may not be solvable! In Philip Guo’s recent retrospective on ten years of PythonTutor, he credits working solo as an important driver of the project’s sustainability. I’ve been grateful to run my own experiment with a few engineers and designers this year. I’ll keep trying.

Crowdfunding depends on highly visible public work

Crowdfunded research is still awfully rare, so I’d like to help others learn from my unusual situation as much as possible. Let’s see what we can glean from another year of this funding experiment.

Last year saw slow but steady growth throughout the year, ending with funding roughly the size of a grad student’s fellowship grant. This year, that growth continued for the first half, then halted for the second half, plateauing at around three quarters of an NSF CAREER grant (a grant for early-career faculty in the sciences).

I can stay afloat with this level of funding, but what should we make of the pause in growth? Should we be concerned?

One important detail here is that the rate at which patrons end their subscriptions has remained constant since mid-2020, at about 1-3% per month. The plateau is due instead to a lower rate of new patrons. And that decline can be explained by a drop in visitors to the Patreon page. Roughly the same fraction of visitors convert into members, across this entire period.

This all adds up to a prosaic, somewhat bleak story for crowdfunding research: growth, and to a lesser extent sustainability, depends on driving new eyeballs to your work. This hypothesis carries some important implications for crowdfunded research. My slowed growth shouldn’t surprise us, because I didn’t publish any major, attention-getting work in 2021.

First, don’t get me wrong: 2021 was a productive year! But my work (like any researcher’s work) varies a great deal in legibility and attractiveness to a new audience. I spent the year running a variety of experiments, none of which “worked” in a showy or summative fashion—but all of which yielded helpful insights that drive my current work. If I were a traditional academic, I’d probably have published papers on these experiments anyway, since I’d be expected to chalk up a handful of submissions each year. Absent these pressures, I’d rather publish my work when I can tell a more complete story.

As it happens, I actually published about 45,000 words in 2021—a new record for me by a wide margin—but all in smaller, informal essays for patrons, rather than highly polished work I’d want to publicize for a wide audience. I’m proud of this writing for what it is, but I don’t expect these minor essays to attract lots of new visitors. In fact, many of these essays can’t attract new visitors: they’re only available for patrons!

The unchanged cancellation rate suggests that my patrons aren’t particularly bothered by what might look like a “slow” year. The problem, again, is that this type of publication pattern isn’t going to attract much new audience. That’s okay for me, since I can at least pay my bills at current funding levels, and I expect to publish glitzier work in 2022.

But the pattern I experienced this year illustrates some of crowdfunding’s limitations. This funding model probably won’t work well for investigators who need to spend a couple years on each major project, without anything to attract a popular audience in between. And that situation may not be predictable in advance. Research doesn’t work on a schedule. “Slow” years are part of the deal.

The crowdfunding funnel’s tight conversion rate implies that the work must appeal to a fairly wide audience. And, alas, researchers can’t completely ignore marketing if they’d like to stay afloat. A cancelation rate of 1-3% per month is fairly gentle, but it means that treading water requires a modest continuous audience growth. How rapidly will these research crowdfunding audiences saturate? Could anyone actually maintain this for their entire career? I have around 650 patrons now, but to maintain my present funding at this churn rate, 1,650 additional members must come and go over the next decade.

We’ll see, I suppose! This has been a somewhat bleak way to close the year’s reflections, but I’m personally quite optimistic for the coming year—and for lots more transformative software environments to emerge.

If you find my work interesting, you can become a member to help make more of it happen, and to get more essays like this one. To my patrons: you have personally enabled the past few years of my life. Thank you; thank you! I hope your 2022 shines bright.

My thanks also to the following folks for conversation which helped shape the ideas above: Adam Wiggins, Andrew Sutherland, Catherine Olsson, Danny Hernandez, Joe Edelman, José Luis Ricón, Kanjun Qiu, Michael Nielsen, Molly Mielke, Nadia Eghbal, Nick Cammarata, Ozzie Kirkby, and Philip Guo.

Finally, a special thanks to my sponsor-level patrons as of publication: Adam Marblestone, Adam Wiggins, Andrew Sutherland, Ben Springwater, Bert Muthalaly, Boris Verbitsky, Calvin French-Owen, Dan Romero, Dwight Crow, Eugene Soltes, fnnch, James Hill-Khurana, James Lindenbaum, Jesse Andrews, Kevin Lynagh, Lambda AI Hardware, Ludwig Petersson, Matt Knox, Mickey McManus, Mintter, Nathan Lippi, Patrick Collison, Paul Sutter, Peter Hartree, Russel Simmons, Sana Labs, Tim O’Reilly, Todor Markov, Tom Berry, Tooz Wu, William Laitinen, Yaniv Tal.