thesephist.com

How I side project

22 July 2019
22 Jul 2019
San Francisco, CA
16 mins

This post started out as a long-form response to this tweet by Raj Kunkolienkar:

… but then, I got busy, a few weeks passed, and it took a while for me to get back to it. On the upside, I have a new place for my ramblings now, which is dotink.co. It’s meant to be a more technical blog for when I want to talk more about software and design, but we’ll see how things go.

Side projects

Over the last year, it seems like I’ve developed a habit of working on and releasing a large number of side projects, from the mundane (like the blocks.css library) to the interesting (like Sounds from Places) to the technical (like making a new programming language). Everybody’s process and working style is different, but I thought it may be interesting to share how I think about side projects, and go from question to idea to execution to promotion.

For me, a side project is always some mixture of learning something new, fixing some problem I have in my life, or making something that I wish existed in the world, but doesn’t. I usually first get the idea for a project from needing to scratch some itch in my life or from an idea someone tosses out, but I let it simmer on my to-do list for a while before deciding that it’s worth spending some time on it to make it real.

Ideas

If I ventured a guess, I’d say I only attempt to build about 20% of the project ideas I write down, and end up working on only about half of that 20% to completion. So in total, given that I don’t always write down all of the ideas I have, less than one in ten ideas I have ever become useful side projects. Some are left on the cutting room floor because I don’t have time, and others are abandoned because I just end up without the time to finish them once I start.

Here are some ideas I currently have on my to-do’s, but may not get to for some time.

(If any of these ideas strike your fancy, feel free to start a conversation with me or go implement it yourself! I don’t own these ideas.)

So… I’ve got a lot simmering on my mind. You’ll notice that most of these are intended to just solve some minor problem or complaint I have with my current workflow. Some of these ideas are more fueled by personal curiosity, or thinking “wouldn’t it be really cool if this existed?” The shopping-cart-for-nonprofits is one such idea, and an earlier project called Sounds from Places is another like it.

This part – getting ideas – feels like the hardest part of my side project puzzle to describe, because I don’t really ever sit down and think, “Okay, what should I built next?” I try to remember and write down ideas when they come up in conversation or in the shower, and remember to check back on that list whenever I have some free cycles in my life.

Naming

Definitively the most important part of my workflow!

Jokes aside, names do mean a lot to me. At the beginning of a project, I don’t really know what its public name is going to be at the end, so I usually end up giving the project some codename that I can use to identify it, until I settle on a good launch name. For example, Studybuddy was codenamed “gemini”, Zero to Code was codenamed “Trident”, and my upcoming personal to-do app is codenamed “sigil”.

And obviously, some codenames stick around post-launch, like Torus and Ink. Some names are also decided from the get-go. Codeframe, Looking Glass, and blocks.css were all named from the beginning.

I try to give projects names that reflect something about the idea, but are relatively unique and sound good to my ear. I get a lot of inspiration from foreign languages, some from thesauruses and archaic names for people and places, and occasionally, made-up words.

Start when the motivation’s high

The number-one advice I have when people ask me how to productively work on side projects is this – when you first have the idea, you have more motivation to work on the project than you will for a while, so start immediately.

This isn’t always practical, and sometimes life just gets in the way. But I find that when I have an idea and immediately get down and dirty prototyping within a few hours, I can always crank out a working prototype that looks usable within the same day. Codeframe, Studybuddy, Zero to Code, Looking Glass, blocks.css, and Lyrics.rip were all mostly built the same day I had the idea and got down to it right then.

There are counterexamples – Torus took about 3 weeks with a 5-month hiatus in between, Ink has been about two months in the making with a month hiatus in between, and Sounds from Places languished on my computer untouched for so long that when I got back to it, I scrapped everything and started over.

But on balance, I find that whenever I get a chance to start on a project the same day I have the idea, I’m the most energized and excited about the idea, and able to get the most done and stay focused easily. When I manage to do this, I almost always get to a presentable prototype in 6-8 hours. Most of what you see on my Facebook and Twitter are prototypes after this 6-8 hour initial sprint, and sometimes it’s in a good enough shape that I don’t really feel the need to touch it after that, like Calhacks.org.

Picking out obstacles from my workflow

After getting started immediately, the next trick that has helped me become the most productive in the last year has been to consciously remove obstacles from my side project workflow. What kind of obstacles?, you might ask. Here’s a non-comprehensive list of things that I may have spent hours doing last year, that are now trivial or take just a few minutes.

Embrace the rough prototype

The first working version of anything is rough. It’s ugly. It works, but just barely, and it’s fragile, and it’s so satisfying. Here’s a glimpse into those moments from the last week.

This is a 2-by-2-pixel image that I rendered by manually writing each byte to a file in the BMP image format.

Here, I’ve just managed to render a blurry, blotchy black-and-white image of a Mandelbrot set. I’d go on to polish the algorithm here to render a full-color 1080p image later that night.

I’ve been trying to share more and more of my projects at this early “it’s kind of mostly working, but only sometimes” stage. I think this is the most satisfying part of early-stage projects. Until this point, I’ve been working on it fueled purely by the hopes and guesses at whether the project idea would pan out, but once I hit this point, it’s easy to see how I could iterate my way to a prettier, smoother, more high-quality, more stable solution. It’s the tipping point – the rough prototype.

Sometimes, I’d be fueled by this moment to keep going, but this is also a safe point to stop and pause, catch up on sleep, get to that meeting I’ve been pushing back, or whatever else. It’s safe to take a break here, because I’ve hit that tipping point, and I know I can pick things right back up when I return, because there’s something already working. But until I get to this point, it feels like an endless sprint.

Once I have the rough, working prototype, it’s also a good time to reflect and refactor or rewrite my initial implementation of something. Torus’s HTML parser went through a big rewrite a week into the project, because I thought of a much faster, less bug-prone design. But I couldn’t take risks in rewriting the parser until it was at least complete.

No matter the project, the bottom line is, if I get to this rough prototype point, I’ve done it. There is a thing that does a thing, and sometimes, that’s enough of a starting point to feel like it’s 80% done.

Don’t be shy about giving up

If you’ve scrolled through my GitHub repository list or my unofficial-official personal projects directory, you’ll know that for every project I’ve “launched” or finished, there’s a graveyard of things I’ve attempted but given up on. I used to be more troubled by that ratio, but I’m less bothered by it these days.

I think when it comes to personal projects, some ratio of finished- to abandoned-projects is a sign that I’m experimenting and playing around with ideas, and more importantly, that I’m prioritizing judiciously and saying no to things when it doesn’t make sense to keep pushing forward on a project that isn’t feeling productive or interesting to me anymore. Heck, I have, like, four different versions of a to-do list manager I’ve attempted and I haven’t got a single working version yet. That’s a bit annoying, but I think that’s fine.

While giving up or stopping work on projects isn’t problematic, I do think it’s important to find or study what went wrong at each instance and try to avoid the same miscalculation in the future. The tricks I outlined above, about how I try to work on an idea the same day it pops into my head, staying consistent with tools, and so on – these tricks are the result of thinking about why some projects didn’t work out in the end, and considering how I could avoid the same problems in the future.

Make it easy to iterate

As I’ve been steadily growing my zoo of active projects, I’ve been learning to place a greater emphasis on easy maintenance and iteration. In a word, this means making it easy to monitor when things are up and things go down, making it quick to push out updates, to find out what’s going wrong, to restore from a backup with minimal effort, and to hit the reset button when something inexplicably explodes without notice.

When I had one or two running projects, if one went down or lost some data, I could just go in, restart it, may be fix a bug that day. But I have more projects in active operation these days than I can keep in my head, and it’s become increasingly important that I figure out the “ops” side of things. I may go into a deeper dive on my personal ops and infrastructure (Tweet at me if you’d like to read about that), but for here, suffice it to say that I’ve written many a scripts and automation configurations so that I can backup, restore, push out updates, and run tests against changes easily for all of the breakable projects I have running. DigitalOcean’s easy backups has been really helpful here, as well as learning some basic shell scripting and automated testing.

Be ambitious, not comfortable

My most successful projects have been, without question, the ones that I’ve attempted with absolutely no clue on where I would even start.

When I began working on Apogee, a Chrome extension to cite webpages that ended up growing to about a quarter million users, I could barely write working JavaScript. The first thing I did that night was Google search “how to make a chrome extension”. When I started writing Torus, I had used frameworks like Backbone before, but had no clue how I would go about writing my own. And my latest pet project, Ink started when I barely knew what parsers were, let alone understand how interpreters and compilers worked deep under the hood.

I compensated for these holes in my knowledge with two useful skills: typing quickly and precisely into Google Search, and not being afraid of reading and re-reading things I didn’t fully understand until I did. For the Ink project in particular, for the last month, I’ve probably been reading 1000-2000 words per day on compilers, language design, garbage collectors, memory management, and a bunch of other related topics. I’m still confused about lots of things, but I have a far better understanding of what’s going on on my computer than I did prior, and I have a mental compass for what topics I should poke into next. (In particular, big thanks to Crafting Interpreters, which was invaluable in initially figuring out where the heck I was even supposed to start.)

Err on the side of being too ambitious, and don’t worry about starting in your comfort zone. I find that the most satisfying, interesting, fulfilling projects are the ones where I had to find and draw the map as I went.

Sharing

Lastly, I’d like to do more of what I’m doing right now – sharing, talking about what I’m working on and what I’m learning, what’s interesting and surprising, and occasionally, about how I work personally and what makes my schedule tick, if it’s interesting to you.

It surprises me what kinds of projects people find interesting. Sounds from Places wasn’t technically interesting at all – just a static site linking to some MP3 files – but that received the warmest reception online that I’ve had for any project except Lyrics.rip. Neither project was technically challenging or involved brilliant strokes of insight, but I think they were artistically interesting and fun ideas, and that seems to be what sparks people’s imaginations more than custom programming languages or frontend frameworks.

I like to joke that, in an alternate timeline, my dream career would to be move to LA to become a professional concert pianist and composer. I’m constantly amazed by what art can communicate – well-crafted art doesn’t necessarily communicate technical knowledge or know-how, but it begs interesting questions and opens up people’s imaginations in unexpected ways.

Even as my personal projects continue to be a way for me to learn and experiment with technologies, I hope I can get to a point where I can also build things that go beyond that, to be tools that artists can use to tell better stories or find better media for their ideas, or even to open up new and interesting questions altogether.

At any rate, I’ll keep hacking and sharing, and I appreciate you being here to tune in when things get interesting. If you’ve got thoughts or questions, you can share it and discuss on Hacker News.

← What to do if you’re a high schooler interested in a career in software

Thank you Repl.it, Hello Hack Club →

I share new posts on my newsletter. If you liked this one, you should consider joining the list.

Have a comment or response? You can email me.