Alex Kurilin

The Code Nobody Reads

Tue, 17 Feb 2026 10:00:00 -0800

Civilization advances by extending the number of important operations which we can perform without thinking of them.

—Alfred North Whitehead

It hasn’t even been a year since the launch of Claude Code, the breakout coding agent of the year. Together with its distant but equally competent cousin, Codex, the agents have proven to the software engineering community that real work can be done with these systems. Anthropic’s Cowork was written entirely by Claude Code, Sora for Android was entirely done in Codex, and a growing share of everyday development tasks can now be handled wholesale by the agent. Practitioners have been experimenting with feeding progressively more work to agents, to see how much they can get away with without touching the code. Minutes, hours, perhaps even days of hands-off work with the human occasionally checking in to see what got conjured. It’s a profoundly different approach to getting code into a repo, and it demands new strategies for dealing with the side-effects.

The more the genie builds plumbing and functionality blocks that you haven’t looked at, or that you don’t fully understand, the more it’s going to hurt once you hit the limits of the agent and you can’t figure out why “Claude, fix it” is just spinning wheels and suddenly no longer making progress, the user seemingly having run out of wishes to be granted. Jolted out of the joy of typing “proceed” into the CLI, you now have to look under the hood for the first time, and understand a system whose internals you’ve deferred studying for days or weeks, with all of the quirks of a codebase that hasn’t had an active gardener in a while.

Just how much of this can you get away with? This deferral of understanding is indeed a form of debt, and just like with technical debt, you can get away with some of it, some of the time, but the exact quantity is still something that we’re trying to figure out as a community. And the target is constantly moving as the tools improve around us and let us stretch the limits of our non-understanding further without paying an immediate price.

If you’ve worked with the tools long enough—a year feels like a century in AI-years—you will have likely started to notice that the more successful the work by the coding agent, the less you feel incentivized to review its work. If most of the time its contribution “just works”, why labor any harder than you have to in order to review every line? Your confidence grows that what you’re getting is going to continue being good, and your desire to second-guess it and review it only lowers with time, as you keep getting better results. Why inspect the work when instead you could be going even faster?

The corporate incentives are stacked against it as well. “Our devs are wizards, they’re pushing hundreds of PRs a day, our job is to empower them and get out of their way” is what every frontier podcast is saying, not “our devs are really good at slowing down the pace of change, at making sure they manicure every PR. We wouldn’t want to go any faster than that, we value pacing ourselves and making sure we’re not running as fast as those crazy AI kids.” The beast rewards being fed, until it doesn’t.

Some call this concept cognitive debt, which, if you’re a fan of the revered Team Topologies framework, will instantly resonate. In the book, the authors claim that an engineering team can only manage so much system complexity before maintaining their dominion requires constant re-learning. The knowledge falls out of the mental cache of the engineers, the larger the territory, the more page misses, the slower the work. Engineering managers will then try to maximize page hits by not cognitively stretching developers beyond what they can reasonably fit into their caches. Too big of a domain? Time to simplify the codebase (unlikely) or split it up into teams that can reasonably digest it (likely, but expensive).

Do we manage the cognitive load the same way as always in a world of genies? Unclear. The agent dramatically expands the territory a single developer is responsible for, well past what fits in their mental cache. I think of this problem as the "fog of war" from the world of real-time strategy games. There’s an area of the codebase you haven’t explored in a while, maybe ever. Nobody has ventured out there to get information on its latest state. You have a limited amount of troops, the more you expect an individual SWE to do with their wetware + agentic augmentation, the more areas will stay in the fog of war. Some of them they’ve never explored to begin with, some they can safely guess the contents of based on spec-level conversations with the bot, some they’ve actually had to look at recently and still have fresh in their memory. Plenty of those areas don’t need active scouting, they’re safe, stable, and don’t cause anybody trouble. It’s unlikely anything nasty is going to suddenly sprout out of there, even if it hasn’t been looked at in a while.

The fog of war isn’t a new phenomenon: many engineering teams already don’t have a perfect understanding of the systems they’re building on top of; that’s the power of good abstraction. Maybe it’s a platform they’re building on, maintained by a big vendor, a complex framework curated by an active open-source community. Sometimes it’s legacy code they’ve inherited from someone else at the company.

I once had to integrate against a system whose creators had all left the company. Nobody even knew who owned it anymore. My manager had to track down former employees by phone, years after they’d left, and by then they could hardly remember how anything worked. The system was almost entirely a black box; all we could do was poke at it from the outside and hope to figure out its behavior from responses and a few old docs on a barely-working internal Sharepoint site.

Sometimes the case is much more benign: the module or service has worked for years. It was a stable, leak-free abstraction that didn’t cause any trouble and didn’t need to be looked at for ages. Whatever tribal knowledge had been built around it simply decayed over time and now nobody knows how any of it works. The people who originally wrote it either moved on to other projects or forgot completely how any of it works. The docs might have gotten lost somewhere in the shuffle. At least you still have the source code and can figure it out, given time.

At other times you’re working on top of a giant pile of code that you don’t have time to understand: you need to learn just enough to be able to get your task done. Nobody’s paying you to become a connoisseur of the whole system. Whether that’s modifying Linux, extending Unreal Engine 5, or even building on top of a buffet of browser APIs, there are not enough lifetimes to understand everything beneath your layer. There’s only so much you can fit into a human’s context window at once, and with large systems you quickly run into those limitations.

But is the product of an agent operating on specs with their human jockey any different from these scenarios? The testing world has long had vocabulary for this spectrum: black-box testing, where you have no visibility into the internals, white-box testing, where you have both visibility and understanding, and grey-box testing, where you have partial knowledge.

The same framing maps neatly onto development postures. With coding agents you’re entering the world of grey-box development, where you could look at the conjured internals of the system if you wanted to, but you really hope you won’t have to. You hope that the agent’s abstraction is sufficiently self-contained, modular and decoupled that it “just works”. You hope that it made all of the right internal design decisions based on the high level spec that you had agreed upon.

One curious side-effect of spec-driven development is that, in my experience, your memory of how something works will decay much faster, as you didn’t have to put in the hours schlepping through building a system by hand that is then firmly lodged in your head for weeks or months to come. Everything was more ephemeral, your connection to the deliverable less visceral, after all, you were mostly approving and tweaking the blueprint, not laying the bricks yourself. The fog of war creeps back in faster than in the artisanal dev scenario.

The obvious comparison is to libraries. Most people don’t actually look inside a library’s implementation after downloading it. They sync it down, wire up against it, read the docs (or have the agent do it), and expect it to work right away.

I think they are indeed different, at least for the popular, well-adopted libraries that most developers gravitate towards. A widely-used library has been smoke-tested by a large number of other developers who used it and were able to report if it worked or not, if it met their use case, if it had any broken corners that tripped them up during testing or production. Your conjured module is more like that brand-new library with 12 GitHub stars that nobody has battle-tested yet. Except worse, because you’re the first and likely the only user of something perfectly bespoke to your needs. The guarantees are essentially non-existent unless you’ve explicitly tested them yourself. You’re hoping the agent got the performance right, caught predictable corner cases, designed the system in such a way that it’s not spreading contagious tech debt onto everything it interacts with.

In the past the developer would have been responsible for knowing every nook and cranny of that system, becoming a deep domain expert in it and being responsible for it in front of the team. But when a developer can crank out a couple of these a day? When the cost of blowing away an isolated module and re-generating it from scratch approaches near zero—assuming no state to migrate, no downstream consumers to coordinate with—is that knowledge worth much anymore?

None of this is entirely new. But it’s accelerating. Coding agents are only getting more independent and capable of delivering predictable, higher quality results with each month. It’s hard to imagine frontier product and application teams—the ones actively racing to adopt the latest techniques for a speed advantage—still writing most of their code by hand in late 2026. The only thing holding teams back by then will be the inertia of large companies with well-established moats that won’t be eroded by a faster competitor. Sure, they could go faster, but competition isn’t going to replicate a deeply entrenched brand and thousands of enterprise relationships built over steak dinners. The existential threat is just not there, short of execs pushing top-down an “AI strategy” with little contact with the needs of the troops on the ground. But for hungry, earlier stage teams still in the exploration stage, the faster iteration cycles will make catching up to the big players product-wise and impressing their customers with quick turnaround times easier than ever.

Are we going to see SWEs move away from attempting to understand most of the code they’re writing, instead relying on the model to figure it out later down the line if something goes wrong? Is the near-future SWE actually ultimately a manager of agent harnesses, one who’s far less technical than their reports? Is the SWE manager going to be expected to roll up their sleeves and fix an issue or re-design a system that the bot complicated to the point of needing torching? Or is the model going to be good enough where the human can just nudge it to fix it and never have to understand the underlying design choices? There are plenty of companies where line or middle managers haven’t made real technical contributions to the codebase in years; they’ve fully delegated the power of changing the codebase to their reports, and operate through those intermediaries. The coding agent could be more of the same pattern we’ve seen for decades, repeated one layer deeper.

My hunch is that the organization’s bottomless desire to go fast and ship more changes will push developers to know less and less about the code territory they’re stewards of, as long as nothing bad happens along the way.

Right now we’re in an awkward transitional period where we still expect developers to know what they’re working on inside-out, have answers when asked to explain how a module works, while also using agents to move faster. But ultimately the speed vs. understanding tradeoff will push teams to accept that their SWEs don’t fully grok their own modules, and that’s ok, that’s what the model is for. The models will keep getting better for years to come, making “the model has the answers, I’m delegating” only more possible with time. Right now? Not quite there.

You can think of an agent as an outsourcing shop who’s now responsible for understanding, explaining, and evolving that section of the repo. It might be a very good outsourcing shop—fast, capable, available around the clock—but it’s still a form of delegating the exploration of the fog of war to someone else. The knowledge of what’s actually in there doesn’t automatically flow back to your team. The coordinator steering their work won’t be in the weeds remembering every line of code written by the shop, and unless they’ve explicitly invested in following along every step of the way, the internals become a mysterious blob that nobody on the team can confidently maintain. It’s a bad day if you’re asking that coordinator to roll up their sleeves and fix things themselves.

The fundamentals of software engineering are going nowhere though, and will only increase how much territory under fog of war a single SWE can manage. The more coupled and infectious the system, the more someone has to really think through it and make sure it’s written thoughtfully. Screwing up the shape of an API at the center of many code paths will really suck. Same with the shape of data that has numerous consumers. Sure, refactoring is cheaper than ever now, but these systems are still potentially interacting with the work of many other teams, plus their agents, and team coordination bottlenecks introduce all of the predictable extra friction that you want to be thoughtful about.

But for something independent and peripheral? You might not have to know anything about it. As long as the tests pass, the module has little to no side-effects, and the cost of mistakes is low, you’re probably better off not wasting time on its internals. For that kind of system, spending time on the guts is becoming more and more akin to the person reviewing the assembly instructions generated by their higher-level application. There’s probably some value to it, but not enough to justify the activity once the compilers got good enough. But all of this only holds if the tool beneath you is predictable, and that’s where the analogy starts to break down.

The tempting analogy is the compiler, but it’s actually inadequate for what we’re seeing here. Compilers are mostly mechanical mappings from a higher level representation of instructions into their fundamental components. But a high level spec discussion between a SWE and a language model leads to an implementation that can take infinite forms. There’s plenty of room for interpretation, creative solutions, stylistic choices, architectural opinions. The more you let go of the wheel the more you’re leaving up to chance, the more the fog of war is likely to hide something that might surprise you.

That’s not a compiler, that’s closer to a junior developer you hired but can’t fully supervise, who works at 100x speed and never sleeps. The cognitive debt with a compiler was near zero. The mapping was deterministic, verifiable, and for well-defined inputs you could trust the output without thinking about it. With an agent, the abstraction boundary is fuzzy. You think in specs and vibes, the agent produces systems, and the mapping is… a mystery. You hope it understood you. You hope its judgment was sound. And the understandable inclination is to leave those implementation details unexplored if they usually work well enough.

The natural counterargument is that automated testing is the torch you throw into the fog of war. If you have comprehensive tests that cover the spec, do you really need to understand the internals? In theory, no. In practice, writing tests thorough enough to substitute for understanding is itself a significant investment. You need to anticipate corner cases, performance characteristics, concurrency edge cases, and failure modes that you might not even know to test for if you didn’t participate in the design. Tests are an excellent mitigation, possibly the best one we have, but they’re not free, and the temptation to under-invest in them grows in lockstep with the temptation to skip human code review.

There’s also a dimension that gets surprisingly little airtime in these discussions: security. The scariest thing hiding in the fog of war isn’t code that doesn’t work. It’s code that works perfectly while being quietly exploitable. An agent that conjures a system you never review is an agent that might introduce vulnerabilities you never catch. It’s possible that this too will be addressed by specialized agents, security-focused genies whose entire job is to audit the output of the coding genies. But that’s still delegating the problem to a different black box. At some point the chain of agents auditing agents has to bottom out at something a human has verified, and whether teams will invest in closing that loop is an open question. At the same time, is this any different from the current state of things, with many early stage teams treating security as an afterthought as they’re chasing product-market fit and more growth? The agent might actually provide them with a more robust implementation than the one they would have bothered with.

So are we becoming dependent on something we can’t understand, and is that dependency qualitatively different from every previous abstraction layer we relied upon? The answer is likely yes, and we’re not sure if it’s fine. It’s unclear if the acceleration in delivery speed at the cost of visibility is sustainable long-term. Will we ultimately reach an asymptote of how much a developer can outsource “low level thinking”, or will the models keep improving to the point where SWEs are describing solutions at a highly abstract level and consistently getting something that “just works”? I’m not sure if that’s even possible, if you can get what you want when you never said exactly what it was, if you perhaps didn’t even know it yourself.

It’s hard to imagine a world in which you’ll get what you want without getting into the weeds of the details, but people have expressed skepticism about the viability of agentic coding the whole way here, and look at where we are now. Dismissing something as just a toy for unserious work is a classic trap of the Innovator’s Dilemma, and we might be in one right now. If the trajectory continues, perhaps the genie learns to grant wishes in a way that never disappoints us, even though it might not be exactly the approach we would have chosen ourselves. And if it does, we should expect engineering judgment to decay, the muscle atrophying with disuse, our dependence on agents becoming complete. But it also may be the tradeoff everybody picks, because getting something that is mostly there, but 100x faster, is better in an industrial context than getting just the perfect solution at a far higher cost.

The code nobody reads might just be the code of the future. The question is whether that’s a triumph of abstraction or a debt we haven’t yet been asked to repay.

What "The Best" Looks Like

Sun, 28 Dec 2025 09:00:00 -0800

Talent hits a target no one else can hit. Genius hits a target no one else can see.

—Arthur Schopenhauer

The second that the next round of funding hits the bank, every new CTO starts obsessing over the same thing: Who the hell do I hire next? The answer is surprisingly non-obvious. You’re told that you always want—scratch that, need—the best of the best, your startup’s future depends on it. You’re told your company is unique, special, and it requires the most hardcore among researchers, designers, engineers, and product managers. You never skimp on who builds the golden goose. You can’t succeed with any less than that.

But, is that actually true?

Every startup-hustle YouTube video, VC podcast and celeb founder interview regurgitates that the key to success is to hire the very best people, no matter what it takes. And yet you live in the real world, with real-world constraints. You have only so much money in the bank. Only so much time and bandwidth to hire. Only so much attention in a given day. And you’re not alone in your hunt. You’re competing with hundreds of other players in your geography, industry, and problem space looking for “the best.” Many of them have a more famous brand, more cash, more promising equity, more charming founders, and maybe even a high production value promo video showcasing happy employees, rare wood office counters and a shoes-off policy.

Will you actually hire the best of the best against those odds? Many years ago I found myself in this pickle and I had to learn all the relevant lessons the hard way. I share these lessons here, so that you don’t have to struggle through that same maze yourself. My pain is your gain.

My first time around the startup world in 2012, I hardly knew what I was doing and relied mostly on luck—and unfortunately, firing—to end up with a team I could be proud of. I had no real point of reference for greatness, for what “the best” in our area could look like, and building that model required lots of experimentation, with high highs and many low lows.

The story of David, our brilliant infrastructure ops engineer at Freckle, has stayed with me ever since. David applied to the company back when we were only hiring for an ops role. We were growing slowly, so there was zero room for anybody who wasn’t absolutely essential on the team. We had no open-ended extra seat for a smart person who just happened to be on the market, unlike some companies these days.

David was 100% what we were not looking for. He had never done any ops. In fact, he had never done anything related to web dev or product engineering. I’m not sure he even knew what AWS was at that point. He was a sharp physics guy working with a professor on simulations in an academic context. He didn’t let that deter him. He wanted to work at Freckle, thanks to our reputation as one of the few software startups in the world using Haskell in production at scale. We were an odd outlier in a sea of buggy and laborious Rails apps, a shelter for people who didn’t want yet another web slop gig. And Haskell was oh-so-hot in the Hacker News programming language theory space at the time, a technology attracting software nerds obsessed with correctness and new, better ways of developing bug-free apps.

I immediately told David that he was not a fit; he had none of what we were looking for. And yet he persisted, emailing me that he would do whatever test project we threw at him, and if he bombed it, no problem, he would go away. But if he nailed it, we would have proof that he was qualified, in spite of what his CV indicated. Fine. I sent him a meaty cloud ops take-home project, expecting to never hear from him again. Importantly, this was in the days before you could have Claude Code slap that together for you in two prompts.

A day or two later, he returned the project to us, and it was pretty much flawless, doing even more than we had asked for. That was not expected. I got curious about what else he could do. We weren’t drowning in applicants anyway, so I figured I didn’t have much to lose. We took him through the usual interview process. He was humble, optimistic, well-spoken, actively communicating and taking feedback well, eager to get to work. He was pumped about everything he could learn on the job, about the doors that would eventually open if he nailed it. He didn’t have much experience, as someone who had never written commercial software before, but he was really quick to absorb everything we threw at him.

We gave him a chance. As predicted, he was stellar, and we had a really good run with him until he moved on to a much more illustrious employer. A few years later he became a senior principal engineer at Stripe, going from a physics lab, then a starry-eyed K-12 software startup, to being a big deal at one of Silicon Valley’s finest firms. An unsurprising path for one of “the best.”

While building the different companies I worked at, I’ve run several times into “Davids” who ended up with meteoric career trajectories, sometimes already pedigreed in all the right ways, sometimes completely invisible to everyone but the trained eye of a CTO looking for gold. What is special about someone like that? And how can you, with your humble hiring budget, identify them before their value is obvious to everybody else in the market?

With David, it was obvious that I had lucked my way into finding him. He had to badger me into seeing what he was capable of, at a time when I was only looking for obvious signs of success. Later on at my game studio, thanks to a brutally skewed job market, I had the total pick of the litter during Hiring Summer and could select one or two stellar engineers from hundreds of perfectly reasonable applicants. There was hardly any competition, with money drying up all around the industry. Again, I found a diamond in the rough. But this time I had the right knowledge and strategies in place to end up with the kind of team I wanted with far less reliance on luck. Even as I was drowning in hundreds of resumes that all started to look the same, “the best” candidate was still in there, like a needle in a haystack. This time, I felt like I was actually skilled enough to find that hidden gem, not simply crossing my fingers and relying on luck alone. That’s a feeling of empowerment I would like you to experience as well.

What do we mean by “the best”?

Of course, when every company says they have the best people, the math doesn’t work out. And of course, if they regularly have to fire staff, something doesn’t add up again. One day you end up landing a job at one of these companies and realize that the braggadocio was hardly reflective of reality, but then again, it’s all part of the startup founder LARP that requires you play the part. “Our team is okay, you will probably like working here, we sometimes know what we’re doing” would be a much more accurate depiction, but you’ll never find that quote on the careers page.

Regardless, you want to be a good Boy Scout CTO and live up to the lofty expectations set for you by the Silicon Valley elders, and at least try to hire “the best of the best” as they say in all of the fireside chats.

But then, a question naturally emerges: what exactly do we mean by the best? The accolades? Pedigrees? Github stars? Hacker News front pages? Job title? Celebrity employers? Number of former YCombinator companies worked at? Hackathon wins? Typing speed? Clout among other Rust developers wearing fuzzy animal costumes? That all sounds good, I guess, but is that all relevant to your company?

It turns out that “the best” is mostly subjective and specific to your situation. The culture, the vibe, the industry, the processes of your company, and the technical choices will all influence who will be a phenomenal addition to your tribe. That person might not be the same individual bringing in a million or two a year at Meta. In fact, that highly pedigreed, highly connected individual might be a net negative for your company, even though they might, at least on paper, seem the most high-end among candidates. They may blow the mind of a recruiter at Netflix, but they may not want to schlep through all of the messiness and chaos of an early stage company still trying to define itself. They’re not the best for you, and they’re likely not the best for startups, but plenty of other people are. Your job is to determine what those traits are and where to find people who have them.

Universally Best Startup Hires

Your company, timing, industry, problem space and founding team personalities are unique, which is why a generic blog post or a book could never tell you exactly what hires are optimal for you. However, experience shows that there are many universal winning commonalities between great hires that will be applicable regardless of your early stage company’s unique DNA. I identify 11 of them below.

The 11 traits:

As a side note, I found Patrick Lencioni’s framework from The Ideal Team Player to survive first contact with both pre-seed and Series A+ realities. It’s one of those dry-as-dust HBR book club management self-help guides that get an eyeroll by your median 20-year-old founder going through YCombinator, but in my experience it confirmed and put a simple model around something I had personally seen emerge again and again in the field.

Lencioni identifies three essential traits: hunger, humility and smarts, the latter of which should have been called EQ all along. You’ll see them referenced below. The three criteria may seem obvious at a glance, but having a simple framework to work with as you’re making hundreds of decisions makes a big difference. Simple things done right turn out to be pretty darn powerful.

Let’s take a closer look at the 11 commonalities among great startup hires I’ve identified. Some of them are simply statistically likely to occur due to the nature of startup hiring. Others are ones you should straight up filter for when you’re doing your search.

Non-obviousness

The best hires in the early stages are usually non-obviously good to the untrained eye. They don’t look as appealing to employers with infinite resources who otherwise would have already hired them. If they were obviously incredible, it would be unlikely you—at your startup—would ever talk to them.

One of the most important skills of a startup CTO trying to hire an amazing team is the ability to uncover awesome talent that others have overlooked. These candidates get skipped often due to being too off-the-beaten-path and under-pedigreed compared to the obvious picks that every other company is making. Often these candidates are bad at marketing themselves, don’t know where to look, and are off-putting in some way to the naive search. You have an opportunity to take advantage of that oversight.

Realistically, someone who looks like the perfect FAANG candidate who has won all of the math olympiads, had all of the stellar internships, and went to the top CS schools will either:

a. get a big cash dump from YC and start their own thing, or
b. go work at one of the career-building brand names in the Valley and make a monstrous amount of money.

The karmic wheel is just about cycling between those two until that person either gets into VC or becomes an exec at a prestigious firm years later. If that’s what the universe has in store for a candidate, taking a chance on a no-name team with 12 months in the bank, poor development practices and sloppy management is a tough proposition.

That leaves a pretty large pool of “everybody else” who didn’t pattern-match. How you go about sifting through them is a big topic I will leave for another chapter, but the key is to look for signals that are less obvious than a Stanford degree and an OpenAI internship. Often that looks like a large volume of work, expertise in unsexy niche areas of technology, an intense work ethic driven by curiosity, and many others.

A Stanford CS degree is no guarantee someone will be a phenomenal contributor at your company, but it’s a reasonable proxy of future potential for large employers with long time horizons that allow them to invest in coaching and nurturing their junior staff. That’s not in the cards for a pre-revenue startup that will run out of money next year.

Not-too-junior

The best people for your startup will most likely be senior, or at least mid-stage contributors with several solid years of experience. Too senior isn’t great either, you don’t need the large scale cross-team mature product jousting chops. There’s likely no staff-level work for them to do, and staff-level engineers aren’t just faster-typing seniors.

As an early stage startup, until you have enough lead developers, senior product managers, senior designers etc. to set the standard, hiring fresh-out-of-school contributors is a gamble. Junior staff are still developing their taste, judgment, ability to work in a team, and understanding of how to follow the process and when to deviate from it. Without an adult in the room you’re in cat-herding territory. This isn’t to say that you won’t get work done this way, but you could have had someone else in that seat who required less supervision. And you’re at a stage where every seat is mission-critical and the opportunity cost is significant.

While it might not matter too much as you throw prototype spaghetti at the wall in the early days, as soon as you have something worth maintaining and complexity rises, you’re now in a race against time. Someone must actively garden the complexity of the work and be ultimately responsible for it, and that someone will tend to be not fresh-out-of-school. Sure, a green dev can care about complexity. But most haven’t yet lived through a soul-crushing, multi-week refactor fueled by years of tech debt, the kind of preventable trauma that earns you “character-building scars.” If they’re learning those lessons on your watch, they’re doing it at the expense of your delivery timeline and your sanity.

I’m fond of mid-stage software developers who demonstrate terrific chops, hunger to learn and prove themselves. Yes, they require more hand-holding, but usually the volume of work they put out, and their openness to re-do it, if needed, is worth betting on. You should be able to manage a couple of them yourself as the CTO in the early days, and soon enough you’ll have other senior people on the team who will pick up the mentorship torch.

Not-too-senior

This also means that the “best people” to hire at this stage will tend to be earlier in their careers. The longer great people are on the market, the more likely they are to be identified by the market as being great. Your job as a startup CTO is to find them before others do. Once the market has found them and has actively started rewarding them, they will be out of your price range.

Now, it’s worth mentioning that in specific well-capitalized niches of the startup market, companies are now able to pay a decent chunk of change thanks to larger VC rounds and quicker time-to-revenue. Thus, the old school notion that BigCo always pays best may not hold true as often. Here’s a quick sample of many Work at a Startup roles in December 2025. Not quite at FAANG-level, but far from starvation.

Category	Count	Mean	Median	Min (Midpoint)	Max (Midpoint)
Regular SWE	45	$157.4K	$150.0K	$85.0K	$275.0K
Senior SWE	32	$209.8K	$202.5K	$140.0K	$350.0K
Founding SWE	33	$189.3K	$185.0K	$125.0K	$255.0K

That also means that, unless your company grows fast or shows tremendous potential for the yet-unvested equity, you won’t get a ton of time with your great early career hires. They will quickly accumulate valuable experience and prove themselves to be stellar, and move on to a prestigious employer with longer time horizons, a great brand name, and a stupendous paycheck that you will not be able to match. Being an L5-7 at BigCo pays a pretty penny, with none of the unpredictability of startup equity and saner hours, but none of the adventure and camaraderie of a pirate ship. For employees later in their careers, that’s not a bad tradeoff. Once they have vested with your company, it makes sense for them to diversify their equities, as they already own one lottery ticket.

Being on the receiving end of a reference call for one of your star employees, while they’re still working at your company, is always bittersweet: on one hand you want what’s best for them, to level up and grow in their career. On the other, they’re your discovery—dang it—and you know you won’t be able to counter-offer the type of employer they can now attract. Or they decide to start their own company, emboldened and informed by the experience at yours.

In the end, that’s okay. Sometimes a great hire’s growth trajectory is faster than a startup’s, and you can be grateful for having given them a chance to prove themselves and to find a new path that they wouldn’t have otherwise. That kind of hire will have likely made a major difference to your company and having trusted alumni out there in the world doing great things should be a source of pride and good industry karma for any CTO who discovered them.

Hunger

The best hires are self-driven, stoked by the ability to learn, to gain mastery of the craft, and by being able to show off their accomplishments to the rest of the team they want to impress. When you look at their track record, even when it’s not filled with household-name accolades and pedigrees, you will typically see breadcrumbs of exploration, toy projects and experimentation fueled by curiosity. It’s that classic desire to hack things and understand them.

You don’t need to ask them to work a little harder, they know this is a big growth spurt for them and they want to take advantage of it. The company is giving them a valuable practice canvas for their skills and they want to make the most of this opportunity, which unfortunately isn’t that common. They love a challenge, they want to prove themselves, they love the work and they are excited by being around others who will push them to be their best. They would have been hacking on something interesting either way, but now you’re actually paying them to do it.

They’re curious about how everything works: your company, the startup world, the industry they’re diving into, other disciplines, the tech they’re using. They’re learning, absorbing. Maybe they want to start their own thing one day, or maybe they’re set on using this gig as a ramp to something else they dream about. Maybe they just want the new optionality granted by having your company on their resume. They’re sponges and will gladly take on the challenges you send their way.

I remember throwing a big Redux experiment at David in what must have been his first month at work. Again, no web dev experience prior to this. Nobody on the team knew the technology and he, hired as an infrastructure engineer with a background in physics and zero product engineering experience, had something up and running for us within days. He was pumped he got to do something so radically new for the team as one of his first assignments. Ultimately we decided the tech wasn’t a fit for our existing codebase, we scrapped the experiment, and he ran off to the next challenge with no loss of enthusiasm.

At Double Dusk we could not figure out why players would regularly de-sync their character positions on the server when using a custom—and wildly complicated, due to needing to replicate Half Life 2-style movement to the smallest detail inside of Unreal Engine—character movement component in combination with our own Unreal Engine networking tweaks, making character movement authoritative on client instances of the game. A big deviation from the defaults.

Our newly-hired star engineer, us being his first ever employer, had to dive into the pit of madness. He almost gave up several times before finally identifying all of the spots in the physics sub-stepping logic that was causing the drift across the network. I’m confident I myself would have gone completely bananas trying to debug that one. He was proud of the trust we had put in him, he was eager to learn, he didn’t want to let the team down, and I knew enough about him to make the bet that the challenge would have been just barely within his abilities.

Hunger is a massive force-multiplier, and unlike many other traits, it doesn’t seem to be teachable. Failures will instill humility. Rejection will hone your EQ. But you’re either driven to compete, learn and prove yourself, or you’re not. No manager’s pep talk will kickstart that drive in you, at least not sustainably. The hubris has to come from somewhere much deeper.

Humility

Startups are all about rapid experimentation, trying countless ideas that don’t work, and bootstrapping the skills and processes of the company as it figures out just what exactly people will pay for. Trust-building is critical in any team endeavor, and members who are unable to admit fault, who take up all of the space, and/or who need to be right at all costs will be a real problem as the company matures.

The best hires are humble and will happily talk about both their victories and their biggest implosions, about what they learned from their misadventures, about talented past coworkers and how their efforts were part of a team. They rarely claim to have single-handedly carried everything on their backs.

As you interview candidates searching for the best, many non-obvious great hires will be bad at behavioral interviews—or interviews in general, for that matter. They didn’t get extensive training on answering questions using the STAR method, didn’t drill stock responses to predictable and tired interview questions. They’re early-ish stage hands-on technology people, not smooth-talking execs jumping from one boardroom to the next. It is your job to fish for key nuggets of insight buried in the rough presentation. After you interview enough people with a similar set of standard questions, you’ll be able to spot the outliers in the bell curve of responses.

An underrated aspect of humility in startups is detachment from one’s work. Companies in the process of inventing themselves need to actively cycle through many ideas, most of which will ultimately not stand the test of time. Design for the trash can and kill your darlings are important mindsets in creative work, and they extend far beyond music, film and games. Quantity leads to quality, and a great hire will accept regularly needing to apply a flamethrower to their work and try again with the new learnings from the latest experiment. They will not be precious about it, nor will they feel diminished by needing to try again. The Davids do not keep score; they keep trying and do the work knowing that there’s a good chance it will not go anywhere.

That is also highly correlated with their ability to take feedback without taking it personally. They’re not afraid of criticism and quick feedback loops. In fact, they seek disconfirmation sooner rather than later. They have excellent feedback metabolism.

High EQ

Building technology companies is a team sport. Everything you do is with the help of other people, for the benefit of other people. As much as the startup world fetishizes the cracked ninja jedi 10x code-poet, if that person makes others never want to talk to them again, cracked they are not. Someone who is able to work with others’ quirks, understand how they operate and what they respond to, and act in a way that brings the best out of their teammates is worth their weight in gold. Brilliant jerks, as great as they are for producing riveting drama in TV shows, quickly become a net negative in the real world.

We’re in the squishy human feeling territory here (aka “soft skills”) and you can’t LeetCode your way into knowing if someone will be an ass. There’s no FizzBuzz for empathy. You have to ask them standard questions, shoot the breeze, interact during the various testing phases of the interview, and outside, and ultimately make a call based on the limited data you have. You will never have all of the evidence, but ultimately you have to make a go or no-go call. Sometimes you get lucky and the candidate discloses a consistent pattern from their past that is obviously disqualifying, but often you don’t.

How they interact with you and your team throughout the interview is usually a good data point, and that, of course, goes both ways. Regardless of how you investigate this side of them, it’s non-negotiable, and the best hires you’ll make will all have a high EQ.

Team-centricity

At the same time, someone highly empathetic, but who hides in a cave and is unable to coordinate their work with the rest of the software orchestra, is not the best.

Great team players anticipate others’ questions and concerns, and are proactive about reaching out and communicating both their status and the progress of their work to those around them. They make their work visible instead of requiring constant polling for the team to figure out what exactly is going on with them.

Remote environments in particular benefit from strong proactive communicators, as you can’t get away with always knowing what’s going on with everybody due to sitting in the same room for most of the week. One great Loom is worth a thousand words, but a thousand words is still much better than having to pull status out of people.

Early stage startups move at a pace that generates tremendous entropy, and someone who’s able to coordinate with the rest of a team in a way that feels natural and effortless will allow the small teams to scale without requiring project management and complex processes meant for lower-performing contributors.

Competence

Perhaps counterintuitively, I don’t fret too much about competence. To me, that’s a given. If your interview funnel is set up well, with a great take-home project, an onsite, or even a work trial, determining if the person you’re interviewing is competent should emerge naturally. After all, you and your team are technical experts, craftspeople who can look at another craftsperson’s work and quickly judge if it is any good.

Yes, AI is getting pretty good these days and it’s becoming easy for candidates of all disciplines to hide behind prompts, but that’s only an extra reason to allow them to bring those tools into the interview itself and showcase their use rather than shamefully hide their existence. How they use the agents, the back-and-forth, the planning, the tweaking, the types of searches they do as part of the exercise are all valuable data points for determining competence, and you should allow candidates to surface them.

In-person interviews and work trials are for now non-gameable, so if you’re extra paranoid, they are a great option. Non-obvious candidates will be in the hiring race with fewer other companies, possibly none, depending on your industry. That gives both you and them more flexibility and time to try working together for a few days before making it official. With a sufficiently relevant set of tasks for the work trial, you can rest assured that they’re competent, and learn a lot more about their other traits in the process.

High-agency problem-solvers

The best start-up hires are problem-solvers, not ticket resolvers. They want to understand what the business is trying to accomplish, what challenge is standing in the way of it, and how to make that problem go away. They’re autonomous in the best of ways. They don’t sit around waiting for someone to tell them what to do; they proactively find challenges and opportunities for improvement, iterate on the feedback, and go out of their way to help their teammates. They figure things out, and they do it without needing their hands held.

Someone like David didn’t take no for an answer; he assumed there had to be a narrow path through the rejections he kept getting from me, that it was only a matter of his resourcefulness before he found a crack through which to get what he wanted.

The communication, coordination and cohesion overheads that emerge with the addition of more and more staff to the roster are probably the toughest parts of scaling any business. A leader’s job is to streamline and remove these emergent dependencies between people and teams as much as possible.

Besides keeping the number of hires low, the other powerful lever that a start-up CTO has is to hire people who will be self-sufficient and require little to no support to get their jobs done. They will ask for help when necessary, but they will take pride in figuring things out on their own, checking in with you when the time is right. They will require little supervision overall, and only direction when it comes to their work. If they’re falling behind, being great communicators, they will let the rest of the team know. If they’re early, they’ll gladly pick the next interesting opportunity to be useful.

If they’re struggling, their pride in their work will make them double-down and overcome the challenge at all costs. In that case your job is to help them pump the brakes when necessary, or their pace can become unsustainable.

Cross-disciplinary empathy

A great startup hire doesn’t only think about their respective lane, but cares about the other disciplines around them that their work is impacting. They understand their angle, their priorities, and their expectations. For example:

A back-end engineer anticipates the needs of the front-end and of the infrastructure developers as they make their changes rather than waiting for their work to be pushed back once it doesn’t meet the standard.
A gameplay engineer doesn’t wait for the game designer to tell them that the feel of what they implemented won’t fly with the player. They put themselves in the shoes of the other discipline and think like them.
A front-end engineer doesn’t just roll out the interface they’re working on, they think through the UX and the UI and the usability and the product management side.

They think about where and how their work will land and anticipate those objections, addressing them in advance. They proactively reach out to those disciplines to avoid yet another coordination chokepoint that will only gobble up time. As more and more skills are compacting into one single individual, we see AI engineers, product engineers, and product managers becoming more technical and moving downstream into programming. This sort of cross-disciplinary mindset buys companies a ton of leverage and ability to iterate faster.

Of course, this becomes progressively easier with experience and seniority. But that’s also why startups moving fast benefit from more mature contributors as time goes on. The reduced communication costs multiplied across more and more team members really add up over time. Unless you have very specific technical needs, having a generalist bias in the early days is a great idea. In fact we’re seeing more and more disciplines get compressed into one, with software developers covering UI, UX, PM and engineering, effectively becoming a cross-disciplinary team of one. Even in later stages, half-a-pizza teams powered by AI can move mountains on timelines that felt impossible just a few years ago.

Tolerance of uncertainty

Early-stage startups are notoriously chaotic and unpredictable. You might be working on one feature one day, it might get cancelled the next day, and maybe now you’re fielding customer support calls for something you shipped that accidentally blew up. The next day you’re taking a trip to a customer site to chat with potential users.

At no point do you know if any of this will make you money, how long until you kill that feature, and how many more of these iterations you’ll do before you either strike gold or the company runs out of money. There are no guarantees in the startup world, except for the fact that your runway will eventually reach zero if you don’t find something worth selling.

As a CTO, it is your responsibility to shield your team from the messy everyday financial reality of where your company is headed. At the same time, I prefer to keep as much as possible in the open so that the team knows how much longer the music will play and when it’s time to start refreshing their LinkedIn profiles. I never want anything to be a surprise to the team when I could have been candid about it far in advance. It’s a fine balance between hiding the daily volatility—mostly of the founders’ moods and their confidence in the company making it—and exposing the long-term trends.

This type of universe requires people who are okay with fewer guarantees.

A startup will never have the time to run comprehensive studies, to build extensive plans, to gain all of the information necessary to make the right decision. Seeking perfection at the cost of an action is generally unacceptable, and you learn much more by failing in a valiant attempt than in delaying a perfect attempt that you might have validated months earlier by just doing the thing.

Great hires respect that process and are willing to operate without all of the information, accepting that the team will figure it out as it moves forward, often deviating from the original objective. Deleting existing artifacts or completely pivoting to a new strategy on a dime is the ultimate startup superpower.

Anybody who needs well-spelled-out plans and continuity in whatever they’re doing will ultimately struggle in that kind of environment.

Wrap-up

Hiring is a gamble, but you can tilt the odds in your favor if you stop playing the same game as everyone else. The “best” hire isn’t the person with the most GitHub stars or the flashiest resume; it’s the person who makes your team better by existing within it. It’s the David who breaks down doors and ends up shaping the direction of the company through their relentless contributions.

Developing an eye for non-obvious talent is the only way to build a high-leverage team on a startup budget. Yes, it takes more work. It requires you to actually pay attention. It’s one of the hardest feats to pull off in startup team building. It means you must trust your own judgment over a candidate’s credentials. But if you do it right, you don’t just get a “nice” team, you get a team capable of doing the impossible. Now stop reading and go find “the best”.

What is a Startup CTO?

Wed, 12 Nov 2025 23:45:18 -0800

If you look at all of the C-level roles out there, the only one that has no template or definition is CTO.

—Charity Majors

It’s 2014, a couple of years since I started as the technical co-founder and CTO at Freckle Education, and I realize I have no idea what I’m doing. I’m running technology, I have a couple of developers reporting to me. We’ve just finished the Imagine K-12 incubator—which would later fold into YCombinator—and have a few million of seed funding in the bank. It feels like I’m doing some of the moves right, but I’m completely improvising my way through this, and I hate that.

Every big decision is nerve-wracking; the day-to-day choices aren’t much easier. Do I ignore everything and just crank out more code? Do I hire more people? Fire someone? Do I adopt a hot new Javascript framework going viral on Hacker News? Do I switch us to holacracy, or whatever other management fad everybody is talking about? Maybe we should adopt Kanban. Oh, look, there’s a new post on how Extreme Programming will make your team even more productive. Perhaps this is all bullshit, and I should be looking for deeper fundamentals. But, if so, what do they even look like?

There’s got to be “the right way” here, some kind of a template. I can’t just keep guessing my way through this. My direct reports are starting to give me that look that says “Oh no, is he totally making it up on the fly again?”, but maybe that’s all in my head. They’re as green as I am, maybe they think this is how a team is supposed to be run.

As early-stage founders going through accelerators, we get flooded with advice: lean startup tactics, how to fundraise, how to talk to customers, how to avoid doing work that feels productive but isn’t. Find the customer pain point. Validate. Ship an MVP. Try to close a sale to get real signal. Don’t fall into tarpit ideas. Read The Lean Startup, The Startup Owner’s Manual, or Running Lean. What I don’t remember ever hearing is what a technical cofounder, a CTO, is actually supposed to do.

The implicit job description always felt like: “Be a hardcore software wizard, leave all the messy business stuff to the other guy, wear a ratty startup T-shirt from some long-forgotten hackathon, lock in, have a Red Bull, iterate, be cracked.” Back then we called them “code ninjas,” long before the arrival of the “10x engineer” and its now-illustrious descendant, the “cracked” developer. Anyways, the message was: “Just do the Woz thing. It worked out great for those guys.”

I would have loved to ask the experts, but I’m fresh to the Bay Area. I could maybe reach out to my former coworkers, but they have been in the warm embrace of a megacorp their whole lives, they’re of little help here. I put all of my belongings into a 1999 3 Series thirdhand beater—the first year they dropped cassettes and put in a CD player—and drove down from Seattle’s Capitol Hill to a moldy apartment complex in the thrilling town of Colma, a couple of miles south of SF. It’s the only place that will take me as a renter with zero income and zero job prospects. Turns out most are not reassured in my credit-worthiness by a big dream, a willingness to grind, and an academic knowledge of Paul Graham quotes.

A decade later, I will either personally know the right person to ask, or have direct access to countless private chat groups of founders and operators who have already figured it out, usually brought together through alumni groups like YC, a16z, small investor networks or former Freckle staff. Worst case I can get a warm intro to someone really competent and get an answer; there’s a sense of camaraderie among those who have done their time in the arena, especially if you’re still in the thick of it and haven’t moved on to more common sense pursuits like VC or a big corpo gig.

But 2014 me has none of that; I don’t know a soul, nor the right questions to ask. At least it’s not as rudderless as it was just a couple of years earlier.

The origins

When you’re completely new to “the scene” and nameless, to boot, nobody serious wants to give you any of their time. I feel like just another hopeful landing in the Bay, one bad year away from packing my bags and flying back to SeaTac. Why spend time on you? The flight isn’t what scares me; it’s walking back into my old office with my tail between my legs. That little internal chorus of “So, how’d the big startup adventure go? Guess you weren’t too good for us after all?” is already playing in my head. Of course it’s all silly and myopic in retrospect, but what does the 20-something me know?

Only years later do I appreciate how unlikely it is for anyone to “make it” here, luck being the only thing missing from many smarter and harder working founders. It felt like until I showed real chops, and that I was going to stick around no matter how many rejections, no one in the Valley would bother wasting their time on me. There were a thousand other strivers they could bet on, the stream of new hopefuls was already steady back then, and it’s only gotten more intense since.

I remember 2012 as I’m trying to at least find a business partner, I think that’s step n.1? In no position to be picky, I meet my first prospective Bay Area co-founder “business guy.” He’s working on Trumpeti, a location-based social networking app, a product that lets you “toot your own trumpet” the instant you walk into a coffee shop where you want to connect with strangers. Think: Grindr meets LinkedIn, minus the payoff. He needs a tech guy to build it for him. He claims the idea will forever change how humans connect. This is the first of many “once-in-a-lifetime opportunities" I will respectfully decline as I move on to my next co-founder date.

A couple of years later I’m in a much better place, I have a co-founder, a startup, some money in the bank, now I just need to figure out the next layer of questions I’m oblivious about. No longer about “how to get started”, but now about “how to actually CTO”.

Getting help

You milk whatever weak personal connections you still have from back home, trying to get answers on what it’s like to do this gig. Someone’s cousin was friends with your former pothead roommate in college and might have raised a round at one point in the 2000s and might still be in the startup game and have something valuable to teach you. All of the events we have access to are chock-full of people like us. Everybody is looking for answers, also having never done any of this before, it’s one big LARP session. Startup podcasts are in their infancy. YouTube isn’t filled with hundreds of YC partner videos handing out advice. It’s TED Talks, Ray William Johnson, Ron Paul videos and epic fail compilations. No countless Startup School lectures hand-holding you through every step of company building. No ChatGPT. A few insiders have the knowledge, with the rest trying to look over the wall for any hint of guidance. I’m having no luck getting real advice here, and the little I’m getting, I can’t assess for quality.

Sure, I can sling pretty decent code, but it turns out that doesn’t take you far. People never mention that if you start succeeding, soon after you’re supposed to grow a team, figure out a process, elect managers and directors, explain what you’re doing and why to the other verticals within the company, design the org and the processes as everything changes, take long-term technical bets, and much more. Somehow you need to figure out how to prioritize all of that. And somehow you must keep coding to stay relevant. But, please don’t burn out, somehow.

Everyone says, “It’s a marathon, not a sprint.” but it sure doesn’t feel like it when you’re six months away from running out of cash and product-market fit (PMF) is nowhere to be found. This is running an ultramarathon at a 5k pace, you feel the pressure to figure this out fast before the music stops. You think that maybe your investors can help, but they know only so many specialists and domain experts, and you’re still a young company, maybe they don’t want to pull all the stops for someone with a 90% chance of cratering. The ROI on that social capital spent might not be there, who knows. Either way they’re off to the next batch, or off to finding the next firm to bring into the fold. You’re mostly on your own now.

Perhaps you could ask for even more help, but your ego keeps getting in the way. “I’ll figure this out on my own” you tell yourself, “High agency! Founder more!” still not knowing what you don’t know, not fully realizing how much you would benefit from guidance. And even if you were open to it, you’re not sure who to trust. Plenty of coaches preach about the philosophy of company building—without ever sharing tactical advice. “Ok, I’ll read High Output Management as you suggest, but I don’t think that’s helping much at my scale..”

“What do you truly want for yourself?” a guru-slash-coach invites you to reflect, exploring your feelings, your relationships, your self-perception, in search of an authentic self. But that isn’t much help when this very week, you have a disgruntled employee who thinks they deserved a missed promotion, but you only had enough budget for one, and someone else was clearly more deserving, and now they’re upset, but you really need them to not leave. You have a really strong hunch they’re already talking to another company, that’s a lot of doctor’s appointments in one week. You want to turn this around before it’s too late—and you need to know what to do next, not ponder the transcendent meaning of your founder journey. Fewer crystals and sound baths and more “this is how you get yourself out of this mess: step one…”

I join a few support groups for early stage CTOs, but it’s mostly the blind leading the blind. At least I feel like we’re in this together, commiserating about our problems. Someone has to fire someone. Someone is having a fight with their cofounder. Someone needs to quit the job, they’re clearly way too miserable for this to be worth it, but they won’t give up, torching everything in their life. Someone is on their 5th pivot and is about to lose their marbles. These are stories you end up seeing again and again in startup land if you stick around long enough.

I finally manage to duct-tape together a small resource buffet. I stumble upon the CTO Summit series of events (nowadays known as CTO Connection), organized in San Francisco for technical leaders of companies of all sizes. I’m the smallest of fries, but at least I get to listen in on others discussing problems that I will soon face myself. I start meeting other people who have made it past the escape velocity of pre-Seed and Seed Round PMF-chasing. They reached the stage where they are now building organizations and figuring out how to run winning teams. They are exactly where I aspire to be.

I eventually find a mentor, Bill Crane of LinkedIn fame through a happy accident. A one-sentence referral from a total stranger on YCombinator’s internal boards, recommending him for anyone who needed help with coaching their company’s technical leader. I manage to catch him right in between interim roles, he’s stoked to sherpa someone just starting out. Here is someone for whom management and process problems that sounded catastrophic to me are just a Tuesday at the office. Bill is practical and pragmatic; skilled at seeing the big picture, but also aware of every tactic under the sun and when to apply it. Focused less on “how does this make you feel” and more on “ how could you make this problem go away, today?” My kind of guy.

Once we hire our first serious Head of Engineering, Rafik Robeal, I end up relying on Bill a lot less; we now have an “adult in the room” who has done the serious engineering management work before. I transfer all of those duties to him, and as a bonus perk, I get to spend years watching him do his magic. He transforms the team from an ad-hoc, reactive improvisation of a first-time CTO into a well-oiled machine orchestrated by a pro. There’s clarity of purpose, consistency to the process, regular feedback cycles and tasteful delegation that make it all anti-fragile. The guy practically puts himself out of a job in a couple of years, proceeds to read books about famous chess matches at his desk. Nobody makes a peep. The trains are running and the team is ecstatic; he’s earned the right to do whatever he wants. It’s an absolute delight and I get to learn how it’s all done from the front row. Chef kiss.

It’s 2024 when I start my second VC-backed company a few years after selling and leaving Freckle. This time Andreessen Horowitz’s new Speedrun incubator is our lead investor, one of the few options out there if you want to take on VC and build a high-growth game studio. A provocative proposition, and history will tell how well that business model stands the test of time, both for investors and founders. Regardless, I get the appreciation of how much easier the job gets the second time around. It’s my first foray back in the games industry in 17 years, I’m effectively starting from scratch, but many things are exactly the same. It’s back to the familiar early stage grind, hiring, managing processes, culture, best practices, architecture, cross-disciplinary coordination. It is now all second nature. What felt completely alien in 2012 is now just another Tuesday. What once felt like guesswork is now just common sense. The climb up the competence ladder is real.

Choose your own adventure

The CTO is one of those hard-to-pin-down roles in the startup world. Ask a dozen developers what a CTO does, and you’ll get a dozen different answers. When you zoom out, it’s not hard to see why.

The role has traditionally been pretty flexible based on the individual’s disposition and the company’s needs. Sometimes you’re the deep tech expert whose ideas, insights and ability to execute them the company bet the farm on. Sometimes you’re an amazing technical people leader, who spends little time thinking about the tech, because you have a trusted lieutenant to do that for you. Sometimes you’re the face of the company selling the product to other developers like you, traveling from conference to conference bearing good news.

Your specific industry influences your role and responsibilities as CTO, too. Running technology at a robotics startup, a game studio, an AI lab or a web dev shop slinging CRUD GPT-wrappers all will require something different from you.

And just to muddy the waters a little more, a CTO’s day-to-day responsibilities vary wildly depending on the stage of the company. At the beginning you’re implementing everything yourself, you then grow and manage progressively more people to do it on your behalf, and one distant day you oversee a complex human network that does that, and much more, under your guidance. At first you are the machine, then you grow the machine, and then you teach the machine to make a bigger machine without you.

No matter what, you’re ultimately responsible for the technical decisions, which means you’re responsible for the success or failure of the research and development side of the company, too. The proverbial buck stops at you, and if it passes you and ends up on the CEO’s desk, you are going to have a bad day. I aspire to have the technology side of the company be an abstracted black box for the CEO; they don’t need to know about how the sausage is made, unless they choose to: the trains are running on time and the metrics are hit, the departments have what they need to move at full speed. People are sticking around and are stoked to get to work every day. It’s one less thing for the CEO to worry about, at least in companies where the tech is enabling the product, and isn’t the product itself—think dev tools.

Hired vs grown

The two most common paths for a CTO to land a gig at a “traditional” Silicon Valley-style VC-backed startup:

Co-found it as the technical counterpart to the business-focused co-founder
Join it (usually at a slightly later stage, around series A or after) once the company has some proof of product-market fit and is seeking a technical leader to take them from that early prototype stage to its full potential

Both paths are perfectly valid, although the dynamics understandably change when the CTO is the technical co-founder and has a much larger vested stake in the success of the business. A hired CTO will always be more mercenary; their professional future isn’t resting on this one singular bet of which they own half the cap table. They can leave anytime and get another gig. As the co-founder though, not so much, the captain goes down with the ship. How they perform in either situation should not change much, but one’s motivation and commitment to stick around for the business and make it work at all costs will be decently influenced by that equity allocation.

Additionally, the technical co-founder has to adapt as the role evolves, starting when they’re the first and only technical contributor. The hired CTO typically arrives later, when more management is required. First-time technical co-founders are typically more green and need a lot more support to find their footing in the role. Professional CTOs hired into this position, on the other hand will have solid pre-existing experience; This isn’t their first rodeo. The point of hiring them is that they’ll lead the way, not demand instant hand-holding and coaching.

When do you actually need a hired CTO?

Outside of deep tech plays, a CTO is overkill for most pre-seed or seed stage companies unless they’re a cofounder doing all of the initial development on the startup’s shoestring budget. In that case the company gets a technical workhorse for free, doing the job of several otherwise normally-compensated developers, which is a good deal for the startup. CRUD apps, GPT-wrappers and other projects innovating more on workflows and distribution than on technology can get away with less technical expertise at first.

Most early stage CEOs would be happy with a product-minded founding engineer (effectively a lead engineer of one) who can take the company from a prototype to a working product they’ll iterate on with their team. You can, of course, bring in an experienced startup CTO but they have to be prepared to be a full-time IC for many years in case the company never finds market pull. Given a sufficiently frothy market, you could raise a ton of VC and hire a sizable team to execute that initial search under your supervision, but premature scaling is risky; commit this cardinal startup sin at your own peril.

If a company doesn’t start with a core technical contributor to grow alongside the firm, then it will benefit more from a founding engineer. There’s less of a fit for a professional technical leader who’s going to turn off most of their skillset for a few years and live inside of vim. Or Claude Code, who am I to judge? The full package—someone to code, lead, hire, manage and beyond—is too expensive, and frankly overkill, for most startups and will need to come in co-founder form, not as a hired gun. Coincidentally, having anybody non-technical on the founding team is becoming rare according to the more recent trends of YC batches, which you can use as a proxy for the rest of the tech startup industry.

The team will benefit the most from a hard working and relentless donkey, not a unicorn. Usually in the form of a founding engineer, unless the founders are celebrities in the technology world and are able to attract a high-end CTO from day 1 with a sizable compensation and a higher guarantee of their equity being worth something. A rare opportunity on both sides of the table. For most 19-year-old Stanford dropouts with a SAFE from hot incubator du jour, or whatever the median trendy founder profile at the time of you reading this, that’s not on the menu.

If you’re considering joining an early stage startup as a hired CTO, the best time to come on board is when the company has about five to ten engineers and is demonstrating that growth will continue. Growth fixes (almost) everything. You get to flex most of your muscles, you avoid spending years in full-time IC-land, should the business never hit PMF. Unless that’s what you find motivating, in which case, don’t let me yuck your yum.

Of course, you can join later, but the earlier you come on board, the less likely you are to encounter a dysfunctional culture you have to deprogram from the time when the inmates were running the asylum. Fewer people to let go of who should have never been on board to begin with, but management didn’t know any better. There’s no right or wrong answer here, just trade-offs.

Stages of a CTO

The only constant of the role is constant flux, and the faster your company grows, the more you have to change. The more flexible you are about your specialization, your identity, and your ability to meet the company’s changing needs, the smoother the ride. The only limits to your success in this role are your own personal interest in new responsibilities, capacity to learn a new job quickly, and willingness to let go once it’s time to pass someone else the torch.

While no two CTO’s trajectories are exactly the same, for the most part, you can expect to go through certain predictable stages as your company grows. Klaas Ardinois’s framework does a great job of outlining the evolution of a startup CTO’s role. I add a few twists of my own on top of it, based on first-hand experience.

Stage 1: The Workhorse CTO Stage

You roll up your sleeves and get to work, implementing as much product as possible, working close to the customer and the rest of the team, seeking signs of life for the product and the business model. You’re seeking a real customer painpoint, and you’re seeking a solution to it. You’re mostly in the Desirability and Feasibility section of the IDEO framework, which you’ll keep refining and expanding with each iteration. Most of what you’ll hypothesize and implement at this point will be throw-away; the only thing that matters is finding a unique insight in the idea maze before you run out of money, morale, or both.

Your initial assumptions about what customers want and how to deliver it to them will likely not work, but schlepping through validation cycles should eventually land you somewhere actually valuable. It’s unlikely you’ll nail the fit perfectly at the pre-seed stage of your company, assuming you’re raising VC. You’ll push to validate that the customer pain is real, raise another round to actually solve it, and eventually figure out how to do it at scale and at a profit.

If you can afford it, you may have a few engineers working with you. Even with a small team, you’ll do most of the work, planting the seeds of culture, workflows, and company strategy that you’ll nurture over many years to come, assuming you find a problem that will sustain the business.

I agree with Dylan Field that the progress of AI-enhanced tools will enable talented people to do far more with less. This is just more automation-enabled augmentation at play, the same as ever, only even faster. With the right tools a CTO can wear many hats, including product manager, designer, programmer, QA tester and more. Throw a few hard-working and self-motivated generalists onto the team and you can shred through iterations faster than ever before, while keeping the scale—and the associated communication costs —under better control than was possible even five years ago. The concept of Tiny Teams and Naval Ravikant’s philosophy around people curation are very much in line with my own thinking here. All of this to say, you can go farther and for longer with a tiny team than ever before, and you should take full advantage of this unique opportunity.

The good news is, you can make plenty of mistakes at this stage without major long-term consequences—as long as you successfully hone in on your customers’ pain point. Your “good enough” workflows, codebase, tooling choice, and maybe even team are good enough for now. They’ll naturally evolve as the company grows, so don’t worry about making them perfect right now. You’re at the messy experimentation stage, and it’s better to find PMF through a little chaos than to run out of funding with the perfect process. It’s a good idea to establish a basic foundation while you’re here, but no need to overdo it. The earlier you establish a culture that values getting things done, simplicity, feedback, and openness, the less you’ll have to undo in future stages. But ultimately all of the branch prediction at this stage will go to waste if you don’t make something people want.

Whatever values you choose to prioritize at this stage, your hires should already demonstrate them on day one. There’s not enough runway to sculpt raw putty. And while you might be strapped for cash, you won’t regret hiring the best-fitting people you can afford, as it will be difficult to attract even better hires down the line if all your staff is “okay-at-best”. Unfortunately that’s not effortless. It requires that you yourself are able to stand out among a crowd of startups all clamoring for founding engineers. And you must learn how to discover talented oddballs that everybody else has passed over. More on this in another post.

Stage 2: The Hybrid Manager CTO Stage

The company has raised enough funds to hire an engineering team or two to do most of the work. The CTO still contributes here and there to stay close to the work, but now, with more than 10 direct reports, your role has shifted dramatically. Your bandwidth to make much of a difference as an individual contributor will be severely limited unless you have an independent, self-driven team and a tech lead or two to help pre-empt requests that would have otherwise bubbled up to you.

Chances are, you’re still the architect, and a backseat contributor, but you’re also running the process, managing the growing team, working closely with product and other disciplines. It’s a gradual transition from being a full-time hands-on workhorse to doing it all at once, as you approach the decision point where you have to choose which long-term path to take.

The hiring and cultural choices you make become even more important at this point. As your growing team begins to solidify around the behaviors you reward, changing course is like stopping a train; sure, it’s possible, but the bigger you are and the faster you’re moving, the harder it becomes.

If you’ve created a relaxed culture of work-life-balance, generally a bad move for a pre-revenue VC-backed play that’s always a year away from hitting zero runway, a cultural overhaul is mostly out of question, short of firing everybody. The environment will not suddenly transform into a Muskian sleep-under-your-desk factory no matter what you do. By now everybody you’ve hired has bought into a particular social contract you’ve reinforced again and again. Changing direction now would require re-negotiating those expectations and likely would lose you every hire except for the most desperate ones.

Not that the transformation is likely anyway: companies are ultimately reflections of the founders’ essence. If you started a company with the natural habit of being harmonious and cuddly at all costs, becoming ultra-hardcore overnight would ultimately be inauthentic and fail as you regress to your habitual wiring.

As you scale here, unless you have the right senior people on board already to help you spread the load, or you abandon trying to do any hands-on work, this stage will suck. Sooner or later you’ll need to make a decision.

Stage 3: The Fork in the Road

Next up comes the Fork in the Road, where the CTO typically chooses a specialization as the team scales. They either go the technology focus route, with no or few direct reports, or the people focus route, with their entire attention on nurturing and growing the engineering organization as the company is scaling. The dedicated post covers that pivotal moment in depth.

Stage 4: Real scale and more

Beyond this point we’re talking about organizations with hundreds, or thousands of employees, and if you’re reading this there’s a good chance this won’t be a challenge you face for at least some time. CTOs at companies as large as Microsoft, Amazon, and Meta spend most of their day doing evangelism, technical strategy, and executive-level management. If you ever get to the stage of your career and you want to share what it’s like to operate at that scale, I would love to hear from you. I won’t speak for this stage, as it’s far outside of my first-hand expertise.

What separates the good CTOs from the mid CTOs?

It’s hard to define one simple set of criteria that will be applicable and define a great CTO across every single stage of the company, but there are traits they often have in common, at least in the early-to-mid stages of a startup.

A generalist able to specialize A great startup CTO is obviously deeply technical, likely a specialist in one or more areas of technology they had a chance to spend years on, but at the same time you’re a generalist. You’re comfortable up and down the stack, you can quickly pick up new technologies and become “good enough” at them to the point where you can ship at least a B+ grade change in them.

At the same time, you know your limitations, and you don’t expect to become a deep domain expert at most of what you touch. You know how to identify a great expert who can pick up the baton and go much deeper than you ever will. You can help them determine how they leverage their skills to be most helpful to the business. At the same time, you know where to cut your losses and where to double down on your efforts and push the team to deliver something better than they thought they could. You’re able to drop in, pick up any area of the codebase, given enough time. You can understand the workflows and the local customs, so you can contribute effectively, advise, and bounce out once you’re no longer needed.

The great CTO never forgets that the business and the mission of the company come first, and the technology is there in a supporting role, not an end unto itself.

An eye for non-obvious talent
A great early-stage CTO is able to identify talent others missed. Many phenomenal hires lack the aesthetics and the eye-popping credentials that less experienced tech leaders will use as indicators of greatness. The CTO identifies the market inefficiency and jumps on the opportunity to recruit undiscovered talent and teach them the ways of the team, giving them an opportunity to join an elite organization, while the company gets a great deal on someone who’s not yet been identified as a star.

Anyone can identify a PhD from MIT with a maniacal work ethic and a stint as a founding engineer at Cursor. The problem is, as a startup, you probably can’t—and likely shouldn’t—afford this type of candidate. A stellar CTO’s superpower is finding the wacky undiscovered geniuses the company can afford.

While this is a make-or-break skill, the fact is, it’s very hard to pull off. Of course, a serious personal brand name and a reputation in the technology community might bypass the need for any of this, but that will be out of reach for most first-timers. For the not-yet-celebrity-CTOs it’s a good idea to build a network of talented product professionals, including designers, programmers, product managers, and scientists etc. This way, while your company grows, you can tap into a pool of great people ready to join your cause with someone they already know.

A people-person
A great startup CTO must be excellent at the squishy people stuff, regardless of whether they’re a manager and growing people, or if they’re fully technical, but still needing to explain, persuade, and bring others on board with their initiatives. Irrespective of direction, be that gnarly technical projects, management, sales, or evangelism, everything the CTO accomplishes hinges on successful collaborations. The lone wolf genius coder in a cave is a myth in a cross-disciplinary team sport like technology. The technology is ultimately just the flavor on top of a dish made almost entirely of human relationships. Or “every problem is a people problem,” as the saying goes.

A great startup CTO communicates technology to every other discipline. They go out of their way to understand other specializations’ challenges and suggest the right tools to enhance each team’s effectiveness. Instead of getting caught up in the minutiae, they take a birds-eye view, looking at the future of tech, how their industry fits into that ever changing landscape, and what the competition is doing. They respond to the changing tides before they get swept up in the current, rather than reacting after the fact.

Wrapping up

The CTO role is weird in the best way. It’s endlessly interesting, full of ways to stretch yourself, and broad enough that you can steer it toward whatever keeps you curious. If what you want to learn happens to line up with what the company needs, it basically turns into a choose-your-own-adventure with a salary.

There’s always too much to do and never enough time to do it. The tech stacks never stops shifting. Management “best practices” reinvent themselves every couple of years. New frontiers pop up just when you are getting comfortable. The job is basically permanent adaptation.

And yeah, sometimes you get stuck owning something you don’t actually care about. Ops cleanup. Compliance swamps. Technical due diligence at the next fundraise. Some cross-functional initiative that feels like homework. But if you treat those detours as a chance to train muscles you’ve been ignoring, they stop feeling like distractions. Those odd corners of the job end up teaching you things you didn’t know you’d enjoy. Sometimes they’re the parts that quietly make you better, bits you’ll learn to appreciate years later, from a different perspective.

What’s there not to like?

Which Game Are You Playing?

Sat, 11 Oct 2025 23:09:03 -0700

If you don’t know where you’re going, you might not get there.

—Yogi Berra

The Paths

In every early-stage startup’s life cycle, there comes a time when you have to decide what game your employees should be playing: the Grow the Pie (GtP) game, or the Grow the Resume (GtR) game.

In the world of GtP we forget about the CV and impressive titles for a few years and agree to do everything in our power to make the value of our respective shares go up. We’re all, to a greater or lesser extent, sizable shareholders in the company, and we want it to grow and be as valuable and desirable as it can be. We hope one day we’re able to sell all or part of our ownership, and be handsomely rewarded for it thanks to all that effort we put into making the company a real meteor.

GtR, on the other hand, is the path most people are used to. Your ability to influence the company is minimal. There’s only so much you can do to make your shares’ worth go in either direction. And your percentage of the company is tiny at best. Best bet is to focus on the resume and plan the next move. Accumulate important titles. Crush impressive projects. Tell a killer story to a future hiring manager. Get a big pay bump at the next gig. Rinse and repeat.

The Early-Stage Siren Call

The magic of an early-to-mid-stage startup is that if your company is going up and to the right—a prerequisite for success as far as VC-backed startups go—then everybody onboard is incentivized to make the company be worth even more. Yes, there’s always the risk that things will not pan out, the hours are long, the pressure is high, and compensation is nothing to write home about. But landing a role at a rocketship is a rare feat, and, if you don’t need the cash right away, you might as well take advantage of it.

Nobody accidentally ends up at a high-growth early-stage startup; it’s usually a career bet on an unlikely horse, mixed with the desire to grow professionally as far as any situation will ever allow. The first few hundred employees at many of today’s rising star companies (along the lines of OpenAI, Claude Code, Cursor, ElevenLabs, Perplexity, etc.) did not get a founder’s amount of shares, but the value of even that smaller slice is absolutely eye-watering.

Of course, most firms do not reach their aspired valuations, but many do. And if you live in the Bay Area for a while, you end up meeting plenty who were just early enough at just the right company at least once in their career. They are recipients of a major windfall, either through an exit or a secondary sale, and they inspire others to try their luck at the early-stage lottery themselves. The cycle repeats with every next batch of hopefuls descending upon the peninsula as they have done since the 1840s.

Selling the Pie

It should go without saying that you yourself as a founder or a leader on the team should genuinely believe that your equity will be worth something one day, otherwise you’re in the business of selling lemons. Some industries will have more desirable equity than others. AI-anything in the peak frothiness of 2025 is a valuable ticket. Other industries like game development will have candidates scratching their heads, wondering what options even are. Many were never issued any in the past, and only ever experienced being paid in cash and bonuses.

Know your field and be honest about the prospects. Are you actually a small business masquerading as a startup? Selling staff on high-growth startup equity returns when your business is not in that race is sketchy, and you’re only delaying a reckoning once the reality catches up to the lofty promises you made to the team. Unfortunately there’s always a new wave of young space cadets entering the field, uninformed about equity and startup outcomes math, and unscrupulous founders can get pretty far without anybody realizing their slice never had a chance of going anywhere. Buyer beware.

Career Progression Complexity Awaits Us All

The reality is that it’s inevitable that a company will eventually have its employees play the GtR game. Given enough scale and an unimpressive option grant, it doesn’t make much sense for you to bet the farm on a big company exit. Even if it happens, the effect it will have on most employees is negligible, short of a trillion-dollar-scale outcome.

There is not much you can do to prevent resume prioritization once you hit scale; it’s just economics at work. The best you can do is delay it, or at least soften its onset. Growth fixes everything, and as long as you’re raising back-to-back rounds, riding the hockey stick and the company’s buzz is strong, everybody feels like their pot of gold is only getting bigger. Throw a couple of secondaries into the mix and people get validation that the magical Monopoly money can be worth something.

But once that fountain of growth dries up, the shift to GtR occurs. So what do you do before that occurs?

GtP In Practice

Some development teams decide that everybody is an Engineer (or AI Engineer, Data Scientist, Product Manager, UX Designer, etc.) until informed otherwise, regardless of experience. Or perhaps you’re a Member of the Technical Staff, or a Software Developer or whatever else sounds good. There’s one tier, that’s it. You tell your staff: “At least for now, stop worrying about your internal title, merit increases and getting to the next level. Focus on building the company and when we all succeed, you won’t have to care about your title ever again.” Some other teams choose to convey seniority with more granularity, and end up with Engineer and Senior Engineer as the only two options. Others provide many levels, but still don’t spend any time on frameworks. They eyeball you to one of the tiers and move on, and if you spend too much time arguing over what title you should have gotten, you weren’t a fit for this company stage anyway. All of these strategies have worked in the past.

Even without internal titles, don’t let seniority be what determines who’s right. Make sure that people’s arguments are what wins, and not their years of experience. If someone presents a better design, better data, or is able to more elegantly argue their side, they should win on those merits. Avoid being eaten by the HiPPO (Highest Paid Person Opinion) or its implicit title equivalent. An Engineer with 20 years of experience should lose to one with two if their arguments are worse. One of my fondest memories of working with engineers at Freckle was how free they felt to tell me, the CTO, when I was wrong, leading to my own thinking being improved at a much higher rate than it would be otherwise. It was a little short-term sting to the ego in exchange for actually getting better at my job longer term.

There are also plenty of titles that do not make sense at early-stage startups. If you’re anywhere prior to Series B and you already have Principal, Staff or Distinguished engineers, something went off the rails. Many super-senior titles are a reflection of the scope of responsibilities and the need to interface with many teams and nurture complex systems which simply do not exist at those earlier stages of a company.

“But wait,” I hear you say, “Don’t titles make sense for coordination? Won’t people get confused about who does what if everybody is just X?” Great question, but it’s mixing up the desire to signal one’s career progression with the need to make one’s responsibilities visible to the team. You can be explicit about who leads what, who owns what part of the company’s processes, or who’s the go-to person for critical module ABC. People can be given those roles regardless of their career progression label.

The same one or two bands of seniority can be applied to other roles as well; this compression of titles isn’t only applicable to product teams. The main difference is that product teams tend to be internally focused and don’t need to impress the outside world with important-sounding titles. The same can’t be said for the wine-and-dine disciplines charming customers into an eventual ACH transfer. There the title someone displays should be part of a broader strategy, not left up to them to decide.

Speaking of external titles, how do you handle that side of the equation?

External Titles

Of course, as far as the outer world is concerned, titles very much do matter, and yet you’re not in the business of controlling what your employees choose to put on their LinkedIn profile.

A popular approach is to allow employees to use whatever title they want to signal externally, and to have their managers confirm those titles in case anybody from the outside ever asks, for example, years later when they need a referral. If that’s something that helps them get hired or present well to future employers, great, no reason not to play ball. Most people will be reasonable and not inflate their title beyond what’s plausibly defensible, so you might as well validate it.

At the same time, you internally bucket people into whatever few categories you create, and those buckets are known and visible to everybody internally. Regardless of what they choose to put on their LinkedIn, their peers know what seniority they have internally to the company.

“Okay, but, what does this mean for people’s compensation?” you might sensibly ask. “Won’t it be super opaque as to whether people are being paid fairly?” I’ll cover this in a future article, but the short version is that I’ve tried both approaches and they each have their tradeoffs.

At Double Dusk, we went all the way to the extreme by making everybody’s compensation and equity visible to the company. Everybody could see that people were compensated fairly and there was a system behind it. Nobody was treated any differently from anybody else just because they happened to negotiate better. No chance of being stuck working for years in the same exact role as someone else, making less than them. This approach requires more titles though, and it will lose you the extreme talent that does not fit into those boxes. If we use the $100M AI researcher as an extreme example, some individual contributors can make 1000x more than others, and people understand that’s just the state of the market. Of course if someone ends up with a dramatically higher pay and performs at the same level as someone else who’s paid less, you have a ticking timebomb on your hands once a Google Sheet of people’s salaries starts circulating the office.

The fewer career tiers you have for a discipline, the more compensation will be a judgment call rather than the result of a strict and perfectly equitable formula. Early stage, that’s arguably for the best. You’re a temporary dictatorship formed to survive inevitable company death within x months, and there is little time for making the perfect fair process with a single- or double-digit sample size of staff. Once you’ve guaranteed foreseeable survival, you can go back to fixing all of the shortcuts you took to get there.

So when do you do that?

Timing GtR

A simple rule of thumb is to not worry about titles and career progression until Series B or later, assuming the company isn’t stuck getting there for a decade. You’re not out of the woods until that point. It might turn out that product-market fit was a mirage, and you were stabilizing and adding process to a company that didn’t have legs. Premature scaling is the root of all evil in early-stage companies, whether that’s with regards to staff, process, or technology.

Once you’ve confirmed a repeatable, sustainable, and scalable cycle for generating revenue, you’re Default Alive (to borrow Paul Graham’s term) and Probably Going To Do Well (to borrow mine). It’s time to add more structure to people’s career progression. Until then, you may be months away from shutting down, in which case their career progression will have to progress at a different company, not a bankrupt one.

Once you start shifting away from the land of pie-growing, it will be time to pay for all the debt you accumulated up until that point.

Paying the Debt

Of course, if you choose to go without titles or with an extremely quantized version of them, you’re going to have to have a day of reckoning when the time to level up comes.

Once the astronomical growth slows down—and it always does, even in the biggest success cases—the former carrot needs to be tossed out. Time to roll out the traditional career progression management tools. Career frameworks, competency matrices, arguing with your staff over why they got 3 out of 5 on one particular box of a complex model. That’s now your reality. True, it’s not as high-adrenaline an activity as adding digits to your company’s market cap each year. That’s not to say that some won’t still do well equity-wise, but for most the math will stop making as much sense.

You must clean up the debt that you deferred addressing. You have to make sure people are placed in the right buckets. You want to make the compensation fair based on those groupings, with all of the predictable messiness of that process. Some might get a better title than they would otherwise because you’ve been paying them more than you should have. Others may be performing at a higher level than their pay, in which case you end up bumping them up in comp to reflect that. Either way, you can never reduce someone’s comp without a serious, and understandable, morale hit. You may have to grandfather them in and hope you don’t have too many of these exceptions.

And watch out for the cultural whiplash of this change. People who signed up to play the Grow the Pie game—“we’re scrappy, titles don’t matter, let’s just build something valuable”—suddenly find themselves in a world where career progression is tracked via a spreadsheet, you’re doing 360 reviews and your impact is translated into a box in column D. Some will adapt and say, “Alright, I get it, we grew up.” Some will breathe a sigh of relief, glad they are given more structure. Others will feel like the original social contract quietly expired. They joined a pirate crew, and now you’re asking them to wear naval uniforms and salute.

Some stick around and adjust to the new reality. A few bounce, looking for more green fields where they can be pioneers again. Most have only so many attempts in them at chasing the thrill of early stage chaos and following an 18-year-old founder with tens of millions in funding into the breach. They remember what you were like when you had just started as a founder and they don’t want to see that movie a second time. Most will seek a more stable, more predictable workplace, especially now that they’re older and have more real-world obligations than when the current company was in its infancy.

As the technical founder/CTO, you need to be prepared for wading through the Resume-Driven Development swamps, fighting against adopting complex tech that brings little value to the business, but looks good to future employers. Growing the Resume understandably pushes engineers towards this, but it’s in the company’s interest to focus their growth on other dimensions of their competency, rather than slapping yet another hot tool onto the CV. Seen much Hadoop on resumes lately? I would rather see senior-plus individual contributors demonstrate the ability to manage serious complexity with elegance rather than playing tool bingo or reinventing the wheel.

Some late stage companies do a 180 and do the exact opposite of what I suggest. Mature scale-ups like Affirm allegedly have their engineers rewrite the core of their product on a yearly basis in order to always have something big and impressive for them to do. It’s a serious investment only to spin one’s wheels, all just to retain competent and hungry staff that are constantly craving for a challenge and an opportunity. With sufficiently successful products and sufficiently hot markets you often see companies fall into the pattern of doing whatever it takes to retain their staff, even if they’re not given much to do, so this is far from the worst option on the table. Still, not one that makes much sense for early stage teams that have plenty of lower hanging fruit to pursue.

Conclusion

To recap, an early-stage company’s best bet is to focus on Growing the Pie rather than Growing the Resume. Delay the distraction of resume-building until much later, lest the team spend time on all the wrong pursuits which will only accelerate its demise. There’s always time for ladder climbing once the firm has reached escape velocity. That’s a fortunate state that the company has to first earn.

The Fork in the Road

Tue, 07 Oct 2025 06:53:03 +0000

When you come to a fork in the road, take it.

—Yogi Berra

The Breaking Point: When You Can’t Do It All Anymore

It’s 2016 and my company, Freckle, is finally seeing real traction with teachers. The self-serve freemium plan is bearing fruit and new classrooms are getting enrolled every day. Kids are hammering the service submitting millions of answers to math problems. We’re fighting to keep the servers running, to add more features, to keep the engineers happy in a market that’s aggressively heating up. Revenue and VC are joining forces to support all this momentum.

Growth is good, but I’m drowning; scaling is kicking my ass. I’m leading close to a dozen developers, reviewing every single pull request, running our bastardized XP-meets-SCRUM delivery process.

I’m investigating support tickets. Is this actually a bug or are the kids trolling their teacher by injecting text into the web page?

I’m the architect, I feel the need to vet every change.

I’m the IT guy. A salesperson’s laptop is choking up on 300 tabs and they need to get on video calls, I think we need to get them a machine with more RAM. Someone needs their password reset. I’m not going to waste a developer’s time on this, but we also can’t afford a person full time to handle it yet. One more thing on my plate, I guess.

I’m doing the first interview for every candidate. Actually, no, I’m doing all of them. I’m hiring, I’m firing, I’m dealing with the aftermath.

I’m blocking the team by taking too long to ship a change that’s been on my plate for days.

I’m in the “exec” Monday morning syncs. I’m regularly pulled into investor conversations for our next fundraise.

Nobody reporting to me feels seasoned enough to pick up any of my batons, and I’m trying to control everything all at once in an effort to ensure success — at least that’s what I’m telling myself. I’m not confident enough in my delegation to trust the junior or mid-level staff with what I’m doing. I have to fire the one guy who had enough experience to be in a lead role; he hasn’t shipped anything in weeks, sigh. Someone quits at the worst time possible and we have a gnarly outage. I stop all caffeine to slow down my anxiety.

I’m losing my marbles, and yet, the cavalry is not coming; I have to dig myself out of this before it gets any worse.

A well-trodden path

Denial is the first response. “You’ll be fine, you can power through, you’re hardcore, something something founder mode” you tell yourself. You have a lot to prove if it’s your first-time in this role, except, you’re human and hit the inevitable limit, the sugar-free Red Bulls can only take you so far. The team isn’t getting enough of your attention. Your individual contributions are rushed or are raising eyebrows. You lack bandwidth to go deep and handle complex tasks that require ample Maker Time. Everything you touch is half-assed; you’re just trying to keep your head above water in a dozen different areas that are all screaming for your attention all at once.

The situation is familiar to every first-time startup CTO who’s experiencing growth and the pull of the market. If you successfully survive the very first phase of your company’s existence—the bit where you’re flinging prototypes at the wall hoping to prove that someone wants what you have—you graduate to traction and further fundraises. You soon reach the limits of how far your team can grow with you in charge of everything. If product management, design, and other roles report to you, you hit that wall even sooner.

Much of what’s challenging for first-time, early stage CTOs here is simply their lack of reps in the realm of management and scaling organizations by adding more people, delegating, forking off teams, and many other tried-and-true techniques. Add identity conflict into the mix, and being so deeply invested in the success of the company as the technical founder and the challenge goes beyond pure management. You care more than anybody else about the company doing well, you have a lot riding on it, and it’s hard for you to let go of control, and yet you must scale.

If you are already an experienced manager and delegator, you can stretch yourself a little further here: you know which reins can be given away with little threat to the trains running properly. But either way, eventually you have to figure out how to scale yourself and the team. There are only so many hours in the day, you can’t cheat physics, and going hero-mode eventually leads to a meltdown.

This is where every early stage CTO faces the inevitable fork in the road.

Technology vs People

For many years now the industry assumed that a growing early-stage startup will divide its engineering leadership into one of the two roles: a CTO and a VP of Engineering (VPE). Years have passed since Fred Wilson wrote that article, but the fundamental principles haven’t changed much. I use the “VP”, “Director”, and “Head” of engineering titles interchangeably here; they’re fundamentally the same role at different seniority levels, at least as far as early stage startups are concerned. Tomahto, tomayto.

This is the choice: do you go all-in on ensuring your company creates and uses awesome technology, or do you choose to spend all day on having the best people and the best processes?

You can’t do both, not well at least; each direction requires full-time dedication and a completely different skill set. Neither path is wrong, but one could be a bad fit for you. Having walked down both of them at different times in my CTO career, I can speak of the nuances behind this choice and how it can impact you. Screwing up this decision can lead to years of dissatisfaction, and yet, knowing in advance which one will be best for you is perhaps not possible. You might only be able to get the answer the hard way.

The good news is that the startup CTO role has always been fluid, flexing to accommodate the unique needs of each company, and the quirks of each person in the seat. As the company approaches PMF and starts to show hints of scale, you get the opportunity to choose how you will partake in this next phase and beyond, and how you will fill the gaps in the team left by you specializing your focus.

The classic Five Flavors of a CTO was a model I often referred to in my up-and-coming technical co-founder days. Your strengths and dispositions influence this a lot. So do the needs of the company. Do you do well in front of customers? Do you like to talk at conferences? Can you sell? Can you be the internal face of the company, acting as a pseudo-COO? Are you a deep technical domain expert that everything is hinging on, or a generalist who is a mile wide and can rally the right people on the right tasks? How well do you manage and lead people, and do you enjoy it at all, or does it feel like a distraction from being deep in the product? Would you rather create routines for computers, or for humans?

Do some soul-searching and pick a direction that feels motivating to you for the next few years.

Why the People Path Feels So Unnatural

Unfortunately, many green CTOs aren’t great at all of the squishy people stuff. You wouldn’t be in this seat if your recent career focus was on growing and managing teams; that’s not what the company needs from you the first few years of the business revving up. You cut your teeth on furious shipping iterations, on managing the complexity of a codebase that’s reaching Product-Market Fit, on knowing how every corner of the product works, on staying close to the customer and relentlessly trying to meet their needs.

That’s got nothing to do with being able to build repeatable, clear processes that involve humans working in unison on a common goal. Little to do with understanding why a direct report is struggling and how to fix it. Orthogonal to knowing how to take your more successful people and make them even better. Little in common with how to create a shared sense of purpose.

The 2016 me would have done just about anything rather than spend a whole day in one-on-ones, having career conversations, moonlighting as a therapist—at least that’s what I thought managers were supposed to be at the time. The company needed to keep moving forward. Spending days understanding people’s feelings to retain them in the frenzied meat market of peak-SaaS San Francisco felt misaligned with the short term do-or-die needs of the company running out of runway. Someone is disgruntled that they didn’t get a promotion. “Ugh, why am I spending time on this? How do I make this problem go away and get back to work?” Two employees are beefing. “Why are you pulling me into this?” Someone took offense at something another person said. “OK.. you guys can’t figure this out on your own?” In all fairness, engineers’ engineers tend to be as low on drama as they come, but even playing the game on easy mode felt like a waste of time to me at the time.

With the benefit of hindsight, many managers from that era eventually realized that coddling and worrying about every minuscule inconvenience wasn’t going to scale or lead to great teams, even in light of a really competitive market. But, that’s for a different article. Managerial maturity and expertise ultimately make a huge difference, and employees not hearing the daily siren song of a different company luring them away doesn’t hurt either when you’re trying to focus on advancing this company’s mission. Not constantly fending off incursions from another SaaS “soonicorn” two blocks down on Market St.

It’s really tempting to short-circuit having to learn all this people stuff yourself and instead bring in that expertise from the outside to finally stop drowning. If you go the people route as a first-time CTO, you know you’ll suck for years as you figure all this “managing and scaling teams” stuff out, and you want to stay in the comfortable embrace of being a technical badass. There’s just too much at stake for you to let everybody down for years before your manager and manager-of-managers skills ramp up, so bringing in a VPE feels like a light at the end of the tunnel.

Hiring a People Leader to Handle It for You

You choose to hire someone to manage on your behalf. This person will likely hold a Head, Director or VP title. Lofty titles are one of the perks of startup life even when you only have one team of engineers to manage.

Identifying the profile of someone ideal for this maneuver is tricky. Sometimes they’re a line manager who believes they can level up to a bigger role. Sometimes they’ve already done this exact gig and they’re eager for a repeat: it’s their comfort zone, they know they’ll be good at it. At times it’s someone who came from managing much larger organizations and believes they will be able to step down in scope and run just a couple of teams for the foreseeable future. Both the first and the last options are the big gambles.

With the first one, you have unproven talent that you hope will pan out. But unless you have good signal that they’ll be able to do it, it is speculative at best. You likely do not have enough experience to properly vet them, evaluate if they’ll be able to move up one level of managerial abstraction, figure out if they’ve ever had to define complex processes for a company that needs to move at breakneck speed. They might have executed someone else’s script at a well-functioning company, but never had to test their ability to create that themselves from scratch.

For the latter, it’s frequently a seasoned manager from a larger company who got bitten by the early-stage startup bug. Unfortunately, they often romanticize the job, and only later realize what they signed themselves up to. It’s hard to step down to a world of duct tape and chewing gum once you’ve learned to lean on the support structures and rails of a larger org. The headcountmaxxing type is going to struggle in a scrappy environment that expects them to accomplish miracles with what little they have.

The “done this before” option, unsurprisingly, turns out to be the most consistently reliable: usually it’s someone who hasn’t gone too far up the career ladder, and who really loves this stage of the startup world where they thrive. The company is small enough that everybody knows each other, but not so small that managers or directors are overkill. They can make real change, shape big chunks of the company first-hand, but they don’t have to convince and coordinate for months to accomplish it. Lots of ownership and control, not a ton of BigCo BS. We ended up with this kind of leader multiple times at Freckle, and it was a game changer for the company each time.

Great! Now you have a seasoned VPE taking care of production, making sure the staff is happy, hiring, firing, unleashing processes, likely reporting directly to the CEO—not you—because they want that direct access. Having an additional layer of indirection at an early-stage company makes little sense.

So what does this new addition to the team mean to you, as the newly-specialized technology leader of the company?

The Trade-off

You experience the delights of a clear calendar. Your counterpart is handling one-on-ones, they’re on top of hiring, they’re running delivery. Sure, you just hired them, they could still screw up, and you regularly monitor how they’re doing to make sure things are in check. You get to run tech now, great, but you realize you no longer have an important lever you were used to.

You have removed yourself from controlling the budget—and that can be a real problem. You can’t make hiring decisions unless you disempower the new people leader from owning the process, and you really don’t want to do that. You usually still have veto power if you’re a founder. You should absolutely have a strong opinion on whether someone is a go or no-go as they get towards the end of the hiring funnel. But quality leaders want to be treated as adults and expect autonomy to direct the future of their team, formerly, your team. And rightfully so. Being accountable but not responsible is a bad place for anybody to be.

Remember, you focus on the technology, they focus on people, so you need to set the expectations around your veto card from day one if you want your relationship with the VPE to last. In practice, assuming you are aligned with them on the quality bar for a new hire, you shouldn’t need to intervene often. Whe it does happen, you don’t want your veto to come as a surprise.

And later if you think someone on the team is below the bar, the VPE thinks they’re fine, you can’t do much about it. You feel the company’s bar lowering, but the best tool you have to deal with it is again influence.

Beyond hiring and firing, you end up not having control of the direction of delivery and how engineers are allocated within it. These are not your teams to manage; these are not your decisions to make. The budget is in someone else’s hands, and the most you can do is advise and influence. This person does not directly report to you, you can’t get them to do anything they don’t want to.

That means that on the technology front, which you are responsible for, you can’t get anybody to implement your vision unless the VPE and their team agree, or unless you’re doing it all yourself. This is not to to say that mandates are a lever you should be pulling often even as someone’s direct manager, now that option isn’t even on the table. You think that rewrite is mission critical and unlocks big benefits for the team long term? If the VPE thinks that’s a distraction from shipping more product, tough luck, not much you can do about it. You now operate exclusively through influence.

Some CTOs find that very frustrating and wish they had more control over the team. They’re used to the codebase being their baby, to having the final say, and losing that understandably feels disempowering. Others are more zen and accept that it’s the requisite cost of growing up as the team scales, it’s not all about what you want anymore. If you hired the right people, having to get them on board with whatever initiatives might be actually for the best, you end up with higher quality decisions. But you introduce extra coordination steps that go against your “let’s just move fast, here’s what we’re going to do” instincts.

The Skunkworks Mirage

“But wait”, you say, “what about the CTO being given a few engineers to run experimental skunkworks-like projects? They can be out there on the frontier, placing bold bets on new approaches and tooling”. In practice, it’s unclear how effective this mode is, and how often this model succeeds. The company is never swimming in excess budget. The startup needs to hit aggressive metrics every single year as it’s growing, assuming VC backing.

Dedicating staff to experimental endeavors is not going to be palatable to the management team trying to hit ambitious revenue goals mission-critical for the next fundraise. At the early stages, there’s little time for “maybe this technical project will pay off months or years from now”, there’s only time for “we need this feature shipped now so we can move upmarket and close the big customer asking for it”. The aspirational “one day this will pay off” bets end up being sidelined and poorly resourced.

Just incentives at play.

A Matter of Identity

Early on your self-image as a CTO is tied to your ability to contribute technically. Your identity as a manager is both underdeveloped and unfamiliar. You have not shed the skin of a hands-on contributor since that’s what got you, individually, into that seat, and it got your company to the stage that allowed you to worry about scaling. Nicely done. But holding on to that identity will not get you to the next step.

Knowing what I know now, it’s possible I would’ve still made the same decision at the time, staying on the technical track the first-time around. I feel very fortunate to have learned from talented directors whose best practices and strategies I could then adopt myself when I eventually switched course. Nowadays, however, I advise first-time CTOs to tread these waters carefully and seriously consider the people path.

It Works for Some

At the same time, it’s worth mentioning that there are CTOs out there who are seemingly happy with their choice of staying technically-focused. Greg Brockman of Stripe/OpenAI and John Wang of Assembled explain why they chose to go that route and how it turned out.

My read is that with the right personality, the right amount of company growth, the right partnership with the VPE and the right buy-in from the company, the technical path can work well. I love to see follow-up articles from them years later describing how their choice panned out, with the benefit of hindsight. Would they do it all over again the same way?

The Case for Staying in Charge (and Learning to Manage)

I remember feeling that, when the time came to pick my own CTO path, nobody on the team had the expertise to replace me technically and to take the lead. In my mind I was special and irreplaceable, nobody could make technical design decisions as well as I could. I thought those decisions were irreversible and company-ending if not done right. I thought we didn’t have time for screw-ups, and my screw-ups would be fewer than others’ screw-ups. Should I have been smarter in my hiring, having a few viable succession candidates? Or did I already have them, but didn’t trust myself to deputize them? Was I way overthinking the whole thing? With more years of management under my belt than most of my team—admittedly still far from many—I was not willing to take bets on my own people. In hindsight, this was a mistake. I overestimated my own competency, irreplaceability, and told myself that others couldn’t live up to that bar.

Only looking in the rearview mirror can I now identify these elaborate stories I created to avoid leaving my comfort zone and avoid mistakes that would have been worth an attempt.

The irony is that being a strong technical contributor and architect as the CTO is the best place from which to pass on those duties to someone else: you’re able to judge the quality of someone else’s work, coach them, intervene if necessary, impart your personal philosophies onto “the next generation” when it picks up from where you left off. You need to let go of the toys you got used to: having all the answers, having complete control, being always in the loop, sticking to what worked before. Your team will also appreciate having someone with your experience and deep knowledge of the codebase and the customer oversee their development. And they will also appreciate you not trying to control everything so tightly at the cost of their own learning and the speed of progress.

I get it, all the non-people stuff makes you feel valued and respected, the work gives you joy, you feel competent, safe. You worry that letting go of what attracted you to the job to begin with will take all the soul out of the role. Etienne De Bruin of 7CTOs talks about the loss of joy in the work as a technical leader transitions from making to administrating. But is that what the company really needs from you at that point? And if control as a CTO is paramount to you, then accepting this new hat as a people leader is very possibly the better option for you.

Your transition to mostly-manager will be rickety at first, that’s to be expected. Get a coach or two to help you through this, and you’ll be surprised by how fast you can learn. Management and processes aren’t rocket science, every scaling team has had to figure this out, and the best practices are well known. Sure, not everybody gets it right, and plenty of managers are atrocious, but you can get good with the right support.

You stay in charge of the team. You stay in charge of product delivery. You can operate through your best people. Have them run experiments if you want to, it’s your call now. You are not adding any additional layers between the CEO and the engineering team, and this forces you to be that better manager that your team might want from you. As the team scales, you do your best to keep scaling with it, keep getting help and feedback.

And when you get the urge to experience the joy of shipping, of learning a new tool, of feeling closer to the codebase, you have the power to make time for it. It won’t be much, but it might be just enough to bring back some of the maker’s spark.

Given enough time you will likely appreciate the people path, and the satisfaction of a well-operating team that trusts you to take care of them.

Other Paths

And if you discover you really don’t enjoy this type of work, you can always get back to the trenches. You can find someone to replace you once you’re sure you hate it. The VPE of your dreams is out there, I’ve found them, and so can you. You become a kind of distinguished engineer, a figure that everybody loves—seems to have worked out for Mitchell—no longer involved in day-to-day people decisions, but still happy contributing in whatever way they find most engaging.

Or you can explore some of the other flavors. You could become more public-facing. You could do more sales. Do more speaking engagements, help build the brand to help you hire high-quality people. Get buy-in and do special projects such as making developer onboarding better, improving dev tooling, improving documentation. Remember: You aren’t gunning for a promotion; you want to make a difference. Get creative, as long as you are honest with yourself about how much control you want over how the sausage is ultimately made.

Hiring - Telling Your Company's Story

Sat, 04 Oct 2025 23:29:15 -0700

Those who built the visionary companies wisely understood that it is better to understand who you are than where you are going — for where you are going will almost certainly change

—James C. Collins

Introduction: Hiring Is Storytelling

Hiring is more than selling a gig, it’s about offering a story that people want to live inside. The founding myth. The gnarly obstacles yet to be overcome. The promise of a better world after the dragon is slain. Your place alongside others through that quest.

When you’re building something early, you’re not just selling a job description. You’re offering a narrative people can project themselves into. The numbers, the perks, the glass-door ratings, those are secondary. What matters is the sense that this company is going somewhere, that this moment matters.

This is part two in a series on hiring for CTOs and other early-stage leaders. And while timing and tactics shift with every hiring season (as I covered in A Tale of Two Hiring Seasons), the deeper truth doesn’t: people join stories, not spreadsheets. The tactics evolve, but everyone’s search for purpose doesn’t.

So we’ll look at how to make that story tangible: who you are, what you’re offering, and how to make others care, especially when no one’s heard of you yet.

Regardless of whether you’re passively hoping to attract the right people, or if you are proactively reaching out, the underlying lifecycle is the same: They must become aware of your existence. Your existence must be appealing enough for them to apply and schlep through the process. They must like the prospect of working with you enough to sign on the dotted line. Once they join, you must continuously deliver your side of the promise. All common sense stuff: the execution is the hard part.

Defining Your Identity: Who You Really Are

First and foremost you need to make it clear what you’re all about. Joining you is ultimately an emotional decision, so you need to dial in the vibes just right. The numbers can only do so much if the idea of your company doesn’t move someone.

Why does your company exist? Is there a genuine mission here? Why does it matter? Do you actually care about it, or are you LARPing, checking off the “mission and vision” boxes that every company out there has to have? You want your enthusiasm to be genuine and infectious, so anything that sounds corporate and overly sanitized has to go. Early stage startups are allowed to express themselves and take a strong stance on the world, and if you’re under 100 people and are already trying to sound generic and inoffensive, you’re practically invisible.

It’s ok if you’re not for everybody. You will not be perfectly inclusive. You won’t make everybody happy. Again, that’s a’ight, take a deep breath and internalize it. You’re not Walmart. At this stage, your personality doesn’t have to scale to millions. It has to put a spell on just a few, maybe a dozen, maybe a hundred if you’re lucky. If you’re in the early-to-mid stages as a startup, you should be unpalatable to the majority. The key is to make that polarizing flavor work for you, passively filtering for the kind of people you want on the team.

VC-backed startups make up less than 0.1% of all US businesses. They’re an extreme sport that attracts a mix of young, restless, wacky, ambitious, otherwise possibly unemployable individuals who might not fit at a traditional corpo gig. Startup employees seek a way to prove themselves, grow faster than they could anywhere else. They also take on the risk of worse pay, terrible management, their employer running out of money, atrocious work hours and more. “Small wages, bitter cold, long months of complete darkness” as the apocryphal Shackleton’s ad promised.

Sometimes it’s not just running towards, but also running from. Many applicants want to avoid the predictability, specialization, compliance and conformity of a big co job. There’s a time and place for that, and if you’re successful at your job as a founder, that will be your company’s lot in life as well in a few years. But for now, you should be offputting to most who would be better served in something less speculative than an early stage gig. Great short video here on that subject from Speedrun’s Jordan Carver.

Once you’ve clarified who you are, the next question is: what are you offering?

Make a Promise: What’s in It for Them

What sort of vision are you promising to people considering joining you? Can they picture themselves as part of it? What are they getting out of it, besides cash, which will be lower than just about anywhere else? Put yourself in their shoes and be honest with yourself. Would you join this company if you were hearing about it for the first time? Would the brochure pique your interest?

You can promise infinite riches. Fame and glory. A mission that will make a real difference to people you actually know. Camaraderie, and an opportunity to learn from spectacular coworkers. A brand that will pay dividends for the rest of their career. An invaluable network which will create opportunities and life-long allies for years to come. Adventure. Cultural relevance and coolness factor. Maybe a move to a location they always dreamed of trying out. The list goes on, get creative. Most of these are intangible. That doesn’t make them any less important to figure out.

Even the aesthetics of your career page matter, and complement the storytelling about who you are. The visuals, the consistent voice, a unified vision for how it all ties together.

One of the first things we did at Double Dusk was to invest into a beautifully hand-crafted landing page using tasteful, organic art and a parallax effect to make it obvious that this was a place for craftsmen and auteurs, that we took the art of making games seriously and wanted to find others who felt the same. We weren’t about quick asset flips and AI slop, we wanted our customers to feel the love that went into the product, feel like the people crafting it viscerally cared about the medium.

Was that the best business decision? Meh, probably not, but we took a stand and were genuine about who we were. We couldn’t be any other way if we wanted to, this wasn’t an artificially focus-groupped conclusion. In the end the people joining us were exactly who we were hoping for, they found a tribe that embodied their same values. Their first encounter with us started with a unique visual impression, especially important for a firm priding itself in its standout art direction.

Solving a seemingly insurmountable problem really helps. Would you rather work on making humanity space-faring, shooting 5000 ton rockets into the stratosphere, or on slightly tweaking an HR software portal with a few ChatGPT calls on top? Life’s short, for many, the choice is obvious, especially earlier in people’s careers when taking bold bets is worth more than safety and predictability of cash flow.

You can lean into a particular philosophy, think TDD and Extreme Programming for Pivotal Labs. Or on a particular technology that offers a unique approach. Or a unique way of working together. Think Gitlab and their early push for remote work in the 2010s. Or Subscript who only work asynchronously, no meetings ever. Weird, right? Yes, but it also attracts a particular type that’s going to absolutely love it there, and turn off many who would be driven nuts by that type of work style.

At Freckle I united the team around the vision of using purely functional programming through Haskell (see my article Haskell at Front Row), the most sophisticated tool on the market for building complex backends that never break, can be iterated on fearlessly, and where bugs are a rarity, rather than everyday reality. Controversial opinion? Absolutely. But it was also a beacon attracting engineers who dreamed of working professionally in that kind of environment, being able to learn from some of the best active practitioners in the industry. The pool of applicants was small, but half or more of those who applied were worth an interview. It was an effective passive filter that did much of the selection work for us.

Don’t forget to paint a picture of the work style you follow. This is understandably top of mind for most people applying. Maybe the culture is mellow and friendly, with a four-day workweek and a focus on balance. Or maybe it’s a 996-style grind for the foreseeable future. Some teams thrive on competition, others on collaboration. Some are experimental and action-driven, while others lean heavily on theory and deliberate design. A company might attract doers who ship fast and learn by trying, or thinkers who obsess over frameworks and precision.

The environment itself could resemble a 20-something hacker dorm in San Francisco: developers taking drags from their vapes with one hand, flipping through TikToks with the other, all while waiting for Claude Code to finish. Or it could be a more mature team with families, soccer practice, and suburban homes. Those crazy-hour startups trade experience for hustle, the more seasoned teams trade hustle for proven talent that doesn’t live in a sleeping pod. Whether that tradeoff fits your current stage is another conversation, but it’s a choice worth making consciously. Now that you know what story you’re telling, it’s time to show it.

Show, Don’t Tell: Choosing The Medium

Getting tactical for a moment, how you go about painting this picture can take all sorts of forms. This is nothing but content marketing repackaged for hiring, there’s nothing mystical about it, just a lot of elbow grease and trying to dial in the language and the channels until you get them right, and until they eventually become shitty.

If you’re handy with videography, there’s nothing quite like shooting OpenAI and Anthropic-style videos to show more about how the sausage is made, showcase some of the work and some of the more interesting people at your company. It’s a lot of effort, but this creates a sense of familiarity with you, and some people will resonate with how you tell your stories. It doesn’t have to be highly produced either, there’s something endearing about the authenticity of an early stage team, the scrappiness of it all. Use it as your strength, rather than trying to copy what $500B companies are doing.

Podcasts are great too, they allow you to go in depth into as many topics as you want, trading length for high production values. Having the founders or a few talented team members showcase their passion for the work and what’s coming next can be infectious.

Blogging is a tried and true method as well, again with its own pros and cons, starting a conversation on sites like Hacker News where team members can chime in and create a sense of community with the broader technology ecosystem.

Conferences are solid as well. Send a few engineers to do a talk about something interesting you’re working on, a non-obvious insight you have obtained, or a fresh take on something people might be already doing. Generate content for your company in the process, re-use it for years to come. A variant of this is running a meetup on a particular topic, positioning your team as the experts in a particular space. Whoever shows up to these will be likely a great candidate for future hiring pushes, just make sure you make time for people on your team to socialize with the attendees.

Social media can work as well here, not my forte, but some people are really artful at it and will generate outcomes, although I’m unclear how well they compare to the other methods here.

Hackathons and other events you organize are also a well-trodden option, they allow people to meet your team, perhaps see your offices, build a more personal connection with your brand and the work that you do, perhaps even win a prize or get to ask your team questions about what it’s like to work there. I’ve always been surprised by how hard some companies make it to ask their employees what it’s like to work at that company and act as ambassadors for your company. I suppose it’s a waste of time in most cases, but carving time for promising potential applicants who are trying to learn what it’s like to work with you seems valuable.

This all sounds sensible, but what’s the catch?

The Tradeoff: Storytelling Is a Long Game

Unfortunately, this all takes time and effort, and is typically not the expertise of a technical CTO who’d rather be in the terminal all day. None of it is urgent, and none of it produces instant results. That’s a cursed mix. In startup land that’s code word for “we’ll never actually do this, way too much other hair-on-fire work we already aren’t getting to”.

Unless there’s someone at your company who owns this, it will always be a nice-to-have. Every team regresses to doing nothing about building the brand until it’s time to post a new job description. Suddenly you realize you don’t have solid warm connections who might be ready to jump ship and what you’re getting in the inbox is far from the bar you were hoping to meet. “Can someone crank out a blog post and push it to Hacker News in the next couple of days” is not what you want to hear as you begin hiring again.

All I can say is try to run tiny experiments here and see what resonates with you and with the outside world.

Does anybody on the team have the chops and the interest in occasionally contributing in one of these formats? Be constantly on the lookout for something interesting you’re doing that you can talk about, whether that’s using Haskell to generate AWS Cloudformation templates or if your artists are painting with watercolors and scanning in those textures to be applied onto your game’s terrain. Someone, somewhere is going to find that inspiring, and will remember you.

Keep the Fire Lit: The Process Is King

Of course, if you temporarily happen to be the hottest company on the block and the wind is in your sails, you may not need any of the above. If you’re Lovable, Wiz, Deel, Cursor, or early days of OpenAI and Anthropic, ambitious people will naturally flock to you seeking challenge, glory, and a mind-shattering equity package at a company raising 2-3 rounds a year.

But your good fortune will not last forever, and once the temperature comes down, you will wish you had your storytelling flywheel dialed in. After all, if you base your whole identity on “this place is where you come to make money”, what are you offering once growth tapers off and people take off for the next rising S-curve?

Wrapping Up

Hiring done well isn’t just attraction. It’s filtration. The clearer your story, the more the company becomes a magnet for the right kind of oddballs. The people who read your manifesto and think “finally, someone gets it.” And the more it makes others go “Absolutely not, you guys are f*****g nuts!”

Even outbound recruiting depends on this. Every candidate you message is silently asking: “Who are these people, really? Is this quest for me? What role will I play?” If your presence online answers that with clarity, conviction and personality, half your job is done before the first call. And for people deciding to opt out, you’ve just saved them and your team a ton of time.

Tell a compelling and authentic story, and the right ones will find you.

Hiring - The Big Picture

Wed, 01 Oct 2025 23:22:11 -0700

It is better to first get the right people on the bus, the wrong people off the bus, and the right people in the right seats, and then figure out where to drive.

—James C. Collins

Growing Pains

My first encounter with startup hiring was a rude awakening. First company I ever co-founded. First time manager. First time hiring an engineer who would be reporting to me. None of us had expectations of this attempt being a home run, but we figured it would be “just fine.” How hard could it be after all? It goes about as well as you would expect.

The candidate, Steve, and I perform an interview rain dance that neither of us are familiar with. Steve is about as green as I, applying for his second job after college. After a brief chat I send him a take-home test I had to slap together the day before. Later, a few standard interview questions scrounged from a Google search. A whole lot of open-ended chatting about programming opinions. I don’t know anybody who has done this before for a startup and asking ChatGPT is still in a galaxy far far away.

In the end neither side learns much about the other, but we’re not exactly swimming in options. He’s only talking to one firm. We’re talking to a few candidates, and he seems “reasonable,” a word that is always a strong telltale. It’s worth a shot, I guess. Part of me hopes I absolutely nailed it, maybe I’m secretly a genius and this hire will prove it. Two months have passed. I have to deliver Steve the bad news in a spartan and dimly lit 3-person conference room that still lives in my head 12 years later. It looks like a broom closet. It actually might be one. The office space owner figured a desk and two chairs (my cofounder had to stand during the announcement) count as a conference room. You can touch both walls if you stretch your arms out. Steve is shocked to hear the news, because, of course, I failed to make it obvious to him this whole time that he was doing poorly. My naive first-time manager self hoped he would get the message somehow if only I gently hinted at it. It wouldn’t be the last time I bomb finding the right person for the team and have to walk someone to the elevator with all of their stuff. Yikes.

Ever since that experience, and many variations of it since, I’ve been tormented by the quest for the perfect hiring process, and why some companies absolutely nail this, and others half-ass it at best. Perfect not in an abstract global optimum sense, but perfect for my company, its industry, its life stage, its people, its tribal customs. I dread letting people go, and I never want another Steve on my hands. I should have known better, been better. What was missing?

That quest is still ongoing, mistakes still happen, but what I do today hardly resembles the first fumbled attempt from 2013. After tens of thousands of resumes and hundreds of interviews, every hiring push I lead offers more opportunities for valuable scars and iteration. And now I approach the process of building one’s hiring flywheel with the same lens of experimentation and rigor as with any other product I build, be that internal or external.

Impossible Tradeoffs

Hiring at a fast-paced startup is all about impossible tradeoffs. You need the perfect hire, yesterday. The team is drowning. The runway is getting shorter by the day. A hiring mistake? You might have killed the company. You delay hiring for a few months? The mountain of work only gets more desperate, and the runway is ever shrinking. You still might have killed the company. As you get bigger the price of every individual mistake goes down, but accumulating many of them leads to a very painful fix down the line.

Every firm must pick their poison. Don’t know who you are, your culture, the role? Wrong hire. Spend forever trying to figure it out, avoiding mistakes? The market might run away from under you. Skip steps, rushing the funnel? You’ll make bad hires. Drag things on? You’ll lose good ones. Your sweet spot depends on a dozen variables: risk tolerance, culture, market conditions, desperation level, the list is vast. And these variables constantly move. What worked during your Seed will crater you at Series B. Last year’s artisanal process is this year’s hyper-scaling bottleneck.

If that wasn’t enough, you have to do it all on a shoestring budget. You might be in Cockroach Mode, and the market might be such that a great hire is already being paid more than your entire pre-seed round. You might have to alter the nature of your company to survive Hiring Winter, potentially hiring abroad and working asynchronously to make it all work.

It’s hard, but it’s also one of the most important areas for you to invest in. It’s so important that some have called it the “other product,” beyond the product you’re actively selling to customers, and my thesis is that you need to treat it with an equal amount of reverence.

Assembling Two Airplanes on the Way Down

Your company’s hiring identity, its positioning, its funnel and the post-hiring follow-up is an essential part of the broader product-market fit search. What most first-time entrepreneurs don’t appreciate is that the product is not actually the only product you’re building. The company itself is to be marketed and sold to a specific niche of potential future hires, with its own narrative, differentiation, aesthetic packaging, and a promise of a better life after the “purchase.”

You have to come up with a thesis of what would be best for your team at this point within your budget and available time, and how you’re going to go about obtaining it. You then proceed to test those assumptions by iterating on the process and the type of person you’re looking for. Whether you succeed or fail with each iteration, you learn something valuable about yourself, the company, and the market. Just like user onboarding, you keep iterating on this for the rest of your company’s life, as the target will be constantly moving.

Reality check: there’s no perfect playbook. Early in your career you hope that there’s a universal formula that you can spam at your hiring problems. With much trial and error–mostly error–you realize that every solution has to be bespoke to your situation. Past lessons are key, but only if you know when to break them for your unique situation. The good news is that any company that has become great at consistently sourcing, attracting, closing and retaining stellar employees has had to figure this out. The playbooks exist and you don’t need to re-invent the wheel. The bad news is that the playbook still needs to be adapted to your own unique circumstances, you can’t carbon copy another company’s essence and expect it to work. You test and test again.

If you run enough of these experiments, with plenty of reflection and course-adjustment, you will get pretty darn close to where you hope to land. You will make mistakes, the only way to avoid them is to never hire to begin with, but eventually you’ll dial things in. At least until it stops working, and then you do it all over again, an important but ultimately Sisyphean endeavor.

The More Things Change

Your gate into the world of early stage hiring is going to be through founding the company or by joining it at a later stage with a mandate to fix whatever has been cobbled together until that point.

The team you’re joining might be 2012 Alex and a homebrew process duct-taped together from whatever fads were hot and trending on Hacker News that year. We were chasing holacratic coding ninjas, then radically candid 10x engineers, then founder-mode cracked 996 Claude-riding context wranglers. Here, your work is going to involve undoing a lot of unfounded assumptions and letting go of those unfortunate enough to have gotten reeled into the company without much rigor. If you’re lucky, they might be great and get to stay. But chances are you’re going to have to rebuild a chunk of the team, or you wouldn’t have been brought in to intervene.

Fortunately the art and science of doing this right is nowhere near as mysterious and opaque as it might have been decades ago. Unfortunately not everything from past eras is going to carry over. Take home projects are blown away by a single prompt of ChatGPT. Spies with gen-AI Zoom faces apply for tech jobs to siphon VC funds for sketchy foreign regimes. H-1Bs cost 100 grand to obtain. Cluely-style cheating apps answer every interview question. $100M comp for the top AI talent.

We’re living in a wacky timeline and many tactics from the past have gone the way of the dinosaur.

Luckily the fundamentals haven’t changed much, as are still very much worth testing for. What also hasn’t changed is the hiring cat and mouse game, and doing the job well requires staying on top of the relentless stream of changes. Part of what keeps things interesting, I suppose, if you want to find a silver lining in it all.

Next up, a look into how to position your team in the relentless cacophony of a hectic hiring market.

Genie, Take the Wheel

Wed, 20 Aug 2025 18:27:17 -0700

The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.

—Donald E. Knuth

How I Stopped Micromanaging Code and Learned to Love the Lamp

Over the last two months, I went from zero to fluency with Claude Code’s take on vibe coding. Prior to this, I hadn’t used any AI in programming outside of a couple of years of GitHub Copilot’s auto-complete feature. In a typical (for me) act of hipster defiance of the popular, I had completely skipped the Cursor revolution until after I had already gotten familiar with Claude Code.

This gave me the rare opportunity to experience vibe coding without any existing preconceptions from intermediate evolutionary links like Cursor or Windsurf. Here are a few peculiar observations from the experience.

1. So much breadth..

Vibe coding is like the spice; it expands one’s horizon beyond one’s natural limitations. It gives one new superpowers.

Thanks to Claude Code, I am suddenly a crappy Swift UI developer. Neat. For a quick experimental app, I don’t need to learn much of anything about how it all works, I can trust the magical code roulette to get it done, and an app with a few views appears in my Xcode.

Thanks to vibe coding I can do basic computer vision with Python. I expected that to be more mystical, but it just works, and I don’t have to wonder how it is implemented unless I want to.

Thanks to the vibe code roulette, I have never written Flask before, but I have a Flask server up and running, doing useful work. The agent recommended I use Hono for my TypeScript backend, made it safer with Zod, tested the implementation with vitest and poof, the implementation appeared out of thin air.

I needed a couple of simple Chrome plugins. One-shot, no problem there. The list goes on and on.

This is a huge change, especially for those of us who remember buying books to pick up a new ecosystem, watching hours of tutorials, or spelunking through official docs for that one needle in the haystack detail blocking us. The barrier to entry for getting started with any ecosystem, at least for code with examples available online, has eroded to almost zero. Whatever it might have seen on GitHub or beyond is now at your fingertips. For quick hacks and small experiments, it’s a big deal.

2. .. at a cost

The breadth of tools that you can now use on a whim is spectacular. At the same time, you’re under no illusion of actually having become an expert in any of them. You’re a nomad doing short stays, not a real local with any depth of understanding. For many use-cases, that’s fine.

What’s not delightful is that as soon as you stop getting what you want out of the interaction with the agent, you are on your own. You have a codebase to maintain that you’ve never looked at. After all, the whole thing was conjured for you by your ambivalent coding genie. It tried its best to please, hit its limits, and now you have to pick up the load. The abstraction balloon burst, and you plummeted back to the previous reality of organic hand-crafted small-batch edits.

Was betting on vibes sensible here? Is this still more productive than the traditional path of needing to build at least some experience with your tools before you start using them? That’s entirely context-dependent; sometimes, yes, sometimes no.

The feeling of working with a vibe coded codebase is that of repeatedly opening a project written by someone else, one that changes every time you look at it. To thrive in this new paradigm, you need to let go of meticulously combing through every line that goes into the project as you edit and code review. That’s not your job anymore as long as things work.

You’re now a stranger in your “own” codebase. You open files as if it was the first time. You don’t know where to look, things are named and positioned in ways you yourself wouldn’t have, nor in a way you would have instructed your team to follow. There’s cruft and scruff in there that you didn’t ask for, hitting you all at once. Not too different to opening another maintainer’s repo for the first time.

This project isn’t actually yours anymore. Someone else is building it, they have their own style and approach, their best practices and biases. You can roll up your sleeves and try to micromanage every line, but now you’re back to the dark ages of pre-2024. With a coding agent, you’re facing the perennial tradeoffs of delegation. You’re having to decide how much effort to spend in the weeds rather than on the bigger picture. And it’s clear to me that the value is not in the microscopic details.

3. Code loses value

As bottomless code pours out of the genie, you have several epiphanies in short order:

Code is now throwaway.

Because you invested virtually zero time building it, you have zero loss aversion to throwing it all away and having it rebuilt from scratch. Sure, you’ll burn a few tokens and torch a few minutes, but someone else is doing the bulk of the work.

In the past, writing a quick experiment, a quick script, a small command line tool would have caused hesitation. You weren’t sure it was worth the bet of spending 15, 30, or 60 minutes executing the task. Reading the documentation for a new library, spelunking Stack Overflow for best practices, and having to remember how to create and activate virtual environments for that specific ecosystem - ugh.

Now? As long as you know how to validate that the script is working, I can have it appear out of thin air in under a minute, and as soon as I’m done, I can throw it away. Glue code is effectively free now, so you might as well spam it.

Over time the LLM architectures and techniques will get better, context windows will grow, the attention mechanism will be perfected, and overall model effectiveness will improve. A 15-minute vibe-coded script will be replaced by a no-brainer 60-minute script, which will be replaced by a no-brainer half-day task, and so on.

It’s possible the progress will not be linear, but will instead hit an asymptote. But people’s predictions about this have been wrong before, so maybe the models will continue to get better at the existing pace. The incentives for capital are all aligned around continuing to improve the tech and gain more leverage.

4. Just-in-time software

We’re not there yet, but as the quality of intelligence-on-tap goes up, and costs go down, there’s no reason why more software is not entirely generated on the fly. It will start small. Naive glue code and simple scripts are already a mostly-solved problem, and the circle of “one- or two-shottable” software will only grow.

Want to put together a quick soft synth for your website? Need a physics simulation, visually rendered by the GPU? Fancy turning a PDF intake form into a whole data entry workflow with validation and user guidance? No problem, given an accurate enough problem description—and that’s a big given!—you will likely get what you want. With more specialized libraries created and uploaded to GitHub every day, stacking legos towards a solution only gets more viable.

Maybe not on the first iteration, but you wouldn’t get that from a human developer either: they don’t know what’s in your head after all, you have to share that vision, explain the constraints, and make sure they fully grokked it. And you probably suck at writing the perfect all-comprehensive spec, it’s tedious and laborious, and you’d rather have a conversation instead where you’re polled for details, you’re discovering the constraints together. LLMs will not develop telepathy, and most often you yourself haven’t thought through every detail until you’ve made some progress and can play with the intermediate implementation steps.

We’re not quite there yet, but it’s only a matter of time. The big question is if we’ll ever get to where a non-expert will be able to get complex software out of these tools without even knowing what to ask for. Possibly not, at least not for a while. But, one day the coding agent might instantly put a dozen options in front of you, and you select the one closest to what you vaguely had in mind, and iterate from there. It’s akin to someone asking for concept art from a skilled artist on their team, but not having the expertise or language to explain what exactly they want. You can often point at an option that feels most right, but you can’t explain it well in advance.

You end up with a much more creative and experimental workflow than what we’re used to today, but one that might actually win in the end, at least for non-critical applications.

Some compare coding agents to the 3D printer “revolution”. Everybody is now able to print physical components for whatever it is that they’re making. But, do they in practice? The revolution never actually happened: 3D printing is still restricted to a niche of enthusiasts, and never gained mass adoption. Most just want to buy finished products, and even if they just need a component, they’d rather get it on Amazon than bother with learning a whole new workflow they might be bad at for some time. It’s likely just-in-time software will follow the same path.

5. Moving up the value chain

With all of this abundance, thanks to your newly-hired hard-working, tireless junior to mid-stage digital SWE genie, you need to move up the value chain. You are now the PM. You’re now the tech lead. You’re now the architect, and your job is to operate at a higher level of abstraction. You’re the UX person. You’re the visionary, the taste maker, the customer representative.

The dream is that eventually you won’t have to look at the code at all because you trust that the implementation of your design is sufficiently well-executed at the micro level. Coding agents are able to take on more and more of the mechanical tasks, and the human is left with matters of feeling, taste, vibe, things that the computer can’t judge, yet.

Of course we’re not there yet, but that’s where everything’s headed. Already today you get to experience the value of being higher up the chain when, instead of spending time thinking about the implementation details of your tasks, you write specs that describe what you’re hoping to achieve and define the constraints for the agent. You then discuss the plan with the agent, reach an agreement, oversee the work, provide feedback, test it, etc. As time goes on the human work will only get more abstract.

There’s a parallel here with what happened with compilers. What’s the last time any of us have had to look at the x86 instructions output by a module that we wrote? Sure, you will pry open the black box once on a blue moon on high-performance applications. Especially if you are a performance specialist, or if you’re trying to debug a strange and particularly nasty bug in some lower level representation of your logic. But for the vast majority of software work, that is not necessary. We trust the compiler to do the good-enough thing. The same way we trust the microarchitecture of the machine below to run those instructions with reasonable speculative execution. No need to scrutinize it.

Again, there’s a whole world of specialists out there who are worrying about these problem. The magic of abstraction allows us to be oblivious to their work for regular application-level tasks. For most of us, the black box you get from the compiler is “good enough”.

I expect that soon entire code modules will become uninteresting for programmers to look at. All you need to know are the interfaces the module provides, everything else can be safely abstracted away. The internals are now the realm of automation, humans not needed, the same way poking at assembly is unnecessary in routine software development. It’s purely functional programming at the layer of modules, assuming no shared mutable global state.

Large, tightly coupled modules that mutate shared state among themselves are going to cause trouble for coding agents and human SWEs alike. They always caused problems, nothing new here. Both software and wetware struggle thinking through all the permutations of state the system can get itself into, and all the branches that take it there.

6. Closing the loop

Andrej Karpathy points out in his AI Startup School talk the importance of letting the agent do its work without a human in the loop. I couldn’t agree more, the difference is night and day. You must do whatever you can to enable the agent to validate that it’s achieving its task without your help, maximizing its autonomy and reducing touch-points with humans.

It’s likely we’ll see a resurgence of TDD/BDD-style testing, given that humans no longer need to be hassled into writing those. Getting adoption of a test-first workflow has always been a pain, but your digital SWE has relentless enthusiasm and will happily oblige. And setting technical constraints has never been more important than when providing guardrails for an unpredictable and sometimes volatile genie.

With a comprehensive set of new tests, reference mock-ups, regression tests and possibly an eval suite, the agent can continue iterating on the implementation until those success conditions are met. Don’t bother humans until you’re done, go away, clanker, we’re paying for your tokens to work, not distract us.

Command-line tools, or anything that can have a straightforward test written for it is a natural fit for this: it is straightforward for Claude to both write these tests and run them as it’s developing the application.

For web apps, it is essential that it is able to open the application in a web page and look at it to make sure that it achieves the UI/UX that you asked it to implement. Or that the page looks aesthetically like what you wanted it to do. There’s no way around giving the agent Puppeteer or Playwright MCP server access in order to open a browser to look at the web page through screenshots.

For mobile apps, it’s once again about being able to open the app and take a look, rather than having to guess correctness on implementation alone, or having to ask the human for an approval at intermediate steps. This is harder, but it looks like the loop is being closed there as well.

Gamedev is lagging behind, although people are making progress there as well. I suspect it will trail behind for some time due to the complex nature of game engine editors and interaction with complex multimedia assets of many different kinds (audio, 3D meshes, animations, levels, VFX, etc.). Testing mechanical scenarios will be feasible, but validating “feel”, “responsiveness”, and emotional impact will be outside of the realm of bots for a while longer.

Ultimately reducing touchpoints with the AI’s workflow is just another variant of workflow optimization you already have to do across individual contributors and teams. The more synchronization rules you create in a process, reducing the team’s ability to ship independently, the less efficient the process.

7. Need for speed

There’s a lot to be in awe of already, in spite of the predictable human tendency to take magical advancements for granted. The truth is however that there’s still much work to be done before these tools reach their full potential.

Model speed is one of those major unlocks.

Right now, even the simplest command run by Claude Code takes 5 seconds or longer. That doesn’t sound like a lot, but you feel it. There’s a reason why it takes that long: every time you ask the tool to do something, behind the scenes Claude Code executes a sequence of steps. It needs to figure out your intent, what you’re asking it to do. It needs to break down the request into manageable chunks. It needs to execute said chunks with the correct tools for the job. It needs to check that the change actually achieved what it set out to do.

Every one of those steps introduces latency between the request and the actual operation being executed. By itself, not a big deal, but add it up over a full day, and it’s significant. But this sum isn’t actually the major culprit.

The real offender is that the delays are long enough to tempt you to context switch to something else, like opening your browser or checking your Slack or email, instead of staring into the void of your terminal waiting for the result. Now you’re context switching between several parallel activities, only half-paying attention to any of them. You are constantly pulled out of the zone as you wait and get distracted, or get bored of waiting for something to finally happen.

I suspect that if we got rid of the delays, if we increased the speed of token generation to 5-10x the current speed, the experience would be more fluid and stream-of-consciousness-like. The bottleneck would be the jockey’s ability to ask for changes and review the proposals emitted by the genie. My hunch is that we don’t yet understand just how different that boost in speed will feel: it will not be just faster, it will be a 0-to-1 towards an entirely different experience, changes at the speed of thought.

8. Ugh, you want me to do this by hand?

Speaking of quick adaptation, I’m still amused that it took me only a few hours of Claude Code to feel hassled by having to write code manually again. The human brain has a seemingly limitless capacity for taking new conveniences for granted, and editing code by hand appears no different. “Massaging characters on a screen? Are we back to the Stone Age? We have assistants for that now.. this is such a hassle! Ugh, why do I have to do everything myself? Why aren’t you better, Claude?”

The assembly-to-C analogy again comes to mind: you get used to working at a higher level of abstraction, but, in the early days, the abstraction often wasn’t good enough. You had to dive down one layer, with the full resentment of losing the conveniences of upstairs, and the need to remember how it all worked below your new zone of comfort.

I wonder if near future newly-minted SWEs will have never had to manually edit source code beyond the tiniest tweaks. Will they just point and explain the change they want, perhaps without much precision? Will everybody become a tech lead of sorts, from day one?

I wonder if the transitional generation of programmers, brought up in a manual editing world, will have a distinct advantage over the younger crop. Or perhaps manual editing will be an anchor holding them back from achieving their full vibe-coding potential, incapable of letting go of the wheel, like drivers who spent decades with a manual gear box, refusing to trade familiar control for convenience.

I wonder if manual editing will follow the path of manual memory management, where a small subset of developers still needs to be familiar with it, but for the rest of the community and the industry, a fully automated approach will be more than sufficient to get the job done. Even in the game development world these days, even with the native implementations like Unreal Engine, the C++ programmer is actually hardly, if ever, thinking about memory management - that being orchestrated for them by the system and done perfectly well as long as one follows the expected idioms.

9. Everything is new all at once

This is to be expected, but the progress of best practices around these tools has been breathtaking, and it takes a tremendous amount of time just to keep up with all of the changes introduced daily. You have to give it to the teams working on Claude Code and Cursor: they’re iterating fast and furious, trying to keep up with each other’s constant output. New best practices emerge all the time, and you’re not sure if you should toss out the old ones or if what is now popular is a real step change.

What do you put in your CLAUDE.md? Which of the many layers of CLAUDE.md files? Should this go into a command instead? Should this be a hook? Should this be run as a GitHub action Claude Code command instead? What’s the best set of Claude Code commands to have in one’s project? Should this be run by one agent or fanned out to a swarm of agents? How many tasks in parallel? What’s the best way to prompt the agent? What format of specs works best? Which model should you be using for which type of task? Can you swap the stock models for third party ones and get better ROI?

It’s all exciting, moving very fast, and very volatile. You can give up and wait for the tools to stabilize, but the FOMO is real, and you worry that you might be left behind, with competition racing past you, whether in the business or in the job market. Much of that pressure is likely not worth giving in to, but it doesn’t mean you should completely sleep on this.

10. In the end

My guess is that in the medium term the average developer will have no choice but to adopt agentic coding as their daily driver. The tools will get faster, more efficient, more autonomous, and staying on top of the latest advancements will keep you employed. Editing logic by hand will be akin to shoveling dirt in an era of bulldozers. After all, why would a company not pay the same for more output, given the option?

The number of developers needed by the industry might shrink over time—perhaps analogous to farming in the 20th century?—and only the most machine-augmented programmers will be left around to orchestrate swarms of intelligent assistants. Fortunately, software developers have gone through these augmentation cycles many times before and are the first to embrace new tools that increase leverage. Unfortunately, it’s unclear if the market will be able to support as many of them as before. Or Jevons Paradox might kick in again and we’ll be just fine.

I can’t imagine a world where human SWEs aren’t the bottleneck for agentic development. Already today with Claude’s sub-agents and parallel Claude Codes running in the same git worktree, the human attention span and context shifting can only stretch so far. Is a wetware orchestrator able to keep up with half a dozen agents all implementing their own separate feature in parallel? Apparently not. Kanban’s rule around limiting WIP tasks seems more relevant than ever.

Regardless of what happens, there hasn’t been as exciting a time in SWE in a while, and we can only guess where the current trajectory is taking us. The growing pains of the technology are both frustrating and to be expected, but the potential is obvious and awe-inspiring. The least we can do is hook ourselves onto the unstoppable train of progress and ride it wherever it takes us.

Why Game Devs Don't Merge Files

Tue, 29 Jul 2025 10:01:25 -0700

My point today is that, if we wish to count lines of code, we should not regard them as “lines produced” but as “lines spent”.

—Edsger W. Dijkstra

In the world of gamedev, versioning mistakes can be expensive and irreversible. Two people working on the same character texture, complex material, or Unreal Blueprint can throw away days of work if they don’t realize both are editing the same file. In the world of business software, which I’ll refer to as Text World, merging two files is usually barely an inconvenience. In the world of multimedia assets, which I’ll refer to as Binary World, it’s practically impossible. This reality ultimately changes the workflows, rituals, and habits of the development team in ways that Text World people may never encounter.

I’ll walk you through the key differences between the broader software version control and how game studios actually manage the chaos of binary assets: from version control systems and asset locking, to CI/CD pipelines, massive file sizes and human coordination rituals.

Binary files, so many binary files

Unlike a traditional web app, a utility mobile application, or most forms of business software, a modern video game is a marvelously complex multimedia system. It blends dozens of asset types—animations, audio, models, materials, logic—meticulously stitched together and projected to the player’s eyes and ears hundreds of times per second. All of it runs in real time, often across the network, many times on disparate hardware, to deliver an experience that’s not just functional, but also deeply moving.

Anyone who’s played a title like Journey can tell you how surprisingly personal it felt. How a few hours with no dialogue and almost no UI left them feeling something transcendent they couldn’t quite put into words. I’m not convinced any other medium is able to deliver experiences this compelling. Anyway. I digress.

Animations, audio, video, visual effects, 3d models, images, materials, AI logic, data files describing stats of innumerable items and abilities, and oh so many others. The majority of these files are in a binary format, a complete reversal from business software, which is primarily made up of text-based code and data.

Of course, code is still omnipresent in a game project, often written in a compiled, close-to-the-metal language like C++. On top of that, you also “code” higher-level logic in one of the many scripting languages supported by your specific game engine. These languages conveniently abstract the gnarly internals of the system to designers and artists who are less interested in the architecture and performance of the underlying machine.

Often this scripting interface is made of text (C#, GDScript, Luau), but sometimes the engine designers decide to take a different set of trade-offs and go with a binary format instead. Unreal Engine’s Blueprint scripting is perhaps the most notorious one in the category.

A Quick Aside on Blueprints

There is much power to visually representing the flow of logic. Connect nodes together, step-debug through the edges of the graph as the game is running, see the connections light up as data flows through them. It’s a powerful paradigm. And the pain of not being able to use git or diff against text is eased with powerful built-in tools for tracking Blueprint history (until you rename the file, womp!) and for showing Blueprint diffs between versions. It’s a tedious manual process of diving through menus, but it’s possible, and it works. Automated or best-effort merging of Blueprints is not (yet?) possible however. You still need exclusive locks.

For UE-based projects, every early team will lean more on C++ or Blueprints depending on their team composition. Programmer-heavy teams and teams with IDE-eager technical designers will be more comfortable with logic living in native code. In contrast, teams without programmers might end up with the entirety of their game completed without a single added line of C++. Both approaches have significant tradeoffs, as one might expect, but both have been shown to work in real-world settings.

I wonder why there aren’t more Blueprint-like tools out there in the broader software world. Is it because most problem spaces are served fine with a text-based interface for building logic? Is it because most companies don’t have the funding and the long term outlook of an Epic Games to iterate on such a tool?

Programmers Play Bass

The natural consequence of the multimedia nature of a game, influenced by the genre and development stage, is that programmers are often the minority of the project’s team. Most of the hires are artists, responsible for delivering an immense amount of carefully hand-crafted assets for the project. Assets that are either plugged directly into the interfaces provided by the engine or into abstractions built on top of those by programmers to make artists’ lives easier. With the 2025-era resistance towards AI-generated art and the positive disposition towards LLM-generated code among most programmers, the ratio of artists to programmers is likely to continue growing.

Programmers play a support class in games, with the stars of the show being the designers and artists who move the player emotionally and deliver a compelling, visceral interactive experience. The programmer is there to help deliver the emotional payload as efficiently and pain-free as possible, making their vision come to reality within the agreed-upon frame budget.

Given that most game projects are driven by artists, the prevalence of workflows centered on binary files is not surprising. Carefully merging lines of text together from different versions of a file is not a native experience for a designer of 3d models, sound effects, or textures. Unlike Text World, this is one of those cases where it makes sense for programmers to accept the needs of the majority of the project and take the back seat, rather than having everything revolve around programming workflows. Some teams try to strike “the best of both workflow worlds”, as we’ll see below, but the jury is out on whether that ends up being a net win.

It’s a Perforce’s World

Perforce dominates source control in the mid-to-large commercial gamedev world. Some studios use alternatives like Git + LFS or Plastic SCM, but most stick with Perforce because it’s the default supported by engines like Unreal out of the box. This spares most of the additional tax of having to support an off-the-beaten-path system that engine developers, such as Epic Games themselves, don’t officially use or test extensively against. For example, many version control features in Unreal Engine work out of the box only if you’re using Perforce.

The Perforce model is that of a central server, the authoritative source of truth, from which every developer fetches their files and pushes them back up to. You download the latest version of each file, unlike Git, where you are getting the whole history at once. The central server’s history can be absolutely massive, and, likely, you wouldn’t be able to fit that on your drive. Unlike Git, there’s no common practice of syncing one local workspace against multiple remotes. Distributed workflows are uncommon, largely because merging binary assets is practically infeasible.

The key aspect of this is that the central server serves as an arbiter of file access. With this central authority, it becomes possible to police who gets access to what file within the Perforce depot, which is the key piece of functionality needed to work with binary files as a team.

If you absolutely insist on not using Perforce, you can still use plugins and other third-party tooling for Git support, but you will be on your own, and inexplicable VCS errors are now your responsibility. Some teams are happy with it and will power through, but most artists in the industry are already used to the Perforce workflow. This is reminiscent of Salesforce in the sales world: a dominant, aesthetically-offensive-but-familiar tool that no one loves but everyone tolerates. A tool that brings the benefit of not having to train your staff on yet another mostly-similar system that will require extra training time and lead to mistakes and confusion instead of more sales.

For most artists, Perforce is already a well-developed muscle, and its workflow makes intuitive sense. One of the key takeaways from working on a multidisciplinary project is realizing just how different workflows can be on the same project and how easy it is to lack empathy for what would make an optimal workflow for someone else on the team who performs a different role than you do. Business software has its own share of disciplines, but it’s not even in the same ballpark as in game development, as delightfully captured in The Door Problem.

During my time at Freckle, we had a fairly sizable curriculum design team, in charge of building thousands of questions to test K-12 students on their knowledge of math, grammar, social studies, and other subjects. The content would go into the main Freckle product, acting effectively as the “game engine”. However, even that team was based entirely on CSVs and YAML files, most of which, again, they were able to version in Git.

Innovating in this area seems generally not worth it. Perforce is not delightful, an artifact of a different era—I checked, it’s actually from 1995, wow!—and the interface shows it. But once you get used to it, it’s like any other piece of software, possibly better than most others for the job, as it has undergone decades of fixes and has had hundreds of thousands of eyeballs on it, triggering every possible accidental corner case.

Matters of Size

Speaking of areas where Perforce again shines, it’s hard to overstate just how gargantuan in size modern 3D game projects get. A single 4k texture can take up 50MB or more, and you end up with hundreds or thousands of those alone, and that’s just for textures. It’s not uncommon for projects to span many terabytes, with some reaching tens of terabytes. Unreal Engine and its build artifacts alone can easily reach 150GB or more. Managing what a developer needs to download on their machine to complete their work that day becomes a real challenge as the project grows, as opposed to in business software, where a project (and its previous snapshots) can be compressed into a few hundred MB and downloaded from GitHub.

The good news is that the final payload shipped to customers is significantly smaller, thanks to a clever build process. You cook the assets to prepare them for the target platform, shrink them to the desired size and format, and ignore most assets that won’t actually be used in the game. You also employ other techniques to keep the final deliverable “small”. That’s why your download from Steam will often be in the 50-200GB range, rather than the terabytes the release build process initially began with.

The Lock Economy

Back to the central problem I alluded to earlier: binary assets cannot be merged if two people made edits to them at the same time (also known as a three-way merge).

What does it mean to merge a rendered music track? What does it mean to merge two PNGs together? What is the correct resolution flow for merging two 3d meshes into a final third one?

You could, in theory, conceive of tooling that would enable this, given the proper preservation of edit history. These tools would allow the developer to choose which edits to keep, what order to merge them in, and replay them; however, most of the time, the result would be either unusable or barely worth the effort. In the world of game development, binary assets are practically always checked out exclusively. Even something as essential to UE-based development as Blueprints only has a diffing tool; the merge itself is up to the developer, as automating it would be fraught with ambiguities that require more context than the asset provides.

The workflow with binary files is exclusive checkouts: a file is uniquely assigned to one developer, and nobody else gets permission to edit it—to avoid later predictable tears—while they’re working. This isn’t dysfunction, this is damage and disappointment prevention in a context where merges are simply not an option.

This is highly strange to someone coming from Text World. What do you mean, I get to hold up the rest of the team on a set of files? What if they need them? Can I hang onto them forever? How do I coordinate? What if they lock some of the files I need to access exclusively later, but to finish their work, they need to access some of the files I’ve already started editing? Time to perform a human-driven deadlock-induced rollback? Fun.

You’re now in human coordination territory; you need to anticipate this sort of issue when working on tasks. The larger the undertaking, the higher the chance you will need to lock files that someone else was working on, which means they’re now stuck on their respective tasks, needing to work on something different until you return those files. Again, bizarre to someone coming from Text World.

Branches Are a Luxury You Cannot Afford

What is surprising to people coming from the world of text-based projects is how little reliance there is in games on topic branches for your workflow. Since you cannot merge most of the files you work with and other developers cannot work on them while you are, there’s no longer a point in having an isolated branch of the project in which to do your changes. The exclusive checkouts are accomplishing that.

The typical workflow goes from backward-integrating topic branches into main, either once or continuously, to always working in main. You simply check out a subset of the files, locking them for everybody else, and do your work.

The major downside is that now everyone else is blocked from proceeding with work on those files. In Text World, you can typically all edit at the same time, assuming one of the developers isn’t completely uprooting the module, and still successfully merge the changes in the end. Not always pretty, but usually tolerable. In Binary World, only one person gets to move forward; everybody else is stuck.

Many a holiday or weekend is spent trying to rush a sizeable change that is blocking several people on the team so that you can finally release those files back into the wild. Unreal Engine projects that heavily rely on Blueprints, UE’s native visual scripting language, for complex game logic are particularly susceptible to this issue. Programmer-heavy teams will port as much as possible to a native C++ implementation, which can then be three-way-merged. That, of course, comes at the cost of slower cycle times and non-editability for game designers, which may or may not be a price you want to pay.

Divide or Be Conquered

One unexpected silver lining of lock-driven version control is that it forces modules to stay small and decoupled, split up in such a way that allows developers to lock a specialized subset of functionality, rather than a giant, monolithic file.

In Unreal Engine, the bottleneck of every character-based game reliant on Blueprints is the infamous ACharacter subclass, which practically every game implements for their main player character. This class tends to bloat with every imaginable piece of functionality, including player inventory, animation, attributes, movement, skills, and communication with other players, among others. It’s a quick and dirty place to park your logic and make it readily available for experimentation.

The naive approach requires the Blueprint to be checked out exclusively for most gameplay changes, but only one developer can edit it at a time. Now changes have to be small, or the file is locked for an uncomfortably long time. Or all changes to that class need to be assigned to one single developer in a “single threaded queue” of work, avoiding issues of coordination, but also losing team efficiency.

This is where the well-trodden Actor Component system comes to the rescue. It allows the functionality of an Actor class, such as the player character, to be contained in small modules that can be attached to any other Actor that needs them. Now, changing a specific piece of player functionality only requires a change to that module. The main character class can finally be left alone.

The main counter-argument to this approach is that it adds an extra level of indirection for something that might as well be in the same module if it’s only ever used by one from one location. “Why did we move health logic into its own module if the only thing using it is the player? We could just keep it there and not have to jump to another file.” Unblocking developers will typically win the cost/benefit analysis.

The Human Protocols

Lock-based workflows often introduce strange rituals, primarily related to coordination and social etiquette.

The most basic one: remembering to release your locks. It’s easy to forget, especially if you were just tinkering with a file. But now it’s 11pm on a holiday weekend, and someone on the other side of the world is blocked. All they can do is ping you and hope you’re awake.

Over time, you develop habits: releasing any files you’re not actively working on before signing off, especially if they’re likely to be needed by someone else. On globally distributed teams, this becomes particularly challenging. The person who locked the file might be asleep when you realize you need it. Your options now are to stop work entirely or make a copy and hope you can manually port your changes over later. Not fun.

Sometimes it’s negotiable. The person holding the lock might say, “My change doesn’t matter, just overwrite it.” Perhaps it was a minor config tweak or a cosmetic adjustment. With their blessing, you work on the unlocked version and clobber their edits when you submit. It’s messy, but manageable.

Over time, implicit ownership patterns form. With maturity and scale, they become more formalized. The audio person owns the ambient audio system. The animation lead owns the blend trees. The systems programmer owns the inventory logic. They’ll often keep “their” files locked for days (sometimes weeks), and no one minds.

Other files become hot zones. High-traffic Blueprints. Shared config assets. You learn which ones are sensitive because you find yourself asking, “Hey, are you still working on X?” often enough. Eventually, a little voice in your head starts chiming in: “Do I really need to hold this file much longer? Maybe I should release it. I bet Dave’s going to need it tomorrow based on what’s in Linear.”

Big CI/CD Energy

Given all of the above, how do you actually test every submission to the project? With the size and volume of assets involved, it’s anything but trivial.

You can’t just spin up a fresh CI dyno and pull several terabytes of data every time you want to run a build. Instead, each CI machine hosts a long-lived Perforce workspace. With every build, you sync that workspace to the latest commit instead of redownloading everything. Of course, now you’ve introduced a new problem: if that workspace’s state gets corrupted, you’re in for a bad day.

Then there’s the horsepower problem. A full build of a AAA-scale Unreal project can take hours, even on beefy hardware. And like in every other software domain, slow feedback is deadly. If your change subtly breaks the build, every developer behind you gets stuck waiting. The faster you find out, the better.

Setting up CI/CD for Unreal isn’t as easy as adding a few GitHub Actions lines to a Rails app. We didn’t want to deal with the hassle, so we used a third-party provider built specifically for UE. The people at Runreal run a fleet of high-end bare-metal machines with preloaded depots, ready to sync and build at a moment’s notice.

In web development, CI feedback on a commit usually takes a few minutes. In modern high-fidelity games, that’s rarely the case. Waiting an hour or more for a build run is typical. That’s why developers do as much local validation as they can before submitting a changelist.

Test automation in games is its own beast. Unlike business software, large 3D games rely heavily on manual QA. That means a lot of regressions slip through—especially the subtle ones you won’t notice for days. On small teams, developers often double as testers. Whether AI-driven QA will transform this in the next few years is an open question, but a lot of people are racing to find out.

Specialized test levels can help manually validate happy-path scenarios, but they’re no silver bullet. Testing a change in games takes far longer than in systems with full test coverage and a clean Test Pyramid. You can’t just toss a change at CI and trust that every edge case will be caught. Especially not when the goal is to preserve the right “feel” across a real-time, distributed system simulating a live multiplayer environment. One day, maybe.

The Bottom Line

As dominant as the Text World’s Git-based workflow is in the world of software development, it’s easy to forget that there are universes in technology that follow a fundamentally different approach to meet their distinct needs. What works in one field of software might not be viable in another.

There’s a lot to learn from the gamedev world: managing massive file depots, collaborating with experts who view the world through a different lens, and whose contributions take an entirely different route.

Gamedev doesn’t “do software wrong” or is behind the times. The field is working within the given constraints and uses solutions that work around the challenges unique to the craft. One could argue it is doing that as elegantly as current technology allows.

Disclaimer: The world of game development is vast, deeply rooted in computing history, and filled with countless unique niches beyond what I’ve described here. After all, you can deploy and control Doom on a printer. What is the best SDLC practice for that? This article offers a broad overview and doesn’t attempt to capture every possible environment, workflow, game genre, or team structure you might encounter in the industry.

A Tale of Two Hiring Seasons

Fri, 18 Jul 2025 20:37:52 -0700

Adding manpower to a late software project makes it later.

—Frederick P. Brooks Jr.

The single biggest factor in your hiring success isn’t your brilliant recruiting team, your world-changing mission, or your A-list investors. It’s the weather.

Not the actual weather, of course. I’m talking about the economic climate. Tech hiring markets are brutally seasonal. There’s Hiring Summer, when talent is abundant and you feel like a recruiting genius. And then there’s Hiring Winter, a desperate, character-building slog where you learn what you and your company are really made of. I’ve survived both, and one of them will teach you lessons you can’t afford to ignore.

Hiring Summer

It’s 2025, and we’re hiring for our game studio. It’s peak layoff season in games. It’s bad. It’s so bad that it has its own Wikipedia page. The games market has never experienced as strong of a whiplash before. Studios closing down left and right. Layoff lists circulating in private group chats. Leadership is trying to find their teams a new home, but many never find one and quit the industry. It’s a game of musical chairs, and seating is vanishing by the day. We interview someone laid off four times in a single year. For job seekers, it’s a disaster. For companies looking for talent, Hiring Summer is officially here.

I’m looking for a stellar Unreal Engine 5 multiplayer physics expert who can do it all. This person is going to be half of engineering. There aren’t that many such specialists in the world, it’s a fairly niche problem and people who really grok multiplayer models and have acquired important scars at it are rarely on the market. Ashby is set up, automation is up and running, the job description is up on game dev boards, Discords, Twitter, you name it. My inbox is exploding with CVs. I start aggressively filtering. There are so many resumes, it all becomes a blur. I have barely a minute to spend on each.

Many don’t seem to have even read the requirements before pressing submit, understandable in this kind of “survive at all costs” market. Or they tell you “Look, I know I’m not what you’re looking for, but maybe?” and throw the resume into the pile just in case. Hundreds of them. Hours later I’m down to 20 who seem in the ballpark. Which ones graduate to a phone call? Crap, more just applied. We’re getting close to 400 applicants in two days. We have to shut the gates. I chat with most of them over Zoom, send half of them take-homes, review the results, do an onsite with half of those, and take one to a work trial. Within a couple of days it’s obvious he’s the one. Three weeks after putting up the job description, we have our dream hire on the team. He’s continuing the few tickets he didn’t get to finish during the work trial week. He’s working the weekend, nobody asked him to, nobody on the team is. His enthusiasm is through the roof, an employer is actually paying him good money to work on games in this market, using his specialization to the fullest. Easiest hire ever.

In Hiring Summer, you’re playing on Easy Mode. Your only competition is time, not other firms. You can be as certain with a hire as you’d like. Take-homes? Presentations? Multi-day interview gauntlets? It’s all fair game now. Need a candidate for a week-long work trial or a contract-to-hire gig? They’ll jump at the chance, grateful to escape the radio silence from every other employer. It’s a powerful position to be in, but remember to be fair and gracious while you’re on top. Hiring Winter is always coming, and the tables will inevitably turn.

Hiring Winter

The year is 2019. ZIRP is in full swing and every software company in San Francisco is juiced to the gills with bottomless VC. Everybody is hiring. LinkedIn is pure recruiter spam. Unicorns are hoarding talent they don’t know what to do with.

A young upstart K-12 EdTech startup, us, is looking to hire a couple of engineers. Growth is modest, as customary in the K-12 tech space, but we have enough capital to expand the team to the next hermit crab shell permitted by the fundraise. Who knows, maybe we get to win the math practice market if only we move faster.

Here’s the problem: the Bay is hot, very hot. Comps are through the roof and growing. The average tenure is two years at best, it’s irresponsible to stay at the same firm any longer when you can 2x your comp in the VC-drunk feeding frenzy. Every other company is better capitalized than our humble education shop.

The market is so steamy that we stop doing reference checks. Whatever you think of them, you don’t expect reference checks to work against you and lose the hire. But that’s exactly what happens to us with one of our dream candidates. He gets to the last stage, he’s about to sign. We do one last call with a former manager–just to be safe–to make sure we did our due diligence.

The next day the candidate drops out. “Sorry,” he says, “The manager you spoke with didn’t know I was on the market. As soon as he found out, he offered me another 70k on top of your offer (which was already 220k). You guys are awesome, but I got a wife and a kid and a house in Seattle, and I really liked working with Tim before, I just couldn’t say no. Best of luck though, rooting for you”. Lesson learned.

It’s Hiring Winter, and you’re not at the front of the pack. How do you survive with this level of competition and barely any resources? What can you do besides lowering the bar? There’s a truism in the startup world: you’re building two products from day one of your company. One is the product itself, what your customers will ultimately pay you for. The other one is the company itself, something you will sell to investors, job applicants, your own team, for as long as you’re selling the product itself.

This second bit is what you focus on. Here’s how.

Step 1. Just like with any product in a competitive market, figure out how you’re differentiated compared to everybody else.

We were in the early wave of remote work. If you wanted to work “in Silicon Valley”, but didn’t want to move your family and two dogs to a beat-up one bedroom in SoMA, we were your ticket. This expanded the pool of accessible top talent far beyond the Bay, which most other companies didn’t do at the time. The team hated it at first, and we failed many of our initial remote experiments in all the predictable ways, but in the end it was a game changer due to being early.

We were among the few companies in the world who would pay you to work professionally in Haskell. If you had been bitten by the bug of writing software in a purely functional paradigm and using opinionated and pedantic type systems, and you wanted to work somewhere small, you came to us.

We were among the few companies in the world who would put your work in front of millions of kids in public schools. You could make a difference in a space traditionally neglected by tech.

You could stay home, you could work on frontier-level tech, and you could make a difference with it. It was a killer combo for the right type of person. Sure, you could have been paid much more somewhere else. You would have found more stability elsewhere. You could have found more growth at other companies. But for many, that wasn’t as important as what we could offer.

Step 2. Be loud and proud about it. We did talks at Haskell groups, blog posts, released libraries. It established the team as a group of world-class experts working on the frontier that you too could join. A group that worked on real-world problems, not one that’s only pontificating from high up in academia.

Don’t pick a technology for this purpose alone, but using a truly unique and hyped technology can save your rear when it comes to marketing your company. It made our job at hiring considerably easier for years to come, but the bet could have easily backfired. It didn’t for us and a few other companies who went with Haskell at the time.

Step 3. Keep updating the team’s unique positioning as the company evolves. Your unique traits will not stay unique forever. Remote work in 2025 is no longer a niche and controversial approach to organizing teams, it’s practically the default. Fully async work with no meetings? Now that’s wacky, and some people will find it fascinating.

Or maybe you are all in on coding with agents, no humans allowed. Spicy, but again, that will attract a certain demographic which may be a great fit for your company.

Or, you could just create interesting work for people to do. Affirm (allegedly) has its engineers rewrite its central data store once a year. It doesn’t need to, but it gives the team an epic struggle to undertake every year, and it keeps people engaged and sharp, rather than resting and vesting.

Survive the Hiring Winter and you’ll emerge grizzled and sharp. The character-building lessons become company muscle. And when Hiring Summer finally arrives, you won’t just find it easier; you’ll be dangerously efficient, knowing who you are, how to find those who resonate with it, and how to tell that story with well-practiced effectiveness. You were forced to trim every excess from your funnel, to waste no time, and to sell your company like its life depended on it. Because it did.

A Leader's Guide to Sudden WFH

Sun, 15 Mar 2020 00:00:00 +0000

In preparing for battle, I have always found that plans are useless but planning is indispensable.

—Dwight D. Eisenhower

Has the latest once-in-every-50-years species-threatening pandemic confined your team to your respective homes? Do you still need to ship product, make customer calls and maintain a regular meeting cadence to avert chaos? Don’t we all.

Every team faces a set of common and entirely predictable issues in the switch to remote work, and I want to share a few core practices that worked well for us at Freckle. We started remote work around 2015 when, as a tiny K-12 education startup, we struggled with hiring locally the kind of talent we wanted on our team. Since then, 70-80% of the company has become remote, allowing us to build an exceptional team we would have never put together in the Bay Area alone. Remote-accepted became remote-preferred, and eventually remote-only, with the unfortunate occurrence of COVID-19.

Here are the basics.

Prepare the new home workspaces

If you spend 16 hours a day in front of your home computer screen like I do, you might find it hard to believe that not everybody already has a dream workstation setup at home. Mind blown, right? A desk, a quiet room, an ergonomic monitor setup, and a stable Zoom-friendly internet connection aren’t a given for many, but are essential for productive work sessions at home. The last thing you want is to fight your environment.

What can you do, then, to support your team members who aren’t properly set up yet?

If you still have access to your office, then I recommend reimbursing last-minute Lyft rides to pick up whatever supplies your employees might want to move to their homes. For anything less urgent, an overnight Amazon or Monoprice delivery will take care of basics such as monitors, cables, and office chairs. Consider going the extra mile and providing your suddenly-remote employees with a home setup budget.

Reimburse your employees for part, or the entirety, of their internet bill. Some will not have a connection that can consistently handle a Zoom video call. This is critical for your customer-facing team members. Encourage them to upgrade their internet speed if they’re maxing out their bandwidth.

Employees who share their living spaces with others will also benefit from a headset with a directional boom mic. At Freckle, we’re partial to the Jabra Evolve 40 UC model, which does a great job at isolating the speaker’s voice in a loud office or call center.

This all assumes that all of your employees have access to an assigned work laptop that they can take home with them from the office. When that’s not the case, you will want to consider renting or acquiring devices ASAP.

Standardize communication tools

Wearing PJs unfortunately does not exonerate you from meetings, team syncs, status reports and ad-hoc group debates. Among many of the social work processes, you still need to run your all-hands and get your teams informed, inspired and motivated to do their best work.

Good news: when everyone is remote, meetings are actually easy. There’s no fiddling with conference webcams, awful large room acoustics — huh, what did the quiet guy in the back mumble? — and having to remember to regularly pause for your remote attendees to make sure they get a word in over people vigorously arguing in the room.

Bad news: you have to overcommunicate. Before, gaps in shared information might have been accidentally overcome through hallway whispers, overheard conversations and lunchtime catch ups. Now, getting the full story to your whole team must be a concerted effort. Many books have been written on company-wide communication alone, so I won’t attempt to squeeze that into a couple of paragraphs. A simple guideline is that you will not be able to overdo it; too much information is better than not enough. Also create a regular cadence for sharing info with your team, if you’re past the point where you can get away with one single Slack channel for all the latest company news.

What about tools, though?

Email is always there for anything that must not be missed, and requires more careful reading and permanence. Good old email is still your Swiss Army knife of information dissemination. For anything less ethereal, group chat tools like Slack are a time-tested solution — at least in startup dog-years. Tandem or Slack video calls can be great for impromptu chats, just be considerate about how and when you pull people out of their flow.

For video conferencing I’m biased toward Zoom, with its gallery view feature. It’s consistently fast, has miraculously not collapsed yet under the extra Coronavirus-induced traffic, and it’s free for K-12 schools impacted by the virus.
Tip #1 : the Zoom calendar integration for O365 or G Cal is essential to making sure there’s always an associated Zoom link with every meeting. The last thing you want is for people to roam your Slack channels asking for a meeting link. It must, must be there in the calendar event itself.
Tip #2 : the /zoom integration for Slack is also nothing to laugh at and adds a quick pathway to starting an impromptu video call.
Tip #3 : this should go without saying, but it’s still pervasive: make sure your video is on during calls. Many teams will devolve to audio-only, but that removes a lot of important communication information that we (mostly) hairless primates rely on to interact with each other.

You should let teams pick their tools, but your company’s core communication infrastructure is something you want to standardize.

Brace for distractions

Unless you’ve done this before, your first impact with the new work environment is going to be full of character-building lessons. Family members and pets will be confused about why you’re stepping on their routine. Your brain will associate home with fridge runs, naps and refreshing Twitter, not with staying focused on the task at hand. Motivation sags, the natural accountability of the office goes away. You wonder how anybody gets work done like this. And if you still have that last stick of string cheese left in the fridge.

Turning home-mode into work-mode is not easy, but it can be done. Set up your desk in a way that screams “work”: keep it clean of distractions and unrelated items. Just your laptop and your notes. Consider keeping your personal phone somewhere else to avoid the temptation. If you’re fancy, have a separate desk or room just for work. It’s tough for those of us who live in 500-square-foot studios in San Francisco, but your situation might be more conducive.

This is trite, but it works: dress as if you were going to work. Don’t go full suit unless you have to, but consider tech-casual or business-casual. It’s hard to negotiate budgets or fix a network outage in your datacenter with no pants on.

Set expectations with those around you that, despite being at home, you’re actually working and your job expects you to be available with minimal distractions. Again, looking like you’re working helps with this. Your cat won’t care, but there are exceptions to every piece of advice.

Hold space for feedback

Let’s go meta. At Freckle, we get a lot of value from having a #remote channel on Slack for surfacing concerns, feedback and ideas on how to make our remote workflows better. It’s a great place to raise issues likely affecting several people and see if anyone has found a workaround.

For example, when we still had an office, our remote staff couldn’t hear one of the conference rooms that well. People didn’t want to bring the issue up, concerned it might be only a problem on their end and it was maybe their fault. Receiving confirmation from others in the feedback channel helped show enough traction to warrant upgrading the mics in that room to a different system.

Additionally, consider having a biweekly or monthly remote hangout, and put feedback exchange on the agenda. With the team no longer having a physical space to voice concerns, having an allocated time to exchange ideas is going to pay dividends over time.

TL;DR

Switching to remote work in a pinch can be confusing, but if you address these four broad categories, you’ll have the fundamentals handled, and will be able to iterate further from there.

Set up your new work environment for optimal ergonomics and availability for video conferencing
Standardize all communication tools across the company
Expect distractions, and to deal with them one at a time.
Share feedback about your experience with remote work with the rest of the team, and iterate on it together.

Haskell at Front Row

Sat, 25 Apr 2015 00:00:00 +0000

Programs must be written for people to read, and only incidentally for machines to execute.

—Harold Abelson and Gerald Jay Sussman

I wrote this post originally for the Commercial Haskell SIG

The mission

Front Row Education was founded to change the way math education is done in a modern day classroom. In the web universe we have all sorts of great tools for tracking, analyzing and incentivising user behavior: complex analytics, rich data visualizations, a/b testing, studying usage patterns over time, cohort analysis, gamification etc. We figured: instead of using the above to have granny click on more ads, let’s make these powerful techniques available to teachers, parents and school administrators to make math education more engaging and effective.

Front Row allows schools to track student progress over time, identify areas of struggle, learn how to address them, all the while encouraging more quality practice. Learning math this way becomes a interactive and compelling experience, providing immediate feedback and adjusting content with every answer. As students practice, they generate rich data that school staff uses to continuously course-correct and fill in the gaps.

Numerous experiments from past years show that making Front Row a regular part of a math classroom leads to improved conceptual understanding, a lower rate of students falling behind, and improved scores on state tests. As of today Front Row helps over a million students in their regular math practice, and has been used in over 30% of US K-8 schools.

Our journey to Haskell

As of today Front Row uses Haskell for anything that needs to run on a server machine that is more complex than a 20 line ruby script. This includes most web services, cron-driven mailers, command-line support tools, applications for processing and validating content created by our teachers and more. We’ve been using Haskell actively in production since 2014.

At the time of the switch we were already familiar with the functional programming world. The central piece to the Front Row system is the JSON API used by both the student and teacher web experiences. I wrote the first version of the API in 2013 in Clojure on top of the Ring/Compojure micro-framework. At the time I didn’t have plans for the API to grow to serve the kind of size and traffic we see today: it was mostly a way for me to really dive into functional programming and understand design challenges that other popular frameworks had to come across.

Building your own framework is a fantastic learning experience, but it is also a significant commitment: without investing a ton of time and effort into the framework, you’ll end up with something very bare-bones and hard to turn it into a production quality, fully-featured application. It takes innumerable iterations to make a framework extensible, modular and well maintained with a team of 1-3 developers, busy with dozens other tasks that a fast-moving startup demands.

Clojure at the time didn’t offer any alternatives as far as web frameworks were concerned, and we were already starting to see the inherent critical weakness behind building large modular systems in dynamically typed languages: refactoring is a serious pain and something you will avoid at all costs because it’s hard to ensure you’re not breaking anything. It’s not that bad if you have ONE codebase that doesn’t have dependencies, but once you get into two digits you’re in for a bad time.

Switching to Haskell and the Yesod framework seemed like a natural step forward: a strongly typed, purely functional, highly expressive language that would finally allow refactoring and moving fast to be painless. On top of it, a beautifully designed, extensible web framework with years of polish, one of the best high-performance web servers in the industry, extreme attention to type safety, and an all-star team of OSS contributors supporting it.

Moving from Clojure to Haskell didn’t feel like a massive jump: a lot of concepts translate pretty closely, although Haskell offers a much richer vocabulary than just maps and vecs. Monads, type classes, IO etc. eventually clicked, and it was smooth sailing after that.

Advantages of using Haskell

Where does Haskell fit into all of this you say? As the development team of a small early stage edtech startup, we have two main goals:

Iterate as fast as possible on new educational concepts, business model experiments and user feedback. Basically, crank out as much code as possible while keeping the quality bar very high.
Stretch our runway, be conservative with our very limited resources

Haskell fits in pretty well with both of goals.

Static typing

First of all, static typing is essential when it comes to keeping the system always in a working state. Coming from a dynamically typed universe, it’s surprising how much time you can save on writing unit tests, because you are getting more certainty from the compiler: no more null exceptions, no type mismatches in function calls, no more forgetting about dealing with the empty list case etc. A whole class of pesky, incredibly common and banal bugs is eliminated from your work: you now have more bandwidth to worry about implementing user stories instead of obsessing that your application doesn’t blow up due to sloppy oversight.

I still remember one of my biggest Haskell/Yesod “aha” moments: not only does Yesod make sure that routes in your HTML are type-safe, but even image files linked in < img > tags are verified to exist on disk by the compiler. No .jpg, no build, it’s that simple. It’s a level of guarantee that dramatically increases your confidence in the code at barely any cost.

Modularity

Modularity is another big one. We have a central module at the bottom of every one of our web applications, APIs, tools and cron binaries. This module wraps the database entities and the SQL logic necessary to access them. It also provides a lot of common shared functionality that should not be implemented more than once. Since the schema changes very aggressively, we need a way to make sure our applications are updated ASAP, we can’t wait for things to blow up in production. Updating our entity definitions in that one module prevents every application built on top of it from compiling again until the change is dealt with.

No more API call mismatches, no more using an old schema, no more apps running against an old deprecated version that can lead to breaking the db state. As many others have stated, Haskell is the first language out there that feels like it manages to achieve true modularity: purity and defining what context a function is allowed to run in ensure that a library call can lead to no surprises. Testing side-effect free functions is much simpler than continuously dealing with system state.

Efficiency

Regarding the second point, why would Haskell stretch your runway? Simple. You’re writing fewer bugs, you’re reusing more code, new developers are causing less damage, and you have more room to deal with technical debt before it bites you. Purity and static types allow a team to aggressively refactor the codebase without having to worry that they might have forgotten to update something: a combination of a light layer of spec-style tests and a very picky compiler provide you with most of what you need to make refactoring a non-issue. More refactoring = more long-term productivity, higher team morale, more pride in one’s work. Doing the same with a Ruby is as fun as pulling teeth.

All of the above adds up to needing fewer developers, as less time is spent on maintenance, which ultimately equals a higher chance of your company getting somewhere thanks to the more frequent iterations. The more stuff you try, the more likely you are to find or expand that business mechanic that will carry your business forward.

Trouble in paradise

Things aren’t all perfect though. There are still quirks and plenty of room for improvement in the ecosystem.

Building

Build times, especially once the whole constellation of Yesod and Persistent packages are brought into the mix, are not insignificant. It still takes a good 5-10 min to build our larger web application on our beefiest machines. There are optimizations that can be made in this space which we haven’t adopted yet, such as caching already build object files to avoid having to re-compile them every time, so I’m confident this will be a non-issue in the nearby future, but it’s still worth being aware of. GHC works hard, you need to provide it with enough juice or time to let it do its job.

Testing

The testing frameworks out there are still fairly spartan from the developer experience standpoint. If you test Yesod with hspec, the premier BDD library for Haskell, there’s currently no way to insert a bunch of rows into the database during fixtures and pass the results into the individual test cases. You have to wrap each test case in additional
function calls to pull that off, adding more boilerplate to your tests.

Additionally, it’s not possible to find out which one of your specific test cases failed when checking for multiple conditions within the same “it” block. This means that if you need to check the state of the system after an HTTP request, you have no clue which one of the checks failed.

Fortunately the developer(s) behind these libraries are responsive and happy to look into improvements. At the very least they’re glad to point other developers in the right direction towards a PR.

This has in general been my experience with the Haskell community: things aren’t perfect, but folks are always looking for a way to improve the ecosystem and want Haskell to be the best language to develop in. People are trying to carve out their little slice of paradise, and are willing to put in the hard work to make it happen.

Docs

Documentation is still not quite there and the initial onboarding of new developers is still rough. There are only so many snippets to Google for, compared to e.g. Ruby and Python. A lot of documentation is very barebones and requires diving straight into the source, which is fine for a proficient Haskeller, but not for an already terrified beginner.

Many times I’ve witnessed senior developers get very frustrated when something wouldn’t compile for hours and they couldn’t find any help to move forward: be prepared to assist them before they get too grumpy. Some projects are better about it than other: Yesod and Persistent have extensive documentation and the FPComplete crew have numerous tutorials out there to help. New books come out once in a while with fresher snippets: the time-tested Real World Haskell is now fairly outdated, but the more recent Beginning Haskell is perfectly relevant. Many channels on IRC are available:
#haskell-beginners, #haskell and #yesod, although sometimes it can take work to get the answer you’re looking for. More than once I heard the comment that documentation seems to be written by wizards for other wizards, and if you’re a lowly initiate, you will have a rough time.

I’ve personally had the privilege to help all of our developers skill up in Haskell and Yesod, and I’ve become a huge believer in the power of having someone more experienced guide you along the way. What took me several months of learning, mostly by myself, now takes our developers a couple of weeks of quality coaching. It took me a while to grok monads, type classes, type families etc., however, properly guided developers can figure it out in a matter of hours. Having a good teacher on your team will speed adoption within the organization immensely.

Strength in numbers

We once experienced a very frustrating issue that got us thinking about our full commitment to Haskell as a company.

When we switched our main API to Yesod (a full rewrite), we almost immediately ran into the issue the API would burn up close to 95% of available CPU on whatever AWS EC2 instance it was hosted on. We upgraded machines, just to see if we could cheat our way out of fixing this by throwing money at the problem, and even with a $600/mo 16 core box, the API still managed to flood all of the available cores with barely any traffic hitting it. I personally spent a good week banging my head against it: was it resource contention? Was it a really big oversight in one of my handlers? Was it misconfiguration? Was it something about the EC2 environment? Why doesn’t this reproducing AT ALL under profiling? Was it our database connection pooling? I threw a lot of screenshots and code samples at the community both on Google Groups and IRC: nobody else had ever seen anything like it. Uh oh.. All the while customer support requests are pouring in,
teachers are aggravated, the team is looking at the devs and “their latest shiny toy”, tapping their collective foot.

This is the part where picking exotic tooling for your stack can be a dangerous beast: “given enough eyeballs, all bugs are shallow”, and when only a dozen teams out there are using your libraries at your scale, you are on your own when it comes to fixing issues. With Rails, there’s enough volume of developers that there will be enough projects of every scale to burn-in your tool of choice. That’s simply not the case with Haskell’s usage numbers.

What this means is that if you’re planning to bet the farm on Haskell, you need to be ready and comfortable with the idea that you might have to get your hands dirty, might be the first person to figure out a solution to the problem you’re seeing. This requirement is pretty much non-existent in .NET / ruby / python at al. Start small, start simple, let the tooling grow on you as you gain experience. Start with tools that aren’t mission critical until you’re more confident.

However..

It bears mentioning that the above concerns are being actively addressed by the community and the state of things is rapidly improving:

Cabal, the Haskell package manager, was a real pain to work with just a few years ago and “cabal hell” is still part of Haskell vernacular. However, with sandboxes and consistent version snapshots provided by FPComplete as Stackage LTS, that problem has been mostly resolved.
Build times are slow, but the community is coming up with improvements such as halcyon that should alleviate things considerably.
Docs have gotten dramatically better over the past couple of years. There’s been a big push towards keeping fresh, community-maintained, easy-to-follow and beginner-friendly instructions such as those provided by Chris Allen’s Learn
Haskell. We now even have IRC channels tailored specifically for beginners, e.g. #haskell-beginners . Today newcomers become more productive much faster than they did a few years ago.
The community has been recently doing a better job at outreach and we’ve seen many new developers come make Haskell a permanent part of their toolbox. With more participants, tools get more fully-featured and more maintained.

Conclusion

It’s a very exciting time in the history of computing to jump on the Haskell train. Yes, the community is tiny and one might get little hand-holding compared to more popular ecosystems, however Haskell offers obvious benefits to software teams who can power through the initial pain period.

Today Haskell offers some of the best tools around for delivering quality software quickly and reliably, minimizing maintenance cost while maximizing developer enjoyment. To me Haskell is that dream of “developer happiness” that we were promised many years ago by the Ruby community: I can write beautiful, short, expressive and readable code that will perform phenomenally and stand the test of time and continuous change. What more can I ask for?

To find out more about Front Row, visit us at https://www.frontrowed.com. You can also follow me at https://twitter.com/alex_kurilin

Shameless plug: we’re hiring. Currently looking for a Founding Systems Engineer with DevOps focus.

Upgrading Yesod 1.2 -> 1.4

Thu, 30 Oct 2014 00:00:00 +0000

There are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies, and the other way is to make it so complicated that there are no obvious deficiencies.

—C. A. R. Hoare

I’ve recently had to quickly upgrade Yesod 1.2 web application from version 1.2 to 1.4. I was looking for resources on how to do this and couldn’t find anything beyond Announcing Yesod 1.4 and Persistent 2.1 Released. This post should be useful if you’re in a similar situation and your app is backed by PostgreSQL, even though it should work just about the same for any other SQL backend.The transition was actually pretty simple, you have to take care of the following steps:

Update the libraries

Installing Yesod 1.4 through hackage is even more of a pain that before. It’s doable, and I have done that before, but I strongly recommend using Stackage Server. Go with an exclusive snapshot to avoid headaches and having to set constraints. It completely takes care of cabal hell as long as your dependencies aren’t highly exotic. Even then there are ways to work around it, but that’s for another post.

Once yoru cabal is wired against Stackage, remove version constraints from all of the packages in your .cabal file, as those will be decided for you by the Stackage snapshot.

Update Foundation.hs

-- replace this
import Database.Persist.Sql (SqlPersistT)
-- with this
import Database.Persist.Sql (SqlBackend)

Then:

-- replace this
giveUrlRenderer $(hamletFile "templates/default-layout-wrapper.hamlet")
-- with this
withUrlRenderer $(hamletFile "templates/default-layout-wrapper.hamlet")

Add the following after all of the implementations of instance YesodAuth:

instance YesodAuthPersist App

Model.hs

-- replace this
share [mkPersist sqlOnlySettings, mkMigrate "migrateAll"]
-- with this
share [mkPersist sqlSettings, mkMigrate "migrateAll"]

Update SQL entities

Love timestamptz in Postgres? ZonedTime SQL type is gone in Persistent 2.1: it’s now officially replaced with UTCTime which was already available before. If you actually need to know the TZ of a certain timestamp, you’re going to need a custom solution, Persistent isn’t going to do this out of the box, but if you don’t, just swap one for the other.

Update Entity Key creation logic

If like me you were manually generating Entity keys in your tests, then you will no longer be able to do so with Persistent 2.1. Replace your old code doing:

Entity (Key $ PersistInt64 1) { foo = "bar" }

with the new:

import Database.Persist.Sql
Entity (toSqlKey 1) { foo = "bar" }

The steps above should take of the most common conflicts that you will encounter when upgrading PostgreSQL-backed Yesod 1.2 to 1.4. Let me know if it’s missing anything.

As always don’t forget to drop by #yesod and #haskell-beginners on Freenode if you have any questions.

Early/mid 2014 update – we’re hiring

Tue, 06 May 2014 00:00:00 +0000

Management is doing things right; leadership is doing the right things.

—Peter Drucker and Warren Bennis

Whoa, a few months have gone by really fast here. Hi world. Guess I should post an update.

Been hard at work as usual at Front Row, it’s been very interesting to watch our numbers go up at a staggering pace while the # of people supporting the project stays the same. Really makes you appreciate the people making it happen, the tools you’re using and the best practices that keep the whole system from falling apart.

In any case, a few folks and we decided that it was time to give Front Row a bit of a budget to start scaling things out. We’re now looking for a couple of engineers to join us and work closely with yours truly. We’re seeking web generalists with strong interest in functional programming: we’re a very small team and being able to pick any part of the stack up in a few days is very important. Here’s more info.

Whatever you like, we got it:

Want to build super-interactive SPAs and get good at data-visualization? Check.
Want to get your hands dirty with ops, configuration management, build automation and continuous delivery? Love Ansible? Check.
Want to crunch a ton of data and serve it to the outside world with Clojure and Haskell? Check.
Want to build tools and pipelines that enable Front Row content creators to publish their work to hundreds of thousands of students? Check again.
Love DBs? We do too! Ours is growing exponentially 😦 If you’d like to contain the beast, we can keep you busy.

That’s it for now!

The hard balance

Sun, 02 Feb 2014 00:00:00 +0000

Plans are worthless, but planning is everything.

—Dwight D. Eisenhower

One of a founder’s toughest logistic challenges is balancing drinking the coolaid of one’s own vision against the iron skepticism that is required for making healthy data-driven decisions.

With the vision, we’re in the land of dreams: Ok, I convinced myself, the team, the people investing in us, the people paying for our services and the people using them that the world is one day going to work according to our bold vision. I claimed that this is how it’s going to be, it’s inevitable, it’s coming and we might as well be the ones executing on it. I claimed that you are all going to behave the way we predict for the foreseeable future.

In the real world on the other hand, a realm of estimates, percentages, decision-making through empirical means, nothing ever goes according to plan. You make a prediction, you test it, most of the time you discover missing a subtle but fatal nuance: one of your assumptions was wrong and you now need to try again with a different set of variables. Rinse and repeat. Maybe you overcome a local maximum, then you run into the next one which turns out to invalidate everything you had done up until that point. Oops, months wasted, time to go all the way back and try again. You just discovered one more way something couldn’t work.

The trick is that you need to convince everybody else of being right, yet at the same time you have to relentlessly try to prove yourself wrong: you most likely missed something and the plan won’t work until all the kinks are ironed out. It’s a very hard line to walk, it’s working every day through cognitive dissonance, getting used to it and not letting it stop you from making progress.

2013 – a retrospective

Mon, 16 Dec 2013 00:00:00 +0000

Those who cannot remember the past are condemned to repeat it.

—George Santayana

The kind of fast-paced, “just get it done now” work we do tends to be very day-to-day. You toil the days away, immediately switching to the next task, and never feel like you’ve learned anything. In a healthy Agile tradition, I would like to reflect on the past year and acknowledge the learnings, keeping things in perspective for those moments where I feel I’ve accomplished nothing. Here are some of them for posterity:

- First MVC JS app in Backbone.js. Since then iterated for close to a year on our existing teacher dashboard. Now using Backbone + LayoutManager to implement the student MVC app for Chromebooks, directly replicating the iOS experience. Really fortunate to be able to chat with folks of the caliber of Tim Branyen and Samuel Reed when it comes to getting some Backbone coaching.*
Got a lot more intimate with JS, CoffeeScript and functional programming in the process. Interesting foray into the require.js / AMD territory with hand-rolled build scripts etc. Dropped that in our latest app, but will eventually slowly build back up a very simple minimalistic pipeline with grunt.js.
- Lots of time spent on CSS and layout work. Good chunk of time working with boostrap 2.3.2 and bootstrap 3.0.x on both responsive sites.*
Wrote first unit test suite in JS thanks to buster.js. Actually extremely pleasurable, somewhat close to the testing experience in Clojure.
- Started working with Clojure half-way through the year. Still very much of an ongoing project, S. Halloway claims it takes at least 18 months for alien superpowers, so I still have plenty of room. Migrated Parse.com-based backend to our Ring app, rewrote all of our clients to use the new REST api instead of Parse’s libraries. Lots of tests on all levels of granularity. App is continuously being refactored as I discover new simpler and more elegant techniques. Infinitely grateful to the folks in #clojure, especially bitemyapp (Chris Allen) for the torrent of tips.*
Finally got some time re-visit the Dragon Book and put some of that to use for our tentative math interpreter with Clojure + instaparse.
- Lots of ops work. Never touched a “datacenter” previous to this year, was always hiding behind something like Heroku. Started with AWS EC2, not using any of their convenience services, feeling pretty comfortable with it now. Initially started CM with Puppet, didn’t like it. Switched on Ansible, no headaches whatsoever now. Switched traditional EC2 to a VPC. The system was all Internet-facing for a while, I had to figure out how to setup a VPN for the first time, which was fun. Now all office machines are on VPN and have direct safe access to the datacenter.*
Lots of Postgres work. Anything from deployment, to administration, to fine-tuning for performance for the given machine. That one was a BIG book, which I still reference every time.
- Good progress in my understanding of computer networks, still a work in progress. Recently had to setup a DNS server at the office for our Chromebook to be able to reach the sites hosted on my dev machine. Networks is such a good skillset to have when doing ops work, debugging connectivity in various locations such as our school networks.*
Plenty of Linux administration work. Again, still very much a work in progress. Lots of time spent on bash and Linux/UNIX tools and utilities. Running Ubuntu full time on all of my machines now, including home. Got very comfortable using tmux, and recently switched from Unity to xmonad so that the mouse is halfway useless. Tiling window managers are awesome, but do require some configuration effort.
- Lots of practice of XP and other Agile practices with a small team. Good experience with cherrypicking which practices to follow and when. Right now the engineering efforts at Front Row are still very manageable, but we’ll need to step things up a bit once more people hop on board and chaos ramps up.*

Looking back this has been a pretty fruitful year. Here’s what I hope to work on in 2014:

Would be interesting to pursue the interpreter-building route a bit more, see where that goes. Still very interested in getting more practice with building languages.
More Clojure work, of course. The language and the ecosystem are like chess: the pieces are very simple, but I can see this taking many years to master.
Would love to experiment with Haskell for some internal low-pressure tools, build some experience before considering it for production.
More work with scaling out the system. Right now our load is very humble, partially thanks to how the API was designed. If we do our job well at Front Row, then it will be time to start engineering for a more interesting load.
More depth with anything infrastructure-related such as Linux, network administration etc.
More opportunities to release some of our tools as open-source. I recently put out an early alpha of a Mixpanel client library for Clojure, so I’m hoping to release more of these tiny helpers as I get more experience with designing quality libraries that others actually want to use.

Clojure’s instaparse TI-style math interpreter

Sun, 01 Dec 2013 00:00:00 +0000

A language that doesn’t affect the way you think about programming, is not worth knowing.

—Alan J. Perlis

There aren’t too many examples of Clojure’s instaparse use out there, so if you’re working on parsing a little language of your own, I hope this might come in handy.

I’ve been working on a little interpreter for some internal stuff part of the Front Row stack, mainly for validating student answers in the more complex middle school math domains. The interpreting the answer becomes pretty much mandatory for validating things like equivalence of two polynomials. This is what came out from the early efforts.

The EBNF grammar itself can be found here, the implementation of the parser is here (still blows my mind it’s under 50 lines), and the tests are all here. Wouldn’t have touched this with a ten foot pole without testing every single incremental addition.

A couple of resources I found useful, in addition to instaparse’s official docs:

The author of instaparse himself was also generous with a few tips on the library’s Google Groups.

Clojure’s instaparse TI-style math interpreter

Correct Content-Type of static assets in Ring apps

Tue, 01 Oct 2013 00:00:00 +0000

Simplicity is prerequisite for reliability.

—Edsger W. Dijkstra

If you’re thinking of serving assets from a Clojure Ring app, then you should be aware of an interesting quirk of one of the core pieces of Ring middleware: wrap-file-info. This middleware is used to automatically detect the file type based on extension and inject the corresponding Content-Type header into the HTTP response.

Now, when developing with it enabled, you might run into an interesting and somewhat hard situation to debug. In development mode the files will be served with the correct MIME type. In production, when the app is packaged together into a single uberjar, IF your resources were being served from within the resources/ folder, then wrap-file-info will fail to correctly identify the file type.

Why? In development ring will be working with real files on disk, in a .jar situation you simply don’t get file extensions. See the following thread where Weavejester clarifies the situation.

Solution? Use wrap-content-type middleware instead. It only cares about the extension specified in the URL and should work correctly in most of the basic cases. I imagine you could have Ring fetch resources outside of the .jar itself, in which case the problem above would not reproduce. I believe you could get the best of both worlds by moving your assets outside of the uberjar and by using wrap-content-type regardless.

As a sidenote, cURL seems to infer MIME type based off of the URL, a behavior that’s inconsistent with the current versions of Firefox and Chrome. If you try to debug the issue above with cURL, you will be deeply puzzled. cURL will consistently correctly guess the MIME type, while the browser pointing at the file will print out a console message saying it’s getting a resource with text/plain, instead a different Content-Type value. The browser still correctly tries to use the resource, as they’re wrapped in a script or link tag with a specified content type. What makes this a tad extra frustrating is that Firefox allows you to generate cURL requests from the debug panel, and the requests are actually inconsistent with the browser that output that command.

If someone knows how to turn off Content-Type inference in cURL, do let me know 🙂