Red Squirrel's Nuts

Excerpts from Reinersten’s “Flow”

I believe it was Aslak who first mentioned The Principles of Product Development Flow: Second Generation Lean Product Development by Donald Reinertsen at speakerconf in February. I finally finished the book on vacation this week. I’m listing the excerpts that resonated the most strongly with me so I can easily reference them in the future. I highly recommend the book if you want to learn why the ideas behind agile development work, and if you want to improve the effectiveness of your development efforts.

Note: I didn’t track page numbers initially, but then started to later.

Another note: You can pretty much substitute “product development” for “software development”.

Final note: You’ll see that Reinertsen likes to use the word “resource” a lot. I tried to give him the benefit of the doubt, but eventually it became obvious he was referring to actual human beings when using that term. I find that term demeaning and counter-productive when used to refer to teams of creative individuals. So, feel free to substitute “resources” for “people”.

Reward generalization, not specialization.

Philosophy of United States Marine Corps: Uncertainty does not inherently favor either side. Instead, the side that can thrive in the presence of uncertainty is most likely to win.

When high quality decentralized economic information is absent, it is too often replaced by the mind-numbing bureaucracy of centralized control.

In product development, our problem is virtually never motionless engineers. It is almost always motionless work products.

To blindly conform to the original plan when it no longer represents the best economic choice is the act of a fool.

Control without [management] participation is control without decision-making delays.

If we make resources available for free, they will tend to be infinitely utilized.

High levels of capacity utilization are actually a primary cause of long cycle time.

When product developers choose to operate their processes at high levels of utilization, they create unnecessary and wasteful variability in their processes. It is important to realize that this variability is a self-inflicted wound.

The key to stable priorities is to link them to economic facts.

We simply cannot rely on randomness to correct the problems that randomness creates. p. 79

We should point out that any specialist can become a queue. p. 82

Either excessive or insufficient probability of failure reduces the efficiency with which we generate information. p. 94

If the cost of improving iteration speed is comparable to the cost of lowering the defect rate, improving iteration speed will have more influence. p. 106

It is usually not productive to work on old bugs, since they may die of old age or irrelevance before we get around to fixing them. p. 108

Batch size reduction can enable us to shorten cycle time, often by an order of magnitude or more, without changing capacity utilization. p. 112

The local efficiency improvements of large batches are completely wiped out by the added rework cause by delayed feedback. p. 116

The 100-fold improvement in manufacturing cycle times was not achieved by finding bottlenecks and adding capacity to the bottlenecks. It was achieved by reducing batch size. p. 133

Zombie projects destroy flow. Kill the zombies! p. 152

Most development organizations have a handful of special people who seem to be able to overpower the most difficult problems. They are the 10% that handles 90% of the most difficult work. If we wish to use such resources to destroy emerging queues, it is critical that we load them to less-than-full utilization. Unfortunately, most organizations love to load these highly profuctive resources to very high utilization rates. We need to change this mind-set. The big guns are not most valuable as an efficient substitute for small guns, they are most valuable for handling situations that the small guns cannot handle. p. 155

The ideal resource is a jack of all trades and a master of one. p. 156

Telecommunication networks are based on fundamentally different assumptions than manufacuring processes. They assume variability is inherent to the problem and adopt flow control approaches that are robust in the presence of variability. These network-based approaches are well-suited to the natural variability of product development processes. p. 169-170

By operating with small queues, we minimize the payoff of [prioritizing], and thus, the need to do it. p. 192

Flexible resources enable flexible routing. p. 202

Companies inevitably feel they can computerize the whiteboard, however, they almost always create a more elegant but less useful system. As simple as the whiteboard is, it makes WIP [Work in Progress] continuously visible, it enforces WIP constraints, it creates synchronized daily interaction, and it promotes interactive problem solving. p. 205

Optimizing the economics of a process is very different than maximizing it’s throughput. p. 208

It is common that we must invest in creating a superior development environment in order to extract the smaller signals that come with fast feedback. p. 222

If you ever have an opportunity to colocate a team, do it. If you don’t, fight hard to get it, and you will be glad you did. p. 231

Human effects of fast feedback are regenerative. Fast feedback gives people a sense of control; they use it, see results, and this further reinforces their sense of control. p. 232

The marines believe that they need the capability to make quick transitions, and, that this capability comes from constant practice. In product development, we can change direction more quickly when we have a small team of highly skilled people instead of a large team. … We can change direction more quickly when we have reserve capacity in our resources. It is very difficult to apply an extra burst of effort when people are already working 100 hours per week. p. 256

What builds trust? We trust someone when we feel we can predict their behavior under difficult conditions. … We actually do not need others to have similar attitudes and values to ours; we just need to predict how they will act. … Small batch sizes build trust, and trust enables the decentralized control that enables us to work with small batch sizes. p. 264-265

Expose Yourself

Derek Sivers just posted yet another excellent story to his blog. It’s a good, quick read. I wish he had written it a few years ago so I could have referred to it in my book. Specifically, in this chapter.

After reading it, I feel compelled to point out something that I think a lot of people will miss as they read Derek’s story. He’s writing about the importance of practicing. But I’ve noticed something in a couple of Derek’s stories: his willingness to approach knowledgeable, accomplished people and ask them questions.

Then I heard a man giving a demonstration of Indian vocal music, and his pitch was so perfect, I went rushing up to him afterwards to ask how he did it.

Do you know why it’s hard to do this? Because by asking questions like this, you’re Exposing Your Ignorance. Derek is letting this awesome singer know that he doesn’t know how to sing very well. That can feel really scary at first. I remember doing something similar with Wyatt Sutherland back in 2002, when he led the Chicago Agile Developers group. I wasn’t as brave as Derek, I did it via email, but it still paid off handsomely.

It’s so easy to overlook this simple but difficult habit. Develop it. Practice it. :)

In May, 2000

I quickly learned that commuting by car from Warrenville to Skokie every day during rush hour wasn’t going to work. Thankfully my new boss, Carolyn, was flexible about it, and let me do an early-in, early-out commute. I would wake up at 4am, do a quick workout, and hustle to Skokie before traffic set in. Then I’d be out the office’s back door at 4pm every day, and home to Staci and Rose before the roads got crazy. Naturally, there were plenty of times when things would get complicated and I’d end up sitting in traffic for what felt like an eternity. This was frustrating, but most definitely worth the hassle.

My workspace was in this cool, old bedroom-like office, upstairs from the main Edventions crew. Wood floors, creaky doors, and an occasional mouse made my silent mornings feel surreal. Once people started to trickle in after 8am, some of the magic disappeared, though the camaraderie was welcome. My time was spent trying to figure out how to use this egrail content management system. I received some occasional guidance from Irv, who seemingly understood everything about everything, though not in an obnoxious way. He was always very curious and eager. I liked Irv. I wanted to figure out how to help him build a successful business.

Egrail felt overly complicated. It was weird, it was actually a web site that managed your web site. I needed to fill in some web forms in the right way, with the right stuff to make starship.com look the right way and have the right content. The craziest parts were these places where I had to copy and paste these strange incantations wrapped in <? and ?>. Not sure what that’s about.

[This year, I’m blogging the year 2000, the year I started programming.]

Going Local

Back around 2006-2007, as Obtiva was growing beyond our initial client, we started asking ourselves what sort of company we wanted to be. We had a couple of successful examples nearby that we could have chosen to emulate. We could have tried to be like my former employer, ThoughtWorks, which had scaled a successful transnational consultancy with a world-class culture around software development. Or we could try to be like Object Mentor, and keep it small and focus exclusively on high margin training and coaching gigs, built around a few uniquely gifted personalities. We liked both of these companies, but for us, the fundamental problem with their business models was that we didn’t want to travel. Kevin and I are both married, with kids in elementary school, and we didn’t like the idea of leaving them on a weekly basis for most of their childhood.

As we pondered our future, I thought back to my tenure at ThoughtWorks (2004-2006). Wow, what an awesome experience that was! Seriously life-changing for me. But I was always perplexed at how few Chicago clients they had, despite their global headquarters’ location in the Windy City. I remember getting on my 20th consecutive Monday morning jet to Detroit and imagining other ThoughtWorkers east of me crossing me in the air, flying west to some other gig, maybe even in Chicago. I remembered feeling lucky to get a Chicago gig, and working on a great team of people who were flying in from all over the country, but then scratching my head when I considered how many Chicagoans would have been just as effective. Finally, I pulled up Chicago in Google Maps and marveled at its size. Surely there was enough business in this huge metropolis to keep us busy for years to come? Why not just do what we already wanted to do and focus on local work?

Well, that’s what we did. We don’t travel for client work. We’ll travel occasionally to do a training. And we actually enjoy traveling to go to conferences. But we’re lucky enough to live near a huge city, so there’s more than enough work to scale several companies like us. What’s a company like us? Well, we’re all unique snowflakes of course, but Obtiva settled into the dual niches of doing end-to-end project work from our studio while also augmenting teams in the some of best development shops in Chicago. I wish I could tell you our full client list, because I’m very proud of it, but I signed some paperwork that forbids me from telling you some of our clients. :| One client I can mention publicly is Groupon, who we’ve been working closely with for a year. (Just FYI, they’ve had quite a year, to say the least.) Lots of software folks will turn up their noses at “staff aug”, but I’ve learned that not all “staff aug” is created equal. Just ask two of our subcontractors, Corey Haines and Dave Astels, currently doing “staff aug” for us right now at two of our awesome clients.

There is an environmental benefit to focusing locally. Harkening back to 2006 again, I remember sitting in the Hard Rock Hotel in Chicago at ThoughtWorks’ internal conference. It was an awesome January day that assembled most of the US-based ThoughtWorkers to learn about all sorts of cool stuff (and a controversial new web framework called “Ruby on Rails”). Anyway, one of the more memorable parts of the day was a lunchtime keynote given by one of the head guys at Greenpeace. This was four months before An Inconvenient Truth was released, so this guy definitely grabbed our attention with tales of shrinking glaciers and projections of shrinking continents. Then he called our attention to the amount of pollution that air travel creates, mentioning that he had flown in to talk to us, but also pointing out the problematic environmental impact of ThoughtWorks’ travel tendencies. Now, I think back to my four years at Obtiva, and how tiny our footprint (per capita) is compared to a company that sends its people across the country (or globe) every week. Consider the energy/emissions it requires to fly hundreds of miles. Consider the energy/emissions required to drive a car dozens of miles. Now consider that almost no one at Obtiva even drives to work! We’re blessed to live in a city with public transportation, so we all take trains, buses, and bikes every day. And what do a lot of us do on the train? We read, we code, we learn!

While the environmental benefits are cool, I prefer to consider the effect that our local focus has on our families, since that was our original motivation. Last summer, I wrote a blog post, citing some of Obtiva’s remarkable statistics over its first 4 years, such as 6 births, 2 weddings, and 0 divorces. I’m happy to say that almost a year later, these numbers are still looking great: 10 births, 2 weddings, 0 divorces. I continue to believe that these numbers are strong positive indicators that we work at a sustainable pace. And for me and many other Obtivians, going local is fundamental to our sustainability.

Thanks to Fred Polgardy for retweeting this article, which inspired me to connect some of those ideas to Obtiva.

Obtiva’s Secret Sauce

Obtiva came from simple beginnings and we’ve grown slowly, conservatively, and frugally over the years. We’ve worked hard, and with small budgets, to reach where we are today. Where are we? We are one of the premier custom software consultancies in the world, and the most prestigious Rails shop in Chicago. How did we accomplish this without a cent of investment capital? Culture. We have a culture built upon the two fundamental activities of what it takes to become someone who delivers great software: the ability to learn, and the ability to communicate well. It’s simple in theory, but must be built intentionally.

Our founder, Kevin Taylor, has a passion for reading. When I showed up for my first week on the job at our first client, Kevin had already established a tradition there of weekly book clubs. A majority of the development team would spend one lunch hour a week discussing the assigned reading. I was impressed. As a consultant, I know how difficult it is to create lasting change in a development organization, yet Kevin had infected this place with a habit of learning. I joined in and read “Ruby for Rails” with the group, which, at the time, was the best introduction to Ruby for Rails developers. This book started Obtiva down the path of Ruby from our roots in Java. (We’ve come a long way, too! One of our senior consultants has just released the beta version of the Rails Test Prescriptions for the Pragmatic Press.)

A year later, I started our Studio practice with my first full-time Rails gig. At first it was just me in a little office in Wheaton, Illinois, where Kevin and I both live. Soon after starting the Studio, we hired a couple of apprentices to support me. What is an apprentice? An apprentice is someone who is willing to put themselves at the bottom of the totem pole for an opportunity to learn from experienced craftsmen. Our apprentices wanted to reach the next level as developers, and we believed that was a key strategy for growing our company: find high-potential, low-credential people, give them a learning-friendly atmosphere with challenging projects, and get out of their way. Naturally, one of the first things we did with our apprentices was start a weekly book club. We studied the manuscript of Refactoring: Ruby Edition. Our weekly discussions in this group slowly evolved beyond the book and took on a life of its own. Three years later, this weekly gathering of Obtiva geeks is still happening every week. We call it geekfest.

Geekfest has become the heartbeat of Obtiva. It is the only regular event that allows us to connect with all of the other developers in the company. (We also just had our first ObtivaCon, an internal conference we hope to repeat periodically.) Within a year of starting geekfest, Obtiva began recognizing its value and started providing free lunch for all attendees. Today, as many as 30 geeks attend Geekfest on a weekly basis. Sometimes we hear trial-run presentations from experienced mentors like Noel Rappin and Andy Maleh. Other times Geekfest provides a safe place for less experienced presenters, like Adam Walters and Ethan Gunderson, to practice. There are times when Geekfest is unstructured and we have an open discussion about current projects, new technologies, and hot topics in the developer community. Finally, we sometimes use Geekfest as a hackfest, spending the hour pairing with each other on our various pet projects, like Spittle or the Mad Mimi Mailer.

Geekfest is a tradition that embodies our commitment to accelerating learning and improving our communication skills. So does our apprenticeship program. Since 2007, Obtiva has been giving apprentices the opportunity to grow to the next level as developers. Over the years we have improved and adapted our apprenticeship program in order to maximize the experience for the apprentice, as well as increase the likelihood that our apprentices will be able to transition into the roles we need them to fill for our clients. Today, our apprenticeships last 6 months. For now, we only have a single apprentice at a time, giving the apprentice the opportunity to have our full attention. The apprentice has a contract-to-hire relationship with Obtiva and must successfully pass 3 milestones to continue through the 6 month program. These milestones give us the opportunity to judge the apprentice on the progression of their technical and communication skills. The milestone wraps up with a retrospective in order to incorporate feedback (from both sides) and adapt the next two months leading up to the next milestone.

Of the 7 apprentices we’ve had over the last 3 years, 5 of them are still with us. Our first apprentice, Brian Tatnall, is now employed at DRW Trading. Our second apprentice eventually moved onto other opportunities. Our third apprentice, Nate Jackson, is doing a stellar job pairing with Corey Haines (Obtiva subcontractor) at one of our newest clients. Our fourth apprentice, Colin Harris, has been consulting for us at Groupon for almost a year now. Our fifth apprentice, Leah Welty-Reiger, PhD., a particle-physicist-turned-web-developer is doing front-end and Rails development at our biggest client. Our sixth apprentice, Turner King, who, like Nate, is an absolute natural back-end developer, is doing JRuby on Rails work. Finally, our most recent apprentice, Ethan Gunderson is in our Studio, contributing to several projects, including Mad Mimi. He is also active in the Ruby community, contributes to RSpec, is being published in the Rails Magazine about HAML/SASS, just completed his apprenticeship, and is now an Obtiva employee. We have recently chosen our next apprentice: Michael Hines will start with us in a few weeks.

Our investments in our successful apprentices have paid off handsomely for everyone involved. Thankfully, the vast majority of our apprentices are successful. We have found that an aptitude for programming is obviously necessary, but that attitude has an amplifying effect on their aptitude. After interviewing countless candidates, and walking with these seven apprentices through their apprenticeships, we have developed a knack for spotting the ideal candidate. Thankfully, our job has become easier over the years: as we’ve risen in notoriety, we’re getting stronger and stronger apprentice candidates. For instance, our next apprentice, Michael Hines, is graduating from the University of Illinois’ computer science program a few days before starting with us.

Apprenticeship is not unique to Obtiva, but it’s something that we’ve embraced for over 3 years. It is a successful business strategy and a vital cultural contributor. We’ve found that apprentices bring enthusiasm and new ideas into a team that more experienced developers sometimes lack. For more on apprenticeship, from the apprentice’s perspective, have a look at Apprenticeship Patterns.

It sounds cliche, but the value of Obtiva is most definitely greater than the sum of its parts. Individually, our people are great. Each of them would be strong contributors in any software development endeavor. What is the secret sauce that multiplies our abilities? We’re all headed in the same direction: we aspire to become great at what we do and enjoy helping each other along the way. Over time, this common goal creates some amazing results, and our satisfied clients are the main beneficiaries.

In April, 2000

On the day before my 26th birthday, I took the day off to interview at Edventions. I drove the long (painful) drive to Skokie (from Warrenville) and found my way to someone named Irv. Craptastic! Irv’s the flippin’ founder and CEO! He had me sit with one of his content editors who sorta quizzed me on HTML, online content editing, and general (non-techie) web knowledge. Then I spent some time in conversation with Irv. Wow, this felt right. The building was this cool 2 story, marble-y old building on the corner of Bronx and Dempster, and for some reason, it just felt like home to me, especially with Irv’s vision for the company. It all felt very alive and vibrant, especially compared to any previous job I’d ever had.

At the end of our conversation, Irv matched the combined income of my day job and side gig, plus a little more, and offered me the job. I accepted it. We shook hands. I got in my car and drove to a gas station. Then I got out of the car and called Staci. Then I called my family, friends, and other people. My life was changing, and although I grew tired of all the times my life changed when I was growing up (we moved around a lot), I had grown to relish change as an adult. I was thrilled to be involved in Edventions, a real, live dot-com startup!

Later that week, I met with my team at work and let them know I was leaving. They were surprised that I would walk away from the field of mental health. Interestingly, some of my clients were less surprised, and encouraged me in my decision. I’ve never regretted or second guessed my decision to change careers, and later that month, that’s exactly what I did. I was building starship.com (don’t bother going there now) using an open source content management system called egrail. This was the first of a series of projects where I found myself in over my head. The work was great. The commute sucked. But it was totally worth it.

[This year, I’m going to be blogging the year 2000, the year I started programming.]

In March, 2000

The Chicago area was starting to thaw. I started spending an increasing amount of time between my client appointments walking around nearby Hidden Lake. As I walked in circles, I pondered the possibilities of this unexpected idea of becoming a computer programmer. The more I thought about it, the more sense it made to me. I prayed about it, and felt peace about what I was considering. So I kept looking for opportunities.

Over a weekend, I was cleaning out my car and found a scrap of paper with a web address on it. I remembered the commercial I heard on the radio about a startup. I stuffed it in my pocket. When I had some time later, I pulled up the web site on my trusty Internet Explorer 5 lickety split with my super-fast new “broadband” connection. Well, the startup wasn’t in Chicago, it was in Skokie, a hellish 40 mile commute only possible by car from where I lived. Yet, what I saw on the site seemed to fit my background. The company, Edventions, was developing a product called StarshipSchool to help parents, teachers, and students use the web to collaborate on homework, progress reports, email, etc. I had a good feeling about the opportunity, so I emailed them, attaching my resume.

Within an hour of emailing Edventions, I received a reply asking me to call. I called and was talking with someone named Irv about what they were looking for. (I later found out he was the CEO. These sorts of encounters are one reason small companies can be so remarkable.) We set up a time for me to come to the office for an interview. This was all moving very fast, and I loved it. I felt like I was living in “Internet speed”, the instinctual decision making I had heard stories about over the previous couple years as dot-coms started dominating the news.

The interview was set for the day before my 26th birthday, April 3, and I was excited.

[This year, I’m going to be blogging the year 2000, the year I started programming.]

In February, 2000

I was a child and therapist, questioning whether I could become a software developer. My attempt at learning Java was stalling. The exercises in Java for Dummies didn’t seem to have any purpose. Even if I could get them to work (which I couldn’t), they didn’t seem to do anything useful. I was filled with doubt. Did I really want to switch careers? I had a good job and had just finished a master’s degree. Was I selling out?

After work, when I wasn’t writing for my About.com side job, I searched the web (sans Google) for local tech jobs. I marvelled at the long and impressive sounding lists of techno-acronyms that I needed to know in order to apply. I quickly realized that I was going to have to apply despite not knowing most of what they were looking for. Remarkably, I got an interview with a placement company who was looking for Java developers. I showed up and was unsurprised (but disappointed) when they cut the interview short when they looked more closely at my “resume”.

I kept looking for tech jobs. I even heard an ad for a Chicago “dot-com startup” on the radio. I wrote down the web address on a piece of paper, but soonafter lost it somewhere in my Chevy Lumina. I gave up on Java for Dummies, but dug deeper into HTML, and started learning some JavaScript. I wasn’t quite sure where this was heading.

[This year, I’m going to be blogging the year 2000, the year I started programming.]

In January, 2000

I was a child and family therapist. I worked in the Intensive Outreach Unit of the DuPage County Health Department. I had just finished my master’s degree in 1999 and was excited about my first gig as a “real” therapist. We worked in pairs and did in-home family therapy to prevent children and adolescents from being hospitalized for suicidal, homicidal, or psychotic behavior. We had a small case load and I was working with some great therapists. Compared to my cohorts from my master’s program, I felt like I’d found a great gig.

But I had another job too. I was the “Teen Advice” guide at About.com. I got the job to help pay the bills, but also because I’ve always been into computers, and was especially interested in the Internet since my uncle introduced it to me back in 1994. This job involved HTML, which felt magical to me, and I found that the time I spent coding HTML was giving me energy. I wanted to dig deeper into this seemingly magical art.

So, 10 years ago, between client appointments, I would hide out in my car in random parking lots reading Java for Dummies. Then I would go home and (when Staci and Rose were sleeping) try out what I learned. I worked hard at it for a while, and would get excited when I could make a button or JSomethingErOther appear on my PC, but it wasn’t sinking in. Applets seemed cool, I guess, but what’s with this javac thing? And all of these rules about public and static?

[This year, I’m going to be blogging the year 2000, the year I started programming.]

Second Nature

A Japanese translator contacted me last week. Apparently, Apprenticeship Patterns is being translated into Japanese. (In other news, Enrique might be working on Spanish and/or German translations.) Well, the translator wasted no time, and started asking Ade and me questions about the meanings of some trickier sentences in our dedication and Ward Cunningham’s foreword.

His question about the foreword forced me to rewrite Ward’s sentence, and I suppose like anyone who has had to rewrite anything of Ward’s, I found the experience enlightening. The translator was confused by a sentence in the last paragraph:

The craftsmanship movement in software recognizes that making this stuff second nature isn’t, well, second nature.

Being the web geek that I am, the first thing I did was look up the definition of “second nature”: “An acquired behavior or trait that is so long practiced as to seem innate.” And after struggling with it for a bit, I rewrote the sentence into this:

The craftsmanship movement recognizes that practicing and acquiring the skills of software craftsmanship are not innate to humans. Not only do we need to acquire software development skills, but we need to acquire the skills of skill acquisition.

This satisfied the translator. I was pretty happy with it too, since it helps explain what the book is about. It won’t teach you how to be a great programmer, it will teach you how to learn to be a great programmer.