"[To outsiders] the functional programmer sounds rather like a medieval monk denying himself the pleasures of life in the hope that it will make him virtuous." - John Hughes, "Why functional programming matters", 1984

I've been working on this blog post for 16 years, ever since discovering Erlang in 1999. For most of that time, I made very little progress. I was a self-taught, fairly lousy programmer with a bad case of expert beginner syndrome.^[1] It took me years to roll back to just an advanced beginner.

Four years ago I finally started learning functional programming, but I didn't know what made it different (beyond the obvious), or how to learn it, or even why to bother. I had, of course, discovered John Hughes' "Why Functional Programming Matters" paper,^[2] but I didn't know enough to understand it. I'd even found Reginald Braithwaite's "Why Why Functional Programming Matters Matters",^[3] and it took me a while to get it. I needed context, and I'm stubborn enough to discover it the hard way. So I realized I needed to write my own version of "Why functional programming matters". And I've done that, several times.

This effort is called "Why be functional?", and it's more or less a persuasive essay on using functional programming.

Here's a spoiler: My answer to "why be functional?" is simple, demonstrably correct code. But that's my answer to most questions about coding craft, so I'll explore it further. We can start with complexity.

Complexity

"I conclude that there are two ways of constructing a software design: One way is to make it so simple that there are obviously no deficiencies and the other way is to make it so complicated that there are no obvious deficiencies. The first method is far more difficult." - C. A. R. Hoare

What is complexity? There are lots of answers to that. In "No silver bullet"^[4], Mythical Man Month author Fred Brooks describes "accidental" and "essential" complexity, putting all code in one category or the other. That's probably correct: the morphology of software, it's form and structure, is complexity by definition, and there is no current way to avoid it.

Fortunately, complexity is relative and we can reduce it if we can identify it. So here are some other places we look for complexity, drawn from various sources.

• Complexity is any code that can be, but hasn't been, decomposed into simpler components.
• Complexity is the interrelationships and dependencies of software components. Complexity is the degree to which one piece of software is dependent on another.
• Complexity is the enemy of reliability. Complexity and reliability are inversely proportional.
• Complexity is the part of our code that isn't beautiful. Beauty is the ultimate defense against complexity.
• Complexity is code. If we could accomplish our goals without programming, we would.

Why is complexity bad? I'll give you four simple reasons: Complexity is code that we can't reason about, can't test, can't prove, and can't trust.

• We can't reason about our code.

We do two kinds of reasoning when it comes to our code. We do "informal" reasoning, which is kind of like crawling inside the code and looking around, figuring out how things should work. We also do "formal" reasoning, which applies math and logic to our code. Complexity actually means we can't do formal or informal reasoning, but in this case I'm referring to informal reasoning, our ability to understand the code.

Being able to reason about our code is important, because we spend far more time and money maintaining a code base than creating it. Code we can't reason about is not only going to take longer to maintain, but we're going to maintain it poorly because we don't understand it. Both problems translate to larger lifecycle costs.

• We can't test our code.

Complex code is difficult to test. We mock the libraries we depend on. We build ourselves a dingus.^[5] We fake our IO - or worse, we don't. We try to identify edge cases and fail. Or worse, we don't.

• We can't prove our code.

This is where formal reasoning comes in, but honestly, formal reasoning almost never comes into our code. It's difficult to prove anything about imperative code, and complex imperative code may fall outside of what's possible to prove. After all, it only takes 24 bytes to represent more different states than there are atoms in the earth.

• We can't trust our code.

This is the bottom line. If we can't reason about our code, and we can't demonstrate the correctness of our code either through tests or logic, then we don't have reasons to trust it, except to the degree we trust ourselves. That used to be enough, but we're learning to build better software.

"[W]e have to keep it crisp, disentangled, and simple if we refuse to be crushed by the complexities of our own making.” - Edsger Djikstra

Simplicity

"It is easier to write an incorrect program than understand a correct one." - Alan J. Perlis

What is simplicity? It's the opposite of complexity. It's the lack of complexity, the other end of the spectrum from complexity. It's achieving our goals and meeting our needs without introducing unnecessary complexity.

I use the same standards described under complexity to test whether code is simple. I call it a "simplicity test".

• Can we reason about the code?
• Can we test the code?
• Can we demonstrate the correctness of the code?

The better our code meets these standards, the more simple I believe it is.

This test works for more than evaluating code we've written. I use it to think about how patterns, languages, or even programming paradigms affect my work. Shortly, we'll use the simplicity test here to functional programming, but for now let's try it out with Object-Oriented Programming.

Applying the simplicity test to Object-Oriented Programming

"OO makes code understandable by encapsulating moving parts. FP makes code understandable by minimizing moving parts." - Michael Feathers

We can use the simplicity test as a lens to view object-oriented programming.

• Can we reason about the code? Yes, we can reason about well-written OO code.

At least, we might be able to reason about well-written object-oriented code, but in practice there's a lot of OO code out in the wild that's extremely difficult to reason about. There are some good reasons for this.

A core principle of object-oriented programming is encapsulation. We hide complexity, even reasonable complexity, inside of objects to make sure that most programmers don't ever have to experience it. Unfortunately this means that we can hide complexity, even unreasonable complexity, inside our objects because no one but us are ever going to see it. Later, when we're maintaining code, we are often inclined to increase complexity of someone else's code because it's hidden.

Object-oriented programming not only encapsulates complexity, but offers a lot of it. Our objects have mutable state, and behaviors to mutate that state. It's often hard to explain why we're in the state we are, because state can be mutated anywhere inside the object by any code that accesses the object. We don't typically keep track of how our state changed in the past, so we can't track how we got where we are.

• Can we test the code? Yes, we can test our code. We do it all the time.

In practice, in order to make our code testable we follow a lot of principles and patterns to try to keep our complexity down. Even then, it's often difficult to test our code, even just unit tests, and our tests often take a long time to run. Our objects are complex, our state is mutable, and our code is hard to reason about.

Of course, most code doesn't have tests, and composing tests for legacy code is often an art form.

• Can we demonstrate the correctness of the code? Yes, it's possible to write OO code that can be formally verified.

In practice, very little OO code is written for formal proof and very few OO programmers would even know where to start.

Using the simplicity test, we can see it's possible to write simple code in an object-oriented language. Of course we knew that, because we've all written simple code in object-oriented languages. We've also seen our OO code go from simple to complex, often feeling like we fell off the complexity cliff and didn't even see it coming.

What is functional programming?

“Inside every well-written large program is a well-written small program.” - C. A. R. Hoare

First and foremost, functional programming is math. A lot of programming is math, but in imperative programming we use that math to make programs that don't need to follow rules the way math follows rules. With functional programming, we figure that the more a language follows the rules of math and logic, the more "purely" functional programming it is.

Functional programming is made up, mostly, of functions. Pure functions have a few characteristics that follow simple rules of math.

Functions

• Functions are referentially transparent.

"Referential transparency" is a characteristic that says a function will return the same value whenever it's given the same arguments. Another way of saying this is that nothing about the state of a program can affect how a function works. Another way of saying this is that a function call and its result are always interchangeable, without impacting the meaning of a program.

This is what we expect from math, too. Math is a hard science; it doesn't change from one moment to another, even when the world around us changes.

• Functions can be composed of functions.

We can apply one function to the result of another function. We can string a bunch of functions together to make a program. In functional programming, that's typically how we make a program.

f(g(x)) // composition
g(x) |> f // or pipelining
x |> g |> f // or more elegant pipelining

This is also what we expect of math. We can reason mathematically about functions applied to other functions applied to variables, without solving for the functions.

• Functions can be distributed.

Because a function will always return the same value for a given argument, it often doesn't matter what order we do things in, or even on what computer we do things. We can distribute functions across a cluster and we'll always get the same result, because the result can not be dependent on the state of the computer.

Again, this is what we expect from math. If we're solving a formula for a value, the order in which we solve uncomposed functions doesn't matter. We can reason about it without solving it. We can even compose functions and reduce them without solving for a specific value.

• Functions can be evaluated lazily.

Often with functions, we have the option of not figuring out a result until we actually need it. Just because a program has code arranged in a certain order, that doesn't mean it has to be run in that order. Instead of calculating and storing a result, a language could just remember how to figure out the result if and when someone actually tries to use it.

There's some question about whether this is an important part of functional programming, but we tend to see it more in functional programming than in imperative programming.

• Functions can be arguments to other functions.

In functional programming, we can pass a function as an argument to another function. This characteristic of functional programming has been pretty widely adopted in imperative programming languages too. In fact, Javascript would hardly work without callback functions.

Common functions that use higher order functions (that is, functions as arguments) are map, filter, and reduce or fold.

• Functions can be recursive.

Recursive functions are hardly limited to functional programming languages, but they are used very often in functional programming. This is because pure functional programming doesn't provide mutable state, and our typical imperative manner of iterating requires mutability. As L. Peter Deutsch said, "To iterate is human, to recurse divine."

Immutability

Functional programming languages don't do assignment the way that imperative and object-oriented languages do. In imperative languages it makes sense to say something like "Set X to True" or "Assign 1 to Y". In declarative languages, including functional languages, we aren't doing assignment, we're describing the world. If we tell a functional language that Y is equal to the value 1, and then we try to say that Y is equal to a different value, the program may just say "No it isn't. You already told me it was 1." (Actually, it'll probably say something like 'right hand and left hand values do not match', because that's what the language is doing, matching names and values.)

Immutability comes to us from two directions.

• Remember how we said a function call applied to an argument is always interchangeable with the result, and can't affect the meaning of a program? That also applies to any reference to that function call. If we say x = f(1), then x always has to be interchangeable with the result of the function call. If x can change, then it's not guaranteed to be interchangeable.
• Most of us tend to think about programming from a single-threaded perspective, but there's nothing about the math that suggests things happen sequentially unless there's causality. So what happens in a multi-threaded environment with mutable data? One thread might change data that another thread is accessing, or intends to access. That's not thread safe, and as a result we can't guarantee referential transparency.

These are advantageous to us as developers because they make it easier to reason about our code. If we know what x means, we always know what x means. Values, references to values, and values returned by functions will always be consistent.

This makes a difference in informal reasoning, but it also makes a difference in formal reasoning. If we have these invariants, if functions are referentially transparent and values can't change, then we can use math and logic to draw conclusions about the properties of our code. The conclusions we're most likely to want to draw is that the code is correct to its specification.

Side effects

"In the end, any program must manipulate state. A program that has no side effects whatsoever is a kind of black box. All you can tell is the box gets hotter." - Simon Peyton-Jones

Functional programming doesn't eliminate side effects. After all, side effects are how we get value from our work. Imagine this conversation:

Functional programmer: "I wrote a purely functional program that will transform our data."

Product manager: "Cool, how do I get data in?"

Functional programmer: "You can't."

Product manager: "Um, how do I get data out?"

Functional programmer: "You can't."

Instead, functional programming isolates side effects, because side effects represent the real world and the real world is messy. It may be governed by physics and math, but the complexity is out of this world! We manage that complexity by isolating it.

One way to isolate it is to keep side effects at the edges. For example, we can bring in data from the real world, and then pass it through a series of functions that transform it, and then return the transformed value to the real world. This reduces almost all the complexity. We have a "given" at the beginning, a series of functions we can reason about, and a way to return a result at the end.

Another way to isolate the complexity from side effects is to wrap it up in a model of the world, and pass that around as an argument. Actions that affect the world transform our model of the world into a new model of the world, which is passed on to the next function.

Some functional programming languages just give us an imperative way to do side effects, and trust us to not use that power foolishly.

Ultimately, getting data in and out of our functional program are imperative, not declarative, operations. We react to events or respond to commands or write to a file. These operations aren't referentially transparent - they can return a different value each time they're called. They aren't immutable, and in fact we may have many different actors changing the data concurrently.

The simplicity test, applied to functional programming

We've already discussed a way of looking at a program and determining if it's simple. We called it the simplicity test. We can apply that test to functional programming as a whole.

• Can we reason about it? Yes. Functional programming can make it easier to reason about our code.

Referential transparency promises us that we can trust the result of our function calls. The results won't change based on anything but the arguments right in front of us.

Function composition gives us a way to look at our code in different levels of abstraction, from the implementation details of each step, to the way the steps are put together.

Immutable state ensures that we always know the value of things. If we transform data, we always get new data back.

Side effects are isolated, leaving us with a very stable view of our system.

• Can we test it? Yes. Functional programming makes unit testing very easy, and integration testing easier.

Referential transparency promises us that nothing but the arguments to a function will change the result of a function. If we write pure functions, we should never have to use mocks in our unit tests.

Function composition with referential transparency gives us confidence that a program tested at a unit level will perform predictably.

Side effects are isolated, and they're almost always limited to IO libraries. We don't need to unit test IO when it's isolated; the IO libraries come with their own unit tests.

• Can we demonstrate correctness? Yes, we can often use formal methods and more effective testing to demonstrate our code will perform correctly.

Functional programming was not only designed to be provable, it comes from math that's designed to be used for proofs. In 1935, several years before the first general purpose computer, Alonzo Church published about lambda calculus, a math for demonstrating the completeness and correctness of algorithms. Today almost all functional programming languages are based on lambda calculus.

Not all programs are deterministic and not all can be proven correct, but with functional programming we are almost always better able to demonstrate correctness.

Programming our brains

"A language that doesn't affect the way you think about programming is not worth knowing." - Alan Perlis

While functional programming meets our simplicity test better than object-oriented programming, that's only part of the advantage. Functional programming also changes the way we approach our work, and think about the problems we're solving. I think the way it changes our thinking is as important as how it changes our coding.

Imperative and declarative language

Functional programming is a subset of declarative programming, and object-oriented programming is a subset of imperative programming. One way of thinking about the difference is whether a variable is set equal to a value, or if we are informing our program that a variable has the same value as something else. In imperative programs, we instruct the computer to do things: set X to 0, calculate a result, print a message. In declarative programs, we describe relationships between things and then ask the program to give us conclusions: There is a function like this, and X has the same value as 3, so what do we get if we solve the function with X?

We speak in declarative language most of the time, especially with other adults. We share experiences ("That meeting sure was a bore!"), and talk about our feelings ("If manager Rob does that one more time, I'm gonna blow my fuse.") We tend to dislike adults who speak imperatively to us ("Make me a sandwich. Sudo make me a sandwich."^[6]) and think of them as jerks.

It turns out that imperative communication is easy to find between adults and children. "Close the door." "Pick up your shoes." One the other hand, declarative language is amazing with kids. "I'm cold because the door is still open." "I see your shoes are in the middle of the floor. I'm afraid someone will trip over them." It helps toddlers develop surprising vocabularies. It helps teens relate to adults and each other. It helps kids with autism increase their verbal and social skills. It literally grows the prefrontal cortex.

Robin Dunbar, the guy who described Dunbar's Number, has done research connecting the size of our orbitomedial prefrontal cortex and the size of our social network, the number of meaningful social relationships we can manage at one time. The upshot is that people with bigger orbitomedial prefrontal cortices have larger social networks, approaching 220 people.

Good news for programmers, the size of our prefrontal cortex also seems related to the amount of complex of code we can hold in our head at one time. So maybe declarative programming languages aren't just simpler, but make us smarter and better at managing social relationships.

"I'm a functional programmer, and gosh darn it, people like me."

Identity at the corner of State and Time

Functional programming changes how we model the world in our code, and as a result how we see the world.

We know that objects make for a decent abstraction in our programming. That's true. It's a reasonably effective way to model the real-world. The problem is that objects have no way to account for the passage of time, and in the real world, "identity" needs to account for the passage of time.

• In object-oriented programming, identity is the present state of an object.
• In functional programming, identity is a collection of immutable states over time.

It's fair to say "I'm not the person I used to be." But who I used to be is an integral part of who I am today. The object-oriented way of viewing objects doesn't account for this. Instead the state of an object is frozen in the current moment.

In functional programming, we don't mutate state, so we have a different state for every changed version of a model. I am who I am right now, but I'm also a collection of all the people I used to be. Functional programming enforces this in its most basic pattern: new state is old state transformed by a function. While we don't have to keep every previous version of our state, each version was created separately and we often do choose to keep them.

The power of a restrictive language

My favorite tech tee-shirt reads "My compiler compiled your compiler". That's the power of a systems language - it can do anything, including creating new programming languages that expand the possibilities. But how much of that power do we need? Sometimes our systems languages feel like driving a semi tractor to the grocery store. More than enough power to transport a dozen eggs, fast enough to hit the speed limit, and really difficult to drive around and park. And I hate to think about what happens in a fender bender.

Languages with a few, well defined, and useful abstractions are limiting in comparison to a systems language. My Toyota can't pull a semi trailer and can't easily transport 60 head of cattle. It can however transport a dozen eggs, hit the speed limit, and it is easy to drive and park.

There are languages that have power and flexibility to do almost everything with a computer that you can imagine. Many of these languages give you wide choice of how to model a problem, how to address control flow, how to structure your data. Unfortunately, all of this flexibility means that there many more ways to solve any given problem, and they don't all have the same merit. All of the power means that we can make bigger and bigger mistakes. It's like Bjarne Stroustrup said: "C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off."

For me and my team, I like a restrictive language. Give me fewer abstractions that fit more situations. Give me abstractions of the other languages' abstractions. A language with fewer abstractions can be learned faster, designs are done sooner and with less code. We focus on the problem instead of the implementation.

“Mathematicians stand on each others' shoulders. Computer scientists stand on each others' toes.” - Richard Hamming

There's another way of looking at this question of how languages present different abstractions. I heard about it from Phil Wadler in a talk about Curry-Howard isomorphism, which is just a fancy term for the way these two dudes realized that mathematicians and computer scientists kept discovering the same stuff.

• Hilbert-style deduction systems (Frege & Hilbert, 1925?) <-> Combinatory logic (Curry & Fey, 1958)
• Natural deduction (Gentzen, 1935) <-> Typed lambda calculus (Church, 1940)
• Type schemes (Hindley, 1969) <-> ML (Milner, 1975)
• Modal logic (Lewis, 1910) <-> Monads (Kleisli, 1965; Moggi, 1987)
• Classical-Intuitionistic Embedding (Godel, 1933) <-> Continuation passing style (Reynolds, 1972)
• Linear logic (Girard, 1987) <-> Session types (Honda, 1993)
• Intuitionistic logic <-> Typed lambda calculus <-> Cartesian closed categories

After hearing Phil Wadler present his talk on "Propositions as Types",^[7] I had a breakthrough. Wadler seemed to desribe that some languages were invented, and some languages were discovered. This fit with my idea on simpler, better abstractions and put it more neatly. Some languages felt to me like they were derived from the rules of the universe, and others felt like they were invented in someone's garage.

The Curry-Howard (-Lambek) isomorphism gave me the proof I needed to trace languages back to the abstractions introduced by both computer scientists and mathematicians independently. Here are some examples:

ML family languages:

• ML (Milner, Gordon, Wadsworth, 1979)
• Haskell (Hudak, Hughes, Peyton Jones, and Wadler, 1987)
• O'Caml (Leroy, 1996)
• F# (Syme, 2006)
• Elm (Czaplicki, 2012)

Lisp/Scheme family languages:

• Scheme (Steele and Sussman, 1975)
• Clojure (Hickey, 2007)

Prolog/Erlang family languages:

• Prolog (Colmerauer, 1972)
• Erlang (Armstrong, Virding, Williams, 1988)
• Elixir (Valim, 2012)

I contrast these languages with Scala (Odersky, 2004), a general purpose programming language inspired by Java's deficiencies and supporting functional programming. I've also heard it called "a slippery slope to functional programming" for Java developers.

Functional-first programming

Having built the case for pure functional programming and even identifying a bunch of languages that expose clean and functional abstractions, let's finish up with ways to bring some of the good things from functional programming into our imperative environments.

I call it "functional-first programming", but I'm hardly the first to make suggestions like these. Some people have simply suggested isolating side effects. Gary Bernhardt has presented it as "Functional core, imperative shell".^[8]

We can drag the low hanging fruit from functional programming into our imperative languages by following two simple steps:

1. Code everything you can without side effects.
2. Then code your side effects.

These two steps are not as simple as they look, but if we follow them we get a bunch of benefits.

Benefits

• Our functions should have referential transparency.

If our functions have no side effects, then the only effect they have on the meaning of our program comes from the value they return. They are referentially transparent, and simple to reason about.

Our functions should be pure, but within the limits of the language. For example, any language that passes arguments by reference risks mutating those arguments. Python, for example, passes object references by value, which means the function will be looking at the same object in memory that was passed to it. If the standard ways of interacting with that object also mutate it, as Python arrays do, it will mutate it in place and have side effects unless the developer knows to avoid this.

• Our program is a composition.

If our functions are fairly pure, much of our business logic will be a composition of functions that transforms our inputs into our outputs. Again, we win on simplicity and reason.

• Our testing story is beautiful.

Pure functions are fast and easy to test. We can write better tests about them. We don't have to mock anything. Each test runs in virtually no time, and when you put all your tests together they don't add up to very much time.

Because our side effects are isolated, the part of our code that does IO doesn't do business logic and doesn't need unit tests. That means we don't have to mock IO libraries and we probably don't have to inject dependencies and our testing is simple.

• Our business logic is separate from our IO.

We don't need functional-first programming in order to separate our business logic from IO. We've been doing this for decades with the MVC pattern and numerous others.

Separating side effects, however, takes this to a new level. It means that our functions can be as pure as possible. Our business logic can be written without even referencing framework or database, and we shouldn't even have a dependency on libraries that do IO.

• Our IO is modular.

Moving all of the IO to the edges has a nice advantage too. Anything that does IO can be easily added or replaced. If we decide to change our database layer, it's all in one place. If we want to add another interface, we can tack it onto the shell with the other interfaces.

Disadvantages

• We're not really doing object-oriented programming any more.

Functional-first programming isn't object-oriented, and that's going to be hard for a lot of teams. Even if you make utility classes and singletons, it's still going to feel weird, and it's going to look weird, and it's going to be weird to an OO programmer.

It is possible to do functional-first programming in more of an object-oriented environment. That's very much the subject of Gary Bernhardt's "functional core, imperative shell" approach. He uses the same concepts in functional-first programming, but uses them inside the class, not at the program level.

• Requires training and discipline

I like languages that make it hard to do things wrong, according to the abstractions that language provides. Doing functional-first programming in an object-oriented language is doing it wrong, because it doesn't fit the abstractions that an OO language provides.

As it happens, a lot of languages don't actually care if you do OO or straight up procedural / imperative code. So it might not feel terribly wrong. It does take training to re-learn how to program in our favorite OO languages, and discipline to keep everything separate. We rely on the developer to do things well, because the language isn't going to help.

• May not fit well with frameworks (but probably will)

It's not just the language that provides abstractions we need to fit into our new workflow. Our frameworks in particular will greatly influence how we think about our code, and fitting the framework to the two principles of functional-first programming can be tricky.

The simplicity test for functional-first programming

Let's go back to the simplicity test, and see how functional-first programming fares.

Is it easier to reason about our code? Yes it is, because of our functions are referentially transparent, fairly pure, and composed; our state is exposed and even though it may not be immutable, we're careful not to mutate it; and we know exactly where to find all of our side effects, and they aren't mixed in with business logic.

Is it easier to test? Yes it is, because our functions are fairly pure and referentially transparent, and our IO and side effects are isolated. And since our side-effect code is pretty much all IO, there's no business logic to test and we can skip unit tests altogether and cover IO with integration tests that don't require mocking IO libraries.

Can we demonstrate correctness? We probably can demonstrate correctness, but I doubt we will. At least not typically. If formal methods of proof are important to our market, we're probably already doing them and can continue to do them with functional-first code. If formal methods aren't critical, then we're probably going to demonstrate correctness with really good tests and monitoring, and both are made easier by functional-first programming.

A month or so later, I was hanging out with Pat Maddox^[1], telling him about my plans for pair programming with a women's internship program I was creating at work. Pat suggested I look at mob programming instead, and I remembered I had already looked at mob programming and I loved the idea.

What is mob programming?

Pat Maddox didn't just think Mob Programming sounded cool, he'd gone to do it with Woody Zuill^[2] and his team of developers at Hunter Industries who worked out the techniques. Pat had a lot of good things to say about the experience, and his excitement was contagious.

You can read about Woody Zuill's experience with Mob Programming^[3] directly, but the short and mostly inaccurate version goes something like this. The team of developers (I heard there were eight) were discussing a project that had been shelved a few months. They kinda fell into a code review trying to remember where things were at, and that turned into a little bit of programming, and because it went well, they did it all afternoon. Then they worked together the next day too, even though they were competing for conference room space. Finally after a few weeks of programming together, Woody made a mob programming space with a couple of projectors and a single computer and keyboard, and the team made it official. This was how they wanted to work. Along the way, they mastered interpersonal communication, destroyed ego, and learned to write some of the cleanest, most correct code ever committed to a repository. You think I'm kidding. I'm not.

Woody Zuill's team is both the prototype and the archetype for what we've done, but I'm a programmer in the vein of Larry Wall's three virtues: laziness, impatience, and hubris. So I scanned the literature, talked to a few people, gave a nod or two to Woody on Twitter, and went off to forge our own path.

How we do it

Over the last year, I've done mob programming with dozens of teams, all skill levels, in more than a half dozen programming languages, on two continents. I can't say that there's one way we do it.

My team at $dayjob has a mob programming area. We have a 50-something-inch television with an AppleTV and an HDMI cable surrounded by comfy chairs and a couch. It's tucked into our work area, so our desks surround it too. We keep a whiteboard and bar-height table nearby too. My team has used the space to mob as many as seven hours a day, often attracting more people from other teams than we bring ourselves. We learned dom.js in a mob, taught each other monads in a mob, created a new product in a mob, and started doing property-based testing in a mob. Right now, my team is small - only five of us - and we're working on multiple projects. We are mobbing about two hours per week on each project. It's not much, but it keeps us all up to date on each of the projects we're responsible for.

Mobbing has caught on in our department. My team is one of three with their own mobbing zone, plus we've converted two conference rooms to mobbing-friendly workspaces. We have weekly testing workshops that use mob programming to help individual developers solve difficult testing questions. We use mob programming to learn new languages, frameworks, and libraries. We mob when new people join our teams, to bring them up to speed quickly. We share our experience with other teams in programming mobs, and invite people with skills we're still learning to our mobs.

Our CTO and VP-engineering strongly support the local tech community. With their support we created a full-time, paid, internship program specifically for women who are retraining and/or returning to the workforce. One of our goals is to take a boot camp graduate to a solid junior level developer in 16 weeks. We have four interns at a time, working with at least two more experienced developers, generally on a greenfield project, in a mob, five hours a day, five days a week. The interns' mob often includes domain experts and developers from other teams sharing their experience.

My primary direction for professional development right now is learning to build teams of functional programmers that productively include junior developers. Part of my study has been to create a workshop series called "An Ounce of Elixir", teaching functional programming (using Elixir) mostly to new developers. Every workshop is interactive and hands-on, and includes about 6 hours of mob programming. In each workshop, the students solve the same exercise: creating an event-sourced, CQRS shopping cart in a purely functional way. In each case so far, the workshop participants, programming in a mob, have succeeded in creating a purely functional shopping cart.

I've done ad-hoc mobs, short-term mobs, and long-term mobs, but I haven't done anything with the level of commitment and longevity that Woody Zuill's team has. They have done mob programming day in and out for years, and in spite of my hubris, there's so much to learn from them. At the same time, I'm confident our experience is useful, so don't stop reading yet.

Our mob programming guidelines, and where they came from

Our mobbing experience started with three months of thinking, discussing, and experimenting. That resulted in handful of guidelines for mob programming. With that, we started mobbing with regularity. Over time, we updated our mob programming guidelines. They're printed off and kept in our mobbing area.

These are our current guidelines:

1. Kindness, consideration, and respect are way better than having anyone in charge.

2. “Yes and” goes further than “no but”.

3. We speak at the lowest level of abstraction anyone in the mob needs in the moment.

4. Declarative language and experience sharing goes further than imperative language.

5. Thinking out loud helps everyone in the mob follow what you’re doing.

6. Drivers type code the mob proposes.

7. We learn differently when we’re the driver, so it’s important that everyone drives.

8. Rotations can happen as often as every five minutes.

9. Learning is contributing.

10. Turn up the good.

Let's look at where they came from, and what they do for us.

How we relate to each other

The first two guidelines are very much about how we see each other, and I'll jump right off a cliff and call it anarchy. Specifically, mutualism, a labor theory of anarchy, but all the same when we mob we are trying to create a non-coercive, egalitarian environment.

Kindness, consideration, and respect are better than having anyone in charge.

I've been fascinated by the way anarchy has been informing software engineering management for years. We have flat organizational charts with everyone reporting to the CTO or CEO. We have engineering-driven organizations, where engineers choose what they'll work on. We have different rules (i.e. none) for engineers' dress code and work spaces. We have game spaces, nap spaces, and a do-whatever-you-want attitude. We have flex-time, unlimited PTO, even unlimited maternity and paternity leave at some places. If we put the team on a farm in Tennessee, we'd have no problem seeing this as anarchy.

Interestingly, our mobbing anarchy also build empathy. When we work together this closely, we have to see each other, like really see where we're at and how we're doing. We can't show kindness and respect without caring. Our teams that mob know each other better, listen better, and communicate better than they did before we started mobbing.

“Yes and” goes further than “no but”.

This guideline comes straight from improv comedy training. My teams do improv comedy training.

Improv is hard work. Improv teams build on each other, take a wacky concept from one or two people and build it into a story that delights their audience and themselves. It's very much the same with mob programming. When someone says "No", especially if they are an authority figure outside of the mob, progress stops. We have to overcome inertia and find a different way to make progress. "Yes, and ..." preserves momentum - it takes the position we're in and contributes not only a new milestone but also how to get there.

How we communicate with each other

These three guidelines help us communicate well.

We speak at the lowest level of abstraction anyone in the mob needs in the moment.

When the interns mob, they often talk about things like which keys to press in Vim, how to structure a Git command, and how to call a particular function.

When our seniors mob, they often talk about things like control flow, patterns, testing strategies, and architecture. Except when there are interns or juniors working too; then, they also talk about Vim key combinations, git commands, and syntax. This is because everyone in the mob needs to have the choice to understand what's going on, to ask questions and get answers they are capable of understanding based on their level of experience.

Declarative language and experience sharing goes further than imperative language.

I've had the opportunity to learn a little bit about child development, specifically how we adults and parents interact with children. One interesting fact is that more than 80% of our verbal communication with children is imperative: "Do this", "don't do that". Yet when we communicate with other adults, almost 80% of our verbal communication is declarative, experience sharing. "I wonder about that", "this thing happened to me".

Can you imagine what our lives would be like if we went around talking to other adults like we do to children? You probably can. I know people like that. And weirdly, when we get in a mob, our imperative communication can quickly escalate. It's almost as if having other people between us and our keyboard turns us into control freaks. "Don't do it that way. Try this. No, type it like I say."

In our mobs we try very hard to not communicate imperatively. Instead, we try to make declarations about our shared experience. "That function looks funny." "I wonder if that test catches all our edge cases." "I don't know how this library works." "There is a typo in line 47." "I just learned something new."

Thinking out loud helps everyone in the mob follow what you’re doing.

Thinking out loud takes our experience sharing and levels it up.

Sharing leads to understanding - we can't understand what we know nothing about. Understanding leads to learning, not just skills but learning about and from other people. Learning leads to empathy. In fact, I think it's very difficult for us to learn with or from other people without feeling more empathetic. Empathy leads to caring, and caring - about other people, about our customers and users, about our work, about ourselves - leads to fulfillment.

Yes, I got all touchy-feely, but this is what happens in a mob. We talk a lot. We think out loud. People understand what we're doing and why. We get involved. We share and listen. We trust each other. We care.

A quiet mob is a red flag. Thinking out loud has amazing benefits, and it as entirely reasonable to encourage constant conversation. This is different from our typical software engineering experience, and it takes practice, but the results are invaluable and at least a little bit beautiful.

Roles

While we've had mobs that choose to define roles, we really only have two roles in our guidelines: the driver - that is, the person on the keyboard - and the mob.

Drivers type code the mob proposes.

Many mobbers have a hard time thinking and talking at the same time. Ask us to think, talk, and type and we fall into a well-established work habit. We stop talking. We go heads down and code, by ourselves. This is definitely not what we want to do in a mob.

In our mobs, we discourage the driver from writing any code. The driver's job is to be the hands and fingers of the mob. This keeps the driver from going solo and leaving the rest of the mob behind while he or she is controlling the keyboard. It keeps the driver engaged with the mob, interacting. It makes the mob into one body, connected and working together.

We learn differently when we’re the driver, so it’s important that everyone drives.

Perhaps the most important reason the driver just drives is that many of us learn with our fingers as much as our heads.

I can remember most of the phone numbers that were important to me as a child. My home phone number during elementary school. My home phone number during junior high. My grandparents' phone number. But, in order to recite the number, I usually have dial it on an imaginary phone.

Many of us learn kinetically. We store data in our fingertips. We store syntax in our muscle memory. Our mobs help honor these other ways we learn things, and by switching drivers frequently, we reinforce what we're learning in the mob, learning faster, and improving our retention, because we're learning it in different ways in quick procession.^[4]

Rotations can happen as often as every five minutes.

We did mobs where we had one driver for hours on end. We did mobs where we barely bothered to sit down, we changed drivers so often. Interestingly, stability of roles in the mob was not necessarily a good thing.

Changing drivers frequently seems to keep everyone engaged - perhaps by keeping them invested in what comes next, since we will be driving the keyboard shortly.

In practice, 20 to 30 minutes seemed to be the ideal rotation for most mobbing sessions. There were mobs where we felt confident we didn't want to change drivers, and afterward decided we were wrong. There were mobs where we changed drivers more frequently and felt moderately successful.

Creating developers, teams, and code

Learning is contributing.

Almost every mob I've worked with has had juniors, in large part because I think juniors are tremendously valuable to any team. And almost every mob I've worked with has had juniors sit idle, refusing to participate, because they don't want to slow down the people who really know what they're doing.

It's possible that "Learning is contributing" is the most important guideline in the list. It doesn't just help the person learning while slowing every one else down. It sets a pace for the mob that best addresses all of our goals.

• Stopping for an explanation forces someone who thinks they know what they're doing to explain it. Often just explaining it will expose many problems.
• If one person doesn't know what's going on, chances are good other people don't know what's going on. I am usually the most senior developer in our mobs, an I often don't know what's going on.
• Mobs are great for training. Juniors aren't just learning the language, they're learning to deliver value. Mobs are an excellent place for seniors to teach juniors how to deliver better value.
• Mobs train seniors too. Seniors learn to communicate and collaborate better in a mob.

Retrospectives

Turn up the good.

Some of our guidelines came directly from Woody Zuill's experience report, and we've stuck with them from the start. This is definitely one. I love it and I want to use it everywhere in my life.

If something is good, do more of it.

Over the last year, we did retrospectives after almost every mob. When we first started experimenting with mob programming, we used these four questions:

• What went well?
• What went poorly?
• What would you like to try differently?
• What still has you puzzled?

In my functional programming workshops, we use these questions:

• How do you feel about your code?
• How do you feel about your team / mob?
• How do you feel about yourself?

Our interns started with these questions:

• What should we do more of?
• What should we do differently?
• What baffled you?

Over time, they got bored with those questions and chose these instead:

• What did you like that happened in the mob?
• What did you learn today?
• What did we accomplish today?
• Do you understand?
• What didn't go well?
• What should be done differently next time?

After trying the new questions a while, they made one more change to their retrospective prompts, and ended with these questions:

• What should we keep doing?
• What should we stop doing?
• What should we start doing?
• What surprised us?

Personally, I like the touchy-feely questions because they get to all of the reasons we mob, on a qualitative and analytic level. How do you feel about your code, your team, and yourself?

The interns got very pragmatic when they rewrote their retrospective questions. I like that too, especially "Do you understand?" It's very direct, and it got better conversation than "What baffled you?"

The questions are just prompts, though. We talked about mobbing every day we did it, and that was important. Learning to mob was a mob activity.

Up till this point, we've talked about how we mobbed. From here on, we talk mostly about how it went, and that information comes primary from our retrospectives.

Everything we did wrong

Team retrospectives

"I started off with something I got stuck on. It didn't start smoothly. If we didn't get stuck on this we could have finished something else."

"In the past I've felt like I had to justify dogpiling on a problem."

Of all the times we mobbed, I can count on one hand the number of times someone outside of the mob tried to compare the productivity of a mob to the productivity of the individual members working on their own. And yet, inside the mob, this was an almost constant concern. We were afraid of other people judging us for ganging up on our work. We were afraid of slowing each other down. We felt guilty when we got stuck, and were "wasting" the time of everyone in the mob.

"I was feeling bad that I got stuck on this and asked other people to help."

We also felt guilty for what we don't know, for not being better programmers, for not having more skills, for needing to learn. One of our biggest challenges was getting people to participate when they didn't know what was going on, because participating meant disrupting the flow of the mob. When we let that happen - when we let the mob continue, even though some portion of the mobbers were lost - what we saw in the retrospectives was that no one felt like it was a good session. The code was poor quality. The practice was useless. The learning was shallow. The energy was lethargic. Code, team, and self all suffered.

"We got stuck on these little things. If we hadn't we could have been cruising along."

This is why we implemented the "Learning is contributing" guideline, and ultimately it's why I think that it is the most important guideline. Regardless of what metrics we use to gauge the benefits of mobbing, learning will come into it. We can't budge the needle on any metric unless the mob is a learning experience.

"Because this was a hack day, I feel like we needed to get more done. If we'd had more time, I would have stopped more to check if everyone know what was going on."

"I think that went well, we did get a lot done. We got a good amount accomplished, we just didn't do a lot of learning and training so that next time we meet everyone could participate better."

"I was going to drive for a while, but I didn't speak up. I feel like I learn more from driving more."

"It was a lot of learning for me because I don't know much about functional programming."
"Did you feel excluded from the mob because you didn't know what was going on?"
"Somewhat."

"Especially for those who don't know the language well, it's easier for them to feel excluded and shut down and even hurt or upset because they don't feel competent already, like 'I don't know the language and now I can't contribute in other ways either.' I think in future mobs we need to encourage them more, to make their competence level go up, and explain why we think this or that idea may not work."

That learning experience is in no way limited to the more junior members, either. I'm the most senior developer on my team, and I learn from my team every day. I often want to hide that fact, but that's hard to do in a mob. We learn from each other constantly, and we learn from each other most efficiently when we do it out in the open.

"We did really high level, difficult Javascript today, stuff I wasn't prepared to understand. There were a lot of times today I was scrambling to keep up, but the team kept moving because the people who were contributing most were able to keep up at the higher level."

"It wasn't the implementation that caused conflict, it was the high level talk."

"I felt there was a lot of pressure on me. I was the only language expert, and the only one who knew the code."

"If we do go the approach of 'just say no, just shut them down' instead of more comforting things, it might be good to rotate people afterwards to help people feel better."

It seems counter-intuitive to suggest that a bunch of highly-skilled developers need to learn how to talk about code, but when we're trying to do it for a half dozen hours straight, for days on end, we really benefitted from practice and reflection. Learning how to discuss code at this level was a new skill for our seniors, and for me. Learning how to do this with interns and juniors was a new skill for everyone.

This is just code. This isn't about empathizing, sharing feelings, being vulnerable.

"I wasn't happy with the choice to use Node, but I feel like it was a circumstance of the mob. It was a hack day. Not everyone knew the languages I wanted to use. It would have been better to use Scala, or Haskell, or Erlang, but there was the issue of bringing people up to speed and that conflicts with needing to get something done."
"I was unhappy we didn't use Java 8. And I got a preemptive shut down before I even brought it up."
"Can we vote then?"
"I'm not the only person who is going to come to the mob with opinion."

"I think it's different for you because you're in a position of authority. If someone else had said that, we could have just said no way, but when you say it, it's different."
"I want to say that no one in the mob has more authority than anyone else, even if I try to make it look that way."
"I think you can say that, but there's still going to be that underlying social factor because you're responsible for hirings, firings, raises, etc."

In the last year, I have mobbed with at least 100 different people. Want to know who was the worst person at mobbing?

Me. More often than anyone else, I was the problem.

Over time, I'd like to think our teams have learned to trust my commitment to an egalitarian mob. Part of that trust has come from me refraining from stating opinions, and instead asking questions and listening to other people in the mob. But you know what? That's not as much fun as expressing opinions.

I have work to do, most of it on myself. This is definitely one of the side effects of mobbing.

Workshop retrospectives

"I saw some people weren't as involved as others."

"I had a harder time following along. They were doing stuff but I wasn't involved because I didn't understand what they were doing."

"I was struggling with the syntax and things of this language so I felt like I couldn't follow along. It's like they were speaking Chinese or something."

"Just because I didn't speak up often, doesn't mean I wasn't engaged. I was just trying to keep up but not in a bad way."

We've seen this at each of the workshops. Some people pick up Elixir quickly and run with it, and other people need extra time to catch on to the language structure. We plan on this dynamic and work with it.

We spend about three hours total with the language before breaking into mobs. We spend 90 minutes in an interactive Elixir shell (the Elixir REPL), and another 90 minutes on test driven development in Elixir. This is not enough time for even experienced developers to learn a new language, and these workshops are designed for people who aren't experienced developers. We cannot expect any of them to enter a mob knowing what they're doing.

While most participants are new developers, I do accept a handful of more experienced developers at the workshop. Every time we've scheduled one, we hear from people with more experience who really, really want to learn Elixir. I usually say yes, but insist they understand that the workshop is for new developers, and that the discussion will be tailored to the experience level of the group. Developers with more experience may be welcome, but in supporting roles only.

This is kind of neat, actually. We get experienced coaches for the new developers who are also sharing the experience of learning a new language. Once we break into mobs, the mobs are on their own. It's not an instructional environment, it's a production environment. They have a project to produce - a real one, something they could take to market with more work - in less than six hours.

"In our group, it was really good when people would explain how they understood things. Not so good when people didn't understand and we were all quiet and we all stared at the screen."

"I think it was kinda frustrating to slow down quite so much, but after I got over that it made sense and we made a lot of progress after that."

I love these two comments. They expose both the difficulty and the solution succinctly. We need to share our experience with each other, even though we're all in the same space and we think we're having the same experience. We aren't, at least we aren't until we share that experience by thinking out loud constantly.

"I had a hard time expressing myself declaratively. I wanted to come in and say 'That function name doesn't really speak to me.' 'I don't like that function name.'"

"I cheated a lot at declarative vs imperative communication. Instead of 'Do this.' I said 'I think you should do this.''"

In the workshop, we discuss declarative and imperative languages, both in programming languages and in our spoken language. This is a neat intersection for our students. Most have a little experience with imperative and object oriented programming, but almost no experience with declarative and functional programming. But since every workshop so far has been hosted by either Girl Develop It or Women Who Code, the groups tend to have a lot of experience with parenting and they relate well to imperative and declarative concepts as they relate to communicating with each other. We get it.

"Because none of us know the language most of us were on doc pages."

"Two computers with two big monitors? One with code and one with docs and help?"

"It'd be cool to try with only one computer working and everyone working off that one."

"I wonder if it's okay for one or two developers to follow their own path, and try things out."

"In a group that big, there are a lot of different ideas. It's hard to tell how long and how far we should go down each path."

"There were too many cooks in the kitchen."

"I've done some mob programming before and I think a little smaller would have been good. Better focus, fewer side conversations."

In the workshop retrospectives, we always get an opportunity to visit the mobbing experience, even if we aren't going to iterate on it together. I'm happy to have the feedback, and some of it has informed later workshops.

"I've never mobbed before. The group seems big to me."

"I don't think mobbing is for me. I like to work by myself."

"I want a beer."

Most of the comments tend toward ways they'd like to try it more, but a small percentage of participants just want to say no - they gave it a try, and that was enough for them.

Intern retrospectives

The retrospectives I run tend to be very informational. Also, I like quoting people, or at least paraphrasing them and making sure they approve of my wording. The intern team (four interns, a junior, and an intermediate developer) ran their own retrospectives without interference from me. I love how practical they are.

What didn't go well?

• Not knowing how to rotate sharing the computer, versus pushing and pulling
• Sometimes we don't want to touch someone else's computer

What would you like to try differently?

• What if we get really good at pushing and pulling?

This was the first day that the intern team mobbed on their own, and they immediately questioned sharing one computer - and worked out an alternative. "What if we get really good at Git?" I love their solution.

What didn't go well?

• Some concern about how much time it takes to push and pull
• Github pulling wasn't as smooth as we'd like.
• Remember to git pull before making any changes.
• Still too much time in transition, so 15 minute turns seemed a bit slower.
• We were lost sheep but not totally lost.

What would you like to try differently?

• Define roles (navigator, facilitator, driver, bug squasher)
• No one else has a computer open
• One computer for the driver and one for research
• Essentially, don't work individually
• Trade more frequently
• Set an alarm for turn-taking
• Maybe 20 minutes each

What didn't go well?

• Turns and roles of cards didn't go well today.
• Total failure of Facilitator's role :(
• Sometimes it's still hard to believe that your questions are valuable
• Group morale is low today

What could be done better next time?

• Remember to put on timer, facilitator should take notes
• Try a cool new ruby timer for mob turn-taking: https://github.com/RubySteps/mob_rotation
• We should have bio break more frequently.

The intern team chose to have more defined expectations of themselves and each other. I haven't seen this in any other mob, but the interns are the only team that mobbed every day all day, and by the end of their internship they had done that for twelve weeks. Their experience with defined roles (they even used index cards to indicate who had what role at any given time) had ups and downs, but they stuck with it most days until the end of the internship.

What still has you baffled?

• Why it didn't work for the first time???

What went well?

• Everything works

What could be done better next time?

• Nothing, today was fabulous

These were from the intern team's last day of mobbing. Their project shipped and has been a huge success with the users.

Everything we did right

We've done mob programming in three different contexts, and the goals for each context - and for mob programming with each group - have been different.

My team

When we've mobbed as a team at work, we are a group of mostly senior developers, often including developers from other teams (mobbing looks like a lot of fun, so other people want to do it with us) and domain experts we invite to join us. In this context, my primary goals are three-fold.

• Team development. I want to see the team improve, and I want to see the individual developers on my team improve. I put this goal first in the list because it's actually my primary goal. I'm a purist and a perfectionist, and I want every member of my team to stand out among the best developers in the state. And we do.
• Value. Developers that write simple, correct code can change the world, but only by shipping it. We don't just ship code, though. We want to ship value for the user, for the customer, for the company, and for our team.
• Code quality. I believe a good developer writes code that is simple and correct. That's all. Easy to reason about, and demonstrably correct in what it does. My goal as a manager is to make developers that write simple, correct code. Our goal for mobbing is to do this even more efficiently, while still shipping value for our stakeholders and customers.

Team development

"It's better for us to work as a team sometimes, even if we don't otherwise work on the same project. It's good for the team."

"The mob experience I had before was very different. This one we did a lot of 'yes' to things. The other one was like 'No we're not going to do it that way, no we're not going to do it that way.'"

"I noticed a few times that the driver would make a statement along the lines of 'This is my opinion but feel free to disagree.' I feel like that should be an implicit understanding, that every statement can and should be challenged."

"I think my favorite thing was that everyone made an effort to make sure we were all on the same page."

"I learned some Scala. I got an idea of the project. I wouldn't have been involved in the project or in Scala except for this mob. Otherwise I wouldn't have learned anything on this."

"I didn't know anything about React when I sat down. I learned a lot more about React in 2.5 hours of mob programming than I would have in a day on my own."

"I learned a lot from this process. About libraries I'd never seen before. Some testing concepts. Even just refactoring the tests, to use a library that we weren't sure about. We weren't sure how it worked, so we refactored to understand the library better. Today we had a limited time frame, and the fact that we took the time to refactor is a definite positive."

"I didn't think I was going to be able to contribute. But I feel like I did, and I learned stuff too."

Team cohesion while mobbing was really impressive to me. I've been on teams that don't need or want to collaborate, teams that turn their collective nose at code reviews, let alone pairing. Mobbing helped us help each other. We didn't just work together, we worked on ourselves and each other. We acknowledged each other's strengths and learned from them. We validated each other's weaknesses and helped strengthen them.

A lot of that cohesion has continued, but one thing I try to do is move my people to other teams as soon as they want. Every developer that moves from my team to another brings skills for code quality - pure functional programming, functional first development, property based testing, etc. - that team probably doesn't have. Likewise, they bring team building skills, communication skills, and collaboration skills that we've learned, in large part, through mobbing.

Value

"The ability for others to look ahead while she's working on a problem, so she can keep her focus."
"With that, different people can pursue different avenues looking forward."
"Faster for me to resolve issues just because people are looking at different things, comparing stuff, instead of just me going back and forth. I might miss something."

"Our driver got called away to a meeting and we continued without missing a beat."

"It was pretty cool that some of the boilerplate we had to do, Nhat could just rattle off the top of his head, where it would have taken me a half hour to figure it out."

"My worst times to code are between 2 and 4:30 in the afternoon. I think it went better in a mob. I think it might be more productive than working individually in that slow period."

"As the driver, more of my mental capacity was used keeping up with all the suggestions rather than trying to think of a solution of my own."
"It was easier for me because I was the one who was trying things, rather than researching and thinking about what'll work."

"I think once we figured out what our idea was and what we were heading for, we all pulled together and worked on the solution. That's what was neat, seeing us all work together."
"We kicked butt. People loved our ideas and they were very complimentary of our work."

"We accomplished more in a short time. We built a project without a bubnch of complexity. Because we worked together we built something simpler than we would have had we worked separately."
"I second that, that's what I would have said."

"I think mob programming means we end up with less technical debt."
"I would agree."
"I agree, and I think it goes along with what was said about the complexity because I kept getting pulled back a bunch of times from what I would have done, and I think what we ended up doing was a lot simpler."
"I agree I think we end up with simpler code, so it's possible. It'd be nice to follow some projects over a longer time to be sure."

Too often, when we talk about value, we are talking about short term value: productivity of the developer and the team in terms of story points delivered. I can see the merit there, but I think that story points are a short-sighted way to measure value.

To me, the average developer is one who can deliver the story quickly. The exceptional developer makes better decisions along the way, decisions that deliver more value in the long term. The exceptional developer delivers the story, but with less code. She writes code that is more secure. She uses fewer dependencies, and has less complexity in her code as a result. We can read and reason about her code, and so can the developers who will maintain it in the future. She uses clear abstractions and they fit the whole problem, not just the story she worked on, so her code can be extended without being rewritten. She doesn't deliver technical debt. She demonstrates that her code is correct. Her code scales in production.

If we have even one exceptional programmer in our mob, then the mob will deliver better long term value. Once they've done that, each member of the mob is also more likely to deliver better long term value in the future too.

Code quality

"I personally think it went better than what I would've gone through by myself"

"It was easier to find bugs. Saved time on that part. Plus multiple people on the same problem. Especially while I'm learning."

"Different people gave me different perspectives for looking at the problem. Fresh perspectives than what I would have thought of."

"We did a really good job of bouncing off each other's ideas at the beginning of the day to come up with our solutions. I don't think it was any one person's ideas that got built."

"This is kinda a plus and a minus. A lot of feedback off comments and what everyone is thinking. Good because I could make multiple hops off of what people were saying. Bad because we were queuing up things to do."
"I like that everyone could feed off other people's ideas. I think things queuing up was a good thing because working individually, it takes more time to think of all those ideas."
"But for what to do next, how to choose? Just in the order people recommend them?"
"I think the problem of which thing to try next is a good problem to have. Having a full queue is better than an empty queue."

"I probably never would have dug into the source code for Jest or jsdom if it had been just me."

"It's nice that I'm not solely responsible for any bad code. There's shared responsibility for good and bad code."
"I think that the result product will be less bad because there's got to be some better coders on the team. Teams with mixed levels of experience will turn out better code. And better coders."

From my perspective, the code we wrote in mobs was dramatically better quality than the code we write individually. First off, we were more strict about process. No one was willing to say, "Hey, let's call this a spike and not write any tests." We were much more likely to use test driven development to explore the problem and create the spike.

Our mobs frequently include stake holders and domain experts. We get closer to what is needed on our first try. But sometimes it's embarrassingly easy to accept a spike that works, even if the code is ugly, especially if we have someone with the next priority waiting for our attention. Sometimes is scarily easy to ship code that delivers on a short term value, without regard for long term value.

This is the primary danger of Agile software development, in my opinion. We ship value, sure. But as soon as the short term need is met, that story no longer has a priority, so we move on a story that does. Yet each story only deals with short term value, and not the long term needs of the code base. Refactoring, technical debt, tests, code quality in general - these are all underrepresented by the system, meaning that we achieve them only through discipline and training. Often, that discipline involves going against the wishes of our product people and stake holders.

Mob programming is a system for turning out quality code, before we even put it in the hands of the product owner. Where any one of us might say "this will do the job", the mob is more likely to support another test, some more refactoring, removing dead code. The mob is more likely to ship code that is simple and correct, even when product is anxious. That means the mob is less likely to ship defects, security flaws, and technical debt.

We all want to ship value, early and often. The quality code our mobs ship represent long term value: code that works now, and has significantly lower maintenance costs in the future.

Functional programming workshops for new developers

"We made a shopping cart! OMG!"

Some of our mobbing happened at my Ounce of Elixir workshops, with brand new developers learning functional programming and the Elixir programming language in a two day workshop. They mob most of the second day (6 hours or so) on a real problem, building a shopping cart service using event stores and command query responsibility segregation.

Our mobbing goals with workshop participants are very different from mobbing with my team. First and foremost, I want the students to feel successful. Two days isn't a lot of time for learning facts about a programming language, but it is enough time to feel empowered.

"I think it's amazing that even though everyone there knows very little we could still figure something out."

"When we started I thought we weren't going to get anywhere. We got a lot farther than I expected. I think we were all surprised every time something works."

"I feel like I learned faster in a mob."

"Mob programming is working well. There's a good constant dialogue that feels productive."

"It's kinda less hacky. You have other people looking at you code, so you become more aware and think things through."

"I don't think it'd be possible to do any of that on my own. Although it was cool to do, I'm not sure on my own I'd be able to do all that much."

"I think it was good to watch your code evolve as well. You start doing it one way and then it goes down another way."

"It's a group effort, everybody is influence the code."

"I felt like my purpose wasn't to build something but to help everyone else build something and to be able to go home and do it more."

"I was amazed that Alicia told me at the end 'I've never programmed before' and I didn't believe her. Everyone's just contributing, and even if you don't know the techical aspects of an idea you can still kinda get there."

"It's great to learn things and do some examples, but seeing how you can use it is really good."

"I don't think I would have been able to end up with the code we have after the same amount of time on my own."

"I'm a newby and I always think 'I don't understand much, I don't know much.' But I kept up with it. And it's more comforting to me, that I want to keep going on this."

"I understand functional programming much better now."

"I feel very energized. I feel like I want to go home and do programming."

We have other goals for mobbing, but they are distant seconds. Participants are learning development skills, not just the language. Retention is worth considering, but as yet we haven't measured it. Networking with other new developers can be beneficial. But mostly, we want students to feel like they succeeded.

"Our code is clean."

"We weren't stumbling over the language. We didn't spend much time in docs. We didn't spend much time figuring things out. When we needed documentation, it was easy to find the relevant stuff really quickly."

"I'd feel comfortable experimenting in the shell and expanding on what I learned today. I learned more today than I could have on my own in the same amount of time."

"I picked up five or six patterns by saying, 'How do you do this?' and somebody telling me how to do it. I very much appreciate being able to pick up on the patterns right in the moment in the problem I'm experiencing right now."

"I feel confident that I can learn to program. It's not completely far away, out of my reach. It's something I've wanted to learn for a long time."

Of course, one of our goals was to learn how to do mobbing well, and the functional programming workshops gave us great opportunities to see how mob programming works. It's a difficult environment. We have mostly inexperienced programmers, working in a language they don't know, on a tight time frame, solving a reasonably difficult program in a completely unfamiliar way. Amazingly, every mob has created a working shopping cart, plus we learned about how the students felt mobbing helped.

"The mob was far less stressful than I thought it would be."
"It was far less intimidating than I thought it would be. You had everyone working, everyone looking for something to work so you never really gave up. The environment was motivating."

"I think it was quite exciting as well to be collaborating to something that looked quite complicated, and that everyone contributed part of their knowledge, it was really quite amazing."

"I did a coding workshop earlier this week, and after this session I think it made everything a lot less daunting. Collaborating this way is brilliant for new programmers."

"It was quite nice that roles weren't assigned, that we just sat down and started talking and swapped roles constantly. It kept it interesting."

"It was also more productive as well. At one time we weren't working on just one problem. People were searching for other things as well at the same time."

"It felt more productive than on your own. Because you're sort of constantly going, it felt like you got more done."

"I think you don't get bored quite so easily. Normally when I work alone I get bored quite easily, but in the mob you keep more focused."

"I felt like I learned a lot more when I was driving, even when I was just like being a typewriter. I think we should have done shorter terms driving and everyone drive more often."

"I think it's a good way to learn in general. Maybe not as productive as eight people who have more experience on their own. But if you don't have eight people who know what they're doing, it's going to be way better for them. And even if you're more experienced, it's gonna keep you more focused."

"It feels a lot more sustainable and engaging. I'm interacting with people the whole time and not just staring at my screen. Not as frustrating."

"Just to reiterate, maybe you gotta drive to learn."

The intern team

Our goals for mobbing with the intern team were kind of a mix. We wanted the interns to come away feeling successful, and we also wanted them to ship quality code that delivered excellent value for our very real customers.

The intern team did so much right it's hard to know what to include. Here's a sample of the things they felt were particularly right over three months of mobbing, roughly in chronological order.

What went well, what you want to do more of?

• We figured out bugs together.
• We accomplished something - we actually wrote some code and solved some problems.
• We were focused on the same thing together.
• We found more different resources to help problem solve.
• It's easier to feel comfortable with your own knowledge level.
• Figuring out problems pretty easily now.
• We all took turns more.
• We noticed the difference between driving and navigating.
• Pushing and pulling from github works well.
• Retrospective are useful to get at deeper issues.
• Using a timer worked really well, when we remembered to use it.
• It's valuable to see that I'm not the only one who gets stuck on a problem. It's valuable to see how others solve problems.
• Turn-taking with the cards was good, especially separating the task of researching errors.
• We remembered the timer.
• We're getting really good at trouble-shooting.
• Watching how other people do things is very helpful.
• Jira training was good.
• We figured out the installed java location.
• Tweaking bash! :)
• We survived without mentors.
• We did turns and we stuck to 15 min turns.
• Taking time to make sure everyone understands.
• We had many issues but team collaboration went well.
• More flexible with the turn taking.
• We noticed taking time for thinking makes the turns different.
• Trying lots of things is good (at least knowing what didn't work).
• Doing the tutorial first helped us gain knowledge of what we were going to do with pouchdb.
• Getting a fast answer from the library author online.
• Refactoring and resolving issue with refactoring.
• We learned a lot about UI, the importance of UX in production, and the importance of ADA compliance.
• Variety of turn taking strategies.
• Julia's code review (she ask us to tell how our code works).
• Changed everything to ES6 today.
• Having everyone else there to help when you get stuck.
• You don't feel so stupid asking questions when you're stuck because everyone is at the same level as you.
• We stopped and talked about communication in the mob.
• Everything exploded!!!!!!
• Thinking out loud was particularly good today.
• Lots of conversation. Off-topic conversation can make it easier to discuss work topics.
• Working with Michael. Having Michael explain high-level concepts.
• Everything works.

Qualitative before quantitative

In the last year, we've learned a lot about mobbing. Every mob has tried different ideas, and reported back on them. Every mob has had people who left excited and wanting more. Some mobs had a few people who left with doubts. And while we have a lot of data about how people felt about their mobbing experience, we've done almost nothing to measure the code they produced.

This was on purpose. Going into mobbing, I didn't know what claims to try to prove or disprove, because I wasn't sure what mobbing would do for us. I just felt confident we would learn something interesting. And clearly we did.

Based on our experience, I can now suggest some testable hypotheses. I think these claims can probably be demonstrated, given an appropriately defined mob and good measurement criteria.

• Code written during mob programming has lower maintenance costs.
• Mob programming produces code with fewer defects.
• Mob programming produces code that is more secure.
• Code written during mob programming is more readable.
• Mob programming meets the functional requirements with less code.
• Junior developers progress to seniors more quickly when programming in a mob.
• Development teams that consistently program in a mob have better understanding of their code base.
• Development teams that consistently program in a mob have more uniform skills.

If you try mob programming, I strongly suggest you start with a qualitative approach as well. The interpersonal skills we learned are significant, and I doubt any attempt to prove the merits of mobbing quantitatively could be successful with an inexperienced mob. Once your team has the experience, I strongly recommend you find metrics and demonstrate improvement. That's what I'll be doing too.

1. First, code everything you can without side effects.
2. Then, code your side effects.

Let's say that side effects are any way that a computer program changes the world, or is changed by the world. This includes inputs and outputs, so really anything that gets data into or out of our program is a side effect. Reading from standard input, or a database, or HTTPS. Writing to standard output, saving to disk, sending to a service, pushing to a mobile client. All of these are side effects, and we can wait to code them.

There's another kind of side effect - saving state in variables. This part is sometimes trickey for programmers who aren't used to functional programming, but it's actually not too hard to do with functional first development. We're still going to have state, but during step one we'll only have state in our tests, and that state won't ever be mutated.

First, code everything you can without side effects:

So before we worry about any side effects, we worry about our program's core logic. Sometimes, it's quite a trick seeing what that logic is without side effects, because we think about our work in terms of how we change the world. But we can find that logic often by looking at the actions our program takes rather than the objects it acts on. We can look at verbs rather than nouns.

Let's use a shopping cart for an ecommerce site as an example. If we look at it from an object or noun-oriented point of view, we might assume that the cart keeps track of it's own state, and it has methods for manipulating that state. So we can tell the cart that we're adding or removing an item and the cart object does all the magic internally ("encapsulated state").

We can also see the core program logic as taking an existing shopping cart, and a shopper's action (our verb), and returning a new shopping cart. This a straight up transformation. We are taking one state and transforming it to a new state.

New state = Old state, transformed by an action

Maybe this looks like we're mutating state, because we have an old state that becomes a new state based on an action, right? I like to think of the old state and new state as distinct. The shopping cart has different states over time. The old state describes the shopping cart at one point in time (before the action), and the new state describes the cart at a later point in time (after the action.) I like to make a data structure for the cart that lets me keep all the versions of the cart, all of the states over time.

Let's try that. Let's say that our shopping cart state is simply a list of events happening to the shopping cart.

[ {add, Eggs}, {add, Milk}, {add, Candy}, {remove, Candy}, {add, Carrots} ]

I've simplified things a bit. There are no prices or quantities, and we could keep track of things like who put the item in the cart, and when exactly that happened. But for now let's just assume that Eggs got added before Milk, which was added before Candy and Carrots.

Now our program's core logic is pretty straight forward. All we need to do is take a list of actions, initially an empty list of actions, and be able to add new actions onto the end of that list. Since we probably also need to display what's in the cart at any given moment, we'll also need to transform that list of actions into a representation of the current state of the cart.

Our program doesn't need to make new shopping carts at this point. That would be a side effect. It doesn't need to save the shopping cart. It doesn't need to connect to an HTTPS port. It just needs to be able to do two things: given a shopping cart, record a new event; and given a cart, reduce the list of events into a list of what the cart contains.

The first is really easy to do, in almost every language. Here it is in Python:

cart.py

class Cart(object):
  def add(cart, item):
    cart.append(('add', item))
    return cart

cart_tests.py

from cart import Cart
import unittest
from expecter import expect

class TestCart(object):
  def test_add(self):
    expect(Cart.add([], {'name': 'item'})) == [('add', {'name': 'item'})]

Because this is Python, we have some side effects that are introduced in fairly hidden ways. For example, when we create a variable cart in the add function, we're actually creating a mutable list. On the next line, cart.append mutates that list, because Python doesn't give us an immutable list. Also, Python passes object references by value, which means it's easy to make functions that change variables that are passed as arguments if we're not careful.

But even in Python, we can write our code as if it is without side effects. We can do that, because that's how we're going to use it. Our test demonstrates this. Our test starts with an empty list, not a variable. We add an item. We expect a new list. And even though Python might pervert some memory along the way, the parts we are allowing ourselves to care about are not mutated.

Here's the same function and test in Elixir, this time without inadvertent side effects:

cart.ex

defmodule Cart do
  def add(cart, item) do
    cart ++ [{:add, item}]
  end
end

cart_test.exs

ExUnit.start
defmodule CartTest do
  use ExUnit.Case
  test "adding to cart" do
    assert [{:add, 'item'}] == Cart.add [], 'item'
  end
end

Regardless of the language, writing our code without side-effects makes our tests super easy. Our functions are simple. They always return the same output for any given input. They aren't hiding state. They don't rely on anything but the information they are provided with when they're called. When we test functions written this way, our tests are simple, fast, and easy to understand.

Writing our code without side effects, focusing on the actions and using functions, has real impact on complexity. With objects we tend to absorb and hide complexity. As our code becomes more complex, we change how we represent state inside the object. State is encapsulated - that's the point of objects - and hidden, so as objects gain complexity, that complexity is also hidden. When we write functions that transform state, we can't hide the complexity. The more complex our state becomes, the more likely we'll expose that complexity in the interfaces to our functions. We can't run and we can't hide, but we can simplify.

Here's another benefit from coding the core logic without side effects. There's an excellent chance that we don't need a framework in order to do this part. No Ruby on Rails, no Django, no Express, no Play. We can probably code this part of our program in the plain old language, possibly with (at most) a handful of third party libraries that have limited, well-defined purpose in our code. This also helps keep our core logic code simple.

That's pretty much our goal with functional first development. We want code that is simple and correct. Coding without side effects as much as possible means that the core code is simpler, and because it's much easier to reason about and test, there's a good chance it will be more correct.

Then, code your side effects:

"In the end, any program must manipulate state. A program that has no side effects whatsoever is a kind of black box. All you can tell is that the box gets hotter." --Simon Peyton-Jones

After we've coded everything we can without side effects, it's time to add in our side effects.

If we've done the first step right, the only code we have left to write is code that changes the world. Code that saves data to disk, or to a database. Code that sends data to a web service, or a message broker. Code that receives messages from a message broker, or on an HTTPS port. Code that goes out and gets data we want to transform. Code that returns data we transformed to someone, somewhere else.

This works to our advantage. Almost all code that changes the world comes in a framework or in a library. These libraries are often hard to test when they're integrated with our logic, because we need to either learn a framework for testing the framework, or we need to mock the things the library does to change the world, because we don't want our tests to change the world.

There is this neat benefit for us. If we're only using the framework or library to change something in the world, and we don't want to write unit tests that change the world, then we don't need to unit test our side effect code! If we don't mix our logic and their logic together, we don't have anything to mock and unit test. This is because the libraries come with their own unit tests that demonstrate (hopefully) correctness of the library code.

There's another neat benefit. If we ever decide to revisit how our program changes the world, that code is all by itself, totally modular and easy to swap out or add to. Maybe today we're using PostgreSQL, but tomorrow we want to use Hadoop. Maybe today we offer a web interface and tomorrow we want to add a message broker. The code for these features performs only these features, and making changes is simple.

There's yet another neat benefit. Our code scales. If we ever need to run it on more than one server, that's not a problem because our core logic does not depend on local state.

It's not functional programming, but it's close.

Functional first development is certainly not the same as programming in a pure functional language, but it gives us some of the advantages of functional programming without learning new languages.

• It's easier to reason about our code, both from the outside in (we can look at the system and it's not too hard to tell what it's supposed to do) and from the inside out (we can look at the code and understand why we wrote it that way.)
• It's easier to test our code, because our tests are super simple and they run fast. Also, we don't have to unit test our side effects, because our side effects only do side effects and we that's the part of unit tests we mock, rather than test.

We get these advantages, but only in exchange for discipline and training. We keep using the same languages our teams have been using, and the languages provide almost no extrinsic motivation to program in a functional first manner.

Functional first development is an approximation - it gets us half-way to functional programming. To go all the way, we need a pure functional language that restricts us to functional patterns. Restrictions are a good thing, but that's another blog post.

More functional first development coming

These two rules give us only a brief introduction to functional first development. There is a lot of room left to explore, including the advantages for teaching these techniques to juniors, for refactoring legacy code, and for building large-scale applications. I'll write and present more on functional first development over the course of the next year or so.

I'm seeing more conference talks and blog posts saying they don't. Surely, the people making these claims aren't saying they can't find two developers who have an order of magnitude worth of skills level difference between them. Instead I think they're saying our industry is changing, that value doesn't come from isolation and the raw code processing power of the programmer. We work together now. We pair, we mob, we pull in our domain experts and develop a ubiquitous language before we even consider how to start coding.

I agree with all of that. Also, I think that 10x developers not only exist, they aren't actually all that rare. And they aren't stereotyped loners. And we want them in our pairs and on our mobs and working with our domain experts.

10x developers deliver 10 times as much value, not 10 times as much code. They do this by making consistently better than average choices throughout the project, and these choices stack on top of each other for a compounding increase in value.

Here are some examples:

Imagine that an average developer makes ten average decisions each of which return an average value. This is our baseline. The average developer delivers 100% of average value.

A 10x developer makes ten above-average decisions, each of which delivers 25% more value than the average decision. This is compounded - 1.25¹⁰ - and the 10x developer ships 931% of average value.

On the other end, imagine a below average developer who makes ten below average decisions, each of which drive value down 25%. This is also compounded, so this developer ships a whopping 5.6% of average value. While not the title character of this blog post, we could call this developer an ^x/₁₀ developer.

As you can see, a 10x developer working on a project where there are 10 major decisions to make can affect the overall value of the project by an order of magnitude, when compared to an average developer. Likewise, an ^x/₁₀ developer delivers value an order of magnitude lower than the average developer. (If you're keeping score at home, that's two orders of magnitude.)

Here are ten decisions, choices, and habits of exceptional developers that compound value on the projects they ship.

10 practices of 10x developers

1. 10x developers write less code, so they do less work.

10x developers do it better the first time. They don't flail around. They make good choices. They know their tools. They know their patterns.

2. 10x developers can reason about their code. We can reason about their code too.

10x developers don't write clever code; they write readable code. They know that troubleshooting code is harder than writing it, so they don't pack their ego into their code.

3. 10x developers use the right abstractions (both programming languages and design patterns).

10x developers use the right programming languages for the problem at hand. They know design and architectural patterns, so they know the established best ways to solve many problems.

4. 10x developers write code that is not encumbered by unnecessary dependencies.

10x developers don't start with a large code base of platforms and libraries because "rapid application development". They write core business logic in the core language, and use libraries for interfacing with the user and the persistence layer.

5. 10x developers write code that scales seamlessly in production.

10x developers make programs that are stateless, functional, referentially transparent, and independent. Their code don't just scale, it approaches linearity.

6. 10x developers write code that is free from defects and does only what's specified.

10x developers admire efficiency and elegance. Their code uses fewer abstractions more completely, does exactly what it's specified to do, and doesn't confound its purpose with undocumented behavior.

7. 10x developers don't ship technical debt.

10x developers like to ship value early and often, but not before it's correct. They don't ship prototypes, and they don't want to support prototypes.

8. 10x developers can extend their code without starting from scratch.

10x developers see architecture and design, strategy and tactics. They rewrite before the code hits production, if they didn't get it right the first time.

9. 10x developers can demonstrate correctness, and like to prove their code is correct.

10x developers write tests that mean something. They would prefer to write code that can be mathematically proven to always produce a correct result. They take joy in functional programming, because math.

10. 10x developers write code that is secure.

Because their programs don't rely on bloated libraries, aren't tightly coupled, don't carry technical debt, are stateless, and do only what they're specified to do, 10x developers write code that is more secure than other developers write.

Some observations

Items 1 through 5 above share a theme of simplicity. Items 6 through 10 share a theme of correctness. 10x developers, and the rest of us, deliver value by shipping code that is simple and correct.

8 out of the 10 characteristics (items 2, 4, 5, 6, 7, 8, 9, and 10) address the ongoing cost of software. Maintenance of our code base is the long tail when it comes to paying for our code. 10x developers shouldn't get their reputation based on writing a lot of code fast, but by delivering long term value through extremely low maintenance costs.

Finally, all 10 characteristics above can be the result of leadership and training. In fact we can foster all of these characteristics by having our 10x developers work directly with the rest of our team. If we put a 10x developer in a mob, the 10x developer may go slower, but the mob will produce much better value than average, and the individuals will go on to continue producing better value.

Most developers are lucky if they ever ship a feature than changes a user's life. Most managers are lucky to work on a product that changes users' lives. Maybe the best we can do is embrace a coding ethic that makes our work meaningful, and maybe bringing that ethic raises our work to a standard capable of changing lives.

This is my talk for Code Mesh 2015 in London, basically as a transcript but more like what I intended to present rather than the slightly more ad-libbed version I actually gave. The abstract I submit reads as such:

We started with 42 Oracle tables, 5200+ configuration values, no default values or templates, logical inconsistencies and circular dependencies, on-boarding times running as long as 10 months, a 1.5 million lines of legacy code, and advice that anyone who tries to redesign our configuration system will be gone within six months. Our solution is a custom-built functional data store with inheritance, flexible projections, backward compatibility with the legacy code, and a Lego™-like framework for building tools. Plus, I still work there. Come learn how we did it.

Talk objectives:

I'd like to share how we replaced a complex, "organically-designed" configuration system that we couldn't reason about with a immutable, accretive, functional, time-variant data store based on event sources and command query responsibility segration. We're particularly happy with our choices to project views into Oracle and to treat Oracle as an event source (which gives us backward compatibility with our legacy code), and to allow queries that compile data from multiple documents (which gives us reusable configuration templates).

Target audience:

Programmers and managers interested in applying functional programming concepts to data stores.

Key concepts

Will Byrd (@webyrd) suggested an important idea to me. When preparing a talk, perhaps a case study in particular, consider what you would want someone who attended your talk to reflect back to you afterward. If you were to ask what someone got from your talk, what do you want to hear in their reply?

Here are the concepts I most want to hear reflected:

• Manage complexity first. Functional first programming reduces complexity.
• Patterns bring surprising wins.

Will's contribution improved my talk substantially. Here are some comments from attendees.

OC Tanner does employee engagement through recognition and appreciation awards. Before I uncover some warts, let me just say we do appreciation well. In fact, we created the field when Obert C. Tanner - O. C. Tanner - went off to Berkeley in the 1930's and earned a Ph.D. in the philosophy of appreciating people. We are the largest company of our kind, with offices all over the world - including right down the road from here in Essex. 28 of the companies on the Fortune 100 Best Places To Work list use our software. We're one of them - we're #40 on Fortune's list of best places to work.

OC Tanner has grown over the years primarily by saying yes to customers. As our capabilities have increased, we've tacked additions on to our facilities like old houses used to get more bedrooms as the family grew. Our basements are a good example: we have seven, and they aren't connected to each other.

I heard a story about a woman who started her first day of work by going through our half-day orientation session, and then asked if she could skip lunch and go say thank you to the friend who had recommended her. She got directions to find her friend's team, down in one of the basements. The person leading the orientation didn't expect to see her again, since she was supposed to report to her manager after lunch. When she didn't report for work after lunch, her manager assumed she had been a no-show for the whole day. Late that afternoon someone found the poor woman in one of the basements, hopelessly lost and crying.

This is what our software is like too.

I mean that our software has grown organically over the years, by saying yes to customers. Can I have this new feature? Yes. Can you make the software do things our way? Yes. What if we want this to be royal blue? Yes. Option by option, tacked on like a new tower on the castle, with a royal blue turret on top. Or purple. We can do purple too.

This is where much of the complexity in our software came from. I suspect we're far from unique in this. Organic growth makes things more complex. At least that's what Alan Watts says.

When I started, my new boss put me in charge of the team responsible for configuration tools on the second largest software platform we offer. He explained to me that there were umpteen hundred settings, and onboarding required individually setting every one of them correctly, and that was a big reason why our onboarding process was running as long as ten months and we need 175 people to do it. My team was responsible for the monolithic web app used by those internal customers to manage those settings. Oh, and the only developer on that team was out on maternity leave and we didn't know when she'd be back.

He also told me that there is a standing prediction in the office that anyone who takes on configuration would be gone within six months.

Here are a few statistics from the beginning of the project. I found it intimidating, but frankly it's probably a pretty typical system.

We have about 75 programmers on staff, most of whom are carving monolithic platforms into microservices. Our software is complex, but you've probably worked software just as complex.

My team is working on a better system for configuration and configuration management, but the process we've been working through would be pretty much the same regardless of the project.

Our configuration data is poorly organized, hard to find, and hard to reason about. It's like a kitchen junk drawer. Most kitchens have one. That's how common our problems are.

Our legacy code happens to be written Java, but that doesn't matter. What matters is there is a lot of it, and it doesn't have many automated tests. We can't fix it all at once, so we have to work with it. We have to support the legacy system.

Reducing complexity is my "zeroeth law of programming", so my goal on this project, and on every other project, is to reduce complexity.

The two tools I use most to manage complexity are design patterns and functional programming. Design patterns prune a project back to a common shape, and functional programming makes it predictable.

I'm not going to cover all of functional programming in one slide so instead we'll talk about a pattern I like called functional-first programming.

But first, design patterns.

You're probably here because you think you might have problems that look something like our problems. If we're all lucky, I might have solutions that look something like the solutions that are going to solve your problems.

That's what a design pattern is. Design patterns are generalized solutions to generalized problems. They help us understand our problems quickly, and the give structure to our solutions early in the process. Design patterns help us reason about our project right away.

Pretty much everything I'm going to talk about today is a design pattern. Functional-first programming, event stores, command query responsibility segregation, prefix tries, static site generators. We didn't do anything new. We looked at the problem we were trying to solve and found patterns other people discovered that matched our problems. Then we took established solution patterns and made them fit our needs.

One of the really neat thing about design patterns is that they bring win with them. If I had sat down to create goals for this project without first having some patterns in mind, I would have come up with a good list. But the patterns we chose informed our design goals tremendously.

Choosing the right patterns not only helped us solve the problem, they raised the bar on what we could expect from our solution.

I stole the term "functional first programming" from Don Syme, without asking him and without checking to see if I'm using it the way he intended. If I am, it's only because he named it really well, because when I heard the term I thought "That's what I'm doing!" I probably owe him an apology.

My version of functional first programming goes like this: First code everything that's possible to do without side effects. Then code the side effects. The first part should be purely functional, easy to test, easy to reason about, and maybe even provably correct. The second part is just side effects, which probably means it all depends on IO libraries that someone else wrote and tested. There's a good chance you won't even need to unit test your side effect code, which is a good thing because we hate writing mocks. Plus, your interfaces to the outside world are now strictly modular, so changing an interface or a database is very concise.

You can do functional first programming in almost any programming language, including object oriented programming languages. Gary Bernhardt has a great talk on that called "Imperative shell, functional core". But using a functional programming language makes it easier.

I like mostly pure functional programming languages. A powerful language might let you do all sorts of amazing manipulations in a computer, and a flexible language might give you tremendous choice in how you solve any given problem. Those languages have their place, but for managing complexity I always prefer a strict, opinionated, constrained language. Power and flexibility tend to lead to increased complexity.

It turns out choosing a language is a lot like choosing a design pattern. The languages we use change the way we think about a problem. My team could have solved our configuration issues acceptably using only Java, but choosing to use an actor-based concurrent programming language created opportunities we don't get with Java.

When we started, I thought we were building a specialized configuration service. It took me a month to see that we were actually building a NoSQL database - a key-value store. When I finally realized this, I went to my CTO and asked him why he approved a project to build our own database.

If I'd realized this sooner, I'm sure I would have dropped the project and tried to figure out how to do it with a generic NoSQL database. The jury is still out on whether that would have been a better move, but by the time I realized what we were doing we already had design goals I couldn't meet with any of the NoSQL databases on the market.

Our data structure is a basically a trie, which is actually called a "trie" because it comes from the word "retrieval" but I prefer "trie" to distinguish it from a "tree" that comes from the word for a big woody plant.

Tries help you find things quickly and store things compactly by building a tree that shares prefixes. In our case, nodes could represent a dotted-notation key. We map each node to an Elixir process - an actor - and give it responsibility for processing events regarding its state, persisting new events, and sharing new events across a cluster.

Our old system doesn't have the concept of default settings. Every new customer needs to have every configuration setting established by an implementation specialist. With our new system, we want to get most settings from a book of defaults, and only get involved when a default setting doesn't fit the customer.

This is where our book metaphor comes in handy. We have a book with system-wide default settings. We have books with the default settings for various option packages. The customer has a book, and each user can have a book.

Composable data is like stacking those books. The user's book overrides the customer's book, which overrides the books of default settings. Conflicting values from less specific books are ignored in favor of the more specific book.

We have a special situation though. Although many of our customers can simply use the default values in the current version of each book, some of our customers want to go through a change review process, so we can't willy-nilly update the default books without warning them. We need to maintain previous versions of each book.

If we look at our prefix trie again, this time I'll show that every node contains not only its current state, but also all previous states of the node. We keep the value of the node over time in a list. When someone asks for a setting without specifying the version, we give them the most recent valid version. But if they specify any specific version, we can give it to them.

Under the old system, when a developer needed to create a new configuration setting, he or she created a row, or maybe a table, and stored the data there. Documentation typically sucked - the name of the table and the name of the column along with the code. Other developers that needed to interact with that setting would figure it out from those clues, or ask the team who made the setting.

In our new system, when a developer creates a setting, they store right alongside it any information needed to find, understand, change, and validate that setting. So our system contains not just data but metadata - a title, a description, and a schema at least, but it can also include tags that identify where the data should be exposed, and access control information defining who can view and change it. The metadata tells us everything we need to know to expose the setting anywhere we'd like, with documentation that comes straight from the creator.

Our trie data structure holds metadata in each node, and when we query the configuration service we can request just data or data and metadata. Either way, as it traverses the trie to get the data, it picks up the metadata, composes it with the metadata on child nodes, and uses it to figure things out like what data types to expect and who can access the data.

We've been carving up our original monolithic code base into microservices, but we're only half way there and have a year or two to go. Our new configuration system has to support legacy code. We don't really have a choice.

I knew if we delivered a configuration system that required other teams to drop their work on microservices and rewrite legacy code, I wouldn't get the buy-in we need to make the project successful. On the other had, if we chose to only support new code and ignored legacy code, then our internal customers would be stuck with the old configuration management tools running alongside a few shiny new tools, and they would hate us for it.

We addressed the problem of supporting legacy code by using a functional data store based on two patterns, Event Stores and Command Query Responsibility Segregation.

A functional data store is a pretty broad label that means a database that doesn't let you change any data, it just lets you transform it. For comparison, we're all probably familiar with a CRUD database. It lets you Create, Read, Update, and Delete data. Functional data stores don't do updates or deletes. Instead of updates, we create a new state for an existing record. Instead of deletes, we can mark things as "gone" beginning at a certain time. Things change over time, and very often we want to know what data we had before the current data, even if the data was "deleted". Sometimes especially if the data was deleted.

Our functional data store is based on two design patterns that work really well together, Event Stores and Command Query Responsibility Segregation.

Event stores are like your checkbook register. With most databases, we overwrite a value when it changes. That'd be like keeping your checkbook balance by erasing it and writing a new one every time you spend some money or make a deposit. Uh-uh. We keep a list of every transactions going back to the first deposit we made when we opened the account. From that list we can figure out what our current balance is, as well as what our balance was at any point since we opened the account.

Command Query Responsibility Segregation is the difference between the checkbook register and the running balance for your account. When we make a deposit or draft a check, that's the command side. When we want to know our balance, we don't start at the first deposit and calculate everything since. We simply look at the running total column. That's the query side.

We built our configuration system with these two patterns, and because of that we got a design win that we needed but hadn't thought of. Our customers don't have to all upgrade default configurations at the same time.

When something changes in the configuration service that also exists in Oracle, we need to update Oracle, otherwise the legacy code won't see the change. Likewise when legacy code changes something in Oracle, we need to treat that like any other configuration event and update our configuration service so that the new code can see it too. We do this by using "data coordinators".

We have two types of data coordinators. Publishers listen for changes in a data source and publish any changes to a pub sub queue. For example, our Oracle data coordinator watches the redo log, and if any of the rows it cares about are changed, it publishes the change to a channel. Likewise with the configuration service. Whenever it changes anything, it sends out a message about the change.

Subscribers listen to those channels and do something useful with the information. On the Oracle side, the data coordinator maps keys in the configuration service to rows in a table. When it gets a message about a change on the configuration service, it updates the row.

Even though it looks a little like we're trying to keep two data sources in sync, I think we aren't. The configuration service is the canonical data source. Oracle is just a place where we store a view of that data as well as an event source about changes made by a user.

Basically we built a platform that watches for data to change, and when it does it publishes a message. We made it possible to subscribe to our data, not just poll it. It's almost too good to be true. I'm rather worried it is too good to be true. If you want to explain to me why it's too good to be true, I'm available after the talk. Seriously.

Our work on this project covers configuration and configuration management end to end. We're not only building a configuration service, we're building the tools to manage it.

Our current configuration management tool is a web app that exposes all the settings for any user in one long list. We don't want that. We want our internal customers to have bunch of small tools that support their workflow. But we don't actually want to focus on builing each of those tools, so we built a tool that builds and manages all of those tools.

We want our internal customers to design their own tools, and bring them to us to build. We expect to be able to turn those tools around quickly - possibly even the same day. If we make it simple for them to work with us, and we deliver new workflows quickly, then they will be able to iterate on their own processes.

One of my goals for configuration management is that juniors should be building these tools. That'll be a good indicator that we managed complexity well. But there's more. When I started, we had no work available that juniors could do, so we couldn't hire them. If we can't hire juniors, then we can't build future seniors. We can't contribute to the growth of our community in that way. And we can't get them started off right. We're a really good company, and people are really important to us. That's why I work there.

Most of industry seems to treat functional programming as something senior developers do, and maybe even just the best senior developers. It's almost as if we don't trust someone with functional programming until they've proven they can go a decade with object oriented programming and still not murder anyone. We're teaching new programmers to live with complexity, rather than how to minimize it.

I want to help change that. Our interns and juniors learn functional programming. I teach functional programming to new developers outside the company too. We're doing a session this weekend with the London chapter of Women Who Code. Ask me about this if you're interested.

Ultimately, whether seniors or juniors build our configuration management tools isn't exactly part of the functional specification, but it's working for us. We even have a team of interns working on the first set of tools now.

There's a natural boundary between server and client software, just like there's a natural boundary between the company and the user. Unfortunately, we've spent much of the history of the internet building systems that violate that boundary.

Companies have goals, and users have different goals. I try to design systems that keep the company's interest - mostly the data and it's integrity - on the server, and the user's interest - meeting their needs online - on the client. Servers shouldn't do client things; they should apply business logic and expose the company's core competencies through a useful interface. Clients shouldn't be constrained in meeting their needs by data restrictions. I hate it when that happens. I call it the tyranny of the database.

Programs on the server should focus on the data, and client-side software should be all about the client. It's the single responsibility principal. It lets the server team focus on the company's needs, and the client team focus on the user's experience. It reduces coupling where it makes no sense to couple things. It reduces complexity.

We're doing this on our project. The configuration service offers an interface for using and managing configurations. The metadata can tell you semantic things about the settings. But the configuration service doesn't have opinions about how you manage it. That's the job of the user-side tools.

When we talked about command query responsibility segregation on the server side, we talked about events. Events are how the server handles commands. The user does something real, and the server updates its model by handling an event about what the user did. But events start with the user doing something. That something is the command.

When we put a form element on a page, we're giving the user ability to issue a command to change that value. Commands are imperative. Commands happen in user-space. Commands are part of our shared reality, what we might call "the real world".

Events are how the server tries to maintain a consistent model of the real world.

Here's another way to look at it. We can call it "sanity". In psychology we consider someone to be sane when their internal view of the world matches our shared external view of the world. Our data is sane when we maintain consistency between our user's experience of the world, and the model of the world we keep in our database.

We had to consider what happens when someone sends us a configuration value that doesn't validate. If we were just a regular database, we could reject it. But if we were just a regular database, we'd be keeping only the current state and it would break our contract with our users to send out data that lacks integrity. As an immutable and time-variant database, I think we have a responsibility to handle all events, not just the ones we like, because events happen in the real world and it's not like we can just reject reality. Even if it means our data is invalid. The decision we made was to use the metadata to flag states that are invalid. If you make a request for just the value of a key, we'll send you the most current valid state. But if you ask for the data and metadata, we'll give you the current state, even if it's invalid. Plus we can use the "invalid" flag to create a report showing which keys are broken, and ask some human being for help.

We've created a slick, functional system for building configuration management tools. And it's bone simple to use.

One of our design goals was for a single point of specification. When a developer creates a new configuration setting, we want them to tell us all about that setting. What it's called. How it's used. Who can use it. How to tell if it's valid. All of this information is stored in the metadata for the key, which means that each key can tell us everything we need to know about how to manage that key.

Metadata is the basis for our configuration management tools. Because the metadata tells us so much about the data, we can use it to automatically render a form element for changing the data. If we want to expose a setting in a configuration management screen, we simply insert one line of javascript and pass it the key. The javascript renders the form element based on the metadata.

It doesn't matter what kind of configuration management tool it is, either. It can be our internal tools, or tools we expose to our customers to administer their programs, or even tools for users to manage their profile inside the software.

Right now, the main tool for our implementation specialists and customer service team is an app that exposes all of the possible configuration settings for any customer in one long list. They designed their workflows around that list. Our new system gets away from that. We don't want their work designed around our tools. Our tools should be designed around their work, and we're going to do it so efficiently that they can iterate on their own processes.

We put together a stack to build their tools. Each tool is defined by a markdown file, and where it appears in the file system indicates where it'll be located in the toolset's navigation. A static site generator called Metalsmith turns the source files into a website of custom-made configuration management tools. The site is built as a collection of static files, so deployment is just a matter of pushing them to the server.

Metalsmith is Node.js, functional, pipelined build tool. The main loop is one long filter chain operating on all the files in the project directory. We start with Markdown files. In one pass, a filter rewrites the configuration keys as javascript calls and wraps them in a form element. In the next pass, a filter converts the markdown to HTML. In another pass, a filter calculates navigation for the site. In another pass, a filter applies our layouts, including calling all the appropriate javascript libraries, stylesheets, and other assets.

The output from each filter step is the input for the next filter, until all the filters are applied and the resulting files are written to a build directory containing all of the HTML, javascript, and CSS files for the tool site.

About the time we finished our design goals and were starting to code, we were kinda thrown for a loop. Another team came to us and said they had a problem that wasn't about configuration, and they wanted to know if the new configuration service could solve it.

Our software is used world-wide. It's translated into I-don't-even-know-how-many languages, and each customer's platform is customized and translated into the languages their employees need. That means we have default translations to a bunch of different languages, each of which might be overridden by the customer.

We spent two hours together reviewing the translation service's needs, looking for the gaps between what we were already building and what they needed. We didn't find any. Our design goals fit their needs exactly.

I think that was the moment I realized that we were actually building a database.

When we started code on this project, I had one programmer available to work on it. He's great at Java but had almost no functional programming experience. We did our best, and the first release of this system - the version we have in production now - is not exactly like what I've described.

We try to work in a lean-development way, to get services into the hands of the developers who are going to use it and see what we did wrong. Our production version is written in Java 1.8 and uses one process per book rather than one actor per key. Because of this, a book is a container for a set of keys, and we can only compose data across books, and books maintain their own event stores and a single projection. It doesn't scale well - the chunks are too big.

Our development version is being written in Elixir. Here there is no difference between a book and a key - a book is just a key, so any key can be considered a book, and compositions can happen at any level. Each key is an Erlang process and maintains its own state across time, which includes metadata and either data or a link to child nodes. Each key manages its own event store as well, although this is distributed around the cluster.

In both versions, clients interact with the service using messages sent through RabbitMQ. The service also publishes data changes to the message queue so that events can be stored across multiple machines in a cluster, and coordinated with Oracle for legacy code.

That's the system, end to end, from the functional data store with immutable, time-variant data to a pipelined static site generator building the management tools. We're in the process of moving configurations onto the new service now, and we're learning from our internal customers and their patterns of use.

We do have permission to open source the code, and we will - once it's proven under heavier production load.

We haven't really done anything novel, we just combined established patterns in a way that fit our problem. We got some surprising results from that - things we wouldn't have asked for if we weren't already looking at a pattern that offered it. But believe me, once we knew we could get it, we were sure we had to have it. We also got some things we knew we needed, like significantly less complexity.

This project is fixing problems with how configuration is managed, but we haven't done anything about the consistency problems within our configurations. We've got great new tools for writing buggy configurations that are impossible to reason about. Moving forward we want to provide a way to make declarative statements about relationships between configuration values. I think implementing a dependent type system may be coming soon.

But that would be a different talk.

Thank you.

This is a transcript.

Hi, I'm Rob Martin, otherwise known as Version2beta pretty much everywhere online, and the guy they asked to talk about where you can go from here.

I hope this weekend has been super successful for you, and that you decide to keep coding, to become a professional developer, or build that app you've been wanting to have, or just express yourself to a connected world. You've got lots of choices in the path you take to get there, and I'm not in a position to give you a road map because I only know a little about where you're at, and I know almost nothing about where you want to be.

But I think our organizers might have wanted me to talk about practical steps you can take to keep learning and keep coding. And that's important, so let's take thirty seconds and discuss a plan.

The facts are that if I knew more, I could help more. If we sat down and talked, I can help you find your path. That's true of pretty much every coach here.

So that's my plan for your next step. Get a mentor. Ask a coach here if there's someone you felt comfortable working with. Ask me if you want, or ask me to help you find someone who's a good match. And then share your dreams with your mentor and find your path.

Now let's talk about changing the world. And why you should hire me.

When you came here today, did you expect anyone to ask you for a job? Well, here I am. I want to work for you.

I have mad skills, of course because why else would I ask you for a job, but since this is a relatively short, large group interview, I'd like to talk about you instead of me. Specifically, why I want to work for you.

I'm kinda an old dude, and I've seen the world change a lot over the past 46 years. And I've spent a lot of time thinking about how that change happens.

I've seen us change from mom and pop downtowns to multinational online supermegastores.

I've seen us go from wall-mount rotary phones - really - to smart phones that put all my friends right here in my pocket.

I've seen us become constantly connected with each other in amazing ways, to have meaningful, rich interactions with each other across continents and decades.

I've seen us go from kids at playgrounds and hamburger joints to something like cyborgs. Our devices are now part of who we are. They put the cumulative knowledge of the human race into our hands. They connect us to each other, and create communities where we used to feel completely alone. And the people we reach through our devices, they are our tribe.

Developers did this. Developers changed how we relate to each other, and created opportunities for us to relate to new people organizing around things that didn't even exist when my kids were little.

Developers changed who we are.

Of course, I know generalizations are always wrong. So it would be inaccurate for me to claim that all developers are white, straight, cis-gendered, relatively wealthy dudes. Still, this generalization is true too much of the time, and it's even more true when you look at the people who are directing the technology we're creating.

There are notable exceptions, but for the most part a bunch of rich, white, straight dudes are responsible for the creation of most of the technologies that have changed the world during my lifetime.

This may be obvious to you, but I'm slow: The less you have in common with the people who create a culture, the less power you have within that culture.

I don't know if you noticed, but I'm an upper middle class white guy. When I go to predominantly black or Latino neighborhoods in Milwaukee or Los Angeles, I don't have much cred. I don't have all that much in common with the people who make that culture so rich and so real.

The flip side is also true. The more you have in common with the people who create a culture, the more power you have within that culture. Maybe you're starting your programming career today, and if you are I applaud you. I can afford to. I have a 36 year head start on you, because I was fortunate enough to be a pimply, geeky white boy from a well-off family at exactly the same time that our culture was saying pimply, geeky white boys make cute hackers.

Becoming a developer was easy for me. The question I've been trying to answer for years now is, how do we make it easier for you, too.

I'm talking about privilege, of course. I have a lot of privilege, and it's because of how much I have in common with the people who have been creating the culture that you and I both live in. I look like them. I have the same background as them. I have some mastery over the same tools they use.

I am them.

Being part of the group of people who create our culture is where I get my privilege, and it doesn't matter much what I do about it, I'm likely to keep a lot of that privilege.

I've seen people do some pretty stupid things with this privilege. You probably have too. I've done stupid things with my privilege, although hopefully not as horrific as some of the things I've seen other developers do.

One thing I can do with my privilege, however, is help change the group of people creating culture.

This is where you come in.

The more women who get into programming, the more influence women have over our culture. The more diversity we have in tech, the more inclusive our culture becomes.

I know this sounds like I'm saying you can do amazing things simply by being a woman in tech and that's true actually, because just by showing up you are helping us change what we think a developer looks like. I really hope it's only one of the ways you do amazing things.

By launching your programming career, by taking your first steps toward being a creator of computer programs and content, you are influencing our entire culture. Today, your influence may be small, but you are already making the world a better place. When you work your rear end off building your dreams, you'll be changing the world.

Maybe you don't want to start your own company. Maybe you just want to share in the fact that they pay us developers stupid amounts of money, and being smart folk to begin with, we don't turn it down.

As an industry, we're trying hard to change the way we work to stop making it hard for you. Really, we are, even though it's slow and isn't happening at the same rate everywhere.

We created our "company cultures" - I hate that phrase, it's usually used as a way to discriminate against people who aren't like us - we created our company cultures around the people who make it to the top. Early on, it was the geeks and freaks like Bill Gates, or Mark Zuckerberg, pimply boys from upper middle class families who identified with the early maker culture. Later, the smart white jocks and frat boys caught on with the opportunity to be rockstar ninjas with huge pay checks. But now we're finally recognizing that programming is a social activity that has huge impacts on society and individual lives. We're working in pairs, even in mobs sometimes. We recognize the value of empathizing with our customers, and we invite them to be part of our creation process. We value "domain expertise" and want to hire programmers who know about things that aren't programming, like community building and mechanical engineering and education and medicine and particle physics and parenting.

If your path is to become a programmer working for someone else, I want to work with you, and - true story - when I decide to hire you, one of the things I'm going to consider is whether or not I would want you to be my boss.

A bunch of people in this room are going to start their own companies - or already have. In spite of the fact that it might not be as easy for you as it has been for me, statistics say that quite a few of you are going to be successful.

I have the impression that it's getting easier, that tech is not as insular as it was in the heyday of brogrammers, that we're calling attention to the fact that women founders and CEO's aren't getting funded at anywhere near the rate that their male counterparts are.

We're calling attention to the fact that women who found and lead startups have 35% higher returns on investment and 12% higher revenue than their male counterparts. We're noticing that successful startups have twice as many women in top positions as startups that fail.

I think this means it's working. From 23andMe and Eventbrite and LOLapps and CodeForAmerica, all the way to the four incredible women interning on my team at OC Tanner, women are not only changing tech, you are changing culture.

I told you I wanted to talk about you, but perhaps I just spent 15 minutes talking about how I want you to change the world. That's a lot of pressure, and it's not fair for me to ask that of you.

There's a rule about women in tech called "the unicorn law". The original version says that a woman working in open source software will sooner or later be asked to give a talk about being a woman working in open source software.

When I learned that rule, it changed my life. Before that I knew there were problems around diversity in tech - not just gender diversity, but all diversity. And I did almost nothing about it, because who am I to represent these under-represented groups of people, right?

That's when I realized that maybe a lot of people might like to get into programming because they like to program, not because they want to become activists. I also realized that it takes privilege to change culture, and expecting the people less privileged to make that change is completely unreasonable.

That's why I'm here today, and when I tell you I want to help you change the world, I mean that I want to help you learn to code. I want to help you get a job. I want to help you become a rockstar ninja, if that's your thing. I want to help you start a company. I want to help you build your dream.

You, being here, being yourself is what will change us. I'm really glad you're here. Thank you.

I want to thank you for the opportunity to be one of the first people to interview for the jobs that you're going to create. I understand that you might not be ready to offer me a job right away. Please, take the rest of the weekend to think about it. After that, I hope we can talk again about how to fit me into your plans for world domination.

I'm Rob Martin, version2beta online, and I want to help you change the world. Call me.

This isn't really a transcript, more like a framework of ideas to discuss while each slide is showing. Sometimes the discussion ideas even have something to do with what's on the slide.

The goal of my talk is to convince you to do more functional programming. We'll start by looking at complexity, and in turn simplicity. Then we'll examine how our languages influence our design and even our thought process. Finally, we'll look at functional programming and how it meets the goal of managing complexity.

Complexity is the morphology of software - that is, its form and structure. All form and structure is complexity - we just accept some complexity.

Complexity is the interrelationships and dependencies of software components, how components are coupled. This is demonstrated by the impact one component may have on another.

Complexity is any code that can be, but hasn't been, decomposed into simple components.

Complexity is the part of our code that isn't beautiful. Beauty is the ultimate defense against complexity.

Complexity is the enemy of reliability. Tony Hoare says, "The price of reliability is the pursuit of the utmost simplicity."

There are limits to our ability to reason about a system, and there are limits to our ability to build complex systems. Our ability to build complex systems far outstrips our ability to reason about them.

Testing is never 100% effective, but the effectiveness of testing decreases as the complexity of the software increases.

There are ways we can use formal reasoning - that is, mathematics and logic - to prove the correctness of a system. But proving the correctness of a system becomes magnitudes more difficult as the complexity increases. Proving correctness quickly becomes impractical, or even practically impossible. Not to mention that it's tremendously difficult to create a complex system that is correct.

If we can't reason about software, can't test our software, and can't prove our software, then we can't trust our software.

Essential complexity is the required inputs and outputs, and the business logic that defines the relationship between inputs and outputs.

Accidental complexity is everything else, including the program that implements the business logic.

Informal reasoning is understanding how a program works, and why. Being able to translate what the program needs to do into how the program will do it.

Good informal reasoning will lead to simpler software. Complex software is difficult to reason about.

Complexity breeds complexity. A program that is difficult to reason about becomes more complex.

Understanding a system from the outside

Leads to more errors being detected

Some limits:

• Each test demonstrates correctness for only one set of inputs. (Property based testing is an exception.)
• Mutable state complicates testing drastically. A mere 24 bytes of state, 6 32-bit words, contains more possible states than there are atoms in the earth.
• Testing only demonstrates correctness in the code, not in the specification

Methods for proving, mathematically, the correctness of a program exist, but they're outside the scope of this talk.

Formal proof of correctness can only be proven mathematically if both the algorithm and the specification are also given formally; i.e. mathematically.

If we can reason about software, can test our software, and can prove our software, then we can trust our software.

The language we use influences the design of our code. It changes the way we think. Alan Perlis says "A language that doesn't affect the way you think about programming is not worth knowing."

Imperative and declarative communication patterns

Imperative programming changes the state of something in the world, or at least in a program's model of the world.

• Could be the state of a variable.
• Could be the state of a user's screen.
• Could be the state of a servo controlling the ailerons on an ICBM.

Declarative programming describes relationships and then answers questions about them

In imperative languages, the current state typically defines identity. That state is changed by code: we say "Set X to 2." Identity is timeless; the only state we can get from it is "now". When we have multiple parts of the code changing state, as complexity increases it can be difficult to guess what values we'll get.

In declarative languages, state doesn't change because values don't change. We don't say that the value of a variable changes any more than we'd say the value of 1 changes. Identity is a collection of states, each transformed into the next, passed to a function or an actor only when a transformation is needed. Time is inherent; even if we throw out previous states, they still existed as separate entities.

We don't move from simple to complex. Our job is to control complexity, to model an inherently complex world in the simplest possible ways. Languages can help us do that.

Power corrupts. The ability to allocate memory, manipulate pointers, or manage garbage collection is sometimes needed, but on the whole it leads to a more complex system, not a simpler one.

Flexibility is a slippery slope. An unopinionated programming language might let you choose to be declarative or imperative, to mutate or not, to use functions or classes. The result is that you have the flexibility to build more complex systems.

There are exceptions, of course. We typically use lower level, flexible, powerful languages to build opinionated, more restrained languages. My favorite geeky tee-shirt is probably the one that reads "My compiler compiled your compiler."

Almost every programming language has functions. In functional programming, functions are the stars, not the supporting players.

Pure functions have exactly one result for any given input and do not generate side-effects. The result of a call to a pure function can replace the call itself for any given value (referential transparency).

Functions can be made of functions (composition), and have functions passed to them as inputs (higher order functions).

Testing pure functions is super easy. Your inputs are known, your outputs don't change unless your inputs do, and you don't rely on mutable state to calculate a result. Look Ma, no mocks!

Distributing pure functions is super easy. Because a pure function will always return the same value when given the same inputs, it makes no difference where we run it.

Parallelizing pure functions is super easy. Two functions that don't rely on the output from each other can always run in parallel.

Even faster than parallizing functions is to never run them. We don't have to evaluate a function before it's needed.

Can we reason about it? Yes. You have an input, and you have an output, and the same input always generates the same output. If the essential complexity of a function is significant, we can build that function out of smaller, simpler functions.

Can we test it? Yes. You have an input, and you have an output, and the same input always generates the same output.

Can we prove it? Maybe. Functional programming is based on Lambda calculus and as such, follows a provable theorem. In fact, both Lambda calculus and Turing machines were created in large part as a way to determine whether a calculation is computable. Turns out it is possible to write functions that will never return, and it isn't always be possible to tell the difference between a function that will never return, and one that just hasn't returned yet. Then there's the complexity of the software itself still. A short function may be provable. A long program may also be provable, given infinite amount of time and effort.

We've already discussed mutable and immutable state. Let's run immutable state through our simplicity test.

Can we reason about it? Yes, because state is typically an input to, and an output from a function, and we can reason about the functions.

Can we test it? Yes, see above.

Can we prove it? Yes, at least to the extent we can prove our functions are correct.

We need side effects. To have side effects is to change the world. Sooner or later we want to change the world. The thing is to do your side effects as late as possible.

One strategy is to have an imperative shell wrapped around a functional core. In the core, do everything you can do functionally. In the shell, do only side effects.

Simplicity test:

• Can we reason about it? Yes, we can often reason about it.

• Can we test it? Not really, but we can mock up how we think the external world is going to work and see how our programs interact with that. By isolating our side effects and imperative programming, we limit the amount of work we need to do mocking the real world.

• Can we prove it? Nope. This is a characteristic of imperative anything. You can imperatively insist on stuff all day long, but in the end, we don't run the world.

• Does it matter? If our imperative shell is providing a minimal interface between our functional program and the rest of the world, chances are good we will be able to trust our imperative code based solely on informal reasoning.

Before we finish, let's talk about how to think about programming in a functional language.

With object oriented languages, we have a consistent overarching metaphor. There's no similar master metaphor for functional programming, but here are some that get us started.

Pipelines are really just function composition. This is the basis of functional programming; we build programs from larger functions built of smaller functions. It's functions all the way down.

Many functional programming languages support concatenation as well as composition. The first example is in Elixir, and the second is Forth.

Concurrency-oriented programming is defined by Robert Armstrong, one of the creators of Erlang.

• COPLs must support processes, which can be thought of as a self-contained virtual machine.

• Several processes operating on the same machine must be strongly isolated, so that a fault in one process shouldn't adversely affect another process unless that's the behavior we want - one process to detect failure in another process, and know why.

• Each process must be identified by a unique unforgeable identifier. We will call this the Pid of the process.

• Processes don't share state, they send messages. If you know the Pid of a process then you can send a message to the process. BUT message passing is assumed to be unreliable with no guarantee of delivery.

Many programs model things in the real world, so they need to support identity.

(One of) Rich Hickey's definitions for Identity: "A putative entity we associate with a series of causally related values or states over time."

My definition, inspired by Rich Hickey: "Identity is a bunch of states over time that we associate with a thing."

Hickey's metaphor doesn't see the world as "becoming" in the way the catepillar becomes a beautiful butterfly. I think he sees it as being a series of unchangeable states over time, that we for our own convenience as beings stuck moving in a single direction and constant speed along the 'time' axis tend to think of as discrete objects. In Hickey's universe, not only is he unbound by time, but he can see how each thing can have its own timeline.

Without getting into specifics, lambda calculus is the basis of most of our functional programming languages. A few, like Forth, Joy, Factor, and Cat, and more based on combinatorial logic.

In fact, when McCarthy wrote about his ideas for Lisp in 1958, he was defining a computing machine that would implement Alonzo Church's lambda calculus.

Shortly after that, Steve Russell - incidentally, the guy who created the world's first video game - went to McCarthy and said he thought he could implement McCarthy's ideas on an IBM 704 computer. McCarthy told him he was confusing theory with practice and that it couldn't be done. So Russell went and did it, and that is how Lisp was born.

Let's explore a recruiting and interview process that is people-centered, authentic, and collaborative - just like we want our work environments to be. In that same spirit, let's make it totally transparent. Let's explore the process from the perspective of both the interviewer and the interviewee, the hiring manager and the candidate.

This is the recruiting and interviewing process I'm using at OC Tanner Company in Salt Lake City, Utah, one of Fortune Magazine's 100 Best Places to Work. It's how I interview for the teams I manage, and it's the process I train other people to use.

We'll start with a generalized recruiting process:

Start recruiting a great developer before you need one. Ideally, start recruiting while the developer is still learning to be a good developer, let along a great one. When you need a developer, it's nice to already have candidates lined up.

You want developers to introduce themselves to you, not the other way around. Speak, teach, coach, mentor, sponsor, host, feed, write, tweet, talk, meet, and get noticed being important to the community.

Make your interview process humane. Set an expectation that the process is collaborative, not competitive. When the candidates you don't hire speak well of your process, you have a powerful new tool for recruiting.

We often have specific needs, and we want to tune our recruiting process to meet those needs. Maybe the company is driving up quality, and you want to attract people with experience in software as craft. Perhaps you're carving up a monolithic application and calving microservices. You might want to attract functional programmers, or developers strong in architecture, or programmers in a more obscure language ideal for your application.

For several years I've focused on diversity on engineering teams, especially gender diversity. I have a specialized recruiting process that helps bring more women into consideration for our open positions.

1. We work with the community to help women get into (or back into) programming, by teaching, coaching, and mentoring groups like Girl Develop It, Railsbridge, and PyLadies. We try to get noticed by women while doing this.

2. We've reworked our interview process to be friendly to the individual and to humans in general. We've gotten rid of some interview techniques that have been demonstrated to have a gender bias. We set an expectation that we help each applicant accurately demonstrate their current skill level, not just their interviewing ability.

3. We're starting an internship program, partnering with Girl Develop It Salt Lake City, that will help to bridge the gap between boot camp graduate and marketable, productive developer.

4. We're presenting our practices within our local tech community, especially in groups that support and encourage developers who are women.

Because of this and other recruiting efforts, the result has been that about half of the candidates we see are specifically targeting OC Tanner for a job, and some are not even applying at other companies.

Before we launch into the mechanics of our interview process, here are some pointers.

Hiring people is a competitive process, and we can't help that. This does not mean that interviewing needs to be competitive too. If you are building teams that collaborate, let your candidates collaborate too. If you are supporting and training your team members to be better developers, do the same with candidates, on a smaller scale.

Share your process with the candidate. Answer their questions directly. Coach them before, during, and after their interview.

Don't stop recruiting just because the candidate is interviewing. Hiring is a two way street: we aren't just hiring them, they are hiring us to be their team and employer. You want the best candidates to choose you.

Going to send a candidate home without an offer? Keep recruiting. Offer a few minutes of mentoring to the candidates you turn down. What did you like about the candidate? What could he work on to be ready for a job with us? When might she try again?

Structure every interview to see the candidate at their best, just like you're going to structure your team to get the best work from your developers. Let the candidate help you with this.

Sometimes an interview just doesn't work. Maybe the candidate is nervous, or has a language barrier, or doesn't gel well with the interviewer. Stay attuned to how the interviews are going, and be ready to change things to make the process successful. At the end, if you didn't get an accurate representation of the candidate's skills, consider whether you want to do that interview over. For example, a candidate who blanks while pairing on a coding exercise may solve a take-home coding interview elegantly.

We want candidates who are "tee shaped" - deep skills in at least one area, but broad skills too. What's the right fit for your team? It's almost never just amazing technical skills with the language. You might want to see your team do more pairing, or better testing. Maybe you want someone who is good at sharing and teaching, or helps build cohesion. Look at how each candidate fits the team along multiple dimensions.

Our collaborative process is only useful if we get your cooperation too.

We want to know where you're at, in both your skill level and your career, so that you'll be happy with the job we offer you.

We want to see you at your best. If we're not getting that, help us figure out a better way of working with you.

Your coding ability is only part of the equation. We also want to see how our goals mesh with your goals.

Here's part of our transparency: we want candidates to know how they will be evaluated, and later, how they performed.

Before you evaluate a candidate's interview, evaluate the interview process. Were you able to get to know the candidate? Do you and the candidate both feel like you got an accurate impression of the candidate's abilities during the interview? If not, is it worthwhile confirming the candidate's skills in another way, like moving to a different space (maybe less casual, like a conference room, or more casual, like a coffee shop?), or with a take home coding exercise, or a corraborating interview with someone else?

I once interrupted an interview taking place in one of our office "living rooms" - small rooms with a couch, a couple of cozy chairs, a big screen TV, and a whiteboard mounted uncomfortably close to the floor. A young woman being interviewed by two engineers (one male and one female) was kneeling on the floor in a skirt and heels trying to solve a coding exercise on the whiteboard. She looked miserable, so we changed just about everything. Instead of the living room, we went to a training room with a desk-height table, office chairs, and bright overhead lighting. Instead of a whiteboard, we had her pair with a female engineer from our team. Instead of being two-on-one, I asked the male engineer to offer the women coffee or tea and then leave the interview. The candidate performed much better under the changed circumstances.

Did you know that the dry erase marker is less than 40 years old? Together with the whiteboard, they are a direct replacement for their predecessors, chalk and the chalk board. A soft rock, and a hard rock.

When we do a whiteboard coding interview, we're asking a software engineer to prove that they have mastered the most complicated devices humankind has created by drawing pictures on a cave wall.

We evaluate candidates using a modified "fist five" consensus method. We rate the candidate after each interview according to this scale:

• A zero (or a "fist") is a block. Blocking a candidate means that you do not wish to see this candidate continue. Either the candidate is clearly wrong for the job, or the job is completely wrong for the candidate.
• A one or a two ("one or two fingers") calls into question whether we should continue the interview process, but leaves the decision up to the hiring manager.
• A three indicates that the interviewer has some reservations, but is willing to support a decision to hire the candidate.
• A four indicates that the interview will support a decision to hire the candidate without reservation.
• A five ("five fingers") indicates that the interview completely supports hiring the candidate and recommends immediate action.

Often we will have more than one goal in the hiring process. In these cases we might also ask for a percentile ranking of a candidate in comparison to the current team.

For example, maybe we don't just need a Scala developer, but one with enough language maturity to guide solid developers on a team who just don't have as much language experience. In that case, we might be particularly interested in a candidate who ranks in the 95th percentile on language skill, and is comfortable pairing.

Perhaps we have specific goals around improving software quality. In that case, we might prefer a candidate who ranks high in the interview who ranks in the 90th percentile for Software Craftsmanship.

At the end of each interview, your interviewers will already be figuring out how to share their impressions of you with the hiring manager. This is an excellent time to ask what their thoughts are.

• Did the interviewers meet their needs for the interview?
• Do the interviewers feel like they got an accurate impression of your skill level?
• Do you feel like they got an accurate impression? If not, be sure to tell them, or bring it up with the hiring manager.
• Will you get the interviewers' recommendation for the job?

Every promising candidate for a job will get at least one interview, and the candidates who receive offers will get seven interviews total.

Let's look at each interview and explore our goals for that interview, along with some ideas for candidates on how to win it.

We don't want to waste the candidate's time, or our time either. The phone screen is a go / no-go interview that determines whether it's worthwhile to bring a candidate in for the on-site interview process.

We want to understand why the candidate is looking for a new job, and why at OC Tanner specifically. Also, we want to make sure the candidate can code.

The best way to win the phone screen is to be an excellent, well rounded developer who knows about the work we do.

Computer programming involves two basic sets of skills: understanding the problems, and knowing how to solve them using computers.

Our problem domain is appreciating people. OC Tanner not only wrote the book, we created the entire field. Chances are good you can relate to the work we do. Most of us like to be recognized for the good we do, and shown appreciation in concrete ways.

There are two sides to a good programmer: discrete skills that help you accomplish specific tasks, and a grasp on theory that lets you generalize your skills to new problems.

A lot of training available to new developers focuses on discrete skills, like creating a web app in Ruby on Rails, or building a user interface in Angular.js. While these are good skills to have, we're going to look for developers who can generalize those skills. Knowing Ruby on Rails is a good thing, but can you do some of the same things in Ruby? What about using another object oriented language like Java? Or a functional programming language like Scala?

The important thing here is not just to know one way to solve a problem, but to be able to efficiently learn other ways to solve problems. This typically comes from studying computer science theory, or from lots of experience.

If you only know one programming language, let's hope it's the one we want you to use on the job. A junior dev needs to know at least that much.

It'd be better to know more than one language though, especially if the job calls for a language you don't know. We'll train a good candidate, especially one who can demonstrate he or she knows how to learn languages.

Writing a new program from scratch is often a waste of time. We want to get boilerplate stuff done within a framework of open source code that other people have written and proven in the field. It's always valuable to have discrete skills and experience with common frameworks in the languages you use. It's often valuable to also understand the patterns they implement, the abstractions they offer, and the opinions they enforce.

In a phone screen, you might be asked about different databases, and it is nice to be able to convey what you do and don't know about them.

Remember, we want to get a feel for where you are in your career so that we can make a good fit for you and for us.

We try to feed every candidate at least once. Some managers prefer to take a candidate to a fancy local restaurant, along with the team of engineers who will be doing the interviews.

I don't. If dinner out is an important part of the recruiting effort, let's do that while we're discussing an offer.

At OC Tanner, we have an excellent cafeteria with an executive chef we recruited away from one of the best restaurants in the area, and while the daily menu may not be a modernist delight, the food is good. More important than the food, though, is that I can make an open invitation to a dozen or more other developers, potential coworkers for the candidate. This gives my candidate a great opportunity to meet the team informally, and gives me a chance to watch how the candidate relates to my team.

The lunch interview is an opportunity for you to learn about us from a diverse group of people working here. Take advantage of that. Ask questions, get to know people. Try to be outgoing.

It's a safe place, and we want you to like us.

We use a variety of programming languages on different projects: Java, Scala, Javascript, Coffeescript, Ruby, and Erlang in my area alone. We want to hire and train developers to be exceptionally competent, often in several languages.

We want to know what language a candidate uses, and how competently they're able to use them. We'll probably look for one or more languages in our list, or similar to the languages in our list. We might also look for candidates who have experience in langauges we think brings a good perspective to their work.

This is a coding interview, and you should expect to have a coding problem to solve. Ideally, you'll do this interview on your own laptop using an environment you're comfortable with, while pairing with one of our engineers.

A lot of places want you to solve coding problems on a whiteboard, as if you're a philosophy professor or a trained monkey. We don't have openings for either in our department. We hire engineers and we want to collaborate with you every day, starting with the day of your interview.

We come up with programming problems from a bunch of different sources, sometimes even our current work. However, if you can solve random problems from ProjectEuler.net, chances are good you can handle our programming problems.

By the time you get through the phone screen, you should have an idea what language or languages we might want you to work with. That's probably not enough opportunity to go learn a new language, but even if you don't know the primary language we're going to use together, it still helps to know at least one similar language and at least two or three languages total. Ideally, you'll be well rounded with an object oriented language plus a procedural language and a functional language.

Some people might not agree with me on this point. We used to act like front end wasn't real engineering, but it's now some of the most difficult programming we do. It's certainly some of the most dynamic programming.

We want to know that you know HTML5 and CSS really well. We'll ask hard questions to test you on this. Also, you'll still need to demonstrate language competentcy in Javascript, understand about shims and polyfills, and show proficiency in frameworks like Angular and React.

As a manager, I'm supposed to communicate with my developers what the problem is and why we're solving it, but not how to solve it. Figuring out the how is engineer's responsibility, so we want to make sure the candidate can do it.

Design patterns are a way of solving problems by seeing how well the problem fits the shape of some known solutions. The idea comes from architecture and the 1977 book by Christopher Alexander called "A Pattern Language". In architecture, a door fits a pattern for entering and exiting a house, and a double-wide French door is an implementation detail.

We have a pattern language for programming as well, and have identified hundreds of different patterns that fit various problems. We want you to know some of those patterns. A junior developer should know the major patterns he or she has used, like a model-view-controller and object-relational-mapper. Senior devs should probably be able to recognize most of the patterns they've used by name, as well as a bunch of patterns they haven't yet used.

Right now, concurrency-oriented patterns and functional programming patterns are particularly interesting to us and to the field as a whole. I'd recommend you learn them.

We want to know whether you're safe around databases, and whether you know how to design and build systems around them. Can you explain why PostgreSQL is a beter open source database than MySQL? (If not, Rob Conery has a great talk on this called "Five things you didn't know about PostgreSQL.")

Can you explain when an ORM is no longer appropriate or performant? When NoSQL is a better choice than SQL? What the CAP theorem is and how to apply it? Why Riak is a better key-value store than MongoDB? Can you make the opposite argument?

We don't have time in a one hour interview to design and build a full stack, highly available, secure, high performance system. We only have time for the fun parts.

What I may not tell you in the interview is that I picked out one of the most important things you might work on when you join one of my teams. I want to see how you approach the problem, and how you related to it. Is this an exciting problem for you? Do you catch on quickly? Do you have experience in the problem domain?

We are improving our quality practices, and while we've got great support from the current members of our team, we also want to hire people who have experience and a commitment to strong quality practices.

Does the candidate write tests? What kinds of tests? Does the candidate write unit tests first or after? Can the candidate red-green refactor? How about acceptance tests and regression tests? Does the candidate know about property based testing or contract testing? How about static analysis? Formal verifications?

Does the candidate pair program? How much? Would they do it full time?

A lot of this interview is about the Extreme Programming (or XP) and Software Craftsmanship principles laid out by Robert C Martin (also known as Uncle Bob). We're kinda hoping you've read these. Either way, there's a good chance you've been training on some of the techniques.

We think the best way to write code is to understand what it's supposed to do, write a test that proves it does it, and then write code that passes that test.

It doesn't actually matter a lot to us whether the test cycle is short term or long term, unit tests or acceptance tests. We want and expect both - short scale, unit test driven development; feature-scaled acceptance test driven development; and bug-scaled regression tests.

Pair programming helps team members learn from each other. It helps keep "institutional knowledge" in more than one head. It distributes accountability. It improves code quality.

We like pair programming and we want to do more of it. We want new hires to expect that, and to participate in making it happen. Some of these skills are aspirational for some of our teams, but we're getting better and part of how we're doing that is by hiring people who bring good habits with them.

Our final interview of the day helps us see how the candidate interacts with the tech community both inside and outside of the office.

How capable is the candidate at receiving help from the community? Does the candidate make use of static resources like tech blogs and community documentation as well as dynamic resources like IRC, Twitter, meetups, and conferences? When he or she describes getting help from someone in the community, do they talk more about the resources, or the people? How valuable is the candidate to the community? Is he or she just a consumer, or also a leader?

None of us got where we are without the help of others, and most of us rely on other people more or less constantly, because we're constantly growing and learning.

Who do you rely on, specifically? There's a big difference between "I Googled this Erlang problem and got a snippet" and "I Googled this Erlang problem and found Fred Herbert's site. He's the guy who wrote 'Learn You Some Erlang' and he works on the routing team at Heroku."

I'm going to enjoy hearing about the other people who help you. Coaches, mentors, community groups, meetups, blogs, twitter accounts, online training courses - the community is full of amazing people helping other people, and recognizing them is an important step in becoming a leader yourself.

Just as a side note, I'd like point out that OC Tanner recognizes and appreciates people. That's our business.

I think Ruby is a wonderful programming language. You know why? Because Yukohiro Matsumoto wanted to optimize for programmer happiness, so he created a language that's easy to think in. Matz was inspired in part by Larry Wall, a linguist who created Perl, a programming language designed to be very expressive. There are poetry contests in Perl.

These aren't just interesting anecdotes. These are some of the reasons you might choose one language or another.

Who created the languages, libraries, and frameworks you like? What kinds of problems were the trying to solve? What kind of elegance and beauty did they bring into the world? If you don't know the stories, you'll never have a full appreciation for the strengths and weaknesses of the languages, libraries, and frameworks you are relying on.

What's your Github username? Twitter handle? Blog URL? Who are you mentoring? Have you contributed to open source projects? How about a major programming language? (My friend Kavita got a pull request merged into Python before she got her first programming job.) Do you coach or mentor anyone? Do you present at meetups, regional conferences, or national conferences? Which books have you written?

These questions scare a lot of experienced programmers, but if it weren't for people who do all these things, we wouldn't be here. We want to see how you see yourself in the community.

If we get to this point, it's because we want to hire the candidate, so we want to write an offer that will make the candidate happy and excited to come work with us. Negotiate transparently as best you can, and communicate a lot. How does onboarding happen? What are the benefits like? What's the drug testing policy? (This one has bitten me several times!)

When you're done with this interview, you should know exactly what the offer will say - and exactly what the candidate will say to it too.

They pay us stupid amounts of money to do this work. It's nice and all, but I couldn't leave tech and go somewhere else and still make anything like the salary I make in tech. Since they're willing to pay us so well, you might as well try to get more for yourself.

Salary negotiation is hard for a lot of people, but here's a tip to make it easier. It isn't your responsibility to make sure you're worth what they pay you. It's your manager's responsibility to make sure you're worth what they pay you. Your manager just needs your help.

Ask for what you want, and let your hiring manager try to figure out whether he or she can make sure you're worth every penny. Research the field, and the area, and the company to that you're in the right ballpark. Ask your hiring manager what the maximum earning potential for the position is. And then ask for the salary you want. Don't hesitate, or waffle, or apologize. Ask.

When you're talking about an offer, take the opportunity to increase your lifetime earning, not just your next salary.

Maybe you need more coding skills, or your need to learn another language. Maybe you want some management experience or leadership training. Maybe you want to do more writing and presenting. Maybe what you really need is someone who can help you figure out what you really need - like a mentor.

What will help you do this job better, and what will help you do the next job better? Propose it as part of your offer.

I've given several variations on this talk, including a version that I gave to the engineers in my department because they are the people who are giving these interviews. I got some interesting feedback.

One day at lunch, for instance, a group of eight developers approached the table where I was eating alone. They wanted to know if the stuff I ask about in the community interview is for real. They asked, "Does this mean I need to use Twitter?" and "Why do I need to know who wrote a library I'm using?" and "Well, I haven't written a book. Am I in danger of losing my job?"

Other people have asked me, "Don't you think this method is a little intimidating?" We talked about famously difficult interviewers for comparison - Google, Disney, Apple. We're interviewing local developers who are also interviewing at these companies, and we're making them offers they want to accept.

I'm finding that this interview process is not intimidating for junior and intermediate programmers. They really appreciate the clear communication and our transparency, and they always leave with good career advice if not an offer. The interview process isn't intimidating for world class developers either, because they expect it. If I came through with an interview that didn't give us all a chance to work together, I think they'd stop paying attention to me.

No matter who is interviewing, I believe it's important to make the process as unintimidating as possible. We're not out to scare people away. We want to attract strong talent at all levels of experience, and with this interview process, we're able to recognize them.

I gave this talk at OpenWest in May, 2015. This is a transcript.

Erlang is a functional programming language, and as it says on the screen, functional programming is right.

That is to say that functional programming is math, which means that we can be concerned about correctness, which means we can prove program integrity.

Contrast this with object oriented programming. Objects encapsulate behavior within the context of a stateful environment. If you change an object's state, you can change it's behavior without changing the interface. You can't reason about what an object is doing unless you can nail down its state.

Here's a neat trick. Let's say you have 24 bytes of state - six 32 bit words. With those six words, you can represent more different states than there are atoms in the earth.

In pure functional programming, we have immutable data. It's easy to pin down data that doesn't change. It's also easy to share it when you know the other party can't change it.

We also have referential transparency - for any given input, our functions will give us the same output. This is super easy to test, and easy to reason about. Also, it doesn't matter where we run one of these functions, we'll always get the same answer. Boom, we have a cluster.

Finally, pure functions don't have side effects. You put a value in, and you get a value out. And nothing else changes. Unfortunately, while mathematics might help us reason about the world, math alone is not enough to change it. For that we need side effects.

Let's look at how Erlang does functional programming.

Erlang does variable assignment a little differently than we learned with imperative languages. In a simple assignment, I tell Erlang that the variable 'A' is equal to one. That's not an instruction, though - I'm not telling Erlang to set 'A' equal to one. I'm merely informing Erlang that 'A' is equal to one.

You can see the difference when I try to tell Erlang that 'A' isn't equal to one anymore, now it's equal to two. Erlang will politely inform me that I'm wrong, that Erlang already knows what 'A' is equal to, and it's not two.

Note the way that Erlang talks about this. "No match on right hand side value 2." It's not an assignment, it's a match. Erlang is looking at the two sides of my equals sign and trying to see how they match. In this case, they don't, so Erlang gives an exception.

The only reason Erlang let me get away with telling it 'A's value was because the first time around, Erlang didn't know what 'A's value was. 'A' was unbound. But then I gave Erlang a logic statement, and from that, Erlang could infer that the unbound variable 'A' had a value.

Erlang will do matching even when it needs to unpack values to do so. However, it's important to note a couple of things. First, the unbound variable needs to be on the left side of the equals sign. Second, bound variables still have to match or you're going to get an exception.

Erlang does a special unpacking job when it comes to lists. If you've ever studied Lisp or Scheme or any of the derived languages, you might be familiar with the terms CAR and CDR, referring to the pointer for the first element of a list, and the pointer for the second element, which carries with it the links for the rest of the list. Erlang refers to the Head and Tail of a list similarly. Head refers to the first element, and Tail gets you the rest of the list.

Erlang does the same kind of pattern matching and variable binding when defining functions. If we define a function with an argument that's constant, like zero in this example, Erlang will only recognize the function when there is a match on that constant. Anything else will crash and burn.

Just as an aside, this example returns a value called 'ok'. That's not a built-in value or anything, it's what Erlang calls an 'atom'. Erlang's atoms are a 'literal' data type, where the name and the value are the same. In Ruby, a similar data type is called a symbol.

This pattern matching business becomes more useful when we add in Erlang's conditional dispatching. We can define the same function name with the same number of parameters in several different ways, and Erlang will use only the one that matches.

In this example, we're defining three different versions of the same function, one for when the input value is zero, one for when the input value is one, and one for when the input value is greater than one. The "greater than one" version of the function is particularly interesting. In that case, we're binding the value provided to a variable, but we're also using a guard that says this version of the function only matches when the value provided meets a given condition.

Just like our 'zero_is_okay' function, if we provide an input value that doesn't match any of our declared functions, we're going to get an error than we don't have a matching function.

While Erlang provides a very rich assortment of test environments, the easiest way to write a unit test is just to create a match that should be true. Erlang's test libraries will catch the exception if the statement turns out to be false.

Erlang is concurrent and uses message passing to communicate between processes. It uses pattern matching to handle incoming messages from other processes. In this example, we send a process we know as Marco a message that is our PID and the atom "marco". Then we wait for Marco to reply with a tuple of the atom "polo" and the direction we should move. Erlang is going to ignore any messages that aren't a tuple starting with the "polo" atom, but when the tuple does start with "polo", it will take the second part of the tuple and bind it to the variable Direction, and return the value of that variable.

Erlang's support for binary operations is extremely powerful, and it is also based on pattern matching. In this example, we are binding a base-16 number to a variable called "Color". You might recognize this color as orange. But then we ask Erlang to unpack Color as a bitstring - that's what the double brackets do - on 24 bits of space and bind each byte to a variable. Now we have the Red, Green, and Blue components of that color. This kind of pattern matching in Erlang is tremendously useful for serializing and deserializing data, no matter where it's coming from.

That was a quick overview of Erlang's version of functional programming, but in just a handful of examples, we've looked at immutability, referential transparency, pattern matching, recursion, function definitions, multiple dispatch, message passing, and bitstrings. That's more than half of Erlang's syntax in about ten minutes.

Notice how every single code example relied on pattern matching in some way?

I do have a confession to make. None of the code in this presentation works as presented. The code is all good, it's just that I've mixed expressions with statements. Erlang's REPL, the erl shell, lets you type expressions but not statements. Erlang programs are pretty much all statements, mostly statements defining functions. So in order to run the code samples, you would have to type the functions into a module and then compile and load the module so that you can reference them in the shell. It's awkward. Actually, the Erlang shell also offers a way to do this, but in my opinion, it's even more awkward.

Concurrency

Let's move on to concurrency and nail down some definitions.

Distributed computing is when our concurrent processes work together to solve a problem.

Concurrency is when more than one process is running. We do this all the time at an operating system level, and often do it at a programming level.

Symmetric multiprocessing means that we can schedule processes and threads across more two or more processor cores.

A process is a sequence of programmed instruction, either single- or multi-threaded, that a computer executes.

A thread is a single sequence of programmed instruction that a computer executes sequentially. Multi-threaded processes execute more than one thread at a time.

In a lot of languages, we consider threads to be lightweight and processes to be kinda heavy. Threads can get started up quickly but processes are slow. Threads tend to share memory and processes tend to be isolated from each other. Threads use a little memory and processes use a lot.

Erlang is multi-threaded, but Erlang threads act more like processes so that's how we refer to them. Erlang processes share nothing, and can only communicate with each other by sending messages. They're isolated from each other, so if one process dies it won't take out other processes. Many people say Erlang feels like an operating system because processes are as isolated in an Erlang program as they might be running from a Unix shell. In fact, Erlang is more like a bunch of Unix shells. Erlang processes behave the same regardless of whether there is a machine boundary, so it's trivial to scale from one server to many.

On the other hand, Erlang processes still act a lot like threads, and this is one of Erlang's greatest advantages. Processes are tiny, using as little as 618 bytes of memory. It's not unusual to run tens of thousands of processes on a single machine, and it's possible to run millions.

Joe Armstrong, one of the creators of Erlang, has said that Erlang is more object oriented than most modern object oriented languages. I asked Robert Virding, another one of the creators of Erlang, about this and he rolled his eyes. But Joe may be right. Alan Kay came up with the term Object Oriented Programming, and he's also said that he's sorry he coined the term "objects" because the big idea is "messaging", not "objects".

Either way, Erlang's primary abstraction is not the object, it's concurrency. Joe Armstrong calls it Concurrency Oriented Programming, and says that Erlang is a Concurrency Oriented Programming Language.

When we're programming in a concurrency oriented programming language like Erlang, we can some nice benefits.

• It's easy to distribute our systems across multiple machines. We just start the processes up somewhere else.
• It's easy to scale a system simply by adding more machines and redistributing our processes.
• It's easy to make our systems fault tolerant by breaking them up across multiple machines and making some of our processes into supervisors that watch over the other processes and clean up after them if they die.

Erlang also comes with the Open Telecom Protocol, or OTP for short.

The name of this library is possibly one of the least fortunate parts of Erlang's history, at least outside of Ericsson where Erlang was originally developed. It reinforces that Erlang comes from the telecom industry and suggests that it's a tool primarily for telecom. When newcomers hear that most Erlang programmers use OTP, it suggests that most Erlang programmers are doing telecom work, or work that's very similar to it.

Erlang is not limited to telecom work. OTP is not limited to telecom work. Just saying, they abstracted telecom out of the library, they probably should have abstracted it out of the name too.

As long as I'm complaining about OTP, I'll add one more. Christopher Alexander's book A Pattern Language came out in 1976, and the Gang of Four book on software patterns came out in 1995. OTP was open-sourced in 1998, and Erlang's developers at Ericsson failed to predict that what they called behaviours, the rest of the world would call patterns.

Aside from that, OTP is great. These are solid libraries with nearly two decades of proven use in large scale, highly concurrent applications. Use it.

Let it crash

In a talk called "Systems that run forever, self-heal, and scale", Joe Armstrong lays out six rules for creating a fault tolerant system. Spoiler alert: Erlang is designed around these.

While Erlang does hot code swapping and the Erlang ecosystem has strong support for fault tolerant data persistence, both of these topics are out of scope for this talk.

I'm pretty good at scope creep though, so I will add just a couple things here.

On the surface, Erlang's hot code swapping is cool, but feels like something they forgot to update over time. Erlang lets you load code in two versions, the running version and the new version. You switch from the running version to the new version by just calling the code. It's an implicit reload. When you want to run explicit code swapping, Erlang has some functionality that'll help with a lot of that. It's a little bit like Ruby's Rake, but it's also sometimes referred to as the 9th Circle of Erl.

So yeah, Erlang isn't perfect. In fact, a lot of people complain about Erlang's tool chain. As it happens there are a lot of incredibly powerful tools for Erlang, but the community around them is small and it can be hard to find and learn them.

Fault tolerant data storage is solid, though. Besides the Erlang-specific ETS, DETS, and Mnesia data storage options, my two favorite NoSQL databases are written in Erlang - CouchDB and Riak. Plus there's RabbitMQ, which is also written in Erlang, and while a message broker is very different from a database, RabbitMQ is a fault-tolerant, stable method for storing and distributing messages.

Isolation and concurrency are the foundation for Erlang processes, as we've already discussed.

When I started thinking about which features of Erlang are idiomatic, which features are so integral that removing them would denature the language, concurrency was first on my list. Joe Armstrong talks about Concurrency Oriented Programming Languages and a style of programming he calls Concurrency Oriented Programming. Robert Virding teaches about the fundamental difference between concurrency through shared memory, the way most programming languages do it, and share nothing concurrency. Erlang, of course, shares nothing and this is the foundation of isolation in the language.

Every program has bugs. Extremely stable systems don't expose their bugs during the useful lifetime of the software.

Erlang does it different. It crashes. All the time.

Marvin Minsky, the founder of the MIT Media Lab, wrote a paper once called "Why programming is a good medium for expressing poorly understood and sloppily formatted ideas." It's a lot harder to know what we're supposed to do as programmers than it is to tell computers how to do it. We can specify the parts we know pretty easily. It's the vast universe of unknowns that bite us in the ass.

I have a guideline I use for programming that I call "de Raadt's First Rule". I'm pretty sure I heard it from Theo de Raadt directly, but I haven't found anyone else attributing this to him so maybe I have it wrong.

Theo de Raadt is the founder and leader of a couple of highly secure software development projects, OpenBSD and OpenSSH. He taught me that the first rule for writing secure software is to make it do what it's specified to do, and only what it's specified to do. Security exploits rarely take advantage of properly functioning code; they exploit software that's doing something it wasn't supposed to do.

In some ways, you could say Erlang embodies this approach. Here's another quote from Joe Armstrong: "Exceptions occur when the runtime system doesn't know what to do. Errors occur when the programmer doesn't know what to do." In Erlang we can code to the specification, and simply let anything outside of that specification be an exception that crashes the process. We don't have to defend against every exceptional condition; we can simply tell Erlang what's acceptable and let everything else crash.

If it's not right, we can let it crash.

Erlang's strengths

To be perfectly honest, I've had a hard time wrapping my head around Erlang. I got the syntax but missed the gist.

I've heard about the telephone switch at British Telecom programmed in Erlang that reportedly achieved nine nines of uptime. Impressive, right?

I've heard about the case study that says Erlang programs are 80% smaller than the same functionality written in C++. That also sounds amazing, like I can do five times as much in the same amount of time.

I also knew that Erlang came from Ericsson and the telecom industry. With amazing pattern matching in a functional programming language, it seems like Erlang would be a great choice when I'm doing heavily network-oriented programming. An API router. A proxy server. Messaging.

I had trouble seeing how amazing uptime and code base reduction fit into the bigger picture. It's like this: a big part of the savings (27% in the study I just referenced) comes from coding for the successful case. And a big part of the uptime is that we let it crash when we don't get the successful case. It's what we do after the crash that makes the system fault tolerant, and not just incredibly stable. And sure, Erlang is good at network programming, but that is a side effect of Erlang being good at a lot of things.

Now I see that this is the big picture, the parts of Erlang that make it Erlang.

At the lowest level, we have Erlang's functional foundation with immutable data and program flow from pattern matching for the successful case.

A layer up from that we have Erlang's concurrency, with supervisors watching workers and cleaning up any messes they leave behind, and OTP providing boilerplate implementation of our most useful concurrent patterns.

Finally, there is Erlang's "Let It Crash" mentality - an acknowledgement that failures happen, and we can deal with them.

The world is a stage and we are all actors upon it. We talk, we grow, we fail, we heal. We're hopelessly separate, but we find ways to connect. We can't change the past, but we can take care of each other.

And that is how we program in Erlang, too.

As always, I write my presentations out in advance, in long form. I think of it as my "ideal transcript" - what I meant to say before I actually started stuttering my way through it. I'm sorry to say this conference did not record video.

You can also see my slide deck.

Hi. I'm Version2beta. That's 'version', the number 2, 'beta', and it's short for 'Life, version 2 beta'. It's my handle most places online.

In meat space I'm also known as Rob, and under a few select circumstances, Dad.

My contact info will be on the last slide too, so don't worry if I zip past this one.

Smell is strong.

I don't mean the smell in this room. I mean that olfaction, our sense of smell, is a powerful sense. It's potent in the moment, sometimes overwhelmingly so, but it's also tightly coupled with our memory - especially our emotional memory. Very often a certain smell will be accompanied by a feeling of safety or fear, love or anger.

I still remember the way this girl in high school smelled. We were at a forensics competition, regionals, and we both got knocked out before lunch. We weren't going to state. Instead we went to the gym, where we played this backwards egg toss game. Every time we caught the throw, we'd move closer together. I still remember how she smelled. I also remember how it made me feel, but we won't get into that.

Turns out we have this thing in our DNA called a Major Histocompatibility Complex, and we can smell it in each other. Histocompatibility is a quick way for us to tell - by smell - whether we have compatible immune systems. My wife has known about this for years and readily admits the sniff test was the first thing she did when we met. I am very pleased to report that I passed.

Despite what we say about code smells, this sniff test may be better for selecting a mate than for selecting an open source library, or even an open source library programmer. But keep this smell stuff in mind. This presentation is about connecting with people authentically, and sometimes that means getting close enough to smell them.

Let's do that now. Take a moment and smell someone near you. Really - I'm not kidding. Smile nervously if it helps. Say something awkward - that's how most people do it. But one way or another, turn to someone near you and smell them.

This presentation is about forming authenticate connections with other programmers, specifically the ones who write the code libraries, modules, plug-ins, add-ons, etc. that you use every day. It's about finding new programmers to connect with and using those connections to help identify new libraries you need.

I'm not suggesting you adopt an open source developer. That sounds patronizing.

I'm not suggesting you befriend an open source developer. That sounds pitying.

I am suggesting we recognize that we are already in interdependent and mutually valuable relationships with the developers of the libraries we use. I am suggesting that we honor those connections.

We won't do it because it's a good idea. We will do it, and we are doing it, because it's part of making what we do and who we are into a single, holistic experience.

Let's shift for a moment to software, to picking out an open source library.

Did you know that Drupal.org lists over 23,000 freely-available modules? Let's say your employer wants you to be an expert on all the available Drupal modules, and is willing to let you spend a whole ten minutes learning what makes each module unique and useful. There's job security in this project - it will take you two years working full time to get through them all, not including the new ones that are released while you're working on the project.

Do you use Wordpress? You have more than 26,000 plugins to choose from, just at wordpress.org. Nearly 4,000 of these call themselves a "widget". Surely there should be a plugin for almost anything you need, but finding it is a needle in a haystack problem.

Are you a javascript programmer? As of today, there are over 37,160 libraries in the npm registry. Need a library for logging? There are nearly 2,000 of them.

I'm a Python programmer, and I have five open source libraries I wrote and maintain in PyPi, or "the cheese shop". But the cheese shop has almost 35,000 libraries, so for every one of my libraries, there are nearly 7,000 maintained by other people. Or not maintained at all.

I used to be a Perl programmer. Thankfully I do not have a job that expects me to know all of the Perl open source libraries. CPAN, the Comprehensive Perl Archive Network, indexes almost 125,000 modules.

When we're looking for a code library, we pay attention to several characteristics.

We need to know that it will do what we need. Those are typically features of a code library.

We might consider whether the project is actively maintained. Commit frequency is useful for this. A project that hasn't been updated in a while is probably not actively maintained. Or maybe it's stable and doesn't need to further development.

We'll often rely on our peers to show us which cliff to jump off, so we might look at the number of times other people have downloaded a library, or the rating other people have given it, or maybe just how many people favorited the library.

We want to know that the programming is reasonably well done, so maybe we'll evaluate the code quality. If we have time. After all, if we rely on the library and the developer doesn't maintain it, we might be stuck fixing the code.

When we choose to use a library, we are also choosing to create new interfaces, new edges, in our code and our process.

Just to clarify, most of the time when I say edge I'm referring to graph theory. You see it all the time in network diagrams. You have nodes connected by lines to other nodes. The lines are called edges. How and where they connect are interfaces. What's true in mathematics is almost always true in life too, so when you hear me say something like "All the interesting stuff happens at the edges," I'm not limiting myself to network graphs.

Okay, back to coding.

• There is an edge between our code and someone else's code.
• There is an edge between our coding practices and someone else's code.
• There is an edge between the needs of our project and the needs that the library was written to address.
• There an edge between ourselves and the developer who wrote the library.
• There is an edge between our needs and the developer's needs.
• There an edge between our expectations and the developer's expectations.

When things go well, the edges mesh together beautifully. But when they go poorly, we get the kind of grinding you hear from a teenager in a car with a manual transmission. Instead of interfaces, the edges represent conflicts.

So we have gears that aren't meshing. We have interfaces that have crashed into one another. We have conflict in our code, in our needs, in our expectations, or even between us and the developer. How do we resolve that conflict? Or a better question, how do we avoid it?

I've had the opportunity over the past decade or so to watch a number of Bollywood films. If it helps, take a moment and picture me dancing around the living room, singing along in Hindi.

My wife discovered this interesting difference between Bollywood films and American films. In American films, when the guy first sees the woman, he sizes her up with that cultural male gaze, and he judges the potential for a hookup. In Bollywood films, when the guy first sees the woman, he also sizes her up with that male gaze, but he judges her potential as a wife. He pictures her in her wedding gown, all hennaed up. He pictures her feeding him, being subservient to his mother, being gal pal with his sister. If it's a particularly racy Bollywood film, he'll picture her in his bedroom with a single bare shoulder.

Believe it or not, when we select an open source library, we do the same thing. We project the satisfaction of our needs onto the object of our desire. I mean the code library of course. The library - and the woman in the movie incidentally - is really just a reflection of ourselves. And like the man in the movie, we are often disappointed once we have real experience with the code.

Therein lies the conflict - we have expectations, and they might not be compatible with the code. The resolution of these conflicts comes from understanding the developer's expectations as well as our understand our own. And that understanding comes from connection, which in turn comes from trust.

The really tough question is, who can we trust?

We can often understand someone's motivations and expectations if we understand their needs. Abraham Maslow helped us organize those needs into a hierarchy. Conveniently and memorably, it's called Maslow's Hierarchy of Needs.

Since we're relying on Maslow's work, let's quickly introduce him. If there's one thing I hope you take away from this presentation, it is to learn about the people upon whom you are relying.

I'm sure you've already heard of Maslow's Hammer. He originated the phrase, "it's tempting, if the only tool you have is a hammer, to treat everything as a nail."

Abraham Maslow grew up in Brooklyn, the oldest of seven children of Jewish parents who fled Russian persecution. As a child, he was bullied mercilessly, much of it anti-Semitic. A psychologist at the time classed him as "mentally unstable". As an adult he became one of our most important research psychologists, teaching at a slew of excellent schools, including Brandeis, Brooklyn College, the New School for Social Research, and Columbia University. Throughout his life, he had a horrible relationship with his mother, which may have contributed to his interest in understanding people's needs. Maslow had a massive heart attack while jogging and died at age 62. I was not yet a year old when that happened.

Maslow created a pyramid describing human needs. At the bottom we have our basic physiological needs, like eating and drinking, peeing and crapping, breathing, sleeping, and sex. Once our physiological needs are met, we are better able to focus on our higher level needs, like being safe and healthy, having a secure home and job, and maintaining a moral code. After we've addressed these needs of our body and our future, we can truly start to work on our sense of belonging - having friends, intimacy, love, and family. Once we have stable social structures, we turn inward and work on our self esteem, self-respect, and sense of achievement. Finally, after we've found ways to satisfy all of these needs, we address our need to be self-actualized, to accept our role within reality and our responsibility to help create the world we live in.

Generally speaking we can use Maslow's hierarchy to understand ourselves and other people. We are all human, we all have needs, and we all need each other. We need each other.

Let's get back to some of the interesting stuff that happens at the edges.

Where there is an edge, there is potential for conflict.

What we need instead of conflict is connection. This is well demonstrated by Maslow's hierarchy. We need to know we belong. We need to hold ourselves in high esteem. We need to be self-actualized. When we work, we are actively meeting these needs and others. When a developer writes a library, she is also meeting these needs.

When we use a library and connect with the person or community behind it, we are meeting these needs together. This builds the connection that replaces conflict. It is the foundation for understanding and trust. It is self-actualization. One-to-one, it is us creating the wholeness of our selves.

We each are part of larger social networks. Sure, this includes Twitter and Facebook and LinkedIn, but it also includes our friends and lovers, our coworkers and cohorts.

Quick review of graph theory: the connections between us are edges, and we are each a node. You might find it interesting to know that "node" comes the Latin word "nodus", which means "knot", as in your node is the spot where the edges are tied in a knot. That's how it goes: You are in the network. You are surrounded by edges. You are tied up.

Robin Dunbar - Robin Ian McDonald Dunbar - has put together some really excellent research on how connected we are.

Dunbar is an anthropologist and evolutionary psychologist, which is pretty much what I want to be when I grow up. Unless I become a farmer instead.

Among other achievements (like being a department head at Oxford University) he has a number named after him. It's called Dunbar's number, and it basically represents an approximate number of people with whom we can maintain stable social relationships. It's suggestive of our cognitive limit to connect. Dunbar's number is commonly accepted as 150, meaning that we can maintain stable social connections with that many people.

In reality, Dunbar's number isn't a single number, it's a range from about 120 to 230. According to his most recent research, the actual value of the number for any given individual depends on that person's "social brain" - specifically, the orbitomedial prefrontal cortex. In fact, Dunbar's 2012 paper has demonstrated a correlation between the size of an individual's orbitomedial prefrontal cortex and the size of their social networks.

My wife and I often talk about our children's underdeveloped prefrontal cortices, and how it's our responsibility to provide that capacity for them. But what does our prefrontal cortex do?

First and foremost, as demonstrated by Dunbar's research, the prefrontal cortex does our social information processing. It helps us keep track of the people with whom we are connected. It's where our theory of mind lives - the ability to recognize that other people have their own experiences, and that their experiences are different than our own.

It's also where we do our planning, both short term but complicated motor planning like tying a shoelace, and long term and even more complicated planning, like me becoming an anthropologist and evolutionary biologist. Or maybe a farmer.

The prefrontal cortex also houses our working memory - the mental stuff we're actively chewing on.

If you're still listening to me, that's also because of your prefrontal cortex. It's responsible for our attention.

Far from least, the prefrontal cortex is a processing center for language and symbolic thought.

Here's something interesting to point out. Every one of these functions are also important to us as developers. We use the same mental skills keeping track of code objects and passed messages that we use keeping track of social networks. We are constantly planning, whether it's keeping track of the current sprint or the next line of code we're going to type. Figuring out that next line of code totally taxes our working memory, and getting it down on the screen is all about language and symbolic processing.

Contrary to the stereotypes, we developers are all about the prefrontal cortex. We are jacked into that network. It's not just how we relate; it's where we live.

Want to hear something really interesting? The prefrontal cortex, specifically the orbitomedial region, is also responsible for olfaction. Our sense of smell.

Feel free to smell your neighbor again.

As you might imagine, there has been a lot of recent work into figuring out what connections are worth.

One of the earlier rules came from Robert Metcalf. You might recognize that name. He graduated from MIT with two bachelor degrees about a thousand years ago, the year I was born. He was so impressed by Arpanet (a predecessor to our current internet) that he made it the topic of his PhD. dissertation at Harvard. They flunked him, and then he went on to co-invent ethernet and start a company called 3Com.

Metcalf's Law says the value of a telecommunications network is proportional to the square of the number of connected users.

David Reed was also educated at MIT, just a couple years behind Robert Metcalf. In the 80's he was an assistant professor there, and since then he's been a visiting scientist at the MIT Media Lab, America's foremost school of wizardry. David Reed invented UDP, though he finds that statement a little absurd since what he really did was co-invent TCP and then decide it might be nice to make a simplified version too.

Reed's Law says that the utility of large networks, particularly social networks, increases exponentially with the size of the network.

Rod Beckstrom is a relative youngster, only 52 years old. He's a Stanford grad and spends a lot of time thinking about organization theory. For a while, he directed the National Cyber Security Center, but he quit when the NSA decided they were in charge. He's also run ICANN, the Internet Corporation for Assigned Names and Numbers, and he's on the board for trustees for the Environment Defense Fund. One of my favorite things about him is that he presented Beckstrom's Law at DefCon, the world's largest underground hacking conference.

Beckstrom's Law says that the value of a network equals the net value added by each user's transactions conducted through that network, multiplied by the number of users.

So at least these three experts agree that networks get more valuable as they grow, and the value increases non-linearly. The more connections you have, the bigger the payoff.

This value is not retained solely by the people who own the networks. If that were the case, we wouldn't use their networks, would we. We use Facebook and Twitter and LinkedIn and Github because they provide value to us. And the value of the network to us personally goes up as our network grows, typically at least until we get somewhere around Dunbar's number.

I'm going to back this up with anecdotal evidence.

I'm not a particularly strong programmer, but for over two years I've made a point of trying to get better in part by knowing the people who write the libraries I use and put on the podcasts I learn from. About a year ago, one of those people heard I was looking for a job and said I should come interview at the place he works.

I said no, I'm not a good enough coder.

I worked my ass off for nine months, learning to be a better coder, and then I tweeted back at the same guy. He took time out from an embedded Javascript conference he was attending to talk with me on IRC. He got me an interview with five engineers at their place - the kind of place where the engineering team decides who they want to work with. That got me a plane ticket and two days of pairing at the office. That got me an offer that met my every need, including moving halfway across the country to one of the most beautiful places I've ever lived.

Now I'm working at a place that let Github, Valve, and TED inspire their employment manual. I make enough money to live the way we want to live. I don't have set hours. I don't have set vacation - if I need time, I let my coworkers know and they cover for me. I pair practically every day. I get a day a week to work on my own projects. If I think a conference is going to make me better at my job, or a better programmer, or just more creative and in touch with the world, I just ask for it. And since I'm not a foosball player, I work on my 9-ball skills at the office instead.

I didn't do any of this on my own. I connected with other people - open source developers who maintain libraries of their own - and they helped make it happen. I love my job and I have no interest in looking for another, but I know me and my network are still going places. For instance, this month I'll have my first article published in a programming journal. This is cool for me - I like talking and I like writing, and someday I'd even like to write a book.

Enough about coding. Let's talk about food. Food as a metaphor, at least.

Many of us buy our food from the grocery store. We look at the tomatoes and check out their color and feel their firmness and decide which ones we want to buy based on the characteristics of the fruit. That's the typical way of choosing open source libraries, too.

Some of us are "foodies", though. Maybe you live in Madison, or Milwaukee, or even Salt Lake City like me. I don't want you to go to the store and check out the fruit. I want you to go to the farmer's market and check out the farmers. Get to know them. Go volunteer to work on their farm. Try to understand what they're choosing to grow and why they grow it. Is that heirloom tomato a specialty, or was it a one-time experiment? Where is the parsley? What happened to the potatoes they used to sell? You only know these answers if you know the farmer.

The same goes for your coding. Instead of a physical location, your community is the group of people who work with the same code base, be it Wordpress or Joomla, Python or Ruby or Node.js. Get to know the farmers in your community, and learn which libraries are their staples and which are experiments. Learn what's important to them, and go back to their code base when you're looking for something new.

After all, we're not here at this conference because we want to buy the tomatoes. We're here because we want to become farmers, and you don't become a farmer by studying the fruit in the supermarket.

Like most projects, this one was a collaboration with a number of other people. Many ideas on this topic come from years of conversation with my wife. The place I'm working has been an entirely positive influence on my thoughts. A number of people helped me compose and edit the presentation. Some of them sat patiently through my practice and provided excellent feedback.

I have relied on these people, and I appreciate their roles in my life.

If you want to discuss these ideas further - or you want to disagree with me - please feel free to drop me a line. If you don't mind me quoting you, please tweet at me. This presentation, with a transcript, is already up on my website.

I'd like to thank you for your attention, and I hope the ideas I've presented are helpful to you.

My job includes responsibility for managing development, continuous integration, customer acceptance, and production environments. I wrote an installer that will build any of these environments. It uses Opscode Chef-solo^[1] with a dynamically-generated JSON definition file that varies based on how the environment will be used. For example, every developer makes their own environment on one or more local virtual machines. Each developer can configure that environment using a Python program I wrote that manipulates the JSON to suit his or her^[2] needs.

I made a design decision in creating this environment-builder that impacts our development workflow. Only production environments lock libraries to a given version. In other words, everything but production always gets the latest and greatest versions of all third party libraries.

In an ideal world, this is probably not the right way to do this. An comfortably funded and staffed development team would probably prefer to lock libraries at a "known-to-be-working" version so that devs only deal with problems they've created, not problems some third party open source developer introduced in their latest update. When a new version of a dependent library is released, a ticket gets generated and someone introduces it to a unit testing or continuous integration environment where the new library can be vetted properly, and any required changes to the code can be made.

Or maybe the ideal is to wait for a new release of our software, and then slam ourselves with all the updates at once. Code it with the known-working libraries, then update everything in one fell swoop. Make it, break it, fix it, ship it.

Not all teams can work this way. The company I'm working for right now is funded by accounts receivables, not angels or venture capitalists. We push a lot of code, and when something is billable, it's often done. Refactoring - hell, sometimes even unit testing - is a luxury. I'm not saying we're short-sighted. We want unit tests and have started writing them. We want continuous integration and we've started building that environment. But the primary target is business viability, not code maintainability. And even though I'm writing about current experience, it is worth noting that every development company I've worked for has been funded this way, with many of the same practices.

I know from experience across multiple teams that updated versions of third party libraries get attention only when there's a major reason to include them, usually a bug we can't work around. When we get around to using the new version, it may be one or more major releases ahead of what we've been using. There's a good chance it will be a non-trivial task to deploy the upgrade.

My solution was to create an environment that can be reprovisioned quickly and painlessly. Every time we reprovision, the installer updates virtually everything - except in the production environment, which has its own list of packages that specify versions explicitly. Any developer who reprovisions his or her local development environment (a virtual machine) gets the latest and greatest of everything, and that might break what they've been coding. If it does, they deal with it. Every night, the CI environment reprovisions and runs tests automatically. This may also catch incompatibilities and other issues related to libraries. If it breaks, I deal with it, or find a developer who can.

This system definitely catches issues. An ETL^[3] library we're using is currently borked. Our code works with the library as of three releases ago, but not with any release since. There are five bug fixes, quite a few new features, and some significant performance improvements we're not getting because our code fails with the new versions. We back-burnered that upgrade while a dev scrambles to make the newest version work.

Getting new releases into the workflow improves our overall quality and reduces our technical debt. Mine is a practical solution, pragmatic even. I don't think that following best practices is an ivory-tower endeavor, but I recognize that it's a journey for a lot of small companies, especially when you already have a code base that evolved over years of hard work, often the product of passion more than programmers.

I write my presentations in advance, and then do my best to present what I wrote, so this post is something close to a transcript of my presentation. Actually, it's a transcript of how I meant to give the presentation, without the 'ums' and 'ahs', nervous pauses, misplaced words, and sections I undoubtedly missed due to various deficiencies in public speaking.

Check out the calculator I made for calculating the CRV Factor, which is the hard part of the formula. My slides are inline below, but check out my slide deck online too.

Here are some positive comments I've received about this presentation:

Hi, I'm version2beta. Some people call me Rob. You can call me either.

My presentation is about figuring out how much to charge for your work. This is one of those talks where you get to learn from someone who really, really knows what they're talking about. I've done it wrong virtually every way you can, so I came up with a system that, so far and without fail, has told me how much I should have charged for every project I've underquoted.

It works proactively too. In fact, that is the recommended way of using this system.

I'm going to warn you right up front, there are going to be mathematics in this presentation. Real math, the kind that requires a scientific calculator app on your phone. Or you can use the special calculator I built for doing this math. It's on my website.

Before we get into the details, I'd like to tell you about my first ever paid programming gig.

slide

• Q is quantity
• D is annual demand
• S is fixed cost per order
• H is annual holding cost

slide

This formula is used to calculate the economic order quantity for inventory.^[1]

It's been around a while. I first learned it in 1983, when I was 13 years old. I don't know if it's true or not, but at the time, I was under the impression that my father was one of the creators of this formula. He was inventory control manager for Simpson Paper Company, Eastern Division. His job was to make sure they had the right raw materials on hand, in the right quantities, for all of the plants east of the Mississippi River.

He had this formula that would help him figure out how much stuff to buy. He also had the only IBM PC at Simpson's Vicksburg, Michigan offices. And he had a very nerdy kid.

The formula isn't too bad. Basically, you take the annual demand for an inventory item, multiply that by twice the cost per piece, divide that by the holding cost for one item for a year, and then take the square root - blah blah blah. I know this isn't actually very interesting, but this formula has a square root symbol in it, so it's going to make the formulas I show you in a little while look easy. So it's important you pay attention.

The short story is that I got paid $30 to make this formula into a program.

Superbowl, one year later. Every year the DeYoungs came to my parents' superbowl party. Mr DeYoung worked with my father at Simpson paper. During this particular Superbowl party, Mr. DeYoung pulled me aside and congratulated me on the program I wrote for my father.

He said, "That program saved us $3 million dollars last year." Even as a freshman in high school, I could do math pretty quick in my head. The $3 million dollars was a very significant return on investment for the $30 I was paid.

It seemed possible I had underquoted that job.

slide

Complexity

Risk

Value

slide

I have a formula for helping people figure out how much to charge for their work. It's called the CRV Model. C-R-V stands for Complexity, Risk, and Value.

I designed this model when I worked for my wife Barbara, in her web development company. Barbara spent extensive amounts of time considering how our pricing models, indeed our entire working environments, were inadequate. She deserves a lot of credit for ideas behind the model, and the conversations inspiring it. She was invaluable in developing and testing this model.

I wish I could say it was a great success, but some credit for the model is also due our largest and most risky customer. They demonstrated part of the need for this model by defaulting on $68,000 worth of contracts.

All of my experience using this model has been focused on web development and programming, but it should be useful for modeling any kind of project work. What matters to this formula is that the project is at least loosely specified in advance, and you're trying to put a fixed price on it.

slide

1. BYO Business Acumen.
2. Damn it Jim, it's a model, not a doctor for your business.
3. When you're pointing fingers, make sure you're looking in the mirror. Kthanxbai.

slide

I have some caveats for you:

1. You're going to need some business instinct to use this model.
2. In fact, the more business acumen you have, the better this model will work for you.
3. If you have a good sense of business, you probably don't need this model.
4. If you don't have a good business sense, this model will help you develop one.
5. This is not a silver bullet. It's a model. It helps you understand the world a little better.
6. At the end of the day you still need to take credit for your own success. No need to call me up and say thank you.
7. Same goes for your failures, by the way.

slide

1. Working for the man.
2. Selling your life by the hour.
3. Client wants a fixed price.
4. Client wants to go out of scope.
5. It can be hard to get paid for out of scope work.

slide

Let's talk about why we're here. What is the problem we're trying to solve. I see it as three-fold.

First. It's very important that you hire the right clients. And in case you get a bad client, it's important to know when to fire them.

Second. Even good clients sometimes suck. They want a fixed price bid. Then they want to go out of scope. Then they want to blame you that they went out of scope, that you should have thought of that. Then they don't want to pay for going out of scope. Then if they do pay you for going out of scope, maybe they resent it.

Third. It's come to my attention that I have a limited number of hours in my life. I know that sometimes it feels infinite, but I've consulted with some pretty good doctors, and they assure me I have an expiration date. This may also be true for you. You should consult with your doctor. If by any chance your doctor confirms this, then you, like me, are in a position to decide at what hourly rate we're willing to sell our limited lives. Or if we want to sell our lives by the hour at all.

I think we can distill these problems down to a list of goals for a pricing model.

Pursue the right work. Avoid the wrong work. Mitigate risk. Charge according to value.

So here it is. Scary. It even has a logarithm^[2] thingamajig in it.

Let's walk through it, one parameter at a time.

On the left side, we have the quoted price. This will be the product of our calculation, and the price the model suggests for your project.

C, R, and V stand for Complexity, Risk, and Value. We are going to look at these terms in detail.

I'm going to call these three little factors your constants. Overhead, days, and materials.

The little 'o' is your overhead. This is just enough to cover you basic costs. This will vary a lot based on how big your business is, whether you subcontract, etc.

The little 'd' is your time estimate in days. Not just any time estimate though - I'll show you how that works.

The little 'm' are your material and other project costs. These are project-based expenses other than time.

So that's the formula. It's not as scary as it may have seemed at first.

Add some numbers together. Those are our factors for Complexity, Risk, and Value.

Apply a logarithm. This is the exponent you'd put on the number 10 to get this number. It's also the log10 key on your scientific calculator app.

Multiple by a couple other numbers. That gets us our overhead for the job and how many days of overhead we need to cover.

Add in the remainder. That's any direct or material costs.

You've got your price. Let's look at the specifics.

slide

o : Overhead costs per work day

slide

While Complexity, Risk, and Value are the real heart of this formula, let's get our constants out of the way first. Let's start with overhead.

In cost accounting, our labor is divided into time-based units of measure and we put a cost on each full unit and use that for our basis. That's the system we're getting away from.

Managerial or analytical accounting uses a variety of accounting metrics to answer specific questions, usually questions about the future. That's what we're doing. We're using a handful of metrics to ask questions about the future cost of doing a project.

One of the questions we need to answer is, "How much of my overhead costs should this project pay for?" This is the little 'o' in our formula. Overhead.

slide

Match your preferred beer to your overhead.

• Pabst Blue Ribbon - $50 / night
• Spotted Cow - $75 / night
• Guinness - $100 / night
• Chimay Blue Label - $125 / night

slide

Moonlighters, raise your hand. Now keep your hand up if "overhead costs" and "beer money" are roughly the same for you. I've been there. I still do some work that way. I admit, this example is tongue-in-cheek. If I were serious, it would have been a chart of whiskeys.

I would like to remind you that there are a lot of people trying to feed families doing the same work. Be careful about setting this value too low. I suggest you use the freelance example instead, and think about pricing your work as if you are doing it full time.

slide

• I want to take home about $30,000 a year.
• Taxes are about $7,500 a year.
• Health insurance costs $6,000 a year.
• I spend $1,800 a year on sales and marketing.
• I spend $2,000 a year on new computers.
• I spend $3,000 a year improving my skills.
• I expect to do billable work three days a week.

Overhead per work day = ( 30,000 + 7,500 + 6,000 + 3,000 + 2,000 + 1,800) / 150

Overhead per work day = $415

slide

Let's walk through an example calculating overhead for a single person freelancing. No employees. No employer.

Let's say this hypothetical freelancer is YOU.

• What's the minimum you could make in a year and still be happy freelancing?

• Now take that, and increase it by at least 25% to cover taxes and self employment taxes.

• If you plan to do this full time, add in the cost of your health insurance. If you have health insurance that's paid for by someone else, for instance your partner's employer, add in half the value of their insurance benefit. If you don't have health insurance, add at least $6,000.

• Now add in any business expenses you expect to have for the year. Sales and marketing, professional development, equipment costs, insurance, business rent, professional memberships, etc.

• Then figure out about how many days you expect to actually be doing production work. Divide by this number. If you plan to do this full time, figure on something between 120 and 180, depending on how much time you need to spend marketing, selling, and running your freelance business.

In the example above, I arrive at a "per project day" overhead cost of $415 for this hypothetical freelancer working something that looks very much like full time on a very modest income of only $30,000 a year take home.

slide

• Operating Expenses (including payroll): $72,000 / month
• Employees engaged in production: 5
• Productive days per month: 20

Overhead per day per employee = 72,000 / (5 * 20)

Overhead per project day = $720

slide

Here's another example, this one for a small-ish agency.

For agencies, I think the numbers are pretty simple to follow. Add up all your expenses, including payroll, even for billable resources. Divide it across the number of people doing billable work.

Again, we're not doing cost accounting.^[3] If you're interested, this formula is inspired a bit by Throughput Accounting,^[4] but the important thing to remember is that we don't want to slip into the habit of thinking in terms of hours of labor.

slide

d : Person-days required for the project under the most ideal circumstances

slide

We are not going to try to figure out how many hours or days or weeks a job will actually take. However, we are going to consider how much time it would take under the most ideal circumstances.

I think this is a real selling point for this model. When we're using this model we get to think in terms of the best possible circumstances. This is a huge advantage because we humans can more accurately predict the future when everything goes perfectly, rather than when all hell breaks loose.

So at this point, we can say sayonara to contingencies. We can ignore plans B and C. We can put in the number that only happens when the stars are perfectly aligned.

Just make sure that "days" means the same thing here that it meant when you were figuring your overhead per day. For agencies, this is the number of person-days for production employees that you would assign to the project - under the most ideal circumstances. For freelancers, this is the number of work days you think the project would take you - under the most ideal circumstances. For moonlighters, this is probably the number of evenings or weekends you think you'll need to work on the job. Under the most ideal circumstances.

Again, the important thing to remember here is to be a paragon of optimism. Plan on rainbows and unicorns.

slide

Examples of materials and direct project costs:

• Travel expenses - hotels, meals, mileage, etc.
• Hosting a development environment for the specific project
• Hiring a project manager
• Contracting specialized work
• Buying project-specific training materials
• Software licensing
• Etc.

slide

Little 'm' is easy. It's any direct costs associated with the project, that apply just to this project.

Travel. A hosted dev environment. Specialized software. Contracted project management. Out of pocket expenses. Etc.

slide

Fuzzy Math

Not actually fuzzy, just logarithmic.

slide

Now we get to the heart of the formula. We're going to estimate Complexity, Risk, and Value with some pretty fuzzy numbers.

This is not an exact science. For example, I can't tell you that the complexity on a given project is exactly 6,423.5.

We really only know how complex something is, if it's not complex at all. We can more accurately predict risk when there's very little. And as well all know, value is subjective. One man's trash and all that?

When I make scales for Complexity, Risk, and Value, I want to have a lot of room to move around, and I want the scales to be more accurate on the lower end than on the upper end. I can't really use "infinity" as a value, but it'd be nice to be able to say "A whole damn lot."

So the scale we're using goes like this: Pick a number between ten and ten thousand. If any of the factors start really ramping up, use big numbers. That's what the "log10" in the formula does for us. If you really have a project that is so risky, or so complex, or so valuable that you feel like you need really big numbers to describe it, use them.

slide

• 10 - Anyone could do it.
• 100 - There are only a few people in the area who can do it.
• 1000 - There are only a few people in the country who could do it.
• 10000 - It probably can't be done.

slide

I think of complexity in terms of who can do the work. Actually, I tend to think of complexity in terms of how far you'd have to go to find someone else who can do the job. In that way, it's kind of like "competition".

You could actually use distance as your complexity number. How far is it to the next nearest person or agency who can handle this project? If bunches of people in this room could do it, let's say 10 since I really never use factors lower than 10. If you'd have to go to Chicago for another place that could handle it, let's say 120. New York, about a thousand miles. A univerity in Tokyo? 10,000.

My grandfather, who was an engineer for over 40 years, gave me this example.

Gus retires from head of maintenance at the local factory. He put in his time and is ready to go fishing.

Shortly after Gus retires, an absolutely critical machine fails. It's making a horrible noise. Nobody in maintenance knows what to do with it, so they tell the plant manager he's gotta get Gus to come back and fix it.

Gus isn't very happy about this, but he agrees to come look. He walks around the machine. He listens to it. And then he pulls a piece of chalk from his pocket and puts an 'X' on a part and says, 'Replace that.'

Gus sends the plant manager an invoice for $50,000. The plant manager tells Gus to submit an itemized invoice.

Gus sends a new invoice with two line items. The first line item was "Chalk mark: $1." The second line item was "Knowing where to put it: $49,999."

slide

• 10 - It's a cake walk.
• 100 - There are some problems with taking the job.
• 1000 - There are significant risks associated with this project.
• 10000 - That's just crazy talk.

slide

I remember the first time it occured to me that I could change the price of a job based on how little or how much the client actually knows about what they want. You know the kind of the project, where the client is figuring out how it works by seeing everything he or she thinks you're doing wrong? This happened way later than it should have. I had my hourly rate, and it was my hourly rate. Maybe I'd add in a few extra hours to compensate for some unknown. But I could have saved a lot of money and salvaged at least one failed project if I'd only realized sooner that I should always charge more for projects that have more red flags.

Let's talk about risk.

First, I want to say risk isn't for everybody. At the low end of the scale, we have "the cake walk." The job is well defined. The customer knows what they're doing when it comes to executing a project like this. They pay quickly and fairly. Maybe they're even an agency like C2^[5] here in town that specializes in contract talent.

This is good, steady, low-risk work. It's nice to have this kind of work. Many of us would do well to focus on this kind of work.

Or maybe you can tolerate a little more risk, so we slide up the scale a little bit.

"There are some problems with taking the job." Here we start to see some red flags, and we need to take them into consideration.

The specification is weak. The client seems uncertain of what they want. They don't respond well to suggestions. Their ideas seem overly fluid. (Scope creep!)

Maybe the project doesn't have a good cost-benefit ratio. It's unlikely to bring value to the client.

The customer seems overly cost-conscious. They may not have the funds required to manage the project well on their side.

The client talks about the previous contractors they had to fire.

Sometimes the risk isn't with the client. Let's say you're not well established yet. Maybe you don't have the skills you'd need to feel confident completing the job. Or taking on this project that's too big, and precludes your ability to take on other work or care for your regular clients. These are all risks too.

"There are significant risks associated with the project." Now we're getting into some serious red-flaggage.

Maybe the project has no specification, and the client doesn't want to do a discovery phase.

Maybe the client's expectation for delivery, price, and / or quality are unreasonable.

Maybe the client seems to be hitting on you.

Maybe you have little confidence in the client's ability to fulfill their part of the contract. You know, like paying you.

Maybe you don't have confidence in your ability to finish the project. It may be too big, too hard, to many moving parts.

"That's just crazy talk." Projects at this level of risk are the ones we walk away from. These are the projects with abusive clients, or projects we have no expectation will ever finish successfully.

But sometimes the client is throwing money at you in spite of the difficulties, and maybe you are pretty tolerant of risk. You might even specialize in difficult clients with deep pockets. There's money in that.

When you're quoting a project, always first try to drive risk down. Build in a discovery phase and price out the full project after there's an adequate scope. Spend extra time establishing reasonable expectations with the customer, in both directions. Discuss your concerns about their ability to manage the project, and how they may benefit from bringing on a professional project manager just for the project. Get a bigger down payment. Schedule invoicing more frequently, and require payments be current before each phase begins.

The risk you're left with drives up your price, and this is entirely reasonable. It does a couple of very valuable things for you. For one, you might not get the job, and depending on how much risk there is, this may be a Very Good Thing.

If the job has significant, unmitigated risks, give it a "walking away" price. Don't put a lot of effort into quoting the job. Bump up the risk to an appropriate level based on the problems with the job and your less-well-considered quotation, and send it over without expectation.

Then, if you do get the job and things start to head south, you were already being paid extra to deal with it and you can take it on with a smile. Scope creep? No problem - you expected it. Plus you can still try to get paid for it. Stiffed you for the last payment? That's okay, the last payment was the second half of your bonus, not your paycheck. The client is a pain in your butt? You charged the asshole tax up front.

slide

Evaluating value

• 10 - The impact of the project will be at least as great as the cost.
• 100 - The impact of the project will be an order of magnitude greater than the cost.
• 1000 - The impact of the project will be two orders of magnitude greater than the cost.
• 10000 - An Arab Prince has already written you a blank check.

slide

My microeconomics professor at the University of Maine was a fish economist. He told us how he'd been approached by Heinz Corporation way back in 1969 to assess the potential market for cocktail sauce. They asked him for a daily rate, and he gave them a price of $100 a day.

He didn't get the contract. They found someone who'd charge them $1,000 a day.

I've worked on big websites for small companies that cost $20,000. I've worked on small websites for big companies that cost $20,000. When it comes down to it, people want to pay a price commensurate to the value they get.

I told you up front you need some business instinct to use this model. That's perhaps most true when it comes to figuring out the value your client will gain from your work. Understanding that number will do much to free you from hourly drudgery.

Don't do work that has limited value.

If the potential gain is small, your client needs to think small, and treat you small. It makes your job much less enjoyable.

If the potential gain is large, your client is motivated to make big things happen with you, and will support you on the project. It makes your job much more enjoyable, and lucrative too.

It boils down to this: Always. Maximize. Value.

slide

Ideal Cost: o * d

CRV Factor: log10 ( C + R + V )

Calculator: http://goo.gl/bCtQT

slide

Maybe you noticed that two mathematical terms make up the heart of this formula.

I call the first one "The Ideal Cost". It's your overhead rate multiplied by your time commitment. That's what the job should cost you assuming everything goes swimmingly.

I particularly like the concept of Ideal Cost. We humans are notoriously bad at making predictions. Our emotions get in the way. Ideal Cost eliminates some of that problem by removing the complexity of the negative emotional factors. We don't have to think about what might go wrong at this point - we get to consider just the positive. And this is a nice way to think about a new project.

The second term is the "CRV Factor". It builds in supply and demand, mitigates your risk, and keeps your price commensurate with the value of the project to your customer. You multiply your ideal cost by the CRV Factor. The higher the CRV Factor, the higher your total price.

But you can use the CRV Factor all by itself too. Let's look at some ways you can quickly analyze almost any project.

If you haven't already opened the calculator and you've got a computer or tablet device, please follow the link on the screen and you can run some numbers with me.

slide

• Complexity: 1000
• Risk: 100
• Value: 1000
• CRV Factor: 3.32

Hard but worth it.

slide

The project has moderately high complexity, well mitigated risk, and great value to your client. You are a specialist in a valuable field. Your clients are lucky to have your attention.

slide

• Complexity: 100
• Risk: 1000
• Value: 100
• CRV Factor: 3.08

Assholes and superheroes.

slide

We've probably all worked on this project, right? It's a good fit for your skill set, and it's quite worth while for your client, but your client is an ass. Or doesn't have the money. Or needs it next week.

The nice thing about these jobs is this: you get to be the superhero, if you're willing to work under these pressures.

slide

• Complexity: 2500
• Risk: 2500
• Value: 10
• CRV Factor: 3.70

Danger Will Robinson!

slide

Sometimes I feel like every job I quote looks like this. I can't think of anyone else getting it right, but the risks are significant and when it's done, the client isn't going to get their money's worth anyway.

These are the jobs you refactor for your customer. Drive the complexity down. Mitigate more risk up front, maybe by doing a discovery phase. Shift the priorities so there is more value in the final project.

Make this project a better project, or embrace the multiplier of almost four and hope they decide you're too expensive.

slide

• Complexity: 100
• Risk: 100
• Value: 100
• CRV Factor: 2.48

The sweet spot.

slide

I like these jobs: Competition's not too tight, the job has manageable risks, and it's going to be worth it to your client. It's a factor of almost 2.5 times your Ideal Cost. You can afford to hold out for the jobs you want. You enjoy your customers and your work.

slide

@version2beta

http://twitter.com/version2beta | http://version2beta.com

CRV Factor Calculator: http://version2beta.com/static/examples/managing-risk-and-selling-value-wordcamp/crv-calculator.html

Tech support is always free!*

• Free as in beer. Means you buy me a beer and I give you tech support.

slide

I want to remind you that this formula is a model for pricing your work. It helps you understand your client, the project, and your self. Along the way it may make you a better businessperson. It's a framework for thinking about the work you do.

Oh, and use my calculator. It helps.

Hopefully I left a little time for questions?

The next month I attended PyCon 2013^[3] in Santa Clara, California, and through a series of mishaps ended up in a tutorial called "Learn Python Through Public Data Hacking", led by David Beazley.^[4] He started off with a warning that "Learn Python" was the important part of the tutorial title, and that it wasn't a tutorial on hacking public data. I was disappointed, since I felt like I already knew Python pretty well and looked forward to pursuing my new-found interest in civic data hacking. I was in for a surprise. Not only did I learn a ton about civic data hacking in the next 3 hours and 20 minutes, but he absolutely kicked my butt with Python, too.

I started attending the Milwaukee Data Initiative meetups regularly, and I got a brief opportunity to demonstrate some of the code I wrote in David's tutorial. That led to this presentation on exploring data sets from the inside-out using IPython and Pandas. It also led to a conversation with Rafael Santana, a founder of the Milwaukee Tech Engine.[^techengine] Ultimately I presented at a joint meetup for members of the Milwaukee Data Initiative, Milwaukee Tech Engine, and MKE Python.^[5]

This presentation is not about learning Python. In fact, using Python is ancillary to the workshop - only enough Python is demonstrated to work interactively with various data sets. To a limited extent, it is a workshop about using IPython^[6], an interactive Python shell that can be run in the browser and supports rich media like charts and graphs. To a greater extent, it's about Pandas,^[7] a data analysis library for Python. These two technologies are presented in the context of civic hacking, with a couple of provided data sets and some challenges that participants might solve using these tools.

I write my presentations in advance, and then do my best to present what I wrote, so this post is something close to a transcript of my presentation. Actually, it's a transcript of how I meant to give the presentation, without the 'ums' and 'ahs', nervous pauses, misplaced words, and sections I undoubtedly missed due to my numerous deficiencies in public speaking. My slides are inline below, but check out my slide deck online too. Keep in mind this was a 90 minute hands-on workshop, so the slides and this writeup are not the important part. The important part was the people at their laptops learning by doing.

Hi, I'm version2beta. Some people call me Rob. You can call me either.

This is a civic hacking presentation, and we're going to work with a couple of data sets that might pique your interest. Hopefully, just the nature of the data we've collected will suggest to some questions that you'll want to answer for yourself. Just in case it doesn't, we also have some challenges.

It's not a presentation on learning to program in Python, but we are going to use an interactive Python shell called IPython, and a data analysis library called Pandas. So along the way you'll probably learn at least enough Python to start using these two tools.

As you're working with the data and tools, you might be inclined to use probability and statistics. If you feel so moved, please go right ahead. But I do want to warn you that I'm a programmer, not a mathematician, so my utility in that area will be limited. I do understand however that we have engineers in the house who are probably pretty good at the probability and statistics stuff, and as a group I hope we'll help each other out.

I think it's best to start a presentation off with a story. Often that story offers a little bit about the presenter. I'm going to start with a story, but it'll be only a very little bit about me, and much more about two guys named David Beazley and Travis Oliphant. And also a bus in Chicago.

Last March, I went to PyCon, the big conference for people who use and hack on the Python programming language. Through a series of misadventures involving bad network connections and other computing problems, I ended up in a tutorial called "Learning Python Through Public Data Hacking" put on by David Beazley. At the time I didn't know anything about David Beazley, which was clearly a gaping hole in my Python education. Among many other contributions, he wrote the Python Essential Reference, the first and possibly most serious reference book published for Python. It's been called "the best programming book I have ever seen" and "the sweet spot of concise usefulness" by people who like programming books. Other people have said "It makes my eyes hurt" and "This is not a book for learning Python. It's a book for writing Python."

This book is for heavy lifting.

David himself does some pretty heavy lifting, but when he introduced the tutorial, he wanted to make sure we all knew it was much more about "learning Python" and much less about "public data hacking". I was disappointed; I felt like I already knew Python programming pretty well and looked forward to the public hacking part of the presentation. And then, for the next three hours and twenty minutes, David proceeded to kick my ass with both Python and public data hacking. I think I crawled out of the room at the end.

He prepared three challenges for us. The first one came from his own experience, and involved Travis Oliphant.

Travis Oliphant is a little bit indescribable.

I interviewed for a job with Google not too long ago, and the interviewer asked me to rate my skills on a scale of 1 to 10. She anticipated my question on what kind of scale Google uses when it comes to rating your skills, since they have some of the best programmers in the world working there. She described how a three or four indicated the kind of competence where you could use a skill independently, and a six or seven is "expert". You don't get to a ten unless you not only created the technology, but it was probably your PhD. dissertation.

Travis Oliphant is a ten. He has degrees in Mathematics and Electrical Engineering, got his PhD. at the Mayo Clinic, and taught at Bringham Young University for seven years. He's the author and primary developer of the NumPy library for numerical analysis, and a founding contributor to the SciPy library for scientific computing. Right now he's CEO of Continuum Analytics, a company that makes the kind of Python tools I use in my dreams where I think I'm a genius and everyone needs me to save the world.

David told a story about a time when Travis came to see him. Travis flew into Chicago where David's office is, and took public transportation to David's office. The last leg of his trip was on bus route 22.

When Travis arrived, he realized he left his suitcase on the bus. A Chicago city bus.

Mere mortals would kiss that bag goodbye, but not David and Travis. They went up to the office and wrote a quick Python program to poll the city data for all of the CTA buses, track the location of all the buses on route 22, and pop open a browser window whenever one of the route 22 buses approached within a half mile of David's office. Then they'd run outside and check for Travis' suitcase.

It took about eight hours til the right bus came by again, but there was his bag. The driver didn't even blink when these two middle-age white guys got on the bus, grabbed a suitcase, and got back off.

This was our first challenge in David's Learn Python tutorial. Write a Python program to find Travis' suitcase. Some of my code is on the slide, along with a sample of the map I generated.

I don't have any great stories like that, which is why you heard David and Travis' story instead of mine. I'm not going to ask you to connect to the CTA website and find a bus. It was a fun program to write, but honestly, it took me as much time to figure it out as we have for this entire presentation. Instead, we're going to use some of the tools that David introduced in his workshop, and a couple of data sets we put together for you, and solve a few challenges that are focus on the data.

First, of course, we need the tools.

If you've already installed IPython and Pandas, please consider making yourself available to other people as they try to work through it.

Anaconda is a free-for-all-to-use distribution of Python, IPython, and about 93 other tools from Travis' company, Continuum Analytics, and others. It's not the only way to get IPython and Pandas, but in many ways it's the easiest and most complete set of data analytics tools you can install on a Linux, Mac, or Windows computer. Use the address on the slide for downloading, or there are a couple of USB keys going around that have the various versions available.

If you're using one of the USB keys, also grab the data folder. If you're not, follow the data links and get the downloads.

Once you've downloaded the package, run the installer as directed for your operating system. For Mac and Linux users, it's just a Bash script to install. Windows users double click on the executable.

Let's take ten minutes or so and make sure everyone gets up and running. When you've got it installed and working, you should be able to run the command at the bottom to get IPython running within your default browser.

Now that you have IPython Notebook running, let's look around a little bit. Start by clicking the button for a new notebook.

If you've worked with other programming languages, you probably recognize that this is different. IPython is an interactive shell, sometimes called a REPL or "Read-Evaluate-Print-Loop", a program that lets you type in programming statements and see the result. IPython Notebook is a much prettier version that runs in your browser and gives you a bunch of extra tools, like the ability to edit and re-run lines, save your work, refer to previous results, and even make graphics right inside your browser.

Try a simple "Hello World" application, right on the first line. Just type the line on the slide, "print 'Hello, World!'"

If you press Enter, your cursor just moves down on the same line. This is so that you can enter a block of code - more than one line that all execute together.

If you press Shift-Enter, IPython will execute your code and underneath that line, you'll get the string "Hello, World!". That's called "running the cell". Now you'll also have a new line (also called a new cell) to type more code.

Before you type into the second line, click back on the first. This time, instead of pressing Shift-Enter, press Control-Enter. Control-Enter lets you run the line just like Shift-Enter does, but keeps your cursor on the same line. This is convenient if you want to stay on one line of code until you get it right.

You can also see that IPython has drop-down menus and a button bar. The menu lets you do things like save your notebook so you can come back to it later, add new cells inbetween the cells you've already made, and run your notebook from start to finish. The button bar lets you do things like save your notebook, copy and paste, move cells up and down, run your cell, and even change cells to text or headlines so that you can turn your notebook into a document with embedded live code and its output.

Pretty slick, huh.

Okay, that's all the introduction to IPython we're going to do for now. It's that easy. Let's start with Pandas.

Pandas is also easy. First we need to tell Python to use it. That's "import pandas".

• import pandas

Pandas has two main data types, Series and Data Frames. Series are one-dimensional, just a list with an index. This slide has some examples on how you might create a Pandas Series. Try one or try them all.

ds1 = pandas.Series(range(10))
ds2 = pandas.Series(list('MilwaukeeData'))
ds3 = pandas.Series(np.random.random_integers(0,100,10))
ds4 = pandas.Series(np.random.choice(list('AEIOU'), 10))

Data frames are two-dimensional, like a matrix or a table. Each column in a data frame is a series, except that they all share the same index. This slide has some examples on how you might create a data frame. Enter all three of these - we'll reference them again.

df1 = pandas.DataFrame(np.random.rand(10,4), index=range(10), columns=list('ABCD'))
df2 = pandas.DataFrame(
    np.random.randn(10,4), index=list('abcdefghij'), columns=list('ABCD'))
df3 = pandas.DataFrame(
    {'A': list('ABCDEFGHIJ'), 'B': np.random.choice(list('ABCDEFGHIJ'),10), 'C': range(10)})

Now that you have a data frame, here are some different ways to look at the data. Try each of them yourself.

df1.head()
df2.tail(3)
df2.index
df3.columns
df2.values
df2.sort_index(axis=0, ascending=False)
df2.sort_index(axis=1, ascending=False)
df1.sort(columns='A')
df1.describe()

I should point out about this point that this is hands on learning. I'm pulling a Zed Shaw, Learning Code The Hard Way on you. I want you to type it all in and learn it with your fingers, not copy and paste. I want you to remember the input, see the output, and make connections in your head. If you're not seeing it, interrupt me and let's talk through it.

Selecting data from a data frame returns either a series if you select one column, or another data frame if you select a subset of the data frame.

Try each of these selections:

df1['A'] # column A
df3.loc[0] # row 0, where index is 0 - 9
df3.loc[0:3] # the first four rows, where index is 0 - 9
df2.loc['a'] # row 'a', where index is 'a' - 'j'
df2.loc['a':'d'] # the first four rows, where index is 'a' - 'j'
df2.iloc[3] # the third row
df2.loc['a':'c', ['A', 'C']] # columns A and C for the first three rows
df2.iloc[0:2, 0:2] # the first two rows and columns
df2.at['a','A'] # the single value at row 'a' column 'A'
df2.iat[0,0] # the single value in the first row and first column
df1.ix[1:3,0:2] # The second and third row, first and second column
df2.ix[1:3,0:2] # The second and third row, first and second column

loc is location. iloc is integer location, regardless of how the data frame is indexed. ix is cool - it will try what you give it first as labels for the row and column, then as integer locations.

One thing I've learned with Pandas is that there are a number of ways to get to your data. Many of them are wrong and end up looking pretty ugly. It's good to know several ways, so when things get ugly, you can try a different way.

With Pandas, you can also select data conditionally, as in, if the data in a cell meets a certain condition, return it.

Try these selections:

df1[df1['A'] > 0.5]
df2[df2 > 0]
df3[df3['B'] < 'h']

You can use the selection statements to assign values. Try these:

df3.at[9,'C'] = 10
df3['d'] = pandas.Series([9,8,7,6,5,4,3,2,1], index=range(10))
df2.iat[0,0] = 0
df2 = -df2
df4 = df2.copy()
df4[df4 > 0] = -df4

There are of course some statistical tools available. Try some of these.

df1.min()
df1.min(1) # other axis
df1.max()
df1.max(1) # other axis
df1.mean()
df1.mean(1) # other axis
df1.median()
df3['b'].value_counts()
df3['b'].rank()
df1.std()
df1.quantile()

Let's take a break for a moment and talk about interesting data sets before we get into the challenges we have for you. Here's a list of some that I find interesting. What are some others?

• Milwaukee County Transit stops and times
• The address of every parking spot in Milwaukee
• The location of every pothole in Chicago
• The location of every drinking fountain in NYC
• A list of every federal goverment agency

Water data set from https://github.com/milwaukeedata/MilwaukeeWater2011:

dfw = pandas.read_csv('MilwaukeeWater2011/Milwaukee-2011-07-14.csv', index_col=False, parse_dates=[4,5])
dfw.iloc[0]

Some challenges:

• Easy: Total Milwaukee water use
• Easy: Average water use
• Medium: Address with most water use
• Hard: Average water use per day

Parking tickets data set from https://github.com/milwaukeedata/MKE_2012_Parking_Violations:

dft = pandas.read_csv(
    'MKE_2012_Parking_Violations/2012_MKE_tickets_issued.csv',
    parse_dates=[1], index_col=1)
dft.iloc[0]

Some challenges:

• Easy: Most ticketed offenses
• Easy: Most ticketed addresses
• Hard: Worst time of day

I'd like to thank you for listening. As always, my presentations come with a 100% money back guarantee plus tech support that is "Free as in beer", which of course means if you buy me a beer, I'll give you tech support.

My favorite example of this was described in the Ruby Rogues podcast, episode 79.^[1] I believe it went something like this:

five = 7; # 12

This comment is funny, but most comment-based lies are much more sinister. They hide, waiting patiently to spring out and bite the next programmer in the ass. Sure the comments were written in good faith, and they probably documented the system accurately when the comment was written. But code changes, often before we even get out of our chairs, and there are few systems guaranteeing that we keep inline documentation consistent with the code base as it changes.

We can make a commitment to avoiding comments that lie. One of the guiding principles of Python is that computers don't care how the code is written (and whenever I read that, I think of obfuscated Perl). Future programmers working on the code base do care how the program is written, because they have to read it. We have an obligation to that future programmer to make our code obvious.

As it says in the Zen of Python,^[2] "Readability counts."

It also says, "If the implementation is hard to explain, it's a bad idea. If the implementation is easy to explain, it may be a good idea."

It also says, "Simple is better than complex."

Let's take those aphorisms and apply them to our commitment about avoiding comments that lie. So now I want to change what I said above. Instead of "Avoid comments that lie", let's try "Avoid comments."

This is a guiding principal, not a carved-in-stone law. Try to write code that doesn't contain inline comments.

Here are two ways to do that.

Write a test instead

When you find yourself wanting to write a comment, write a test instead. Not just a test, but a spec. Tests don't lie - they either pass or they fail. If you write it as a spec, any future developer on your project (including you in six months - or maybe just after lunch) can view the test and see exactly what the code is supposed to do. Even better, you can see whether or not the code and the spec are still in sync with each other.

Here's a function written in Python. I'm not going to tell you what it does, and it doesn't have any comments.

def matcher(i):
    group = {}
    for element in i:
        sorted_el = ''.join(sorted(list(element)))
        if group.get(sorted_el):
            group[sorted_el].append(element)
        else:
            group[sorted_el] = [element]
            group[sorted_el].sort()
    return sorted([v for k,v in group.iteritems()])

Since this is a public function written in Python, it should actually have a docstring.^[3] I'd give it one like this:

def matcher(i):
    """Group items that have the same letters."""
    ...

Docstrings can lie just as easily as any other comment, and we want to document this code in a way that proves both the code and the comment are correct (or at least both wrong in the same way.)^[4]

I learned Test Driven Development and Behavior Driven Development using Ruby and RSpec,^[5] so in Python I tend toward tests written using Nose^[6] and Expecter^[7]. Here's my test for this function:

class TestMatcher(unittest.TestCase):
    def test_that_strings_are_grouped_by_component_letters(self):
        expect(
            matcher(['abc', 'def', 'aca', 'acb', 'fed', 'edf', 'c'])) == \
              [['abc', 'acb'], ['aca'], ['c'], ['def', 'edf', 'fed']]

So now my code has an example, and the beauty of it is that this is a specification can be proven, over and over again. This test tells me what kind of result to expect, and lets me demonstrate that the code actually achieves that result any time I wish.

One disadvantage of the "write a test instead" option is that the documentation (specification, really) is not in the same place as the code. It may not be in the same file or even the same directory. This can make it difficult to find. So here's an alternative to writing a comment that puts the appropriate documentation in the code at exactly the place you wanted to put the comment. Bonus, it complements the "write a test instead" directive.

Write a function instead

Write a function instead, and name your function for whatever you were going to put in your comment.

Here's an example, this time in Ruby. (Caveat: I'm still learning Ruby, so try not to laugh if I'm Doing It Wrong. Actually, laugh if you want, but please leave me a comment with advice.)

class Matcher

  def initialize
    @my_hash = Hash.new
  end

  def make_hash(list)
    list.each do |e|
      # Add the list element to my hash keyed to
      # the element's component letters, sorted
      key = e.split(//).sort().join()
      if @my_hash.has_key?(key)
        @my_hash[key] << e
      else
        @my_hash[key] = [e]
      end
    end
    return @my_hash
  end

end

See that comment in the middle? It explains what the next six lines of code do, the six lines of code that are making my method long and ugly. Let's refactor that comment into a method.

class Matcher

  def initialize
    @my_hash = Hash.new
  end

  def add_to_hash(a_hash, key, e)
    if a_hash.has_key?(key)
      a_hash[key] << e
    else
      a_hash[key] = [e]
    end
  end

  def make_hash(list)
    list.select{ |e| add_to_hash(@my_hash, e.split(//).sort().join(), e) }
    return @my_hash
  end

end

So now instead of the comment that described what the code was doing when I wrote the class, we defined a new method. Plus, the new method can be tested separately.

We still have a bit of code in the middle of the list.select block that looks obscure and a little confusing. Maybe I feel inclined to comment that, too. Let's make another method instead.

class Matcher

  def initialize
    @my_hash = Hash.new
  end

  def add_to_hash(a_hash, key, e)
    if a_hash.has_key?(key)
      a_hash[key] << e
    else
      a_hash[key] = [e]
    end
  end

  def sort_letters(a_string)
    return a_string.split(//).sort().join()
  end

  def make_hash(list)
    list.select{ |e| add_to_hash(@my_hash, sort_letters(e), e) }
    return @my_hash
  end

end

As I described above, this works in conjunction with tests, so as I'm refactoring my code-in-need-of-comments, I'm also running my tests to make sure that the code meets the spec all along. Here is a test for this code that meets the requirements above, specifying what the code is expected to do and giving an example.

test_data = ['abc', 'def', 'aca', 'acb', 'fed', 'edf', 'c']

hash_data = {
  'abc' => [ 'abc', 'acb' ],
  'def' => [ 'def', 'fed', 'edf' ],
  'aac' => [ 'aca' ] ,
  'c' => [ 'c' ]
}

describe Matcher do
  it "creates a hash that groups strings by component letters" do
    matcher = Matcher.new
    expect(matcher.make_hash test_data).to eq hash_data
  end
end

Stop commenting your code

I've written tens of thousands of inline comments in the last 30 years. I can absolutely promise you that most of them are useless. They are all written from good intentions. Many of them inject humor, or at least personality. Quite a few of them were written to demonstrate how clever I am. Some of them might even help the poor sod who is now stuck maintaining that code understand what I was trying to do. The majority of them are probably no longer consistent with the operation of the program.

My motivations demonstrate why comments are often bad. My personality and attempts at being funny have little to do with shipping code. My clever programming was rarely an asset, and usually just made my code less understandable. And comments that no longer document the code as it stands now do nothing to enlighten the current maintainer. They confuse the issue.

I've been on the other side too, stuck with a code base that may have hundreds of lines of inline comments that made sense to someone at some point, but are useless to me now. Often I want to start from scratch rather than wallow through someone else's wall of text figuring out what's true and what's not.

Using these directives, my code is cleaner. It's not perfect, and I'm still learning, but at least the person who has to read it has a fighting chance.

Give it a try. Stop commenting your code, and see if it doesn't make you a better programmer.

In order to round out the conversation, I also asked Mike Bayer^[1] to review Paul's notes and provide any commentary he thinks may be helpful. Mike is the author of SQLAlchemy, as well as a number of other powerful Python tools.

Paul's notes, and Mike's responses, are interleaved below.

Paul Morel:

SQLAlchemy does not wrap the Boolean column for MySQL so that 1's are True and 0's are False. This means that all code must be updated to change conditions from == 1 to == True and == 0 to == False. Postgres does this right by having booleans while MySQL only has tinyint. The SQLAlchemy class should probably mask this from developer so the Boolean column only returns True, False, and None.

Mike Bayer:

SQLAlchemy does not wrap the Boolean column for MySQL so that 1s = True and 0s = False. This means that all code must be updated to change conditions from == 1 to == True and == 0 to == False.

The SQLAlchemy Boolean type specifically handles the True/False constants, and translates them to 1 and 0 on the backend side, not the public side, for backends that don't have a native boolean of their own, e.g. MySQL. However, if your code deals with all == 1 and == 0, the first thing to note is that the True/False constants in Python are the integers 1 and 0 (try True == 1, type(True).mro to see). The second thing is that SQLA's Boolean type when used against psycopg2 doesn't do any coercion, and psycopg2 doesn't either, but for any type you can add whatever coercion you like using TypeDecorator:

from sqlalchemy import create_engine
from sqlalchemy import MetaData, Column, Table, Boolean, TypeDecorator

class CoerceBool(TypeDecorator):
  impl = Boolean

  def process_bind_param(self, value, dialect):
    if value is not None:
      return bool(value)
    else:
      return None

t1 = Table('t1', MetaData(), Column('boolvalue', CoerceBool))

e = create_engine("postgresql://scott:tiger@localhost/test", echo=True)
conn = e.connect()
trans = conn.begin()

t1.create(conn)
conn.execute(t1.insert(), {'boolvalue': 1})
assert conn.scalar(t1.select()) is True
assert conn.scalar(t1.select().where(t1.c.boolvalue == 1)) is True

trans.rollback()

Postgres does this right by having booleans while MySQL only has tinyint. The SQLAlchemy class should probably mask this from developer so the Boolean column only returns True, False, and None.

I don't understand this statement since this is exactly what the Boolean type does do. Boolean only returns True, False and None, assuming you're using it against a boolean column on postgresql. If you point it at an int, that's the wrong usage. Use a TypeDecorator around Integer for that.

Paul Morel:

SQLAlchemy does not alias the func.ifnull function as COALESCE for Postgres. So, any code using the ifnull method breaks until it is changed to func.coalesce(). You can use func.coalesce with MySQL so it is safer to start with that but any legacy code still using ifnull must be changed for Postgres use.

Mike Bayer:

SQLAlchemy does not alias the func.ifnull function as COALESCE for Postgres. So, any code using the ifnull method breaks until it is changed to func.coalesce().

You can redefine any func. in any way you'd like per specific backend. It's easier in 0.8 which I recommend for new projects:

from sqlalchemy.sql.functions import GenericFunction
from sqlalchemy.ext.compiler import compiles
from sqlalchemy import create_engine, func

class IfNull(GenericFunction):
  name = 'ifnull'

@compiles(IfNull, "postgresql")
def coalesce(element, compiler, **kw):
  return compiler.process(func.coalesce(*element.clauses), **kw)

e = create_engine("postgresql://scott:tiger@localhost/test", echo=True)
assert e.scalar(func.ifnull(None, 8)) == 8

Paul Morel:

MySQL treats Text column types as BLOB. So migrating data from MySQL to Postgres introduces \015\012\ characters in place of line break characters. If the data stored in a Text type column all have lengths under 255, the column can be converted to varchar(255) before trying to migrate the data. If not, the binary characters need to be corrected after migration.

Mike Bayer:

MySQL treats Text column types as BLOB. So migrating data from MySQL to Postgres introduces \015\012\ characters in place of line break characters.

I'm not aware of any reason why that would be true. Whatever data you have in your MySQL BLOB comes out just as it is, and goes straight into the PG TEXT column in just the same way. There shouldn't be any line break conversion behavior.

If the data stored in a Text type column all have lengths under 255, the column can be converted to varchar(255) before trying to migrate the data.

Postgresql supports VARCHAR of any length, including unlimited. So feel free to declare a column as VARCHAR on PG, without a length:

from sqlalchemy import *
import random

t = Table('t1', MetaData(), Column('data', String))

e = create_engine("postgresql://scott:tiger@localhost/test", echo=True)

data = "".join(chr(random.randint(ord('A'), ord('Z'))) for i in xrange(10000))
conn = e.connect()
trans = conn.begin()

t.create(conn)
conn.execute(t.insert(), {"data": data})

assert conn.scalar(t.select()) == data

trans.rollback()

If not, the binary characters need to be corrected after migration.

This sounds like something that's local to your own software.

Paul Morel:

There is no self correcting auto increment feature for Postgres since auto increment is achieved through sequences. Therefore, after migrating data you must update all sequences to the proper position so the nextval() command will return a valid number and not cause key violations. I had to write a routine that would update all sequences to their latest max corresponding value for the table key where they are used.

Mike Bayer:

This is true.

Paul Morel:

Granting rights to a database for a user does not actually do much. You must grant rights on all tables specifically. In addition, if you have auto increment sequences being used, you must also grant USAGE permission on the sequences. While these can all be granted at the schema level, they do require several commands and it is rather opaque when coming from the simplicity of the MySQL security model.

Mike Bayer:

Wow I feel the total opposite, MySQL's model which inexplicably takes into account the "hostname" and weird things like "user@." has never made sense to me. PG's is clear, and there's no need for a per table grant as he says, just do this:

GRANT ALL PRIVILEGES ON DATABASE <database name> TO <username>

The syntax diagram at http://www.postgresql.org/docs/9.0/static/sql-grant.html shows all the options.

Paul Morel:

Database objects in Postgres are all expected to be lowercase. If for some reason, you chose to use non-lowercase values for table or field names you must enclose them in double quotes. So all of you Oracle and SQL Server developers that use all upper or camel case are going to be in for an unpleasant surprise. SQLAlchemy handles this for you but if you have any hand written SQL those queries will need to be updated. If you leave a field or table unquoted it will automagically get converted to lowercase when trying to access the database objects. For a database that strives to not do magic things, this one is sort of puzzling.

Mike Bayer:

Database objects in Postgres are all expected to be lowercase.

Totally untrue. Postgresql will consider a name to be case insensitive if it isn't quoted, that is CREATE TABLE SomeTable (), which causes the symbol "SomeTable" to be represented as "sometable", and it will match any casing in any statement. If you quote the name, i.e. 'CREATE TABLE "SomeTable"', now it's case sensitive and has to be specified as "SomeTable" in any statement.

http://www.postgresql.org/docs/9.2/static/sql-syntax-lexical.html#SQL-SYNTAX-IDENTIFIERS

Quoting an identifier also makes it case-sensitive, whereas unquoted names are always folded to lower case. For example, the identifiers FOO, foo, and "foo" are considered the same by PostgreSQL, but "Foo" and "FOO" are different from these three and each other. (The folding of unquoted names to lower case in PostgreSQL is incompatible with the SQL standard, which says that unquoted names should be folded to upper case. Thus, foo should be equivalent to "FOO" not "foo" according to the standard. If you want to write portable applications you are advised to always quote a particular name or never quote it.)"

If for some reason, you chose to use non-lowercase values for table or field names you must enclose them in double quotes.

See documentation above.

So all of you Oracle and SQL Server developers that use all upper or camel case are going to be in for an unpleasant surprise.

This is an odd statement because Oracle and SQL Server have nearly the same behavior - Oracle instead will fold case to UPPERCASE in the absence of quotes, SQL Server will maintain the casing you specify without quotes but still applies case sensitivity based on whether or not you supplied quotes (which also, conveniently, are the bracket character [] up until very recent versions).

MySQL IMO has by far the most unreasonable and outrageous casing behavior, basing it on the operating system in use :

http://dev.mysql.com/doc/refman/5.0/en/identifier-case-sensitivity.html

SQLAlchemy handles this for you but if you have any hand written SQL those queries will need to be updated.

If your schemas are created using case insensitive names, that is without quotes, then you can refer to any kind of casing convention you'd like in your statements. No need to update hand written statements:

test=> CREATE TABLE FooTable (SomeCrazyCasING integer);
CREATE TABLE
test=> INSERT INTO fooTABLE (someCRAZYcasing) VALUES (3);
INSERT 0 1
test=> SELECT SOMECRAZYCASiNG FROM FOOTABLE;
somecrazycasing
---------------
              3
        (1 row)

If you leave a field or table unquoted it will automagically get converted to lowercase when trying to access the database objects. For a database that strives to not do magic things, this one is sort of puzzling.

Because the unquoted identifier name is not actually uppercase or lowercase, it's "case insensitive". Any case matches it, schema inspection tools will display it as lowercase and it will be represented in the information schema views with lowercase names (this is the one area where the case sensitivity comes back again, when searching through those tables).

Paul Morel:

No simple way to perform upsert like actions in Postgres. MySQL has a handy way to performing updates inline when key violations are detected without having to write a function to do so.

Mike Bayer:

That is true, PG is really lagging on supporting the SQL standard MERGE. Then again so is MySQL which has it's own homegrown partially-compatible version of MERGE.

Paul Morel:

UPDATE queries using INNER JOINs don't work in Postgres. There is a new syntax for joined table updates using the UPDATE ... SET ... FROM ... WHERE rather than UPDATE ... INNER JOIN ... SET ... WHERE. Note the change in position of the FROM and INNER JOIN so no quick search and replace. The FROM also takes no join conditions directly so they all need to get moved to the WHERE.

Mike Bayer:

This is true, and SQLAlchemy has support for UPDATE ... FROM. But to refer to the Oracle background, those of us coming from Oracle just use standard SQL correlated criteria in the WHERE clause, which is what I usually do with UPDATE.

Paul Morel:

INSERT IGNORE is not available Postgres. Sure they have a way to do it but it, again, requires writing yet another specialty function. When we strive to not put too much logic in the database, writing a function is moving away from that goal. There are a lot of pronouncements why IGNORE is a bad idea but there are some current use cases we have that absolutely make IGNORE a far more efficient way to import like data into tables.

Mike Bayer:

INSERT IGNORE is not available Postgres.

IMHO this is one of MySQL's crazy hacky keywords that people shouldn't be using

Sure they have a way to do it but it, again, requires writing yet another specialty function. When we strive to not put too much logic in the database, writing a function is moving away from that goal. There are a lot of pronouncements why IGNORE is a bad idea but there are some current use cases we have that absolutely make IGNORE a far more efficient way to import like data into tables.

Sure, if your codebase is based on that, and you really need to maintain those semantics, write a stored procedure, yup.

Paul Morel:

GROUP BY in Postgres follows the SQL standard requiring all non-suummary fields to appear in the GROUP BY clause. MySQL allows for some laziness here where you don't need to list all the fields unless you feel like it. So, all the GROUP BY clauses had to be revisited and updated in many cases to add missing fields.

Mike Bayer:

Right, this is another MySQL behavior (apparently SQLite does it too) that I'm pretty strongly against, as it produces random results in many situations. I'd recommend strongly against adding large lists of columns to the GROUP BY, as this is not how GROUP BY is meant to be used and it will lead to inefficient queries - you should only GROUP BY the thing that actually needs to be grouped, in a subquery. A great article on this technique is here: http://weblogs.sqlteam.com/jeffs/archive/2005/12/14/8546.aspx.

Paul Morel:

Postgres treats dates as dates and does not accept invalid dates. MySQL stores bogus dates so during the data migration several errors occurred and cleaning of data was required.

Mike Bayer:

Very true!

Paul Morel:

There were also a lot of other little things that Postgres does right and MySQL doesn't. That translates into a lot of little fixes to make syntax "right".

All in all, I think we will be happier with Postgres but it was not a slam dunk converting despite extensive use of SQLAlchemy, which I thought would make the transition naively transparent except for the hand written queries.

We still need to run the performance testing to determine if we get any better performance out of Postgres for our analytics.

A final note from Rob Martin / @version2beta:

I'd like to thank both Paul Morel and Mike Bayer for allowing me to document their conversation here. I've changed almost nothing from the emails each provided me, with the exception of making some characters capitalized and adding some puctuation marks.

I fully expect this discussion will be valuable to other developers migrating from MySQL to PostgreSQL. If you should use this article and wish to add to the thoughts, please leave a comment below.

I.

In my first draft of this blog post, I started by listing what I do to support women in tech. But the reality is, I have no authority in this area, so I'm skipping that.

White people in the 1950's accepted that black people were disadvantaged. Today, not so much. A suprisingly large number of middle class white people believe that they have it every bit as bad as any black person. As a result, it's more difficult for many of these white people to recognize the advantages and privileges they have that others do not.

I believe this is also true for men and women. When oppression of women was practiced (even more) openly, there was no stigma in recognizing it. It was the norm. Now we accept that this form of oppression is wrong, and the result is that we can no longer see it in ourselves. Seeing it means accepting we are not the men we think we are.

II.

There was an incident at PyCon. A woman chose to speak up about inappropriate behavior in violation of the conference Code of Conduct. She has my complete support. I will not for a moment second guess her handling of the situation. The conference leadership also has my complete support. They (specifically the PyCon chair) were deliberate in their actions and consistent with the code of conduct.

Both the woman who asked for support, and the conference chair, have received vitriolic messages. I've seen these. I've heard that the conference chair has received death threats at home. I don't know that is the case. I have seen threats against the woman that scare me.

III.

I have a friend in San Jose. We dated in high school. To be perfectly honest I still have a crush on her.

She visited me at the PyCon hotel bar. Among other things, we discussed booth babes. She's been laid off from a Silicon Valley startup in the hospitality industry. They also did a big annual trade show, in Vegas. To save money they asked her to work the company's booth. There was no question about whether she would be a booth babe. They openly discussed her chest size as an unfortunate compromise. She did not accept their offer to go to Vegas.

IV.

At about that point in our conversation, a woman came up to the bar near us. This is the same woman who later would ask the conference for support. I commented on her in the context of our conversation about booth babes. I said I found her very attractive, and this made me suspicious that she may be more "babe" than "tech".

I texted with my friend yesterday and got her recollection of what I said. Her memory of our conversation did not treat me as kindly as mine did.

[me] Hey remember the woman at the bar with the awesome hair that we talked about being tech or booth babe? That I didn't know who she was, just that she worked for a company I know? She's definitely a tech. ... She's been interviewed by Rachel Maddow. She's invited to speak all over the place, and is apparently amazing to hear. She is in the middle of a shitstorm right now because she called out three men for sexist comments in the closing session at PyCon.

[my friend] Oh my!

[me] She wrote about the incident. She's catching a lot of idiots I guess but also getting, and got a lot of support from PyCon and the Python community. She is going to be one of my heroes very, very soon.

[my friend] Well good thing she didn't hear your comments about booth babes.

[me] Did I accuse her of being a booth babe? I thought I said she was attractive but I had no idea whether or not she has tech skills. That I hoped she wasn't a booth babe. In part because I like the company she works for, and I want to respect them.

[my friend] Ah, I thought you suggested she was a booth babe.

[me] It sucks that the question is relevant to me at all though. It'd be better if people assumed, and I trusted, that any attractive female rep at a vendor booth is technically competent. Not that I assume the men are either. I try to ask who is able to discuss technical details regardless of whom I'm speaking with.

[me] I suspected she might be [a booth babe], so that's close enough.

[me] Anyway having learned who she is and the incredible work she does both in the tech and black communities, I thought I'd share this with you. You may find her post interesting. I knew I had mentioned her attractiveness and questioned whether she was there for those qualities.

V.

"Smile but don't be creepy."

"Don't make sexual advances towards women."

These are two of the instructions given in teacher training for Railsbridge. It's good advice. The point is to create an environment where women can feel safe participating. I've taught only one Railsbridge-style course, and I don't think these guidelines were a problem for me.

There is an interesting side-effect though. It should come as no surprise that a geeky guy, especially one who wishes to help create an environment where people feel safe and are treated equally, might find a geeky woman who believes the same things really quite attractive.

A Railsbridge session is absolutely not the right place to bring up that attraction. It's quite possible that, having worked with a woman in such a session, there is never an appropriate way to approach the topic. I really don't know.

VI.

I once saw author Derrick Jensen speak. I was particularly troubled by his assertion that men have a responsibility to announce and prove on every occasion that they are not a threat to women. That I must be presumed dangerous, and that I have responsibility to demonstrate definitively otherwise.

After I started writing this post, I saw yet another threat against the woman from PyCon. This one was posted on Twitter, and it contained her home address and phone number with a photograph that I would find disturbing even in a horror movie. It seems that Derrick Jensen was living in the real world, and I'm still learning how sheltered I've been. "Check your privileges."

Being male is only one of my advantages. I'm white. I'm heterosexual. I'm big - tall, strong, imposing. I'm married. I have children. I have some advanced education. I have a white collar job. I live in a house. I have a car. I have connections. I come from a "decent family". I'm sure the list goes on.

To be perfectly honest, I have trouble living with the messages I get. Sometimes my depression over just being who I am is debilitating. This depression is informed in large part by the degree of truth in the message.

I am the oppressor. Believe me, I can probably overpower you, male or female. Mentally I'm quick-witted and argumentative, and in an argument I can cut you down. Physically I am powerful. My hands alone are twice the size of most - my pinky, for example, is bigger and stronger than my wife's thumbs. My wife tells me I should include here that she is neither small nor weak.

I am the patriarch. I have tried to be a loving husband and nurturing parent, and I've been very successful in many ways. And yet I will power-over my children to get done what needs done.

I am entitled. Not only do I have all of these advantages, not only have I used all of these advantages, but I believe at various times that it is my right to use all of these advantages.

I'm Hulk in a kindergarden. Sometimes I can't turn around without hurting someone vulnerable.

VII.

I've never worked with a woman developer.

I'm not the only one, either. I've been collecting stories about this, asking hundreds of people in real life and online how many women developers they've worked with, and in comparison, how many male developers they've worked with.

These stories will become a blog post eventually. Maybe even a presentation, if I can figure out how I, a privileged heterosexual white man, can get up and talk about the position women hold in the tech community. So far the best I've come up is this: I can call for us to throw our hands in the air and reject the premise that a small number of (mostly) white men should be responsible to create our society and our culture.

But even there, I have looked in the mirror and recognize the enemy.

So then I wonder if I can just go to teach coding in some predominantly minority neighborhood, to anyone who wants to learn. I imagine maybe they won't notice I'm white, and hope that someone from the community quickly takes over and I can play only a supporting role.

VIII.

No one has solicited my help to solve the issues around minorities in tech - at least not beyond speaking up in support. I do not expect to be drafted in the future to solve these question either.

So far as I can tell, I have two responsibilities:

1. I have privilege and power, which puts me in the class that is considerably better able to create change without violence.
2. I have children who are dependent on me, and responsibility to bring them into the community better prepared and with fewer hangups than I have.

I have four children, two from my first marriage and two from my wife's first marriage. Two of them are adult women. Two of them are young men. All of them are dependent in part on me to help them learn how to check their privilege. I continue to try to teach them by example, even as I'm learning this myself.

IX.

I'm going to put it out here for discussion, not because it's enough: "The role of women in tech is relevant to men because we have wives and daughters."

I've heard it many times. I've said it myself. It even has some truth to it (especially if you ignore the patriarchal way it defines a whole group of people by their relationship to some man). I want my wife and daughters to be recognized and valued for who they are, and for their contributions rather than their potential as sexual partners.

I started getting closer, I think, when I realized I want the role of women in tech to improve because I have two sons, also. I want them to have the experience of working with women as equals, to get the benefit of different perspectives and experiences. I want all of my children to live and work in a world that treats women as people.

The homogenous tech environment we have now has not only limited the opportunity for women, it limits opportunity for all of us. The role of women in tech is not relevant to me because I'm a man, it's relevant to me because I am a person. Each of us lives fuller lives when our communities include, and are built by, all kinds of people.

X.

I don't think the woman who asked PyCon to enforce their Code of Conduct intended to be brave. She is brave, and she's proving it repeatedly, but her courage should never have been necessary.

I was at PyCon, and I'm not one of the people she called out. But over a period of three decades, I've contributed to her dilemma. I'm trying to do better. I've undoubtedly been one of the jokers, inappropriate and unprofessional, at points in my career. I've never been one of the people threatening to rape or murder. Sometimes it takes the latter to even notice the former, if you're one of the privileged. I understand women and other minorities find it much easier to recognize our biases.

This hurts me. I've spent two and a half days half-broken in a depressed funk over this specific situation at a conference I just left. I was just there. I was with these people, beautiful and kind and caring for each other. It wasn't all unicorns and rainbows, but I do prefer my peak experiences stay where I put them.

Sexism in the tech community hurts me. I go to work and to tech meetups and I see variations of myself - white men, many of them younger than me these days, building the products and services that I put my heart into. We can't do this by ourselves. We need the diversity of women, of people who weren't raised in the same way I was, thinking many of the same things. When I look at this community, I don't want to see myself. I want to see us all.

Some sexual banter between these guys at a tech conference ended up hurting us all, but none more than this one woman at PyCon it seems. The situation continues to develop. As if threats of rape and murder were not enough response to her request that the conference enforce their code of conduct, she lost her job about ten minutes ago. SendGrid's announcement.

We not only share this world, we co-create it. This means that you and I are responsible for how it has turned out. I've done my share of damage, in spite of my best efforts. I try to do my share of construction too. I'm still looking for my role, and I'm afraid it will ultimately only be clear when it's become my legacy.

Thank you to @KuraFire, @zspencer, @angelaharms, and my wife @senvara for her help in editing this post, and to my friend in San Jose for allowing me to share our conversation. While my post has been improved by their input, I don't speak for them or anyone but myself.

I've decided not to use footnotes or name names in this post since it is not my intention to focus yet more vitriol on the people involved in this story.

On March 24, 2013 I edited sections VIII and IX based on feedback I received in order to better communicate my ideas.

Surely I'll have opportunity to give this talk somewhere. Meanwhile, it's here for you.

My slides are available online here and as usual, this is my "intended transcript" - the way I meant it to be, before I messed it up by actually presenting it.

Data represents reality.

Maybe just a small slice of it, but data is a slice of reality, and it's packed away inside of a database.

Databases model our world. Practically every electronic representation of the real world is in fact some kind of database.

If you use a computer, or are a customer of almost any company, then there is a database that models you.

As models go, databases lack a lot. They tend to store only what they expect to need. They often store that information poorly, trying to squeeze the world into a set of tables like putting an elephant into ballet shoes.

I use my first initial and my middle name. I get squeezed into databases all the time.

Many of us are developers, and we build applications that are database doorkeepers and data validators. Our job is to build that toddler's toy that turns star shaped cylinders into square blocks.

The short-sightedness of a database's design can lead to all sorts of misadventure. I laugh when my wife gets mail that has my name as her last name. Not even my last name. My first name as her last name.

It's funny when we're talking about junk mail, but try standing at an emergency room counter explaining why your insurance card has the wrong name on it.

This is, of course, our fault. Us, the developers and architects and thinkers about this model of our world.

We design databases based on our subjective understanding of the parts of the world we need to model.

Our subjective understanding is informed by culture, and because of this we often encode culture into our database.

Even today it's hard for my wife to create a joint account for us that doesn't list me as the first and primary account holder.

The database distorts reality to fit its limited ability to model the world. As a result, it changes how we describe ourselves. It changes our story. And when we change our story, we change our selves.

That is the tyranny of the database.

I want us to do better. We have the storage space, the performance, the bandwidth, and the programming skill to build databases that are informed by behavior and observation.

Here is a JSON document that represents me.

I picture how this document could be the product of a call to a service. Maybe this service is backed by PostgreSQL, or by someone else's API, or maybe it's flat files. This document doesn't need to be drawn from any one data source at all. We can make it richer by pulling data from other services and merging them into one document. It's no longer a join so much as a mixin.

I call this data Arbitrary. I want to even further decouple the database from the application. Let's make fewer objects and more services that can become richer and better over time.

Let's look again at the document that describes me. What's your app supposed to do with all of that?

Start by using what you can, what you recognize as relevant data, but approach it semantically. Be prepared to look for data in more than one place. Coalesce from multiple sources.

Silently set aside the data that isn't relevant. It's what our brains do, a thousand times a minute, in the real world.

Gracefully accept what you don't know. Write tolerant code that recognizes that data has holes. And when you can, make your software help fill those holes to build a richer, more complete model of reality.

16 years ago my wife used a web authoring tool to build a botanical encyclopedia. That software is long gone. That data is locked up in a bunch of three and a half inch floppies. But the information is still relevant. I'm pretty sure that amanita virosa - the destroying angel mushroom - is still poisonous even if somebody stopped supporting the software.

The botanical encyclopedia she built is legacy data. If we do this right, we can instead build data legacies. Lets plan for that. Lets build rich models that demonstrate their own intrinsic value, that improve over time, and that provide services to the applications that call on them.

Thank you.

I happened into the conversation, but I'm afraid I may have taken it over a bit. See, I already have a system for pricing software development, so I simply shared it, in condensed form, over a period of ten or fifteen minutes. The mechanical engineers were very kind in their appreciation, which is always cool to have, and the software developer said I should present this at one of the tech meetups.

This hadn't occured to me before, but I was happy that Web414 accepted my proposal and invited me to give this presentation on pricing project work.

I write my presentations in advance, and then do my best to present what I wrote, so this post is something close to a transcript of my presentation. Actually, it's a transcript of how I meant to give the presentation, without the 'ums' and 'ahs', nervous pauses, misplaced words, and sections I undoubtedly missed due to various deficiencies in public speaking.

Check out the calculator I made for calculating the CRV Factor, which is the hard part of the formula. My slides are inline below, but check out my slide deck online too. I've been using reveal.js^[2] and it's pretty sweet.

Dusty Filippini, one of the @web414^[3] organizers, streamed this presentation using Google Hangouts, and sent it to YouTube. The video quality is not ideal. The presentation doesn't actually start until about 17:30 and goes for about 55 minutes plus questions and answers. About six minutes into my presentation, I have computer problems and we switch machines. But once it gets going, the sound quality is pretty decent. Watch "Managing risk and selling value" on YouTube.

Hi, I'm version2beta. Some people call me Rob. You can call me either.

My presentation is about figuring out how much to charge for your work. This is one of those talks where you get to learn from someone who really, really knows what they're talking about. I've done it wrong virtually every way you can, so I came up with a system that, so far and without fail, has told me how much I should have charged for every project I've underquoted.

It works proactively too. In fact, that is the recommended way of using this system.

I'm going to warn you right up front, there are going to be mathematics in this presentation. Real math, the kind you need a scientific calculator to do. Or a log table, if you're over the age of 50 and know how to use them.

Or you can use the special calculator I built for doing this math. It's on my website.

Before we get into the details, I'd like to tell you about my first ever paid programming gig.

slide

• Q is quantity
• D is annual demand
• S is fixed cost per order
• H is annual holding cost

slide

This formula is used to calculate the economic order quantity for inventory.^[4]

It's been around a while. I first learned it in 1983, when I was 13 years old. I don't know if it's true or not, but at the time, I was under the impression that my father was one of the creators of this formula. He was inventory control manager for Simpson Paper Company, Eastern Division. His job was to make sure they had the right raw materials on hand, in the right quantities, for all of the plants east of the Mississippi River.

He had this formula that would help him figure out how much stuff to buy. He also had the only IBM PC at Simpson's Vicksburg, Michigan offices.

And he had a very nerdy kid. I didn't know it then, but I was going to write the student council's computer dating program and run the annual computer dating fundraiser every single year for the next four years.

The formula isn't too bad. Basically, you take the annual demand for an inventory item, multiply that by twice the cost per piece, divide that by the holding cost for one item for a year, and then take the square root. The resulting number is the ideal quantity to minimize your total ordering and holding costs for the year.

I know this isn't actually very interesting. But this formula has a square root symbol in it, so it's going to make the formulas I show you in a little while look easy. So it's important you pay attention.

The short story is that my father had me code this formula. He paid me $30 and took it to the office and that was the last I heard about it for a year.

Let's fast forward to the Superbowl, about a year later. Every year the DeYoungs came to my parents' superbowl party. Mr DeYoung worked with my father at Simpson paper. During this particular Superbowl party, Mr. DeYoung pulled me aside and congratulated me on the program I wrote for my father.

He said, "That program saved us $3 million dollars last year."

Even as a freshman in high school, I could do math pretty quick in my head. The $3 million dollars was a very significant return on investment for the $30 I was paid.

It seemed possible I had underquoted that job.

slide

Complexity

Risk

Value

slide

I have a formula for helping people figure out how much to charge for their work. It's called the CRV Model. C-R-V stands for Complexity, Risk, and Value.

I designed this model when I worked for my wife Barbara, in her web development company. Barbara spent extensive amounts of time considering how our pricing models, indeed our entire working environments, were inadequate. She deserves a lot of credit for ideas behind the model, and the conversations inspiring it. She was invaluable in developing and testing this model.

I wish I could say it was a great success, but some credit for the model is also due our largest and most risky customer, whose $68,000 default not only demonstrated the need for this model, but also contributed to the downfall of my wife's company.

All of my experience using this model has been focused on web development and programming, but it should be useful for modeling any kind of project work. What matters to this formula is that the project is at least loosely specified in advance, and you're trying to put a fixed price on it.

slide

1. BYO Business Acumen.
2. Damn it Jim, it's a model, not a doctor for your business.
3. When you're pointing fingers, make sure you're looking in the mirror. Kthanxbai.

slide

I have some caveats for you:

1. You're going to need some business instinct to use this model.
2. In fact, the more business acumen you have, the better this model will work for you.
3. If you have a good sense of business, you probably don't need this model.
4. If you don't have a good business sense, this model will help you develop one.
5. This is not a silver bullet. It's a model. It helps you understand the world a little better.
6. At the end of the day you still need to take credit for your own success. No need to call me up and say thank you.
7. Same goes for your failures, by the way.

slide

Pro-tip: Ignore everything I'm about to tell you and do Agile development instead.

slide

It's also important to note that there are better systems. For example, don't do fixed price quotes.

I'm a fan of Agile methodologies.^[5] I strongly recommend you ditch fixed price quoting and pursue Agile development and project management. I promise you, properly done, it works better. All you need is a lot of trust, excellent communication skills between you and your client, and the knowledge and experience to do Agile properly.

slide

1. Working for the man.
2. Selling your life by the hour.
3. Client wants a fixed price.
4. Client wants to go out of scope.
5. It can be hard to get paid for out of scope work.

slide

Let's talk about why we're here. What is the problem we're trying to solve. I see it as three-fold.

First. It's very important that you hire the right clients. And in case you get a bad client, it's important to know when to fire them.

Second. Even good clients sometimes suck. They want a fixed price bid. Then they want to go out of scope. Then they want to blame you that they went out of scope, that you should have thought of that. Then they don't want to pay for going out of scope. Then if they do pay you for going out of scope, maybe they resent it.

Third. It's come to my attention that I have a limited number of hours in my life. I know that sometimes it feels infinite, but I've consulted with some pretty good doctors, and they assure me I have an expiration date. It's true.

This may also be true for you. You should consult with your doctor. If by any chance your doctor confirms this, then you, like me, are in a position to decide at what hourly rate we're willing to sell our limited lives. Or if we want to sell our lives by the hour at all.

I think we can distill these problems down to a list of goals for a pricing model.

Pursue the right work. Avoid the wrong work. Mitigate risk. Charge according to value.

So here it is. Scary. It even has a logarithm^[6] thingamajig in it.

Let's walk through it, one parameter at a time.

On the left side, we have the quoted price. This will be the product of our calculation, and the price the model suggests for your project.

C, R, and V stand for Complexity, Risk, and Value. We are going to look at these terms in detail.

I'm going to call these three little factors your constants. Overhead, days, and materials.

The little 'o' is your overhead. This is just enough to cover you basic costs. This will vary a lot based on how big your business is, whether you subcontract, etc. We'll discuss this in more detail.

The little 'd' is your time estimate in days. It doesn't need to be exact. We'll cover this more too.

The little 'm' are your material and other project costs. If you are going to have project-based expenses other than time, put it in here. If you mark up your direct costs, put the marked up amount in here.

So that's the formula. It's not as scary as it may have seemed at first.

Add some numbers together. Those are our factors for Complexity, Risk, and Value.

Apply a logarithm. This is the exponent you'd put on the number 10 to get this number. It's also the log10 key on your scientific calculator app.

Multiple by a couple other numbers. That gets us our overhead for the job and how many days of overhead we need to cover.

Add in the remainder. That's any direct or material costs.

You've got your price. Let's look at the specifics.

slide

o : Overhead costs per work day

slide

While Complexity, Risk, and Value are the real focus of this formula, let's get our constants out of the way first. Let's start with overhead.

Just in case you're wondering, our formula is managerial accounting, not traditional balance-sheet type accounting. For example in cost accounting, our labor is divided into time-based units of measure and we put a cost on each full unit and use that for our basis. That's the system we're getting away from.

Managerial or analytical accounting uses a variety of accounting-type metrics to answer specific questions, usually questions about the future. That's what we're doing. We're using a handful of metrics to ask questions about the future cost of doing a project.

One of the questions we need to answer is, "How much of my overhead costs do I need to assign to this project?" This is the little 'o' in our formula. Overhead.

slide

Match your preferred beer to your overhead.

• Pabst Blue Ribbon - $50 / night
• Spotted Cow - $75 / night
• Guinness - $100 / night
• Chimay Blue Label - $125 / night

slide

Moonlighters, raise your hand. Now keep your hand up if "overhead costs" and "beer money" are roughly the same for you. I've been there. I still do some work that way. I admit, this example is a little bit tongue-in-cheek. If I were serious, I'd have used whiskey in the example

I would like to remind you that there are a lot of people trying to feed families doing the same work. Be careful about setting this value too low. I suggest you use the freelance example instead, and think about pricing your work as if you are doing it full time.

slide

• I want to take home about $30,000 a year.
• Taxes are about $7,500 a year.
• Health insurance costs $6,000 a year.
• I spend $1,800 a year on sales and marketing.
• I spend $2,000 a year on new computers.
• I spend $3,000 a year improving my skills.
• I expect to do billable work three days a week.

Overhead per work day = ( 30,000 + 7,500 + 6,000 + 3,000 + 2,000 + 1,800) / 150

Overhead per work day = $415

slide

Let's walk through an example calculating overhead for a single person freelancing. No employees. No employer.

If I am this freelancer, this is what I need to know:

• What's the minimum I could make in a year and still be happy freelancing? Or moonlighting?

• I need to increase that number by 25% to cover taxes and self employment taxes.

• If I plan to do this full time, I'll add in the cost of my health insurance. If you have health insurance that's paid for by someone else, for instance your partner's employer, add in half the value of their insurance benefit. If you don't have health insurance, add at least $6,000.

• Now add in any business expenses I expect to have for the year. Sales and marketing, professional development, equipment costs, insurance, business rent, professional memberships, etc.

• Then I figure out about how many days I actually expect do production work, and I divide by this number. If you plan to do this full time, figure on something between 120 and 180, depending on how much time you need to spend selling.

In the example above, I arrive at a "per project day" overhead cost of $415 for this hypothetical freelancer working something that looks very much like full time on a very modest income of only $30,000 a year take home.

slide

• Operating Expenses (including payroll): $72,000 / month
• Employees engaged in production: 5
• Productive days per month: 20

Overhead per day per employee = 72,000 / (5 * 20)

Overhead per project day = $720

slide

Here's another example, this one for a small-ish agency.

For agencies, I think the numbers are pretty simple to follow. Add up all your expenses, including payroll, even for billable resources. Divide it across the number of people doing billable work.

Again, we're not doing cost accounting.^[7] If you're interested, this formula is inspired a bit by Throughput Accounting,^[8] but the important thing to remember is that we don't want to slip into the habit of thinking in terms of hours of labor.

slide

d : Person-days required for the project under the most ideal circumstances

slide

We are not going to try to figure out how many hours or days or weeks a job will actually take. However, we are going to consider how much time it would take under the most ideal circumstances.

I think this is a real selling point for this model. We'll talk about this more in a few minutes, but when we're using this model we get to think in terms of the best possible circumstance. This is a huge advantage because it's easiest for us to accurately predict what's going to happen if everything goes perfectly, rather than if all hell breaks loose.

So at this point, we can say sayonara to contingencies. We can ignore plans B and C. We can put in the number that only happens when the stars are perfectly aligned.

Just make sure that "days" means the same thing here that it meant when you were figuring your overhead per day. For agencies, this is the number of person-days for production employees that you would assign to the project - under the most ideal circumstances. For freelancers, this is the number of work days you think the project would take you - under the most ideal circumstances. For moonlighters, this is probably the number of evenings or weekends you think you'll need to work on the job. Under the most ideal circumstances.

Again, the important thing to remember here is to be a paragon of optimism. Plan on rainbows and unicorns.

slide

Examples of materials and direct project costs:

• Travel expenses - hotels, meals, mileage, etc.
• Hosting a development environment for the specific project
• Hiring a project manager
• Contracting specialized work
• Buying project-specific training materials
• Software licensing
• Etc.

slide

Little 'm' is easy. It's any direct costs associated with the project, that apply just to this project.

For example, if you need to hire a specialist, put his or her costs in here.

Actually, I've done this a bunch of times. On one project, I was programming the controls for an automated heat treating line. This particular system used huge vats of molten salt for heat treating the parts. The system was basically robot arm on a 120 foot long track that picks up baskets of parts, each basket weighing about 300 pounds, and moves them through the heat treating process. Each basket got its own recipe, based on the kinds of parts inside. So a given basket might go in the hottest vat for 30 seconds, then quench for 30 seconds, then do something else.

Believe it or not, that was the easy part.

The hard part of this project was adding the ability for the heat treating company to tell the computer what parts were coming up, so that the computer could decide when each basket should be loaded in order to maximize throughput. For that I hired an outside consultant, a guy named Kim Duk-Won.

Kim is great. Before I went to visit Korea, he taught me to read and speak enough Hangeul so that I could get around town, order food, and find a bathroom. When he did his Masters thesis, I helped him with the programming. When I hired him for this project, he had completed his Ph.D. in Operations Research, done a few years optimizing delivery schedules for FedEx, and was teaching at the University of Tennessee at Oak Ridge.

On this particular project, Kim gave me a fixed price quote. You know, because I'm that kind of customer.

Paying Kim to work on that project was a direct project cost.

slide

Fuzzy Math

Not actually fuzzy, just logarithmic.

slide

We're going to estimate Complexity, Risk, and Value with some pretty fuzzy numbers.

This isn't an exact science. I can't tell you that the complexity on a given project is exactly 6,423.5.

We really only know how complex something is, if it's not complex at all. We can more accurately predict risk when there's very little. And as well all know, value ranges from nothing, to a lot, to priceless, right?

When I make scales for Complexity, Risk, and Value, I want to have a lot of room to move around, and I want the scales to be more accurate on the lower end than on the upper end. I can't really use "infinity" as a value, but it'd be nice to be able to say "A whole damn lot."

So the scale we're using goes like this: Pick a number between ten and ten thousand. If any of the factors start really ramping up, use big numbers. That's what the "log10" in the formula does for us. If you really have a project that is so risky, or so complex, or so valuable that you feel like you need really big numbers to describe it, use them.

slide

• 10 - Anyone could do it.
• 100 - There are only a few people in the area who can do it.
• 1000 - There are only a few people in the country who could do it.
• 10000 - It probably can't be done.

slide

I think of complexity in terms of who can do the work. Actually, I tend to think of complexity in terms of how far you'd have to go to find someone else who can do the job. In that way, it's kind of like "competition".

You could actually use distance to the next nearest person or agency who can do this job as your complexity number. If bunches of people in this room could do it, let's say 10 since I really never use factors lower than 10. If you'd have to go to Chicago for another place that could handle it, let's say 120. New York, about a thousand miles. A univerity in Tokyo? 10,000.

My friend Kim Duk-Won is an example, right? I needed a highly-specialized algorithm that he was uniquely suited to provide. Given that the competition for that position is very limited, he can charge more for his time. It's simple supply and demand.

Here's an even better example. I first heard this example from my grandfather, who was an engineer.

There's this guy Gus who retires from his position as lead maintenance person at the local factory. He's put in his time, he's ready to be done working and focus on his fishing.

One day shortly after Gus retires, an absolutely critical machine fails. It's making a horrible noise. Nobody in maintenance knows what to do with it, so they tell the plant manager he's gotta get Gus to come back and fix it.

Gus isn't very happy about this, but he agrees to come look. He walks around the machine. He listens to it. And then he pulls a piece of chalk from his pocket and puts an 'X' on one part. 'Replace that,' he says.

The plant manager is thrilled until he sees Gus' invoice for $50,000. The plant manager tells Gus to submit an itemized invoice.

Gus sends a new invoice came with with two line items. The first line item was "One chalk mark: $1." The second line item was "Knowing where to put it: $49,999."

slide

• 10 - It's a cake walk.
• 100 - There are some problems with taking the job.
• 1000 - There are significant risks associated with this project.
• 10000 - That's just crazy talk.

slide

I remember the first time it occured to me that I could change the price of a job based on how little or how much the client actually knows about the project. This happened way later than it should have. I had my hourly rate, and it was my hourly rate. Maybe I'd add in a few extra hours to compensate for some unknown. But I could have saved a lot of money and salvaged at least one failed project if I'd only realized sooner that I should always charge more for projects that have more red flags.

Let's talk about risk.

First, I want to say risk isn't for everybody. At the low end of the scale, we have "the cake walk." The job is well defined. The customer knows what they're doing when it comes to executing a project like this. They pay quickly and fairly. Maybe they're even an agency like C2^[9] that specializes in contract talent.

This is good, steady, low-risk work. It's nice to have this kind of work. Many of us would do well to focus on this kind of work.

Or maybe you can tolerate a little more risk, so we slide up the scale a little bit.

"There are some problems with taking the job." Here we start to see some red flags, and we need to take them into consideration.

The specification is weak. The client seems uncertain of what they want. They don't respond well to suggestions. Their ideas seem overly fluid. That's a nice way of saying they are a high risk for scope creep.

The project has weak prospects. The project doesn't have a good cost-benefit ratio. It's unlikely to bring value to the client.

The customer seems overly cost-conscious. They may not have the funds required to manage the project well on their side.

The client talks about the previous contractors they had to fire.

Sometimes the risk isn't with the client. Let's say you're not well established yet. Maybe you don't have the skills you'd need to feel confident completing the job. Or taking on this project precludes your ability to take on other work or care for your regular clients. These are all risks too.

"There are significant risks associated with the project." Now we're getting into some serious red-flaggage.

Maybe the project has no specification, and the client doesn't want to do a discovery phase.

Maybe the client's expectation for delivery, price, and / or quality are unreasonable.

Maybe the client seems to be hitting on you.

Maybe you have little confidence in the client's ability to fulfill their part of the contract. You know, like paying you.

Maybe you don't have confidence in your ability to finish the project. It may be too big, too hard, to many moving parts.

"That's just crazy talk." Projects at this level of risk are the ones we walk away from. These are the projects with abusive clients, or projects we have no expectation will ever finish successfully.

But sometimes the client is throwing money at you in spite of the difficulties, and maybe you are pretty tolerant of risk. You might even specialize in difficult clients with deep pockets. There's money in that.

When you're quoting a project, always first try to drive risk down. Build in a discovery phase and price out the full project after there's an adequate scope. Spend extra time establishing reasonable expectations with the customer, in both directions. Discuss your concerns about their ability to manage the project, and how they may benefit from bringing on a professional project manager just for the project. Get a bigger down payment. Schedule invoicing more frequently, and require payments be current before each phase begins.

The risk you're left with drives up your price, and this is entirely reasonable. It does a couple of very valuable things for you. For one, you might not get the job, and depending on how much risk there is, this may be a Very Good Thing.

If the job has significant, unmitigated risks, give it a "walking away" price. Don't put a lot of effort into quoting the job. Bump up the risk to an appropriate level based on the problems with the job and your less-well-considered quotation, and send it over without expectation.

Then, if you do get the job and things start to head south, you were already being paid extra to deal with it and you can take it on with a smile. Scope creep? No problem - you expected it. Plus you can still try to get paid for it. Stiffed you for the last payment? That's okay, the last payment was the second half of your bonus, not your paycheck. The client is a pain in your butt? You charged the asshole tax up front.

slide

Evaluating value

• 10 - The impact of the project will be at least as great as the cost.
• 100 - The impact of the project will be an order of magnitude greater than the cost.
• 1000 - The impact of the project will be two orders of magnitude greater than the cost.
• 10000 - An Arab Prince has already written you a blank check.

slide

My microeconomics professor at the University of Maine was a fish economist. He told us how he'd been approached by Heinz Corporation way back in 1969 to assess the potential market for cocktail sauce. They asked him for a daily rate, and he gave them a price of $100 a day.

He didn't get the contract. They found someone who'd charge them $1,000 a day.

I've worked on big websites for small companies that cost $20,000. I've worked on small websites for big companies that cost $20,000. When it comes down to it, people want to pay a price commensurate to the value they get.

I told you up front you need some business instinct to use this model. That's perhaps most true when it comes to figuring out the value your client will gain from your work. Understanding that number will do much to free you from hourly drudgery.

Don't do work that has limited value.

If the potential gain is small, your client needs to think small, and treat you small. It makes your job much less enjoyable.

If the potential gain is large, your client is motivated to make big things happen with you, and will support you on the project. It makes your job much more enjoyable, and lucrative too.

It boils down to this: Always. Maximize. Value.

slide

How to discount your work:

• Look for snacks, not huge meals.
• Quote full price and then apply a discount.
• Your discount may be a marketing expense. Talk to your accountant.

slide

We have one more factor that we could have added to our formula, but if we did, it'd be there strictly to help you undo the benefits of using a pricing model like this at all.

Sometimes it doesn't matter what you know you should charge, you really need to get the work. Or at least some work. I propose it's still best to know what you should have been charging, and then discount it, rather than ignoring good sense and pricing the work artificially low.

"Hunger" is that factor. The hungrier you are to get the job, the more you'll knock of your price to make sure you get it.

Obviously the best bet is to keep hunger at bay. But since that's often not possible, I have a couple of suggestions that may be helpful:

• If you're hungry, look for snacks, not huge meals. Once you get something in the pipeline, that sense of hunger is going to start going away. You don't want to be paying for it long term. Sometimes you can even turn a big job into a snack. For example, it may be beneficial for everyone if you got paid for a 40 hour discovery project and use that to spec out the big job at regular price.
• Quote your work at full price and then add your discount as a line item. Offer a new client a "new customer discount". Apply a coupon. Do a cross-promotion with your client. Your client gets the price they need, but you're instilling an expectation about your future charges.
• Talk to your accountant. It's possible that discount you're giving might actually be a marketing expense, and this might improve your tax situation.

slide

nested CRV

slide

I'm just going to throw this out there. You don't have to use the same complexity, risk, and value factors for every part of a project. You can break the project into meaningful pieces and apply the model to each piece.

For example, let's say we're quoting a new website for a heat treating company. Most of the project is straight up design and implementation, but they also want their customers to be able to log in and retrieve quality documents about previous jobs. To do that, you need to tie into their internal document management system running under Microsoft Dynamics.

Now we have a project that involves two kinds of CRV profiles. The web site is relatively easy and moderately valuable, but the document retrieval system is significantly more complicated and valuable. Depending on how much experience your team has with Microsoft Dynamics, it may also add a bit of risk.

slide

Ideal Cost: o * d

CRV Factor: log10 ( C + R + V )

slide

You may have noticed that two terms make up the heart of this formula.

I call the first one "The Ideal Cost". It's your overhead rate times your time commitment. That's what the job should cost you assuming everything goes swimmingly.

I particularly like the concept of Ideal Cost. We humans are notoriously bad at making predictions. Our emotions get in the way. Ideal Cost eliminates some of that problem by removing the complexity of the negative emotional factors. We don't have to think about what might go wrong at this point - we get to consider just the positive. And this is a nice way to think about a new project.

The second term is the "CRV Factor" is your multiplier on the Ideal Cost. It builds in supply and demand, mitigates your risk, and keeps your price commensurate with the value of the project to your customer. The higher the CRV Factor, the higher your total price.

Let's look at some ways you can use the CRV Factor to quickly analyze almost any project.

slide

• Complexity: 1000
• Risk: 100
• Value: 1000
• CRV Factor: 3.32

Hard but worth it.

slide

You're a specialist in a valuable field. Your clients are lucky to have your attention.

slide

• Complexity: 100
• Risk: 1000
• Value: 100
• CRV Factor: 3.08

Assholes and superheroes.

slide

We've probably all worked on this project, right? It's a good fit for your skill set, and it's quite worth while for your client, but your client is an ass. Or doesn't have the money. Or needs it next week.

The nice thing about these jobs is this: you get to be the superhero, and if you're willing to work under these pressures, you can really shine.

slide

• Complexity: 2500
• Risk: 2500
• Value: 10
• CRV Factor: 3.70

Danger Will Robinson!

slide

Sometimes I feel like every job I quote looks like this. I can't think of anyone else getting it right, but the risks are significant and when it's done, the client isn't going to get their money's worth anyway.

These are the jobs you refactor for your customer. Drive the complexity down. Mitigate more risk up front, maybe by doing a discovery phase. Shift the priorities so there is more value in the final project.

Make this project a better project, or embrace the multiplier of almost four and hope they decide you're too expensive.

slide

• Complexity: 100
• Risk: 100
• Value: 100
• CRV Factor: 2.48

The sweet spot.

slide

I like these jobs: Competition's not too tight, the job has manageable risks, and it's going to be worth it to your client. It's a factor of almost 2.5 times your Ideal Cost. You can afford to hold out for the jobs you want. You enjoy your customers and your work.

slide

@version2beta

http://twitter.com/version2beta | http://version2beta.com

CRV Factor Calculator: http://version2beta.com/static/examples/managing-risk-and-selling-value/crv-calculator.html

Tech support is always free!*

• Free as in beer. Means you buy me a beer and I give you tech support.

slide

I want to remind you that this formula is a model for pricing your work. It helps you understand your client, the project, and your self. Along the way it may make you a better businessperson. It's a framework for thinking about the work you do.

Thank you for listening. As always, my presentations come with a 100% money back guarantee plus tech support that is "Free as in beer". That means you buy me a beer and I give you tech support.

My slides ran on Reveal.js and are available here. Because I write my presentations long-hand before I give them, I can post something very close to a transcript of it below.

Update 2013-05-08: Tonight I'm presenting this same talk for the Fox Cities PHP Users' Group. I chose to update this post with the latest version of my slides and presentation. It feels a bit revisionist, but I did it anyway.

/static/

Hi. I'm version2beta. That's version, the number two, beta. I'm version2beta on twitter, github, linkedin, skype, etc. I'm at version2beta.com online.

In meat space, some people call me Rob. There are also some short people who call me Dad.

Since we're talking about Vagrant, I decided to dress like a vagrant. I even have a hole in my armpit. Since the last time I gave this presentation, I've developed a hole in my other armpit too.

We're talking about Chef tonight, so I'm going to run this like a cooking show. We'll start by putting the casserole in the oven.

That's how it works, right? The TV chef makes up the entire dish in advance and tosses it in the oven just before the cameras start rolling. Then, live in front of a studio audience - that's you - they make the dish from scratch. And they put it in the oven. And then they pull the other one, the one they've had cooking all along, out of the other oven.

That's what we're going to do. Preparing this dish live right in front of you sounds a little scary, so instead, we'll just toss this one in the oven and let it bake, while we just talk some about the ingredients.

I think this is actually going to work out pretty well, because I actually made a whole bunch of these casseroles in advance and stuck them up at Github. All you have to do is grab one and throw it in the microwave for about six minutes. The first step is to grab a frozen casserole from Github. There's the link if you want to follow along real-time, we're going to go slow enough. But no worries, it'll be up again.

Actually the zeroeth step is to install VirtualBox and Vagrant if you haven't already. That's the other link, Vagrant Up Dot Com. We don't cover installing these two programs because it's easy and they cover it just fine on the website. If you want to install them right now, that's fine. You'll have plenty of time. I haven't even told you about my grandfather yet.

But first, I'm going to git clone this repository -

[Shell: git clone git@github.com:Version2beta/vagrant-chef-wordpress.git]

and then change directories into the repository -

[Shell: cd vagrant-chef-wordpress]

and then run vagrant up. I love that. Vagrant Up. I always wanna say Vagrant up, muthafucka, Vagrant up. I don't know why. Or when I'm in a family friendly venue, I just say Vagrant up, yo.

[Shell: vagrant up]

That'll be done cooking in about 6 minutes. We'll come back to it.

Okay, I'm Version2beta. I know a lot of people start off their presentations with the "Why should you listen to me anyway?" slide. "I'm blah blah blah and I blah blah blah so you can blah blah blah."

I haven't written any books, or keynoted any conferences. I have a few cool open source repositories and a small Twitter following. So instead of bragging about the little things, I decided to tell you about my grandfather instead. And my mom, too.

So grandpa always said he was a total screwup, but what he really meant was he was a geek. He almost got expelled from high school for his senior prank. He made explosive BB's and tossed them on stage during the finale of the senior talent show.

A fews years after he graduated from high school, he and his brother Bill both signed up for the Navy. It was world war two, they both had new spouses, and Grandpa even had a baby girl at home.

Grandpa was stationed in Rhode Island testing missiles. I don't know what Bill did in the Navy, but I do know that when they'd done their two years of service, they got four years of college paid for on the GI bill, and they squeezed every penny they could get out of it.

In four years at the University of Pennsylvania, Grandpa did a bachelors in engineering, a masters in business administration, and a masters in the brand new field of operations research.

In fact the university had created that Operations Research department specifically at the request of the federal government. I heard it was the Office of Strategic Services that asked for it.

At the end of his four years at University, in 1949, Grandpa went to work for RCA. He started out as an engineer for their military contracts. When he retired from RCA in 1989, he was RCA's chief technical liaison to the Pentagon.

Hey this is cool. I recently discovered that Grandpa wrote a book in 1951 called "Music in Industrial Plants." Like a boss.

In comparison, my Uncle Bill was a slacker. In four years of college all he managed was a bachelors and a masters in chemical engineering. But look at this. At the beginning of college Grandpa and Grandma had only one child, my mother. At the end of the four years, they still only had one child. When Grandpa and Bill started at the university, Bill and June didn't have any kids. Four years later they had four kids. So Bill was maybe more productive during his college years than my grandfather was.

Uncle Bill went on to become a senior engineer on the team that invented PVC. And later he died of cancer. I could be wrong, but I've always suspected a connection.

The story about my grandfather doesn't really have anything to do with our topic for tonight, I just like telling it. But I do have a story that can tie it together. On Valentines Day in 1946, the Moore School of Electrical Engineering at the University of Pennsylvania announced they had built the first general purpose electronic computer. You've probably heard of it. It was called Eniac. Eniac was Turing complete, back in the days when Alan Turing himself was still Turing complete. Eniac was capable of solving a wide variety of problems, all of which had to do with the firing of artillery.

Around this same time, Grandpa had a habit of hanging around the Moore School, and my mother had a habit of hanging around him.

I know when I was a kid, I had a lot more freedom than kids have today. When I was in elementary school, I ran the streets ’til dusk, rode my bike wherever I wanted, hung out with friends, and only sometimes got in trouble for it.

My mother apparently had a little more freedom. When she was in elementary school, she was allowed to hang around with college boys and play tic tac toe against the world's first electronic computer. In fact my mom beat the computer at tic tac toe. At least, that's her story. She also says that Grandpa accused her of lying and said no one could beat the computer at tic tac toe. When I asked him about it, Grandpa denied ever doubting her.

I spent the 90's doing industrial controls, some ladder logic, some programmable logic controllers, and then the best part, human machine interfaces. Except they were still called man machine interfaces back then.

I had a photography hobby though. My specialty was tastefully photographing attractive women without their clothing, and sometimes I photographed flowers. I even won a few awards and was invited to lecture at a local university.

The Internet was getting pretty interesting so I decided I should have a website for my photography. So I did the only logical thing. I built myself a web server.

Don't tell anyone but back then, I was running IIS on Windows NT 4.0 Server.

For like the first six months. I hosted my own Windows web server in my office at home, connected to the internets with two 56K modems. Shotgun.

This was my first internet dev environment. It was also my first internet production environment. I broke it a lot. Soon, I replaced IIS with Apache, and Windows with Red Hat and then NetBSD and then FreeBSD.

These were exciting times. I got a lot of experience.

These were the days when just being there on the ’net would get you visitors. Having one modest web site was enough to keep the average hobbyist busy. Plus my website had naked people on it.

Let's zoom forward a few more years. By 2001, most of my work was on the Internet. In fact, I was doing development and systems administration full time for a company I owned part of. We hosted a few hundred websites, had email for a couple thousand accounts, and provided Internet access for about a thousand families.

Yes, we were a dial-up internet service provider. And no, we didn't last very much longer.

I had an interesting experience last weekend. I was teaching a Ruby class, and one of the participants works for a company that bought a company that bought the company that I owned part of. So this kid was telling me that he's now responsible for maintain code I wrote more than ten years ago.

And he didn't even try to hurt me.

I don't have any screen captures of those days to share with you, so I made one up.

You've seen file structures like this, right? Where there are four versions of "Final"? Backups upon backups? No good way of knowing what's actually what?

Okay, lets jump a few more years. Now the dev environments are getting organized, although still not on version control, since that's actually still kinda hard to do with a CMS and relational database.

By the way, has anyone here solved that problem? Who's running Wordpress, or Drupal, or Joomla or a similar CMS?

Keep your hands up if you're also using Git?

Okay now keep your hands up if you're also versioning your database?

For about five years most of my web work was done in an application framework and content management system called MODX. It's pretty cool, works okay.

Here is a site that is running on MODX now. Over the past four years, it has gone through three different major rewrites. The first one I saw was static. I didn't build it, that's what I inherited. That's version 1.0.

Version 1.1 is on MODX and Wordpress, but it's really just the static site cleaned up and put into a content management system.

Version 1.2 is an incremental improvement from there. Still MODX and Wordpress.

The thing I want to show you here is that version 1.1 was the dev site, up until it launched and then it became the production site.

The way I built this system, launching was really easy. I just point htdocs at the correct version with a symbolic link.

After version 1.1 became the production site, version 1.2 got started up as the new development site. Obviously if we continued along this path, version 1.3 would be the next development site.

This setup works. It's good for small sites on shared servers where you don't have to worry about breaking too much shit.

Oh, and when you're the only developer on the project.

And you don't need very many versions, or branches, on the code.

And you always have internet access.

And when you don't have to worry about dependency hell. You know, when one version of the site needs one version of a package, but the development version needs a newer version. That's why god made virtualenv. And rvm. And rbenv. And chruby. And so on.

This way of doing it also works really well when you want your development site to use the production site's database. Which of course you never do. Especially not by accident. There are some stories there.

So yeah, this was the state of the art for me a couple of years ago. It's not like git and mercurial and svn and everything else weren't available, I just wasn't using them.

I work now for a company called Tartan Solutions and they make a platform for running programs that do profit and cost management stuff for big companies. For instance, one of the programs will calculate how much every product in a product line will cost for the next year given current sales forecasts. Even if you took these three little parts that are used in a bunch of different products and combined them into two parts and got them from a different provider, say in China.

When I was interviewing for my job, I hit on a real winner for selling myself. I said, "My problems are not big enough."

My problems are not big enough.

Sure, I can make my current problems bigger. That's easy. You've probably seen this.

Let's say you a problem you need to solve with an internet application. So you install Drupal. Now your problem is twice as large.

I'm trying to go the other way, though. I've adopted this new philosophy. It's my Twitter bio right now.

It says, "I think I'm free for lunch."

No, wait. That was the old one. The new one is "When you only have two ducks, they are always in a row."

So that's my strategy now. I eschew complexity.

It's actually really cool. It means I can do things like make a completely separate development environment for every project, and they build themselves automatically.

It means I can make a separate server for every production site. Unless of course I can manage to host it without a server at all.

I'm also avoiding complex, monolithic application frameworks and content management systems. I'm focusing on microframeworks and services that collaborate.

This all fits together with our dev environment, right? It's done cooking.

[Browser: http://localhost:8000/wp-admin/install.php]

So here's a brand new Wordpress installation ready to happen.

[Browser: Fill it in.]

That was git clone and vagrant up. Two commands. If you want to work directly on your new server, it's just vagrant ssh.

[Shell: vagrant ssh to development environment. ls -al blog]

Pretty slick. Let's look at it more closely.

Discuss:

Vagrant and Chef are Ruby. Ruby is optimized for programmer happiness. Really, when Yukihiro Matsumoto created it, he made programmer happiness a primary goal. Here's one think I've learned from observing the Ruby community: You know you're a Ruby programmer when you stop buying cheap booze for medicinal purposes and start buying expensive whiskeys instead.

Chef is a DSL, a domain specific language, which is a way of saying we can make our own commands in the language of our problem domain. In this case, managing servers.

Does anyone here read TheChangelog, or listen to the podcast? Vagrant was created mostly by Mitchell Hashimoto, and yesterday they recorded a podcast with him as the guest. It was his second appearance on the show. I got to listen to it live, and I asked Mitchell several questions, including whether there was anything he wanted me to tell you tonight. There wasn't.

Vagrant recently came out with version 2. Not really, but that's what the part that says "configure('2')" means. It is version 2 of the configuration, even if Mitchell is building up to Vagrant version 2 slowly, starting with versions 1.1 and 1.2.

This Vagrantfile specifies Ubuntu 12.04 from vagrantup.com. Vagrantup.com doesn't offer very many choices for boxes, but there are other sources too, and it's easy to make your own.

We're forwarding port 8000 to port 80 on the Vagrant guest. This is happening on all addresses for the host computer, so you can access the guest's web server from any host on the local network.

We share a folder between the host and the guest operating system. This makes it convenient for passing files in.

Outside of config.vm.provision is all VirtualBox magicness.

Inside of config.vm.provision is the Chef magicness.

This is Chef Solo. We don't have a Chef server. We're not using hosted Chef. We don't have a persistent store of each Vagrant boxes' configuration. This is kindergarten-level Chef.

In kindergarten they teach you some best practices, right? Like to say excuse me after burping and wash your hands after using the bathroom. We're doing the same thing with Chef - following some best practices, even though we're totally introductory level right now.

So here's a best practice we're using in this recipe. We're following a chef pattern called "Library cookbooks and Configuration cookbooks".

This means that we're using a handful of standard libraries created by Opscode (that's the people who make Chef) or other members of the community. We don't mess with these cookbooks, we just use them. We don't change the settings. We don't twist any knobs. We keep our hands off.

That way if the maintainer of one of those cookbooks wants to change something, they can do that, and we can upgrade to it, and it should affect us, but only in a good way. We should get the new features, the new coolness the maintainer put in, but it shouldn't break our code.

We also have one or more "Configuration" cookbooks. These are our in-house "application" cookbooks, the cookbooks that set up our stuff.

See, we know what our raw materials are - maybe a little apt, a little build-essential, some nginx, a fat dose of composer, or wordpress, or Jekyll, or whatever. Those are our library cookbooks.

But the meal comes of out how we put it all together. You know, presentation is everything. Sure, I made some rice, but rice tastes a whole lot different when you put my sausage-coconut milk-curry on top of it.

That's what the configuration cookbook is for. It puts the curry on the rice. Yum.

Just real quick, under a Vagrant project you're gonna have a few folders. Cookbooks live in the cookbook folder. Both library cookbooks and configuration cookbooks, all mixed up together.

Anyone here like Korean food? There's a Korean dish called Bi Bim Bap. It's a great name. Not only does it sound cool, but it exactly describes the dish. Bap means rice. Bim is vegetable. Bi is 'all mixed up'.

Files that contain data about the system as a whole like to hang out in 'databags' folder.

A databag is just a collect of JSON files that you can use to store information that one or more of your recipes might need. We can reference these values inside our recipes. Like this information includes a public and a private SSH key that my github account knows about. That way I can work my git repository from the Vagrant box. I don't actually do it this way anymore though, because private keys are a pain to put into JSON, so I just do it manually. If you use one of my dev environments, you'll have some choices of how to get connected to Github. Ask me about that later if you want to know.

In Vagrant, the cookbooks are straight up Opscode Chef cookbooks.

Actually, in the interest of full disclosure, Vagrant does give you the option of using other systems too, like Puppet. I'm using Chef, and I promised we'd talk about Chef. I did not promise to talk about Puppet.

Cookbooks have a README, a place to put in some attributes (you know, those tunable knobs I told you to leave alone?), a place for recipes, some resources maybe, and some providers maybe.

Here's some definitions:

A Chef RECIPE is a set of instructions describing how to install the software on one or more kinds of systems.

A Chef RESOURCE is a piece of information about the state of the machine. For example, a resource might indicate that a given piece of software is installed, or that a configuration value is set, or that a service is supposed to be running.

A Chef PROVIDER is a tool that puts the system into a given state. It's basically just a Ruby program. So a provider might set a configuration value, or tell a service to reload its configuration, or restart itself..

Let's look at a configuration recipe, since that's where all the interesting stuff happens.

[Shell: vagrant-chef-wordpress configure default recipe]

We're gonna breeze through this, and it's going to be easy, because it's just a collection of Chef standard providers. That's because Chef is a domain specific language, so we just use these little domain-specific commands to get exactly what we want.

First we include a bunch of library recipes. I did this with a little loop in Ruby, because cookbook files are in fact Ruby programs.

All of these are cookbooks that someone else wrote and someone else maintains. Some of them are from Opscode, the company that makes Chef. Some of them are from people like you and me.

By including these recipes without any more specific instruction, we're telling them to perform their default tasks. In each case, that's the install action.

Next we have the "directory" provider. We pass it some attributes and it creates the directory.

Next we have a "remote_file" provider. This goes out and gets a file from a remote location, and puts it on the virtual machine in the right place with the right credentials.

Now we have a couple of "cookbookfiles". Just like remotefile, but it comes from the cookbook, under the "files" directory.

Finally we have a provider called "link" that creates a file or folder that's a symbolic link to some other file or folder.

I started this off by telling you how simple it all is, and now I probably just went and made it look complicated. So let's back up and recap.

There'a a github repository. It's called vagrant-chef-wordpress.

We cloned it.

We changed into its directory.

We ran vagrant up.

About 6 minutes passed.

[Browser: http://localhost:8000/]

We connected to a brand-spanking-new Wordpress.

Clone my repository and use it if it helps. If you improve it, please remember to send me a pull request.

I made a couple more repositories that are similar to this one, but run Vagrant, Chef, and Composer. One of them is a nearly-empty Composer project, and the other one installs Drupal7.

[Shell: vagrant halt vagrant-chef-wordpress]

[Shell: vagrant up vagrant-chef-flask]

Let me tell you a story about my blog, version2beta.com. It's not a PHP project - it's runs from a static site generated by a Python microframework. And, of course, my development environment uses Vagrant, Chef, and Git.

A couple of years ago I decided I'd start blogging again. I felt like the Cheerleader on Heroes, I don't know if you remember that show. No matter how many times she tries to kill herself, she just keeps on healing.

No matter how many times I try to kill myself with blogging, I just keep coming back.

So a couple of years, I decided to use Wordpress. That wasn't a new decision. In fact, a couple years after I first started blogging, the platform I was using was discontinued. That was B2 from CafeLog. They dropped it in favor of a complete rewrite under a new name, Wordpress.

I built my theme, etc etc. You all know how baby Wordpress sites are made.

A year or so later I got sick of Wordpress. Actually, I got sick enough of Wordpress to do something different. At the time, I'd been learning Django, but I'd also been checking out this cool Ruby-based blogging tool called Jekyll. It is a static site generator, so it builds all of your pages in advance so your blog is way fast.

Since I was working with Python, I went looking for a Python version of Jekyll. I found Hyde. Where do they come up with these names?

Hyde was cool, but the dev environment you needed for it was a bit heavy for me. In fact, I broke my dev environment almost year ago, and never quite found the energy to go fix it. So I stopped blogging from May of 2012 until the end of January of 2013.

In January I decided I needed to fix my blog. I almost used Jekyll, which would have made sense since I've been learning me some Ruby here and there.

Apparently I still must be inspired by Larry Wall, the creator of the Perl programming language. He says the three characteristics of a good programmer are laziness, impatience, and hubris.

Fits me like a glove. I was too lazy to rebuild my blog development environment for almost a year.

Once I got over my laziness, I was too impatient to learn enough Ruby to use Jekyll. Or even look at how much Ruby I'd need to learn.

Then I polished up my hubris and built my own blog engine. In Python.

[Browser: http://version2beta.com]

It wasn't hard. In fact, it's only 125 lines of code. And it's really easy to use. And the development environment is a breeze.

So let's talk about the development environment.

My blog development environment works much like the Wordpress dev environment we already went through. I have a repository on my github account that sets up a Vagrant box with all of the tools I need: basically, Python and a handful of Python packages.

But this environment works just a little bit different. It doesn't bring up a clean, new environment. It deploys a local development version of my blog.

Let's walk through my configuration cookbook.

[File: vagrant-chef-flask/cookbooks/configure/recipes/default.rb]

My configuration recipe starts much like the Wordpress version. We start by including some library cookbooks for Python and Vim.

The next Chef command is new though. It's called "package" and all it does is use the virtual machine's package manager to install a package. My box is running on Ubuntu 12.04, so it's probably going to use apt-get. If I were on one of those other, weird versions of Linux, maybe it would use yum.

I have a handful of Python packages to install, so I'm going to use the Python package manager called pip. The Python cookbook even gives me a provider for this, python_pip.

My dev environment needs a few directories. I'll take one for the blog, one for my SSH keys, one for my Amazon Web Service keys, and one for this program you might have noticed called vcprompt. That's the git notifier I'm using in my Bash prompt. It's included in the Wordpress and Composer Vagrant boxes I made too.

Next we need to put some stuff in the filesystem. vcprompt, my bash profile, my vim rc file, public and private keys, a configuration file for s3cmd for Amazon S3, and my configuration file for git.

Then the last step tells Chef to go to Github, as me, and clone or pull my blog repository.

[Shell: vagrant ssh]

My workflow is really easy then.

I run a simple command to launch my dev server.

[Shell: python sitebuilder.py]

I create or edit my content.

I check it out in a browser.

[Browser: http://localhost:8000]

I commit my changes and push to Github.

I like it so I launch it.

[Shell: python sitebuilder.py deploy]

I think that's pretty damn easy to use. I'm happy with it.

I've been building a development environment for our developers at my day job too. It involves multiple virtual machines, many more library cookbooks, four configuration cookbooks, at least a dozen Github repositories, dozens of Python packages, preloading over a gigabyte of data into the database, and it's all written to work with Vagrant, or a virtual private server, or bare metal - whatever you choose.

I'm happy to say it works, even scaled out, comparatively.

Let's wrap it up. There are four repositories on my Github account showing off how you can make dev environments with Vagrant and Chef. Three of them are set up so that you can just clone them and go.

The first one is called vagrant-chef-wordpress, and if you clone it, you can go straight from vagrant up to picking your wordpress name. Clone it, star it, use it, make it better, send me pull requests.

The second one is called vagrant-chef-composer, and it gets you in a LAMP environment with composer and a starter composer.json file. Clone it, star it, use it, make it better, send me pull requests.

The third one is called vagrant-chef-drupal7, and it gets you the same composer environment, but this one is ready to run Drupal. Clone it, star it, use it, make it better, send me pull requests.

I really like stars, by the way. I hope to get more of them.

The fourth one is called vagrant-chef-flask, and it's my blog development environment. I recommend against working with it right now. It's really a single purpose kind of thing. My daughter Miranda and I are working on making it into a general purpose static site generator called Kobol. It's in Python, not PHP, but maybe you'll have both her and I back to talk about it sometime.

I want to thank you for listening. As always, my presentations come with a 100% money back guarantee plus tech support that is "Free as in beer". That means you buy me a beer and I give you tech support.

So after a mere 8 months of false starts and procrastination, I managed to move my blog. And make it prettier.

The new setup

The new platform is based on a handful of interesting technologies that I wanted to learn more about:

• Amazon Web Services Route53 and S3 Simple Storage Service
• Vagrant and Chef
• Python Flask microframework
• YAML and Markdown

Amazon Web Services

Amazon Web Services^[7] offers a bunch of technologies at affordable pay-per-use prices. I did a largish project on AWS a couple years ago but haven't been using them much until recently, when I decided to take advantage of the "free" tier. I'm using an EC2^[8] micro instance virtual private server right now as an online coding server pretty much specifically so I can invite other devs in on tmux^[9] for remote pairing, but that's not actually relevant to my blog platform. For that, I'm using Route53^[10] and S3^[11], but it's all giving me a good opportunity to come up to speed on a lot of their tech.

S3^[11] provides file storage in the cloud. With the right permissions on files, you can share them with the world just like a web server would. Since my blogging platform is a "static site generator" (it builds a collection of static files that make up the site), I need someplace to store the files that's accessible via the internet.

Route53^[10] is the special sauce that makes my static files accessible via my domain. This is a fairly recent addition to the AWS product line. First, it provides Domain Name Services or DNS for any domain pointed there. Second, it let's you associate a host in your DNS with a bucket in your S3. With that, any file I upload to S3 can be served at http://version2beta.com. Mission accomplished.

Vagrant and Chef

Chef is a server management tool that let's you associate a server (node) with cookbooks and recipes that install, configure, and maintain the software on the server. More advanced use lets you associate your server with one or more roles (like application server, or database server, load balancer, or super-secret-international-spyperson-hacking-platform) in an environment (like development, QA, production, or maybe destructive-test-grounds).

Chef works well with Vagrant^[12], which is a tool for quickly deploying local virtual machines using VirtualBox.^[13] Vagrant deploys the virtual machine and then uses Chef (or Puppet,^[14] a similar tool) to provision it.

What I've done this time around is create a Vagrant project (available on my Github^[15]) that sets up my development environment on any machine with Vagrant installed. The blog is also on Github^[16] of course, so in one command, I build everything I need in a development server to work on my blog. It's an on-demand environment. Pretty slick.

I've been killing birds with stones all over the place with Vagrant and Chef. One of my current $dayjob^[17] projects is to create a portable development environment that's quick and easy for a dev to deploy. Another is to create QA environments that are disposable and fast. A third is to set up our production systems on Opscode's Hosted Chef environment.^[18]

I'm also preparing a presentation for the Milwaukee PHP User's Group^[19], one of our excellent local (Milwaukee) tech meetups. My presentation is on creating and using virtual development environments:^[20]

Virtual development environments keep your workspace clean and predictable in dev and production, and creating a virtual development machine with Vagrant is quick and easy once you have all the pieces in place. Rob Martin (@version2beta) is going to put all the pieces together and hand them to you, and then show you how you can use them.

Python Flask microframework

I've mentioned Flask^[21] in other blog posts. It's a Python microframework originally inspired by Ruby's Sinatra^[5] and written by Armin Ronacher^[22]. I'm a big fan of Armin's work, which includes not only Flask but also Werkzeug,^[23] Jinja2,^[24], and Sphinx.^[25]

I find writing web applications and restful services in Flask pretty easy. But it wasn't until I was looking at my options for replacing Hyde on this blog that I discovered Frozen-Flask,^[26] a Flask extension that takes a static snapshot of a Flask application and stores it in a file system. Frozen-Flask makes Flask into a static site generator.

My Flask application is really quite simple.

• If I run it directly (python sitebuilder.py) it launches a development server on port 8000 (which Vagrant has forwarded to port 8000 on the host computer.)
• If I run python sitebuilder.py build it freezes the website into the ./build directory.
• If I run python sitebuilder.py deploy it freezes the website and copies the contents of the ./build directory to the S3 bucket, taking the site live.

YAML and markdown

Finally it comes down to content, which if I understand correctly is an important part of a blog.

My content is stored in flat files. I like this. It makes it very easy to do version control and store them at Github. In fact with Github's new online editing,^[27] I can even create and edit blog posts through their web interface.

My content is divided into "pages" and "articles". Pages are manually linked to the navigation, or referenced on other pages. Articles are blog posts and always sorted in reverse chronological order (most recent first).

Each page or article is in it's own text file. The raw file is YAML^[28] over Markdown.^[29] There's a YAML header that contains a short and long title (the short title is used for the H1 tag, the long title is used for the title attribute in links), and an excerpt or summary. Articles also have a published attribute with the date and time to show as the published timestamp. These YAML fields are arbitrary though - what they're called only has relevance if they're referenced in a template or the program logic, so any fields can be added at any time. In fact, I'll probably add tags at some point, which will require updating the controller and the templates.

After the YAML header, there's a blank line. After that, there's markdown defining the main content for the page or article. Easy-peasy.

What's next

I hadn't intended to use YAML as my metadata transport. I wanted to use JSON for all of the content definitions, and support a variety of backends including CouchDB.^[30] But JSON doesn't support multiline strings (you must embed the carriage returns) and I don't have a JSON+Markdown editor that I can connect to CouchDB. So that's a project I might take on.

Blogging is inherently organized by time, and my platform supports that. I think it'd be nice to also support tags. If I do that, I'll just use Javascript to make a faceted filter - check or uncheck tags or categories and articles appear or disappear.

Check it out

Check out the code for my blog platform on Github at https://github.com/Version2beta/version2beta.

Check out the Vagrant + Chef setup at https://github.com/Version2beta/vagrant-chef-flask. I'll be building a couple more open source versions of that getup for PHP stuff in preparation for my February presentation.

http://docs.python.org/.