Oh no, not more of the same TDD discussion Tedium.

Every once in a while, otherwise reasonable people get together to argue about TDD with religious zeal. In the most recent flare-up, I’ve been disappointed that on all sides, as nobody is saying anything new.

Yawn

At the risk of adding yet more noise, I did have a two nuanced thoughts that are at least new to me and I thought I would share them, along with a recent personal anecdote.  If this is obvious or old-hat to you, then I’m sorry.  If you think I’m  too stupid for words and that I’m drinking and/or selling snake-oil enriched kool-aid, I look forward to what will undoubtedly be informed and insightful feedback.

1. s/driven/aware/

The hardcore position of “never write a line of production code without a failing test for it” probably does more harm than good. Different kinds of code require varying degrees of effort (cost) to write and maintaintests for. Different kinds of code give varying degrees of benefit from test automation. Without always realizing it, developers make cost-benefit decisions all the time, and good development organizations empower their developers to act on those decisions.

That said, the cost-benefit decisions developers make must at least be informed decisions. A professional developer who hasn’t taken the time to learn how to use the appropriate test frameworks for their language/environment (jUnit, nUnit, whatever) is just plain negligent in 2009.  Test automation is just one tool in a competent developer’s toolbox, but a critical one.  I wouldn’t trust a carpenter who didn’t know what a hammer was, or a cardiologist who hadn’t bothered to learn about this newfangled angioplasty business.

Test-driven may not be appropriate for every context, but everyone needs to be at least test-aware.

2. s/first/concurrently/

Test-first is a really helpful approach, but it doesn’t work with the way everyone thinks, and mandating that everyone must always think in exactly the same way is the worst sort of micro-management.  The other extreme, writing test automation for a large system after it’s complete, is often prohibitively difficult and (frankly) boring as hell.

My advice is to always at least think about how you would write tests for your code before writing it. That will help keep you from painting yourself into untestable corners. Also, interlacing test writing immediately after you get a small subset of your system done is going to be much easier than testing the whole thing after the fact.  Personally, I move back and forth between writing the tests first and writing the code first. The key for me is that I’m working in short code-test-code-test cycles, using persistent (that is, I don’t throw it away when I’m done) test code as the primary mechanism for executing the code I’m writing as I’m writing it.  I don’t think of the process as being test-first, I think about testing concurrently with coding.

Recent Anecdote

Sure, anecdotes aren’t data, and they can’t prove anything, so take from it what you will

I just finished a pretty big refactoring project (that is a “pure” refactoring, the external interfaces and behavior of the existing stay the same but the underlying implementation was improved) of a system that had some decent test automation. Every time I got an edge case behavior wrong, introduced a side-effect, or removed a necessary side-effect (yuck), a test would go from green to red. This saved me at least a few days of development and testing time, and reduced the chances that I would release bugs to our QA guy (bad) or our production system (even worse).

It takes a good journalist to make software development interesting

Emily (who is best described as the heart of I Can Has Cheezburger) came into my office yesterday and looked around. “KING 5 news is going to be here, so we just want to make sure that there’s nothing offensive up anywhere”

There wasn’t. We used to have a copy of the Hot Chicks With Douchebags book as part of our display library, but that’s missing now. I was displaying my “Wrong-Face Panda” picture, but that’s meant to be disturbing, not offensive. Nonetheless, I tidied up the office a little bit and sat with better-than-normal posture as the news crew visited Ben in his little closet of an office, and then went around, poking their giant camera into different workspaces. They came to my office ,where Ben introduced us as the development team.

The news guy just shrugged and gave a sarcastic “Oh, software development, that’s really interesting” before walking away.

So I yelled back “Yeah, so is local news!”

That’s right, I actually came up with and executed a Winston Churchill-style zinger at the right time, not the day after.   I’m usually not so touchy, but that guy was just being a jerk. Think about it: would you come into some professional’s office and insult what they do?

The truth is, that the story of how we do software development is interesting. Some examples:

Our user activity is massive. We serve six million page views daily. We have a database of over four million funny pictures. A thousand new people register to participate in our community every day. We tally and act on hundreds of thousands of votes every day, in real time.

We’re a small team and we have to generalize and do more with less. We don’t have a dedicated test person, so we make our own robots to test our sites for us. We move quickly (turning around new ideas in days or weeks instead of months or years) yet we have really high stability with fully redundant systems that repair themselves if something goes wrong.  We deploy new code with zero downtime by pulling a server out of service,  upgrading it, and putting it back into service after we’ve tested it.

All of our office infrastructure is virtual. There are no desktop phones or LAN servers. We use Internet-based tools like Skype,  Tokbox, Google Docs, and ReviewBoard to collaborate. Some key members don’t even work from this office, yet they are just as productive as someone who works right here. Our office is essentially paperless, the only time I’ve ever used an office printer was for personal use (sorry).

We have created a public API which opens up our systems to any developer who wants to contribute or use our content in their programs. You can take pictures on your iPhone, drop text on them, and send them directly to us using a program that some guy made for free just because he wanted to.  We’re plugged in to the emergent web ecosystem; we integrate with WordPress and Twitter and YouTube and Digg and Facebook. We’re working on all sorts of new stuff that will be really cool.

We’re living and working in the bright and shiny future of tomorrow, as a profitable Web 2.0 company.   The truth is, I don’t really care if some old-media guy and his ever-shrinking audience doesn’t get it.

Script Frenzy 2009 – Writing a Screenplay in 30 days. Here I go.

When I was about to start my first attempt at National Novel Writing Month, a friend of mine said “Martin, you’ll have no trouble with this, you have a skill for telling things as they are“

“Thanks, but I’m writing fiction”

“Oh.Never mind, then.”

In the same spirit (and organized by the same lunatics) as National Novel Writing Month (every November) is ScriptFrenzy (every April). I participated in ScriptFrenzy in its premiere year and managed to finish a flawed but readable script for a feature film about an insecure superhero investigating a crime involving his former partner who was framed for taking performance enhancing drugs.

This year’s script is for a two-hour pilot for a television drama. It’s set a few years after a mysterious natural disaster which has destroyed every large city on Earth. My characters are (so far) a former Ju-Jitsu instructor turned fisherman turned militia leader, an agrarian bible scholar turned underground spy, a cokehead airline pilot with political ambitions, a fugitive former CIA agent taking refuge with old mafia connections, a shameless disaster profiteer, a teenage girl trying to break away from the UFO cult her family is in, a HAM radio geek turned talk radio superstar, and a cute young journalist who rides a Vespa (her character needs some more depth, obviously).

I’m working under this basic formula, but it’s changing from moment to moment:

(Left Behind – Kirk Cameron/evangelical message) + (Battlestar Galactica – Cylons/Space Travel) + (The Sopranos – pop psychology)  + ( World War Z – Zombies ) + (X Files – creepy sideshow episodes/aliens)

The best thing about writing a TV pilot instead of a feature film is that you don’t have any pressure to wrap things up. I’m going to just open up a whole bunch of messy loose ends and leave them there with an outlandish cliffhanger ending. It seems to work for J.J. Abrams.

After the first two days, I’m off to an OK start. Last night I literally fell asleep at the keyboard and typed an entire sentence with my eyes closed. Upon further inspection, I had to throw it out as it had no vowels in it. Tonight was better and I managed to get up to 8 pages (target is 100 pages in 30 days, so I’m a page or so ahead of schedule).

A big part of making a writing project like this succeed is the visible public commitment to meeting the deadline. This is the same reason that many teams get benefit from doing daily stand-up meetings, is that coders can stay focused more easily when they commit to action in front of their peers. By telling everyone that I’m doing this, it’s a lot harder to just give up silently and go back to watching Hulu or playing iPhone games.

Wish me luck.

One of the Coolest Things I’ve seen in a while.

Construx to Offer Free Training to Laid-Off Software Developers

This is a total win-win.

Laid off people get some training to keep themselves sharp when they’re not working, and a recent line item to put on their resume.

Construx gets some additional brand awareness/mind share/word of mouth (just think of all of those resumes going out to their target market). They also get the advantage of filling up their classes at a hard time to do so. Trainings like theirs (I’ve been to many) work better with a critical mass of participants.

Here’s hoping other companies will do creative things like this.

Monospace Coding Challenge

It turns out that my rant on Monospaced typefaces for coding is somewhat controversial. Although the post was supposed to be about challenging all of the arbitrary things that we do based on obsolete traditions. Jeff Atwood (who seems like a much more reasonable person on his blog than he does on Twitter) tried a similar thing when he challenged the wisdom of #region tags in C# and code folding in general. I think Jeff’s right about Region tags, by the way.

Anyway, this gem recently showed up in my comments.

Dude, fixed with fonts make code more readable, period.

And while it is difficult to argue with the the persuasive technique of sandwiching an unsupported claim between “dude, ” and “, period”, I try to be evidence-based when I can.

So, for the sake of advancing the state-of-the-art, I’m offering a challenge. I’m going to give a $20 coffee card (from your coffee shop of choice) to the first IDE screenshot which provides concrete evidence that monospace fonts are better for coding.

Official Contest Rules

Examples of letter differentiation don’t count. I’ve been coding in Calibri for more than a year now and I’ve never mistaken a zero for an () or a pipe for a lowercase l. Yes, I know that those letters may be easier to tell apart in Consolas or Courier New. Dude, people who can’t tell their letters apart need glasses, period.

The example needs to be in a contemporary (no FORTRAN, no ADA) and mainstream (no Haskell –sorry Paul, no LOLCODE –sorry Cheezburger fans) language. Dude, people who code in Haskell are academic snobs, period.

The example needs to be from a contemporary IDE such as Visual Studio, IntelliJ IDEA, Eclipse, etc. While I respect those of you who write code in vi, we’re living in such different worlds that I doubt what you have to bring is relevant.  Dude, people who don’t use IDEs with automated refactoring tools are walking dinosaurs, period.

Enforcement? You’ll just have to trust me. $20 is a small price to pay for science.

It’s Like Working in a Record Store

Here I am, posting for the first time in a very long time, which is kind of a shame as I had been meeting my at-least-one-post-a-week schedule for a while. Here’s the story:

A few months ago, I decided that it might be time for me to change jobs. While I had learned a lot working with both my client and my staffing agency, I felt there really wasn’t anything exciting on the horizon. The project I was working on was described as “wholly unremarkable” by Gizmodo, which I thought was about right.

The biggest issue for me, however, was the structure of the development organization. The people who created ideas and the people who implemented ideas were never the same people, by design. And while there was a possible opportunity for me to move into doing more idea creation and less implementation, I don’t want to go post-technical yet.  I like writing code.

I did enjoy playing the role of full-on code monkey and improving my chops at the bottom of things. I’m a much better hands on-coder now than I was.  Not only do I write better code than I used to, I’m better at talking about code. I did a handful of presentations on the whole tdd/refactoring/code qualities/design patterns universe. Trust me: nothing helps you understand where your weaknesses are better than having a room full of developers asking you tough questions about a new concept. I also got to do SDET stuff for a while, which is something that all developers should do from time to time (I have more to say on that concept in a later post).

Anyway, I cautiously started looking for something new. I put the RSS feed of a Craigslist job search into my Google Reader and waited. I only responded to a small handful of listings that seemed like interesting companies. One company could have been really good for me, but they were planning on moving from Fremont to East Bellevue. While I don’t want to come across as the latte-drinking Seattle snob that I probably am, I’ve really come to enjoy not driving to work and I’m seriously not ready to ride my bike from Ballard to Bellevue.

And then, one day, this comes up in my RSS reader:

Be a .NET Developer for Cheezburger (http://icanhascheezburger.com) (Lower Queen Anne, Seattle)
Help us make the intarwebs a better tube for millions as we develop some amazing tools and features (we’re more than just a blog under this fur!).

I’m not even huge lolcat fan, but I have realized that I’m an internet culture person. I read in an article about Internet Trolls (which I’m not, just to be clear) that their favorite video game is Portal and their favorite TV show is House. I thought “Oh no! That’s me!”. Besides the Portal/House connection, I blog (of course), have a Flickr stream, I subscribe to few dozen RSS feeds, I listen to streaming radio on my chumby. I listen to podcasts on my iphone. I use facebook/linkedin/twitter to keep in touch with people. Heck, even the times that I specifically unplug from the network (NaNoWriMo and ScriptFrenzy) are essentially internet culture things that happen largely asynchronously.

So, after three weeks of working to make the intarwebs a better tube, what’s life like at the Cheezburger factory? It’s like working in a record store, only for internet geeks instead of music geeks. There’s still a lot of real work that needs to be done, but it all essentially centers around helping people do a better job of wasting time online. Or, if you want to take more uplifting view on it: we make a few hundred thousand people happy for a few minutes every day.

Besides just being in a position to both create ideas and implement ideas, the favorite part of my job is when we get together for Thai food on Thursdays and just talk. We talk about our favorite webcomics (in what would come as a surprise to nobody, Garfield minus Garfield and XCKD are both universally loved). We wonder aloud what kind of people will come to the Fail blog meetup. Someone noticed the “NEDM” inscription on a picture of Happy Cat. It turns out that “NEDM” is an acryonym from the YTMND community which stands for “Not Even Doom Music”, which is to say that something is so horrible that not even the soundtrack from Doom can redeem it.  Yeah, Internet geeks, srsly.

One specifically cool thing about working here is that all of the infrastructure is on the cloud. There’s esesentially no LAN. We use gmail and google docs instead of exchange/office. We use the Mercurial distributed source control system. We use ReviewBoard for code reviews online. The result? I can work just about as well from home or from a coffee shop or from Europe (as one of our developers does) as I can from the office.

And now that things have settled down, I’m renewing my pledge to keep a blog posting schedule and try to write at least one thing worth reading every week.  Wish me luck.

Clearwire: Don’t get my hopes up like that!

I got a call today from a number I have never seen before from 404 area code.  Rough transcript below:

Rep: Hi, my name is [x] from Clearwire.

Me: Hello.

Rep: I want to talk to you about your clearwire service. Got a moment?

Me: Sure. 

Rep: How is your ClearWire service these days?

Me: Not great, actually. After any intensive usage (downloading, streaming, uploading) it slows down a lot. I think I’m being throttled.

Rep: Do you use a desktop or a laptop?

Me: Uh… both.  I use a wireless access point.

The full answer is that I use the wifi with two laptops, an iMac, a Wii, a PS3, a chumby, and an iPhone, but that’s pretty wordy.  At this point, I thought she might tell me to move the access point far away from the ClearWire modem. Something I had already tried which solved nothing. Instead, I get this:

Rep: Do you want to try our laptop card for free? it’s the new “2.0″ and it works much better.

Me: Uh, that won’t work, as I’ve got multiple laptops.

Rep: Well, I can send you multiple cards.

Me: No thanks.

Rep: OK then. (hangs up abruptly)

The worst part of this is that it was a sales call pretending to a be friendly check up on how I felt about the service. I thought that maybe even someone at ClearWire had read my earlier post and was calling me to say that they’ve seen the light and aren’t going to provide such terrible service to perfectly legitimate internet users. When I wasn’t going to be a sale, there was no follow up about my problem or how I really felt. Shameless. Shameful. Whatever.

All of that, and I doubt that getting the slightly faster laptop card would even solve my problem. The issue isn’t with the base download speed (1.5MB vs. 2.0MB, theoretically), it’s with the aribitrary and opaque throttling. I don’t really need to be able to hit the mysterious limit of my ability to download things at a reasonable rate just that much sooner.

Writers who don’t read, software creators who don’t use software, and The Zune Express

I write sometimes. I’m not a very serious writer, and I’m not trying to get my work published. It’s just nice to have a challenge and to use a different part of my brain. I’ve written three short novels, one is unreadable, one is borderline-readable, and the other is actually kind of OK.  I’m genuinely proud of my one Screenplay, Marc X and the Legion of Benign Disdain.  It’s a comic-book-style superhero story, told from the point of view of an insecure hero whose partner is framed for taking performance enhancing drugs.

To liven up the bus commute on non-bicycling days, I’ve put the audiobook edition of Stephen King’s On Writing on my iPod. I’m aware that Stephen King is somewhat out of fashion in some circles right now (I mentioned that I was making a novelist the main character of a story I was writing, and a good friend said “That sounds like something Stephen King would do” while wrinkling his nose) but until the best of my stuff surpasses the worst of his stuff, I’ll reserve snobbery. 

One thing that has come up in his advice to writers more than once is that you need to be a serious reader if you want to have any success as a writer. He’s clear that he doesn’t read fiction to directly improve his plotting, dialogue, or prose. He reads fiction because he enjoys reading fiction. His prose and dialogue just get better naturally with more exposure.

I’ve noticed a parallel in the software realm, where people who wouldn’t ever use a particular kind of software tool, are in charge of designing and building one. The results are predictably lackluster. The end product is stilted, like the work of a non-reading writer with a clumsy vocabulary or a flat ear for dialogue. You won’t get to greatness this way, you consider yourself lucky if it works at all.

I’ve made a conscious effort to make sure that I actually use software products similar to the ones that I’m building. I’ve been doing some media related work, and whenever I get stuck on an interaction, I ask myself, “What would iTunes do?” Not that I slavishly copy everything Apple does, but I know that they at least put a lot of time and energy into their interaction design. If humans didn’t solve problems in the context of how other people have solved similar problems, we never would have progressed beyond cave painting.

Now consider the iPod Amnesty Bin at the Microsoft Zune HQ.  I know it’s mostly a joke, but there’s still a message behind it: You had better not be seen using our competitor’s product, even though it’s a huge commercial success and generally considered one of the most well-designed machines in modern history.

And while the Zune is only in its second generation and has been steadily improving, it hasn’t exactly set the world on fire. I’ve only seen a small handful of them in the wild, while I see iPods everywhere. I sometimes joke that the Microsoft Connector commuter shuttles in the Seattle area should instead be called The Zune Express. It’s a magical, insular place where Microsoft employees don’t have to deal with the disturbing reality that nearly everyone on regular public transportation is using an iPod.

Can you imagine the scandal that would break out if you saw Stephen King reading a book by Dean Koontz? Or if you saw Michael Crichton reading a book by John Grisham? What? There’s no scandal at all. Just a recognition of the community of experts learning from each other’s successes and failures.

What’s next? I’m going to use as many software programs as I can get my hands on, and I’ve deviced to re-read The Shining with a critical eye, even if it is yet another story with a writer as the main character.

Alternative to AlternatingItemTemplate Redundancy in ASP.NET

One of the facets of good coding practices that almost everyone seems to agree on is that redundancy in code, usually via copy and paste, is a dangerous thing.

I saw a great example of this in some ASP.NET code I was working with recently. A tester had reported that some of the items in a display were being truncated inappropriately, while others weren’t being truncated at all. I looked at the screen and could immediately tell what was wrong. Every other row in the table was truncated. It corresponded perfectly with the ledger-paper-style alternating background colors.

Looking at the code, I saw the following things (simplified):


<asp:Repeater id="repeater" runat="server">
<ItemTemplate>
<tr bgcolor="#FFFFFF"><td><%# DataBinder.Eval(Container.DataItem, "Title")%></td></tr> </ItemTemplate>
<AlternatingItemTemplate>
<tr bgcolor="#CCCCCC"><td><%# DataBinder.Eval(Container.DataItem, "TruncatedTitle")%></td></tr>
</AlternatingItemTemplate>
</asp:Repeater>


 
The vast majority of the code in AlternatingItemTemplate has the same intention as the code in ItemTemplate. Whenever you have duplicated code (or near-duplicated code) you will eventually forget to update one of the two things that need to be updated in parallel. Hence the bug.

An alternative, would be to not have the duplicated/duplication-heavy AlternatingItemTemplate, but instead, call a method to get the row color, like this:


<ItemTemplate>
<tr bgcolor="<% = GetCurrentRowColor()%>">
<td><%# DataBinder.Eval(Container.DataItem, "TruncatedTitle")%></td></tr></ItemTemplate>


In the code behind (or even someplace more generalized, if you are going to use this layout more than once, of course), you could have this simple logic:


private const string initialRowColor = "#FFFFFF";
private const string alternatingRowColor = "#CCCCCC";
private string lastRowColor = initialRowColor;
public string GetCurrentRowColor()
{
 if(lastRowColor == initialRowColor)
{
lastRowColor = alternatingRowColor;
return alternatingRowColor;
}
else
{
lastRowColor = initialRowColor;
return initialRowColor;
}
}

 
One Courtesy Note: in the off-chance that my comrade who introduced this problem is reading this, I don’t mean to single you out or pick on you. It’s the sort of mistake that I’ve made a billion times myself, and wanted to capture what I felt was a reasonable solution to the AlternatingItemTemplate redundancy problem.

Also, in case any ASP.NET gurus are out there, I generally don’t like to use the DataBinder.Eval syntax, this is a simplified example, remember?

Kind of Chunderwhelmed

For my birthday (I’m now 100000 in binary) my lovely wife got me a Chumby to place in the kitchen. Specifically, it was to replace the low-quality radio that I had in there. I wanted to be able to listen to internet radio (mostly the KUOW stream) as well as view various photos from flickr.

It does both of the things that I want it to, but neither in the way I actually want.

Internet Radio

Turning on the radio consists of (1) squeeze to bring up control panel. (2) touch  ”Music” button, (3)touch to scroll down to “My Streams”, (4)touch to select my favorite station. (5)touch play. Optionally, if I want to get back to the channel/widget display, I have to (6) touch “done”, (7) touch “done” again, and finally ( 8 ) touch “hide control panel”. 

Yes, that’s 1 “squeeze” and 7 touches to turn on the radio and get back to where I was. Sometimes I want to do this with wet and/or soapy hands. 

Flickr Photos

Flickr has a cool feature where you can get the RSS feed of just about any collection of photographs. You can see the photos your contacts have posted, photos uploaded to a particular group, photos tagged with particular tags, etc. It’s a very cool system. I had assumed that I could just point the chumby at a flickr RSS feed for some of my favorite groups and always get a chance to see new different things. Unfortunately, the Flickr widget doesn’t do this. Instead, I created a simple “Chumby set” in my account, and I’m using the chumby like a simple digital picture frame, looping through the same small set of pictures. It’s cool, but not what I had in mind.

What does this mean to me as a developer?

As someone who both produces and consumes software, this speaks to the difference between a feature and a use case. “Flickr Support” is a feature, while “As an off-camera lighting geek, I want to view new strobist group photos, because I enjoy getting new ideas” is a use case. “Internet Radio Support” is a feature, “as a news junkie, I would like to switch on NPR the moment I enter the kitchen because loading/unloading the dishes is a boring low-stimulation activity” is a use case.

Too often, people are thinking/speaking in terms of features when they should be thinking in terms of use cases. This is not to pick on the good people at Chumby, I think what is more likely is that they were thinking in terms of an entirely different set of use cases than I was. 

To be fair, a lot of things work really well. The chumball game is mesmerizing. The motion sensor works much better than I would have expected. The selection of clock widgets is awesome. My favorite is the death star clock. The sound quality is quite good for such small speakers. Much better than the radio I replaced with it.  I’m still planning  on writing a few chumby widgets of my own, and I plan on getting one to use as a bedside clock-radio. I’m just a little sad that this cute funky toy is so close to what I actually want.

Progressively More Disturbing Contextual Search Results

 

I was recently searching to see if there’s a standard/ideal way to catch the “potentially dangerous input” exceptions that ASP.NET likes to throw if there’s anything that looks like script injection in the request.

Note: these did not come up in my Google results, but in a page hosting a generally useless forum discussion.

OK, this one is kind of funny. I’ve never watched the show, but I hear that it’s pretty good. I live in Ballard and drive by the signs for the boat tour all the time. I was trying to catch “potentially dangerous” exceptions, and not “deadly” ones, however. 

 

Ignoring for the moment that our culture is already awash in meaningless catch phrases and we shouldn’t encourage people to use more of them, aren’t catch phrases free? If they somehow weren’t free, wouldn’t you be better making up your own?

 

This third one is the most disturbing. Using GPS to track someone you suspect of cheating on you? Added bonus: having an active GPS installed on someone’s car would come in handy for the post-breakup stalking phase. Here’s some free advice to anyone looking into this solution: if your relationship is so bad that you feel a need to track their every move, it’s best to move on. If you need evidence, hire a private detective, because at least that’s kind of cool.

Alas, I didn’t find anything useful with the specific problem I was searching for. I’m just turning off the request validation and dealing with the potentially dangerous stuff manually.

Another 25 employees? No thank you.

I recently read an outrageous claim from a company that makes software project management tools. I’m leaving the company name out of this post as I don’t want to either single them out, or to give them publicity.

[Product X] companies were 25% more productive.  That’s like adding 25 people to a 100-person department, free!

Let’s just set aside the notion that you can accurately measure productivity of software teams in a meaningful way (you can’t). My immediate thought take on that was, “No, it’s like removing 25 people from a 100-person department. Hasn’t anyone read the Mythical Man-Month?”

Automated Test Distinctions: The Food Pyramid

This is something I came up with a while ago when I was working on a large integration/orchestration project for a now-defunct MVNO (mobile virtual network operator). It was a pretty complex project, which had many external collaborators and end-points into the system. The project was having some issues with being able to reliably deliver incremental functionality on time. I had a directive to use my judgment to essentially turn the project around.

I had just come off a project which had excellent pure unit test coverage, but not very good overall test coverage with actual collaborators. The integration points were much more difficult and painful than they needed to be. On the other extreme, I inherited a bunch of existing non-unit tests as part of the MVNO project. These were brittle, took a long-time to run, and failed for a bunch of different reasons. As a result, they weren’t run very often, became neglected, and eventually worthless.  There was no continuous integration in place, so the feedback loop between breaking something and finding what you broke was very loose.

To make matters just a little more dicey, I was working with another test-focused developer who had a strong preference for writing epic end-to-end tests, while I was trying to make the code more able to be tested in isolation. I resented the fact that “his” tests were keeping me from running the tests as part of the cruise control build.

I came up with what I called the “Food Pyramid” as a way of balancing these different forces and points of view. I broke the one massive test project into three distinct assemblies with their own goals.  The differences between the projects are best explained in a series of pictures:

So, did it work? Yes and no. We eventually had much smoother and more confident deployments thanks to the end-to-end tests. I also managed to refactor and add new code more quickly with fewer bugs thanks to my unit-test safety net. But, just as I was starting to get a handle on the project, the client filed for bankruptcy and stopped paying their vendors (which unfortunately included me). It seems that having a viable business model trumps all.

I have, however, used these and similar distinctions successfully on subsequent projects, and I know that at least one development organization I’ve explained the idea to has adopted the pyramid as part of their overall testing strategy.

Not Sure the World Needs Another Named Prescriptive Methodology

This, on the very same day that some tries to create a definitive list of methodologies, Net Objectives (the one-time sister company and training provider for the company I contract through) has just announced their new flavor/version/whatever of Scrum, which they are calling Scrum# (pronounced “sharp”, like C-sharp). It is (as far as I can tell) essentially Scrum + Lean Thinking + Emergent Design + Focus on scalability. Many things that a lot of us are doing already.

To be fair, the approach they are taking is exactly the one that I would advocate, focusing on and being mindful of the principles/values of what works and what doesn’t work. I’m just not sure I can get behind the name.

Respect for People, and Mean-Spirited Trash Talk

I was at lunch with a few developer friends a few weeks ago when a story of rather horrible code experience came up. Someone mentioned that they should send it to The Daily WTF.

“I don’t really like The Daily WTF.” I said, off-handedly.

Everyone looked at me as if I had said that I didn’t like kittens, or apple pie, or democracy.

“What?” one of them asked.

“I think it’s too mean-spirited” I said, lamely.

I didn’t always feel this way. My position on code-quality-improvement-via-harsh-mocking changed after working with a great developer (the kind that reminds me that I am merely a good developer). He was very focused on code quality and would say things like “I should hop on a plane and fly to where this guy who wrote this is and punch him in the nose for writing this method this way” or just cry out “Jesus Christ!” when he found something that he didn’t like.

Even though we always got along, it made me nervous. If this is what he’s saying about other people’s code, what’s he saying about my code? It created a sort of climate of fear. Not only that, it didn’t do any good to actually improve the codebase as a whole. We still had a team member who insisted on writing things in his own way. He brushed off valid concerns as “well, that guy just hates everything”.

This culture of harsh criticism is what leads people to be generally closed and insecure. Personal example: I thought about releasing the source code to my little Survival-Horror Asteroids game, but I haven’t yet, for fear of having some sharp-tongued rock star developer come along and tear me to pieces. 

None of this is to say that you shouldn’t be discerning. I’m as concerned about the distinctions between (and definitions of) good code and bad code as much as anyone else. I’ve literally had nightmares about having to work with a particularly nasty procedure. I’ve just learned to not be so insecure that I have to insult and belittle people over it.

This isn’t just a problem with developers, either. I enjoy reading about design and usability, and I found a great article about a particularly mean-spirited Flickr thread in response to a screenshot from an iPhone app. The app itself may not look great, but the developer had done nothing to warrant the vitriol of the responses. 

How to make things better:

1. Be kind and respectful, even to inferior coders. When I first read about the Lean Software Development concept of “Respect for People” I read it as “Respect the developers, especially Martin”. But it’s actually omnidirectional. Respect the other developers. Respect the folks in marketing, respect the pointy-haired-bosses (the first step, of course, is to stop calling them pointy-haired-bosses). 

2. Think incrementally. If you’re working with a developer who needs to improve, select a few distinct values-based things that need to be changed when you provide feedback.  If you try to fix absolutely everything all at once, you’ll fail.

3. Be constructive. Comments like “my eyes bleed” or “you, sir, are too stupid for words” don’t help do anything besides inflate the ego of the insult hurler. Comments like “if you do an extract method refactoring instead of copy-paste, it will be easier to understand and maintain” or “some stronger lines of alignment would make this easier to understand” are helpful.  

3a. (update, July-19-08) Realize that reasonable people can have different opinions and pick your battles. I’ve stopped getting hung up on stylistic things such as formatting or casing. I’ve learned to adapt to the consensus of the project. It’s a useful skill.

Also, realize that there’s generally no one right way, most things are tradeoffs. In this de-normalization example, the author is looking into trading one kind of pain (arguably slow joins and more complex queries) for another kind of pain (risk of data anomalies). It’s generally not the tradeoff that I would make, but it surely  doesn’t warrant the “you have no business designing databases” hate that he got in the comments. 

4. Realize that mistakes are OK and a necessary part of progress. Some developers (including me sometimes) get defensive or insecure when people find bugs in their work. The best developers I’ve worked with never are. It’s not that they never make bugs, but they don’t take their bugs personally.

Is Clearwire right for you? Take this simple quiz!

Alas, the curse of the early adopter.  I was really excited to switch to Clearwire, especailly after my beloved ISP for many years, Seattle’s own Speakeasy Networks was purchased by Best Buy. I’m not exactly a Best Buy fan.

I should have done some more research before signing up, though. It turns out that Clearwire is actually not for everyone, and the things they don’t support well are not really called out in the terms of service. To help future consumers, I’ve put together a simple guide:

 

Clearwire is not for you if you’re a pirate.

Intellectual property pirate, that is. I think that the company has no problem with peg legs and eye patches. Reading some reviews online, I found that ClearWire openly and aggressively throttles traffic from P2P apps such as BitTorrent. No big deal, I almost never use BitTorrent, and I never ever pirate movies, software, or music. 

 

Clearwire is not for you if you want to watch perfectly legal web video.

Sure, BitTorrent is used largely for piracy and/or pornography, sure the copyright status of YouTube videos is often dicey, but what about Hulu? Hulu is the Republican National Committee of web video, as square, legal, and business-friendly as it gets.  Hulu also provides some of the most conclusive proof that My connection is throttled, as I can reliably watch the first half of an hour-long drama no problem, but the second half is unbearably slow.  Letting me watch the first half of a really engaging show like House but not the second is just mean-spirited.  Now I’ll never know if it was Lupus, Vasculitis, or Paraneoplastic syndrome!

 

Clearwire is not for you if you want to listen to internet radio.

OK, maybe streaming video is too much to ask for even if the video bitrate is well within the download bitrate I’m paying for. How about streaming audio? Well, if I’ve been doing anything network-intensive in the last few hours, the live audio stream from my favorite NPR station to my Chumby stops working.  

 

Clearwire is not for you if you’re a digital photographer.

Every time I try to upload more than a dozen or so pictures to Flickr via the Uploadr, I hit whatever mysterious upload cap there is and the last of them just completely fail with the “Houston, we have a problem” error message. Just like with Hulu, it works fine for a little while before some opaque limit is hit and then it grinds to a halt. How important is this really? It’s not like very many people actually use Flickr.

 

Clearwire is not for you if you need to work from home.

After several years of working from home exclusively, I try not to work from home any more than I need to, but when I need to make a  VPN/Remote Desktop connection to my PC at work, it works out pretty poorly. I know the problem isn’t with my company’s network connection, as when I do VPN/Remote Desktop from a coffeeshop with Wi-fi, it’s like I’m at my desk. Maybe this is meant as some sort of feature? Helping me improve my work-life balance? 

 

Clearwire is not for you if you’re a Linux user.

As linux users tend to be power users who not only download huge ISOs, but also download ISOs using the infamous BitTorrent network. Fortunately, for me, I haven’t been a Linux user for some time.

 

Clearwire is not for you if you’re a Mac user.

This was particularly painful. The first day with a gorgeous new iMac, some software I wanted to run required a large OS X update… which by the end of the download, was coming down at sub-dialup speeds. 

 

Clearwire is not for you if you’re a Microsoft Windows user.

Even more painful and disruptive to download than the OS X updates are the frequent, vital, and huge updates that Windows XP needs.

 

Ouch. That has turned out to be a very negative summary above, and I usually try to be a positive person. So, who would Clearwire be ideal for? Here are some groups of people that I can think of for whom Clearwire would work well enough:

• The last holdouts of the gopher protocol, those who are convinced that this whole web/multimedia business is just a passing fad. 
• People who want high-speed internet but don’t actually have any computers or network devices.
• Dead people.

Agreement on the Lean-Agile Connection

I’ve been arguing (mostly unsuccessfully) for some time that Lean Software Development and Agile are two different intellectual frameworks for understanding the same underlying concepts. The artificial distinctions that people draw are often based on connotations of the words themselves, such as “agile is about fast while lean is about cheap”.

Now, in a shameless appeal to authority, I find that Martin Fowler sees things the same way. From a recent post:

You can’t really talk about them being alternatives, if you are doing agile you are doing lean and vice-versa. Agile was always meant as a very broad concept, a core set of values and principles that was shared by processes that look superficially different. You don’t do agile or lean you do agile and lean. The only question is how explicitly you use ideas that draw directly from lean manufacturing.

Right on!

One thought on drawing ideas directly from lean manufacturing: The ideas that work in both manufacturing and software (such as empowerment, incrementalism, tight feedback loops) don’t necessarily work in software just because they work in manufacturing. That’s a naïve mindset that can lead you to all sorts of bad conclusions.  Some ideas work in both due to the nature of the people involved in both. 

It’s easy to get ideas from manufacturing as it’s a human endeavor that’s more mature and has been very well studied and documented.  An observant and mindful person could surely draw lessons from all sorts of places.  It’s possible that someday we’ll all understand software development well enough that we won’t need to borrow concepts from other disciplines. It’s clear to me that we aren’t there yet, though.

Finally, I get to talk about Lean Software…

…even if it’s only for a few seconds.

I occasionally do training seminars on software development technologies. Some are internal to my company, some are external.  Most are on the more technical aspects of Agile software (using the definition of Agile as “any modern software development practices that work”) such as TDD, design patterns,  and refactoring. What I really care about and want to talk about are Lean Software Development and other more overarching values-based approaches to making software, but there doesn’t seem to be the same level of interest. Until today.

This morning, while I was in the elevator at work, I overheard one passenger (total stranger) ask another “so, what does “kanban” stand for anyway?” the other passenger said she didn’t know.

So I pounced!

I managed to cram a pretty good history of how kanban came from the Toyota Production System into a few floors of elevator travel. And I get the sense of satisfaction that comes from making someone’s day a little more strange.

Checklists are Your Friends (surgery, photography, testing, coding, marketing)

The headline of the local Seattle Newspaper yesterday caught my eye. It was about how surgeons at the UW (my Alma Mater) are now using aviation-inspired checklists to make sure they don’t, you know, leave stuff inside of people. This resonated with me, because I had resolved to making checklists for photo gear packing after a near-fiasco last weekend.

I was taking wedding reception photos as a favor for an old friend. For the most part, I hate the entire idea of doing wedding photography. The risk/stress/reward/effort ratios don’t work out right.  This one actually worked out really well, as the couple and wedding party were all great. They also had a “real” photographer working (who had more invested in one lens than I have in my entire setup, alas) which freed me up to take some more experimental pictures, such as HDR still lifes of the venue, controlled motion blur shots of people dancing, and grainy black and white candids (I love tmax 3200 because of the grain, not in spite of it).

I took special care to make sure that all of my camera and flash batteries were charged before the event, but I didn’t spend a lot of time packing the bag before I left. I just threw everything into the camera bag and ran out the door. When I got to the venue and started putting all of the things together, I found I was missing a small but vital piece, the caddy that holds two batteries and slides into my battery grip. Without it, my DSLR wouldn’t work. Panic!

After calling home 15 times, I finally got in touch with my wife, who graciously drove across town with a small piece of plastic so I could actually turn the camera on. Fortunately, I brought a film SLR as a backup, and had just finished my last roll when she pulled up.

If I had made a simple packing checklist, much pain and fear would have been avoided.

And while I’m not a fan of oppressive standards, heavyweight processes, or detailed artifacts in software development (my thinking is that if you’re constrained by your conventions, the best you’ll ever be is conventional) simple “have I forgotten this” checklists are insanely valuable.  Based on what I’ve seen, they’re also underused. 

My first job in software was split between testing and support for a small company making technical graphics software. The testing department was pretty unstructured. We had reasonable automated test coverage (horrible by today’s standards, but OK for the time) but all manual testing was, “Hey Martin, I just fixed this bug, go poke at it!” and “We’re realeasing a beta next week, test everything!” and “We’ve got a release candidate, get the people from sales and marketing to play with it!”

So, for no other benefit than my own confidence, I made a some checklists, just to keep from forgetting to test specific features/permutations.  Eventually the company adopted my checklists and handed them around when we did our “all-hands” pre-release testing.  Just asking people (including me) to be mindful of the feature list when they were doing their exploratory testing helped us find many problems as well as places where we could improve the user experience.

I saw this again a few days ago when I was looking at a web application that made a lot of pretty common security mistakes (no SSL for login, emailing passwords in cleartext, etc.). “These are all essentially solved problems,”  I thought, “Shouldn’t there be a checklist for this sort of thing?” 

Many experienced folks have a sort of mental checklist. Things they know instinctively to look for. Like my old mentor who would always enter “O’Brien” into name fields to catch inappropriatley escaped SQL input. This is one of the reasons why domain knowledge is so valuable. How do we get people to capture and share this domain knowledge? Couldn’t a development group be able to use a set of mature “don’t forget to think about thing X” checklists as a competitive advantage for design/development/testing?

While writing this, I’m reminded of my favorite marketing professor at the UW. After spending a whole quarter discussing different theories, reading case studies, and pulling examples from real-life companies, the last day of class she said (and I’m paraphrasing here, because that was some time ago).

In the real world, when you start in marketing professionally, it’s just checklists: Have I identified the market for my product or service? Have I explained my offering to someone who doesn’t understand it? Have I underpriced? Have I overpriced? Am I saying something stupid or offensive in this ad? How will my competitors react to this change? How will my customers react to this change?  The answers are all easy, but bad marketers forget to ask the questions.

Sure, that doesn’t get you to greatness (or even guarantee goodness), but it at least keeps you from forgetting about the obvious.

Another Decompression Artifact Example: Domain-Specific Languages (DSLs)

This is a particularly odd example, as the three-letter-acronym “DSL” is already in wide use as the name of a technology nobody really understands (Digital Subscriber Line). It reminds me of the time Microsoft rolled out the acronym “DNS” for Digital Nervous System. This isn’t quite that bad, but is still pretty confusing.

Martin Fowler is been writing about Domain Specific Languages for some time now. He’s apparently even writing a book on the subject at the moment. For the longest time, I just ignored blog (and bliki) posts on the subject, as I read the title and immediately assumed that it was something that wouldn’t work for me. It was only when a friend said “no really, you need to read about this” that I actually got it.

In the style of my previous decompression artifacts, I’ve created one for DSLs.

Once you get past the almost inevetable initial misunderstanding of what it actually is, creating a domain-specific, fluent, expressive interface for your domain objects is a really cool idea. It’s also something you can work with incrementally. If you just change every void method ot a “this” method, none of your existing clients will break, and people who aren’t down with DSLs don’t have to do anything any differently.

On a side note: I recently did a TDD presentation for a large-ish company in Portland that has been working with ThoughtWorks for a while. One of the attendees said to me, before I started, “Oh, someone said Martin was doing a presentation and I was expecting Martin Fowler.”

Ouch. Talk about living up to high expectations. I’ve been a big fan of Fowler’s work (and of ThoughtWorks in general) for a while now. There are actual new ideas in his writing, while I just organize, package, and give context to existing ideas. 

While daunted, I think I managed to put on an OK presentation.

The solution here is not process dogma

The other day I was discussing a process/dev workflow problem with one of my friends. I managed to get a basic understanding of what the problem was (team disagreement about the importance and sequence of sprint reviews, retrospectives, and planning) but we were both too pressed for time to brainstorm for a solution. He had a meeting to attend, and I had a demo to give.

The only advice I gave was, “The solution here is not process dogma. You can’t just fall back on the Scrum rulebook and say ‘we’re supposed to do it this way’, you have to get to the value of why you should do the previous review/retrospective before the next planning.”

The value, of course, is that you should be taking what you learned from the last sprint and using it to inform your actions of your next sprint. It’s Scrum’s larger-scale feedback loop (the small feedback loop is the daily meeting).  Inside or outside of Scrum, feedback loops are important for making software well.

When I was thinking about it later, I realized that process dogma is never the solution. Software development is intellectual work, and to make a persuasive case with skeptical people, you have to do better than “because the book says so.”

Code Smells, Correlations, and Poisoned Coffee

Winston Churchill had a much faster and sharper wit than I have. Consider this* famous exchange:

Lady Astor: “If you were my husband, I would put poison in your coffee.”

Churchill: “If I were your husband, I would drink it.”

I, on the other hand, always think of the correct thing to say in an argument a few days later. It’s not just for ex-post-facto arguments, either. I’ve been doing some technical trainings lately, and I try to have a more conversational style than just reading lots of bullet points from slides. I figure that if I don’t understand a topic well enough to speak about it extemporaneously, than I have no business talking about it.

In the last presentation I made, I talked about the practice of refactoring and the concept of code smells.  I gave examples of a few of high-value smells and then gave this little wishy-washy disclaimer.

“When you encounter these code smells, it doesn’t mean that you have to change it, it just means that you should look at your code closely here, as you may have problems.”

It’s not so bad, it’s pretty much what everyone says about refactoring. What I should have said, was this:

Code smells are correlations to quality problems. Heavily commented code blocks aren’t necessarily bad, but lots of comments very strongly correlate to readability problems. You don’t fix it by deleting the comments. You fix it by making your code readable enough to stand without the “how does it work” comments.

Long methods aren’t necessarily bad, but long methods very strongly correlate to cohesion problems. It’s possible, and sometimes required, to have a long method that’s perfectly cohesive, but it’s outside the norm.

And, of course, you don’t fix the problem based on the correlation to the problem, you fix the actual underlying problem. Breaking a long ProcessThings() method into three arbitrary methods called StepOne(), StepTwo(), and StepThree() doesn’t actually make the code any better.

You see, that’s not wishy-washy at all, and it appeals to the distinction between correlation and causation. It’s not as funny as poisoned coffee, but it has some concreteness to it.

—

*After looking up the Astor/Churchill exchange, I found that it’s very possibly an apocryphal story. Oh well, it’s still funny.

Testing Abstract Classes In Isolation

In my post about guilt as a code smell, a comrade pointed out that it’s perfectly possible to test abstract classes in isolation, you just make a concrete derived class as part of the test (thanks Craig!). Having a distinct concrete subtype for testing is something I’m already doing a lot with endo-testing, so it’s not even totally without precedent.

It does still bother me, though, and I’m not 100% sure why. Some thoughts:

Let’s say your abstract class is using the Template Method pattern, where it has a public method which just delegates “downwards” to abstract methods with different overridden implementations. This is a perfectly good use of an abstract class, yet it seems kind of pointless to test in isolation, as you’re going to be testing each implementation and will be testing those base classes anyway. 

The scenario that I had in the other post had a different kind of abstract class, with no public methods, just protected ones. I’m just achieving zero redundancy by moving code logic upwards in the tree. Testing here becomes trickier, as there’s no publicly exposed design surface. Should I just make one up? The right solution, for me, for that time, was not to test the abstract class, but to move that logic into a distinct service class so I could work with it more directly.  It’s textbook “favor composition over inheritance” design patterns stuff.

In any case, I stand corrected. It’s absolutely possible to test an abstract class in isolation. It is impossible, however to test a base class in isolation without testing its abstract class. Which could make testing the abstract class in isolation kind of pointless.

Narcissism

I was reading the post on Narcissism over at the (always excellent) Coding Horror Blog and I was reminded of something that happened to me.

Many years ago, I was disappointed by my job. I was working at a small consulting company which had been mostly a software development outfit and was transitioning to being more of a general marketing consultancy. I didn’t mind that as much, I have a greater love (or tolerance, depending on your perspective) for marketing than the average developer. Actually, I was driven out by a top-down mandate to “improve our process” which was manifest around trying to adopt development methodologies that were state-of-the-art thirty years ago. It felt as if they were saying to me “all of those things you did to make the last project you ran a big success (tight feedback loops, close developer-to-customer collaboration, design for change, light up-front-documentation)… well, don’t do any of them again.”

So I quit.

In retrospect, I think it would have been possible to improve the situation, but I didn’t yet have a solid handle on why I was doing the things I was doing and how to help a software development team be more effective. I was still essentially “in the closet” –embarrassed that I didn’t want to write 200 page functional and technical spec documents before doing any actual coding. This was before the Agile/Scrum concepts were as mainstream as they are now.

When I was interviewing for another job, I found myself absolutely enthralled by one of the interviewers. He seemed like the single most brilliant developer I had ever met. I was so eager to work with this guy, that I let myself overlook a bunch of warning signs about the organization (legacy code base, previous-generation languages/tools, uninteresting problem domain, deplorable office space, disrespectful management, etc.). I wound up taking the job.

Many months later, as I was trying to figure out how I came to be in the horrible situation I was in, I came to the shameful realization that I thought this guy was brilliant because he reminded me of myself. It was narcissism, plain and simple. Ever since then, I’ve been cautious to think about why I think someone is so amazingly smart.

The same phenomenon came up again last summer, when I did a “Pragmatic TDD” seminar presentation for a handful of development companies. After one presentation, a guy came up to me and said “Your presentation was great. Just brilliant. This is exactly what I’ve been advocating we do for forever.”

Of course I brilliant, I was just like him.

Now, I’m trying to do a better job of being honest with myself, challenging myself, and listening intently to those I immediately disagree with.

My continuing love affair with ReSharper: Indicating Recursive Calls

I generally believe that comments should be about “why is this code doing this” instead of “what is this code doing” because if you feel you need comments to explain what your code is doing, it could probably be refactored to be more readable and/or intention revealing.

One exception that I’ve often made is for recursive calls. I generally write a comment indicating that the method is about to call itself and why. This may be left over from my first CS class at the University where I had some kind of mental block around recursion, I don’t know.

So, I was delighted to see ReSharper give a little visual indication of a recursive call in the margin.

This useful when doing recursion on purpose (this example was using reflection to populate test data into properties, I would recurse when a property was a complex type), but it’s even more useful if you do recursion on accident, as in this public property returning itself below.