To TDD, or not to TDD?

Summary: Some thoughts on Test-Driven Development (TDD).

A few weeks ago, Stack Overflow co-founders Joel Spolsky and Jeff Atwood discussed criticized certain aspects of Test-Driven Development (TDD) and SOLID Principles of Object-Oriented Design (OOD) [the latter advocated by Robert C. Martin, AKA Uncle Bob]. Check out the excerpts from the original podcast and the follow-up talk (you may also want to listen to the Scott Hanselman's interview with Uncle Bob, which prompted Joel's rant):

• The Stack Overflow: Podcast #38, Excerpt (14:32) [see also podcast page and transcript]
• The Stack Overflow: Podcast #39, Excerpt (18:35) [see also podcast page and transcript]

As expected, the discussion criticism triggered rather emotional rebuttals from TDD supporters, including Uncle Bob himself; see (and read comments):

• "Quality Doesn't Matter That Much" -- Jeff and Joel
• An Open Letter to Joel Spolsky and Jeff Atwood

And then, a counter-rebuttals by Jeff Atwood:

• The Ferengi Programmer
• Real Ultimate Programming Power

Eventually, Joel, Jeff, and Uncle Bob met for a makeup session, which you can find -- along with the summary -- at:

• The Stack Overflow: Podcast #41

Although all participants of the podcast #41 seemed to agree on many issues, I have a feeling that the opposing sides stuck to their points of disagreement: uncle Bob remained convinced that TDD could do only good (the more TDD, the more good), while Jeff and Joel continued emphasizing that other aspects of software development may be more important than TDD.

I happen to lean more towards Jeff and Joel for the following reasons (in addition to the reasons mentioned in The Stack Overflow podcasts).

First, Joel mentioned the issues related to GUI testing, but it's not just the GUI: the database layer poses another challenge. When unit testing database-driven apps, a common approach is to rely on mock objects (in which case, you totally skip database testing) or use test-specific data in a simplified version of the database (here you will have to figure out how to maintain this database and integrate it into your build process). Even with the 100% code coverage of the business layer, if your unit test skip the GUI and database, about half of the application (more or less) will remain untested, and you'll have to address this gap using functional testing, which will overlap with unit tests. So why spend time writing unit test for the code that will be tested functionally anyway?

Second, claims that TDD improves code quality are rarely, if ever, substantiated by data. You may hear statements like:

"Since I (we) started using TDD, my (our) code has become much better."

I'm not claiming that these statements are false, but without data they are just personal opinions. Even assuming that code did in fact improve, could it be due to other factors, such as better programming experience, tools, team composition, processes?

Analyzing effects of TDD on code quality is extremely difficult. One of the better studies on the subject was conducted by Microsoft and IBM (see the 8-minute interview with one of the researchers and the original paper). The study found that TDD improved code quality, but it also increased development time.

There is a trade-off: either use TDD and spend more time writing, maintaining, and running unit tests now or do not use TDD and spend time fixing bugs later. The question is: how much time? If TDD activities take 2 weeks out of an 8-week development cycle (a typical 25% TDD penalty) to prevent defects that could've been found and fixed in 3 days during functional testing, would TDD give you the best return on investment? By the way, it is worth noting that the team with the smallest test coverage (62%) in the study achieved the best results (90% drop of defects), while the team with the highest test coverage (95%) achieved the "worst" (as compared to other teams) results (40% drop of defects; still not bad when compared to the control group). I would also emphasize that the use of TDD in the study was close to ideal: the projects were appropriate for TDD and the teams were not pushed to achieve the maximum code coverage.

The Microsoft-IBM study had a few problems, though. First, the number of participants was rather small. Second, and more important, it tried to compare similar teams and projects, but there were many difference between them, such as team/project sizes, and project durations, which is expected in a study focusing on real life projects. Studies performed in academic settings -- with undergraduate students of approximately the same grade working on identical projects -- attempted to minimize some of the differences and make experiments more controlled. According to an abstract from one of such studies:

"[T]est-first students on average wrote more tests and, in turn, students who wrote more tests tended to be more productive. We also observed that the minimum quality increased linearly with the number of programmer tests, independent of the development strategy employed."

How this sounds depends on how you interpret the study's findings. If you're a TDD supporter, you may join Phil Haack and announce that Research Supports The Effectiveness of TDD. Or, you can look more closely at the results of the study and come to a different conclusion. For example, Jacob Proffitt suggested (see also post comments) that:

"[W]ithout question, testing first leads to having more tests per functional unit. The question is if this is valuable. This study would seem to indicate that this is probably not the case, at least if quality is your intended gain. But then, I'm not that surprised that number of tests doesn't correspond to quality just as I'm not surprised that the number of lines of code doesn't correspond to productivity."

I would also point that the code in the experiments was not written for database-driven apps with web-based GUI, so again it was an ideal usage of TDD. So I would speculate that if either of these studies focused on database-driven apps with web front ends, the results would probably be even less encouraging.

I'm not an opponent of TDD and agree that it can be valuable when applied to certain types of projects. My gripe against TDD is that it is being often sold as yet another silver bullet and forced into adoption indiscriminately, in the projects where TDD offers minimal, if any, benefits.

I also resent the assumption that TDD always leads to good code. Your My project can have 100% code coverage, but it can still suck. It will pass all unit test, alright, but at the same time it can be unreadable, unnecessarily complex, uncommented, resulting in an app with bad GUI and unintuitive behavior, an app which both the users and support teams hate. But don't you dare to criticize my code: since it's 100% unit tested, it's good by definition. Usability testing? Never heard of it. And as far as everything else goes (like comments, better design, etc), do I have time for this when I spend my better hours working on unit test? Something's gotta give.

Finally, what about all those great apps written by developers, who did not use TDD or any other silver-bullet-like methodology? Take Google developers, who apparently do not follow any software development methodology or approach, but somehow manage to write apps that people love. If they can write great apps (which is one of the goals of good code), why would one want to impose TDD on them?

Now, Google developers are good, but what do you do if your developers are bad? [I'm not talking about developers who make occasional, sometimes serious, errors (this happens even to the best of us); I'm talking about developers who consistently write bad code.] Wouldn't TDD help them write better code? I wouldn't expect it, but if you can share a success story, please leave a comment.

If your organization suffers from bad code, consider adopting TDD (when applied correcly, it may help), but more importantly, consider other factors, such as:

• The Joel Test: 12 Steps to Better Code by Joel Spolsky
• What Makes Good Code Good? by Paul DiLascia

Once you the adopt common-sense practices, you may not need TDD after all.

See also:
Hanselminutes Podcast 146 - Test Driven Development is Design - The Last Word on TDD
Hanselminutes Podcast 31 - Test Driven Development
TDD Tests are not Unit Tests by Stephen Walther
Test-After Development is not Test-Driven Development by Stephen Walther

Blogging and podcasting: personal syndication for fun and profit

Summary: Presentation with links and references, which can help you find, use, and produce blogs and podcasts.

I recently gave a presentation at work which my teammates seemed to like. The presentation discusses blogging and podcasting and addresses such questions as:

• Why would you want to read blogs and listen to podcasts?
• Where can you find interesting blogs and podcasts?
• How do you start a blog?
• How dow you produce a podcast?
• Where do you store the podcast files?
• How do you stream audio and video podcasts?
• And many more.

Coincidentally, at about the same time, a couple of (non-technical) friends asked me about the very same topics, so I thought I would make this presentation available online to help both technical and non-technical -- okay, make it semi-technical -- people, who are just starting on the path to blogging and podcasting.

Here is the inline version of the presentation (the best part of the presentation is links to resources and articles, so don't miss them):

Blogging and podcasting from Alek Davis

If you want to access the presentation directly, use the following links:

• Blogging and Podcasting presentation (view the Flash version)
• Blogging and Podcasting presentation (download the PowerPoint version)

By the way, I was not sure which presentation sharing/hosting service to use, so I started investigating by reading two excellent reviews at Digital Inspiration:

• Web Based PowerPoint Presentations: Upload and Host PPT Slideshows
• Where to Upload PowerPoint Presentations On the Web and Why?

First, I tried the services not mentioned in the reviews: ThinkFree's Online Office and Microsoft's Office Live Workspace. I did not like either of them. ThinkFree removed all hyperlinks from the presentation and did not offer any way to put them back, while Office Live's sharing screen looked rather ugly (dunno, maybe I did not use it correctly). Anyway, I tried Scribd recommended by Digital Inspiration, and I really liked it. I suspect that there may be better options (such as SlideShare), but Scribd fulfilled my immediate needs; I may try other services in future, though.

UPDATE #1: Apparently, Scribd turned off hyperlinking capability from the online presentations, which is a shame. I'm now back to square one: looking for a better online presentation hosting service.

UPDATE #2: I just found out that Docstoc offers pretty much the same features as Scribd, but it also supports embedded hyperlinks. Kudos to Docstoc. I updated the embedded presentation (and reference to it) to point to the Docstoc version.

FM transmitters or cassette adapters?

Summary: Pros and cons of using FM transmitters and cassette adapters for listening to digital music in a car.

I do not get auto manufacturers. At the time when Ford is promoting SYNC (voice-activated audio) and BMW is pushing iDrive (entertainment system, which is so high tech that few drivers manage to master it), it is hard to find a popular model (Toyota, Honda, etc) which comes with a USB or AUX line-in port. If you want to listen to digital music on the go, you'll either need to buy an expensive upgrade, install an after-market audio system, or use an add-on, such as FM transmitter or cassette adapter.

I did not want to mess with my factory-installed stereo, and being too cheap to buy an overpriced upgrade, I tried my luck with FM transmitters. First, I wooted a couple of AudioBugs ATB-350 made by Aerielle (they seem identical to certain Phillips, Kensington, and iRiver models). I also bought another cheap model from, I think, Meritline (unlike AudioBug, which could be set on any FM frequency within the common range, this model supported just a handful of preset frequencies). I have been using an AudioBug FM transmitter for about six months until it suddenly died. Having had other problems with AudioBug before (one of the transmitters I bought was defective, so I had to send it to the manufacturer for a replacement), I decided not to push my luck and bought a cassette adapter (actually, I tried a couple of different models). Here is how I would compare these devices.

FM transmitter has a better form factor: when plugged into a cigarette lighter, it is more pleasing to the eyes (the wires of a cassette adapter hanging from the car stereo look gross). In all other respects, FM transmitters mostly disappoint.


First, in urban areas, it is hard to find an FM frequency not taken by a radio station. Even in a such semi-rural location as Sacramento, CA, I could not find a single available frequency. The one I ended up using was free most of the time, but not always (when it was not free, the station signal was weak enough to be overtaken by the signal from my FM transmitter). A word of caution: if you decide to buy an FM transmitter, never get one with a limited number of presets. The other one I had was totally useless because none of its built-in frequencies worked in my area.

After finding an available frequency, synchronizing your transmitter with the car radio becomes a challenge. First, the LCD of a typical FM transmitter is really tiny and, when the transmitter is plugged into the cigarette lighter, it is not clearly visible from a normal driving position. Switching the active frequency when driving is only possible from the back seat of the car and requires either an outside help or Mr. Bean's driving skills. With cassette adapters, this is not an issue.

Once you synchronize the frequency between the FM transmitter and radio, you will find out that the audio signal is rather weak. To get to a decent level, you must either raise the volume of the car's stereo or the volume of your digital music player (raising the volume of the music player beyond certain threshold, about 70-80% of the max, will cause sound clipping). Cassette adapters normally provide a stronger signal; in my non-scientific experience, it is about 20% stronger.

An FM transmitter may look nice in a cigarette lighter, but when plugged in, it will not let you charge the audio player unless you use a splitter (finding a splitter that works is a challenge; none of the cheap ones I tried worked). When using a cassette adapter, you can always charge your device (I find this very handy).


The only negative aspect of cassette adapters (other than aesthetics) is a clicking noise made, I assume, by the tape rolling mechanism. The noise made by the first unit I bought was so pronounced that I had to return it. The second model was quieter, but still I can hear the noise at low volumes; it does not bother me much, though. I'm not sure if more expensive cassette adapters (my model is very cheap) have this problem.

The bottom line: for listening to digital music in a car, get yourself a cassette adapter; just do not get the cheapest model and if not satisfied, exchange it for a different one. If you cannot use a cassette adapter (say, your car stereo does not have a cassette player), get an FM transmitter which allows you to tune to any FM frequency (not just to a limited set of frequencies).

Additional references:
How to Listen to Digital Music in Your Car

Wait wait... fix this podcast!

Summary: See how you can fix unplayable or corrupted podcast files; learn how to split long podcasts into shorter segments.

Every Sunday, a local NPR station plays my favorite show Wait Wait... Don't Tell Me!. Because the 11:00 AM-12:00 PM broadcast time is rather inconvenient (for me), I subscribed to the show's podcasts. The podcasts play fine on computer, but my MP3 player cannot handle them: after playing several seconds of introduction, the player pauses briefly, then makes a loud screeching noise and moves to the next track. I tried these podcasts on other MP3 players with no luck (although the same players play podcasts of other shows just fine). After several weeks of troubleshooting, I finally determined the root cause of the problem: apparently, the MP3 player does not know how to handle a short (less than 2 seconds) silence gap that follows the introduction, so it jumps to the next track. Eventually, I figured out how to make podcasts playable on my MP3 player and more manageable. These are the steps I follow.

1. Subscribe to the podcast RSS feed (optional)
If you like a particular show, instead of looking for new podcasts yourself, you can get them via an RSS (or Atom) feed. If you computer is always turned on, you can use a desktop-based aggregator, such as Juice. The aggregator will check if any new podcasts are available for your subscriptions and download them automatically.


Alternatively, you can use an online aggregator, such as Bloglines, which will monitor your subscripions and display new podcasts when they become available. An online aggregator allows you to access podcast feeds from any computer, but you will need to download MP3 files yourself. When using Bloglines, click the Enclosure link to download the podcast (MP3) file.


To subscribe to a podcast show, you need to know the address of the podcast feed, which should be listed among the subscription links. For example, NPR provides a directory of available podcasts, where you can find links to such shows as NPR: Wait Wait... Don't Tell Me (http://www.npr.org/rss/podcast.php?id=35), NPR: Car Talk (http://www.npr.org/rss/podcast.php?id=510208), and others.

2. Remove silence gaps from the podcast
Once you download a podcast (MP3) file, you need to make sure that it does not contain excessive periods of silence. If you don't know any better, you can remove silence using the free Audacity tool. Notice that to use Audacity for MP3 editing (including silence removal), you have to install LAME (a popular MP3 encoder). [These instructions and screenshots apply to Audacity 1.3.2 (Beta); newer versions of the application may look differently and require different steps.]

After installing LAME, do the following (you need to do this only once):

1. Start Audacity.
2. Select the Edit - Preferences menu option.
3. In the Audacity Preferences dialog box, select the File Formats option from the list box on the left side.
4. In the MP3 Export Setup section, use the Find Library button to locate the lame_enc.dll file (this file must be in the same directory where you installed LAME, such as C:\Program Files\Lame).
5. Make sure that the Bit Rate value in the MP3 Export Setup section is set to 64; a larger value (such as 128) will increase the size of the modified podcast files. Notice that while the 64 bit rate is good enough for podcasts, it is insufficient for music files; when using Audacity for music editing, increase the bit rate value to at least 128.
   
   
6. Click OK, to close the Audacity Preferences dialog box.

Once you make sure that the Audacity's MP3 export settings are configured correctly, open the podcast, trim silence, and export results to an MP3 file following these steps:

1. Open the podcast using the File - Open menu or by dragging and dropping the MP3 file onto the Audacity window (Audacity may take a minute or longer to open the file).
2. After loading the podcast file, press CTRL+A keys to select the whole track.
   
   
3. Select the Effect - Truncate Silence menu option.
4. In the Truncate Silence dialog box, set the value of the Max Silent Duration (milliseconds) to 250 (you can use a slightly larger or smaller number) and click OK (truncating silence takes about one minute).
   
   
5. If you find the volume of the podcast too low, you can increase it by selecting the Effect - Amplify menu option. Set the amplification level to the appropriate settings (use the Preview option if needed), check the Allow clipping box, and click OK (normally, you should avoid clipping in music files, but you will probably not notice its effects in podcasts). Amplification takes a couple of minutes.
   
   
6. Once the track is ready, use the File - Export As - MP3 menu option to save the modified file. If you want, you can rename the file in the Save As dialog box or you can use the same name (I prefer to use the name of the show and the date of the broadcast in the file names, such as WaitWait_7_15_2007.mp3). Exporting to an MP3 file takes a few minutes.

At this point you will have an MP3 file which your MP3 player should be able to play. However, there is one additional step I recommend.

3. Split long podcast file into multiple MP3 files
If a podcast file is long (20 minutes or longer), you may find it more convenient to split it into shorter segments. After trying several applications which can split MP3 files, I would recommend mp3DirectCut written by Martin Pesch. [These instructions and screenshots apply to mp3DirectCut 5.02; newer versions of the application may look differently and require different steps.] To split an MP3 file using mp3DirectCut, do the following:

1. Start mp3DirectCut.
2. Open the podcast using the File - Open menu or by dragging and dropping the MP3 file onto the mp3DirectCut window.
   
   
3. Select the Special - Auto Cue menu option.
4. In the Auto Cue dialog box, define the length of the segments into which you want to split the podcast (10 minutes is a reasonable value) and click OK.
   
   
5. Select the File - Save Split menu option.
6. In the Split file dialog box, define the format of the names of the generated track files in the Filename creation field. I use the %F (%N) format to append sequence numbers to the original file name, but you can use a different naming convention. You may also want to change the destination folder where the files will be saved, e.g. directly to your MP3 player. Click OK to start splitting the file (splitting takes about one minute).
   
   

I realize that the process may seem a bit tedious, so if you have recommendations how to improve or simplify it, please submit them via comments.

Additional references:
MP3 encoding primer

UPDATE: Since switching to Zune, I have pretty much abandoned the process described in this post. Both the Zune device and software (after the latest update) have been excellent tools for subscribing, managing, and listening to podcasts. I'm not sure if other media players offer the same podcasting capabilities, but if not, I would highly recommend Zune for its excellent features. (Note: This is not a paid advertisement.)