The Stack Overflow Podcast

This is the 45th episode of the StackOverflow podcast, where Joel and Jeff discuss what a program manager does, the value (or lack thereof) of a functional spec and vision statement, building developer community, and planning your development time. Joel and I will be at the upcoming MIX09 conference. We're also trying to set up a live podcast there on Tuesday, March 17th in the evening. Joel's essay How to be a Program Manager attempts to explain the essential role this person plays on a software project. It's a shame the job has such a nebulous title. Is writing a functional spec at the heart of agile development? What is a spec, exactly? There has to be something between sitting down and pounding out the code with no planning whatsoever, and meticulously, bureaucratically documenting every tiny detail of your application. Not all storyboarding has to be painful. Wireframing the user interface with tools like Balsamiq can take the pain out of a lightweight "functional spec". Describe every screen, and have some annotations about how stuff is supposed to work. I call this UI-first software development. The often cited article No Functional Spec doesn't actually mean no functional spec, if you read it closely. At least in our interpretation of the text. Can your team pass the elevator test? You also need a vision statement or "elevator pitch". Everyone on your team should be able to explain what your application does, in a few simple paragraphs, to a layman. If they can't, it's sign of deeper problems on your project. The book Dreaming in Code, which documents the Chandler project, might be a good example of a project that had a vision statement that hurt the project instead of helping. It described where they wanted to go, but not how they planned to get there. Flock might be another example -- what does "we're a social web browser" mean? Dave Winer maintains that, if you read the description of some new technical thing and can't understand it after the first readthrough.. it ultimately isn't important and can be safely ignored. Has Joel Spolsky been honest about his time at Microsoft? Realize that the article in question is one of Joel's first non-blog blog posts, way back in 2001, describing something that happened in 1992. So it's ancient history. Joel maintains that Greg Whitten's 2005 email is simply the other half of the story. There's no conflict, just two sides of the same coin. For all the talk about how Reddit comments have degenerated, we felt the programming reddit comments on the Joel article were generally quite insightful. Joel and I are big fans of Hacker News. Although I have criticized the lack of downvotes in the Hacker News system, it's important to note that there is a secret cabal of 30 editors that will kill flagged articles. So it's not entirely subject to the whims of user voting, either. Joel thinks that every hacker who maintains a community comes up with a manifesto that puts them squarely in Clay Shirky a Group is its Own Worst Enemy land. Even he has done it, with Building Communities With Software. You have to make very different decisions based on the size and the composition of the group at any particular time. Joel and I both dislike threaded discussion formats. When I delved back into threaded discussion this week on the programming Reddit and Hacker News I was reminded how awkward they are. I think developers have a myopia about tree structures, which are incomprehensible to the average person but a daily part of their programming work. I was shocked to discover that SQL Server will sometimes look at a parameterized query and come up with an incredibly bad query plan, which it will then store in the query cache, and (even worse!) use over and over! The trick is to use the optimize for unknown hint, which tells the query plan generator to use a statistical sampling of potential inputs rather than basing its decisions on whatever random paramet...

This is the 44th episode of the StackOverflow podcast, where Joel and Jeff discuss the enduring influence of C, the questionable value of the title "Software Architect", and the evolution of Java. Joel brings the YouTube video Write it in C to our attention. "Pascal won't quite cut it, write in C." Did you know that there's a new version of C++ on the horizon, C++0x? It even has its own tag on Stack Overflow. Speaking of C, Joel had lunch (and just guest-taught a class at Princeton for) Brian Kernighan. Brian is of course the co-author of the classic K&R book, and the creator of the Awk language. And his second favorite language is Visual Basic, surprisingly. I'm a little bitter that so many languages (C#, Java, JavaScript, etc) followed the "look" and many of the important design decisions of C, but Joel blames Algol-68. One C design decision I agree with: using carriage return as your programming line terminator is not a good idea. Having an explicit line ending character like semicolon gives you so much more flexibility, and is far less awkward than weird line continuation characters. Extension methods in C# are a poor man's way of extending the underlying language. One new feature in C# 4.0 is named parameter arguments. Joel notes that Excel's VBA implementation went through this same evolution, and it can potentially mask problems, like "why the heck do we have a function with 7 parameters in the first place?" We implemented the fantastic open-source Cacti tool to graph how much bandwidth and CPU time we're using on our servers. Our ISP does burstable billing at the 95th percentile, and this graph is built into Cacti. Right now we're doing about 750 KB/sec or 6 megabits/sec under those terms, which is (unfortunately) more than we agreed on with our ISP. This throughput number is 99% GZIP compressed text. What is the rationale for expressing all network bandwidth units in terms of bits? I prefer bytes. And while I'm at it, what the heck is a kibibyte? Remember Alexa? They're still around. It is sort of a mystery how sites like Alexa, Compete, etcetera can infer web traffic for any random website without access to the webserver's logfiles. It's essentially client sampling, but the accuracy of this approach is anyone's guess. Joel's classic article on unnecessary choice in software reminds us how customization is a double-edged sword. We've resisted a lot of per-user customization options on Stack Overflow for similar reasons. Joel and I both are unsure that the title "Software Architect" is a good one. We're leaning towards it being almost.. a net negative. "It's almost disrespectful of the actual architects who work in construction, to use that word to refer to some kind of high-falutin' big-picture UML-drawing code monkey." To the extent that the architect is not in the trenches with you doing the work, they don't have enough context, and will inevitably make the wrong decisions. If we had the power, we'd do away with the title "Architect". But if you're stuck with it -- and the architecture astronomy that it frequently engenders -- what is the proper role for a so-called Architect? They could work to connect disparate groups at large organizations, to provide context and reduce duplication for disparate groups that are working in isolation. It can be hard for groups working locally to see the context of the larger organization. But I traditionally think of this role as an evangelist and educator, not an architect. The catch-22 of rekindling a nascent programming career is that.. good programmers can't stop programming. So if you can give up programming, it sort of almost means that you shouldn't be doing it anyway. This is Joel's tough love answer. Bottom line: if you want to be a programmer, get out there and start writing code.. yesterday. Joel thinks that Java on the desktop is essentially dead. I don't necessarily disagree, but I think Java is still a fine choic...

This is the 43rd episode of the StackOverflow podcast, where Joel and Jeff discuss dealing with incompetent programmers, whether salaries should be public, dealing with technical debt, and programming for small businesses. Joel is away this week in Florida at the Future of Web Applications conference, where he was a speaker. He mentioned that the new Atlas web GUI builder was particularly impressive. We will try to be more careful about our use of Begs The Question. Joel asks about the rationale behind requiring 50 reputation to leave a comment, but allowing a brand new user to post a question or leave an answer. The reason is mostly because we have no mechanism for voting or marking offensive on comments, because they're ultra-lightweight. One way to avoid the dilemma of dealing with bad programmers is to be selective who you work for -- only choose employment at companies where they Hit the High Notes. It's even in the Joel Test itself: do new hires have to write code? I'm not a fan of puzzle-based interview processes. I met with a Stack Overflow user this week, Chris Jester-Young, and he revealed a clever and potentially more useful strategy: give interviewees a C program full of bugs, and have them try to debug them! Of course, Chris is a big Code Golf enthusiast, so of course he nailed that one. Sometimes you have to try to change your organization to fix the root problems, otherwise you're just fighting the symptoms and not the disease itself. This can be quite a challenge when you have no real authority. Joel offers some advice in Getting Things Done When You're Only a Grunt. Are there happy incompetents? I argue that there are; Joel argues that there aren't. Among the Inept, Ignorance is Bliss. Perhaps the better question to ask is, how can you help this marginal programmer find a career they'll enjoy? Many so-so programmers can make outstanding testers, for example. We wonder: what would it be like to work at a commercial for-profit company where everyone's salaries were public knowledge? I imagined it as something like The Lord of the Flies. Just make sure you aren't Piggy! At Fog Creek, salaries aren't technically public, but they have a formula through which everyone's pay can be derived. This is the Fog Creek Professional Ladder. Fog Creek also does profit sharing, so when the company does well, everyone does well. Is there a two-class society at Microsoft, between Testers and Developers? Joel wonders why they need different titles. I had always heard Microsoft did a great job of giving test engineers a viable parallel career track. Joel believes manipulating compensation to motivate people does not work, almost by definition. Top developers will do a good job no matter what compensation system you put in front of them. It's a "blunt instrument" that can cause as much harm as good. Joel is an advocate of "taking salary discussion off the table, paying people fairly and justly and well, so they can stop worrying about it so much." We have a lot of anti-bot code on Stack Overflow. What we didn't think about was human-entered spam! Now we do -- yet another example of the incredible power of rate limiting techniques. On matters of customer service, we do endeavor to not make the situation worse. Start practicing saying "I'm sorry, it's our fault." Updates may slow down on Stack Overflow for a little while. We have built up a lot of technical debt around our database, and we have to hunker down and refactor a few core database tables that affect 80% of our code. If you don't pay down your technical debt every so often, you could end up like Twitter -- a reliability laughingstock. But, somehow, still successful. There's something liberating and energizing about going in and tearing down huge parts of your application to rebuild it and make it better. Unlike our previous discussion on learning languages, I'm more of an advocate of the big bang model here, whereas Joel...

This is the 42nd episode of the StackOverflow podcast, where Joel and Jeff discuss ethical email, backup strategies, how to learn new programming languages, and dealing with underperforming developers. The Conversations Network, a non-profit organization that graciously underwrites the bandwidth costs of this and many other great podcasts, is looking for a sponsor. Email us at podcast@stackoverflow.com if you know of any We finally rolled out email support at Stack Overflow. If you haven't been to the site in 7 days, and have provided a valid email address, we include all the responses to your questions and answers (if any) in that period. And of course there is a true one-click unsubscribe. We're still tweaking the parameters of how it works -- what is the optimal email relationship between a user and a website? Sending email these days is a bit of a minefield. How do you avoid instantly going into people's spam folder? One key piece is having a Reverse PTR record, which is set up at the ISP level. There's a whole "Deliverability" industry around sending email to people. We're encouraged by the emerging standard of entering your OpenID provider's address as your OpenID login. For example, "yahoo.com" works for Yahoo OpenIDs, and eventually "gmail.com" will work for Google. (Today you must use "google.com/accounts/o8/id" for Google, which is not optimal for hopefully.. obvious.. reasons.) Microsoft is also coming on board, though their OpenID support is in private beta. We also implemented gold and silver tag-based badges, based on upvotes within a tag. It's a way of rewarding people who participate heavily in certain topic areas. We did have to rule out discussion based questions for this algorithm to work. We are also considering a tag leaderboard, as suggested by Greg Hewgill. Our backup strategy has been half-hearted so far. To improve this, we invested in an inexpensive embedded Linux based 1u Network Attached Storage device, the QNAP 409u. This will become our dedicated backup device. It has four drive bays and supports RAID 6 (dual parity). Kind of a neat little device; there's a whole subculture of inexpensive NAS devices I hadn't explored until now. Drobo, for example. As it turns out, the cost of bandwidth ends up being the gating factor for us when dealing with our daily multiple - gigabyte database backups. Jim Gray had an eye opening piece on the economics of bandwidth and the surprising effectiveness of "sneakernets", even today. How likely is it that your datacenter is going to explode? Unless you have a fancy multiple datacenter setup for redundancy, it might be more effective to do some trickle uploads to services like Amazon S3, or even some monthly datacenter driving runs to copy data off using a cheap USB 2.5" hard drive. Luckily, one of our team members lives a mile from the data center, so that's the approach we'll be using. We had some semi-serious issues with our IBM ServeRAID 8k controller, having to do with write-through versus write-back caching. Write-through blocks on actual disk writes, whereas write-back writes to a fast RAM buffer, returns very rapidly, and spools the writes over time (e.g. "lazy writes"). The performance of write-back is dramatically better, but we were seeing some eventual system-wide I/O blocking under heavy write load with write-back caching on. Supposedly this is normal for some RAID controllers, but we opted to downgrade to write-through because the nightly backups would always trigger this behavior for us. Speaking of blocking: it's funny how many of the techniques discussed on the High Scalability blog boil down to hashtables in memory. Memory is one of the fastest things you have in a computer, and it almost never blocks for any significant amount of time. Unlike, say.. hard disks or network. The act of trying to learn a new language will make you a better developer. Where do you go if you only know PHP? I think you sho...

This is the 41st episode of the StackOverflow podcast, where Joel and Jeff sit down with Robert Martin aka "Uncle Bob", and discuss software quality, the value of software engineering principles, and test-driven development. Joel clarifies that some of his comments in Podcast #38 were a bit unintentionally ad-hominem, and apologizes to Uncle Bob for that -- see Bob's open letter blog post. But on the positive side, it did get us a podcast with Uncle Bob! This was a big week for Stack Overflow; we moved to a new hosting provider -- PEAK Internet in Corvallis. We did have a few blips with DNS but other than that the move was relatively smooth. Increasing our servers from 4 GB (web) and 4 GB (database) to 8 GB and 24 GB, respectively, opened up tons of breathing room and unleashed a lot of latent performance. Memory is incredibly cheap right now; there's no reason not to install ridiculous amounts. It is (almost) free performance. Bob reminisces about when he bought memory by the bit! When I said "quality doesn't matter", I didn't mean it literally. If you deliver a software product that nobody likes or wants to use, it doesn't matter how great the quality of your code is. You can always fix code quality -- but fixing "nobody gives a crap about our product" is far more difficult. That's what you should be worrying about most of all. Quality has many dimensions. The cleanest code in the world could utterly miss the point on usability, scalability, performance, and meeting users' expectations. On the other hand, as Bob points out, there are companies that have shipped broken products which permanently damaged their reputations and, in some cases, even forced themselves out of business. Bob's SOLID principles are based on some well known conventions. I talked about the first one, the Single Responsibility Principle, in Curly's Law: Do One Thing. You may have heard this before as Don't Repeat Yourself (DRY), Once and Only Once, or Single Point of Truth. We wonder if some of these guidelines -- such as "deploy independently" -- are obviated by the inevitable forward march of technology, such as software delivered through the cloud, and virtual machines. What happens when principles fall into the hands of people who don't really know what they're doing? Or people who become bureaucrats, rigidly enforcing rules on everyone? We think the existence of rules, in and of itself, isn't necessarily a net good. The types of developers who need those rules are often immune to them. Often, software developed internally doesn't have to be good; users are forced to use it. This software would never survive as a real product that had users who actually had to want to pay for and use the software. Bob is fond of asking "why is open source software so much better"; part of the reason is that this software has to survive in the real world on its own merits to garner users and attentions. It's not isolated on some peculiar little corporate Galapagos island where it has no competition. Joel worries that excessive TDD (say, going from 90% to 100% test coverage) cuts time from other activities that could benefit the software product, such as better usability or additional features users are clamoring for. Unit tests are absolutely useful as a form of "eating your own dogfood", and documenting the behavior of your system. Even if you disregard the actual results of the tests completely, they're still valuable as documentation and giving you a fresh outside perspective on your codebase. Joel notes the difficulty of testing the UI (web or executable) versus testing the code behind the UI. The classic method of doing this is probably documented in The Art of UNIX Programming, where you start with a command-line app that takes in and spits out text. The GUI is simply a layer you paste on top of the command-line app, which leads to perfect testability -- but perhaps not such great apps, in the long run. Which is...

This is the 40th episode of the StackOverflow podcast, where Joel and Jeff sit down with Michael Lopp, aka Rands, to discuss how a geek manages other geeks, the dangers of working remotely, the pitfalls of offshoring, and some techniques for continual learning. You may know Michael Lopp from his excellent blog, Rands in Repose. We highly recommend the Best Of category if you haven't visited Rands before. Heck, even if you have. Rands is a Manager of Humans, and many of the entries on his blog are a fantastic resource for other Managers of Humans. "A lot of people stuff on a daily basis, it's 60 to 70 percent of my day." He wrote a book on this topic, Managing Humans: Biting and Humorous Tales of a Software Engineering Manager. If you transition from being a software developer to a manager, you will almost never write code. As a manager, how do you balance writing code -- if only as a form of daily exercise, to keep your skills sharp -- with the rest of what you do? The #1 management skill? Listening. Which means Joel and I should probably never be managers. The magical number of people one person can manage without having serious problems is seven. Beyond that, you're probably not giving the people on your team enough attention. On managing geeks and nerds: it should be done systematically. They like structure, and you have to put that around this messy business of us being human beings. Perhaps the handbook will be of use. Leading by example can only get you so far. What do you do when you inherit the world's best programmer in some language you don't even know? It's much more hands on than that. Your strengths are not your team's strengths. Building a so-called Nerd Playground is one way to attract talented developers. But as Joel says, "10% of the joy of coming in to work is that it's a great place to work, and 90% is that I love these people." The danger of working remotely is that there's a lot of non-verbal communication that typically goes on. Our tentative recommendation for working remotely, if you must do it, is to have frequent intervals where you all come together multiple days and work together. In order for the remote relationship to work, there has to be a face to face relationship in place to support it. Michael asks about the Stack Overflow badge system, which was explicitly modelled on the Xbox 360 Achievements system. The Bronze, Silver, and Gold badges are there to support beginners, intermediates, and the hardcore respectively. The idea that you have to figure out why you got a badge is there very much by design; the mystery is part of the fun. There were complaints that answers came too fast on Stack Overflow, which we have hopefully addressed -- there is an art to answering questions quickly and effectively. If anything, Stack Overflow "games" you into learning how to progressively answering other people's questions more and more effectively over time. How, exactly, is this a bad thing? A brief discussion of offshoring. "If people were getting as much value out of remote developers as local developers, then the price would be the same." It's also difficult to get good results out of teams when their skin isn't in the game. While I am deeply uncomfortable with using Stack Overflow as a hiring tool, it is totally reasonable to use it as a "breadcrumb trail of your awesomeness" as a programmer. If your questions and answers are well written and clear, and getting upvoted by your peers, that is something to be proud of. How do you manage continually learning? Reading, writing, and practice of the things you enjoy. It is possible to take this too far, though, and become a magpie developer. One advantage of older developers is that they have more failures under their belt. Michael looks for a combination of ego and humility when he hires -- these are somewhat contradictory traits, but they balance and complement each other. Our favorite Stack Overflow ques...

This is the 39th episode of the StackOverflow podcast, where Joel and Jeff discuss database design and the shell game of performance, the value of short, focused presentations, and the importance (or not) of a prestigious degree for software engineers. Joel insisted that we run the stackoverflow.com books on QuickBooks. I was more open to using an online accounting solution, but we were worried about someone holding our business finances hostage. Joel also maintains that "every accountant in the world knows QuickBooks". I'm not happy about it, but small business reality. Now that we can (theoretically) afford to pay ourselves to work on the site, we deployed two new features: reputation bounties on questions and new reply notifications. Joel expresses some concern about the value of panel discussions at conferences. There are lots of variables: who is on the panel, what are the topics, and how skilled is the moderator? It's almost better to give each person on the panel a 10 minute window to talk about some topic they think will have value for the audience. I often think of small, focused presentations as "Grok Talks". One presentation format which hews closely to this model is Pecha Kucha. It's generally a good idea to err on the side of keeping it small. The larger the talk, the more material you should have whittled away and deleted to get it down to size. In building Stack Overflow, we copied a key part of the Wikipedia database design. This turned out to be a mistake. We are due for a massive database refactoring, which is going to be very painful, but it is necessary to avoid excessive joins in a lot of our key queries. We also begin to appreciate why the giant multi-terabyte table schemas (like Google's BigTable) are completely join-free, basically simple hashtables or tuples spread across a server farm. My benchmarking shows that CPU speed is surprisingly important on our database server. Going from 1.86 GHz, to 2.5 GHz, to 3.5 GHz CPUs I see an almost linear improvement in typical query times! The exception is queries which don't fit in memory, but right now with 4GB RAM most of our DB fits in ram; with the new 24GB RAM server I expect the database to be entirely in memory for quite a while. Joel talks a bit about queueing theory, and its relation to OS scheduling. I propose that performance is a shell game with bottlenecks, where you're constantly trying to decide whether the bottleneck should be memory, CPU, or I/O. More discussion about unit testing: when it's appropriate, when it isn't, and what it is for. Remember, 100% code coverage (or even 90% code coverage) is not free. You're playing another shell game, so decide what axes of that resource balancing equation are important to you. Perhaps the greatest risk is being dogmatic about it. If you enjoy this podcast, you might also enjoy Hanselminutes -- Joel gives it his seal of approval! Joel revisits the SOLID principles, and compares them to designing replaceable batteries, or a headphone jack, into your product. Appropriate in some narrow cases, but not all the time. Imagine a consumer product where every single part of it could be plugged in and replaced with another compatible part. Is that realistic? Perhaps new language features will advance programming a lot faster than any given set of programming principles, no matter how good they are. I've often thought that design patterns are how languages evolve. Joel says you don't have to go to a prestigious name brand school, necessarily, but you need to choose schools that have a rigorous selection process. "These people are a tiny little bit more likely to succeed at our rigorous selection process." I concur -- if you aren't seeking out challenges, whether they are large, small, or somewhere in-between -- what are you doing? Our favorite Stack Overflow questions this week are: Jeff: Is mathematics necessary for programming? It can't hurt, and often helps,...

This is the 38th episode of the StackOverflow podcast, where Joel and Jeff discuss YSlow optimizations for large websites, the value of unit testing, and the hidden pitfalls of asking questions to programmers. Joel notes that simply paying attention to what your coworkers are doing is an effective way to build and lead a team. If you aren't interested in what your teammates are doing.. why are you on that team, again? I've followed the excellent Yahoo YSlow tool for a while. It really is intended for large scale websites, as I've noted before. But Stack Overflow is exactly the type of website that can benefit from YSlow recommendations! We've been using the Expires or Cache-Control Header since we launched. This saves the browser round-trips when getting infrequently changing items, such as images, javascript, or css. The downside is that, when you do actually change these files, you have to remember to change the filenames. A part of our build process now "tags" these files with a version number so we no longer have to remember to do this manually. We also integrated the YUI Compressor into our build, to minify our CSS and JavaScript resources. We had bad luck with the .NET port of this tool, so we just shell out to Java in the build. Works great, although we had some crazy pathing issues that made us put the JAR file in the root. It's also possible for Google to host your shared javascript files, if you're using a popular third party JS library. We chose not to do this because we package related JavaScript together, so it would defeat the benefits of packaging. Browsers will parallelize their requests, but only so many requests can be "in flight" to the same domain. So it can be wise to split your website components across domains. Simple aliases such as a.mydomain.com seem to work fine for this purpose. Joel explains CSS sprites, which is an effective way to minimize the number of HTTP requests your website is generating. This is particularly useful on toolbars and the like which contain a lot of related images. There are analogs here in the Strings tab of Process Explorer, and the UNIX command Strings, as well as classic Windows resource browsing -- spelunking for whatever icons and image resources you can find in a file. This is all important because I want Stack Overflow to be as fast as possible. Performance is a feature, and I think often underestimated. Even half a second delay can cause a 20% drop in traffic. We discuss differential database backups and DNS time to live, to make the upcoming site transition to new harware as painless as possible, and minimize downtime. We'll also update the old site with a static HTML page that tells you you need to flush your DNS. Joel notes that his partner Michael had to order thermal compound back in 2000 when he built up PCs for Fog Creek. This stuff is important! Please don't use vegemite. We finally fixed our paging algorithm, which had some aggravating edge condition bugs. I dedicate this fix to John Topley. Joel and I have some reservations about unit testing, at least the dogmatic test-first kind, where your goal is to have code coverage in the 95%+ range. It seems to us that this works best for legacy type applications that aren't changing very much. At some level, the tests become friction preventing you from making changes, as every change results in a stack of failing tests. Joel talks about Robert Martin's (aka Uncle Bob) Solid Principles, as explained on a recent Hanselminutes podcast. Joel: "it sounded like extremely bureaucratic programming" that "could theoretically protect you against things, but You Aren't Gonna Need It." On the other hand, if you're building a framework or API, something designed to be used by thousands or millions of developers, then having a lot of unit tests -- or Uncle Bob's Principles of OOD -- might make sense. Principles and rules are fine, but thinking about what you're doing should alw...

This is the 37th episode of the StackOverflow podcast, where Joel and Jeff discuss the expansion of Stack Overflow into non-programming IT topics, the pernicious problem of "systemitis", and how to reach the next generation of programmers. I was star-struck that Alan Kay actually participated in Stack Overflow, both answering a question and even asking a question of his own. We finally reverse engineered the WMD source code, thanks to the noble and herculean efforts of Dana Robinson. If you're interested you can pull the latest version from Dana's Git repository. Joel recommends Eric Raymond's Understanding Version-Control Systems. Our (very, very limited) experience with Git emphasizes the importance of editing code with the goal of creating easy to apply patches. If your changes are hard to merge, they're likely to be ignored. I've been having fun (for particularly small values of fun) configuring and building the new server hardware for Stack Overflow. I was moderately surprised to find that live rebuilding a RAID array with a 500 GB drive takes around 8 hours; rebuilding a simple 1-1 mirror array in offline mode takes 4 hours. Joel recommends the built in Windows Server Network Load Balancer (nlb); there's also the open source equivalent, HAProxy, which the Reddit guys mentioned in our podcast with them. We are planning to launch an IT-centric Stack Overflow in the next few months. This will be a place for System Administrator and IT professionals -- people who work with computers in a professional capacity, but aren't necessarily programmers -- can go to get their questions answered. Our first challenge with the IT-centric Stack Overflow is naming it. Naming is extremely difficult, whether you're naming functions/variables, businesses, websites, or new human beings. We also need to find the leaders and moderators who will drive the community and set the tone for everyone else. Joel heard the world population of computer programmers is 4 million. I can't find a source for this; does anyone have one? Joel thinks the current downturn is unlikely to affect the tech sector, except possibly as a broad excuse to cut dead wood out of companies. It's interesting to contrast the Web 1.0 crash in 2000-2001 with the current environment; it certainly doesn't feel the same to us. Joel and I don't agree with rigidly defined Project Manager, Programmer, and Test roles; how can you judge other people's competency in a particular discipline if you have zero competency in it yourself? Obviously this varies by company and person, but cross-training in related disciplines will make you a better programmer. Joel talks about "systemitis", programmers who spend the bulk of their time creating giant universal programming solutions to business problems that don't really make sense. This is perhaps a sign of programmers who aren't being challenged in their jobs. Rather than letting them spend their time creating another Universal System, try to recognize systemitis, and encourage these programmers to improve their skills in related disciplines instead of building "the system" We remember the classic BASIC programming that a whole generation of programmers grew up with. Typing in and modifying these simple little games was our first programming experience, an experience that launched a lifelong career. What is the equivalent for today's young programmers? Our favorite Stack Overflow questions this week are: Joel: Significant new inventions in computing since 1980? Joel picked Alan Kay's question, even before Joel knew it was from Alan Kay. We swear! We answered several listener question on this podcast: Alfred: "What will happen to the open source movement in a sluggish economy? Will it grow or shrink?" Shawn: "On the Business of Software: why do companies sell only small personal pizzas instead of individual pizza slices?" Daniel: "Large companies have...

Joel and Jeff, with special guest Eric Sink of SourceGear, discuss source control present and future, why writing a compiler is an important rite of passage for programmers, and how budding software engineers should be educated. I was convinced Eric was going to agree with me on whether or not software developers should know C. Unfortunately for me, he agrees with Joel. Doh! Eric's Source Control HOWTO should be required reading for every software developer. Eric literally wrote the book on this, and in fact this series is being converted into a book. Deservedly so; it's fantastic. The best person to explain source control is someone like Eric who literally wrote a source control system from scratch -- SourceGear Vault. An extended discussion of the evolution of source control systems, into distributed version control (DVCS) such as Git, Mercurial, Darcs, and BitKeeper. Eric feels Linus Torvalds' video "evangelizing" Git does the opposite for many. He's also uneasy about branching and merging becoming easy, common operations, which is much of the promise of DVCS. Eric feels DVCS solves certain edge conditions very well, but those edge conditions should only be explored when you feel the need, not promoted as broad feature bullets to the average developer. "The harder you try to explain DVCS, the worse it gets, so stop!" Eric points to Eric Raymond's evolving Understanding Version-Control Systems as a pretty good survey of today's source control landscape. Often, features aren't the point -- discoverability is. I was amused to find that Eric discovered how convenient background compilation is in Eclipse. This is something that Visual Basic developers have had for 8 years! Java and C# developers don't appreciate what they're missing because it hasn't been surfaced in the product, until now. Eric and Joel note that, at some level, source control is about putting obstacles (let's call them safety barriers) in front of developers -- and the question is, how many does your shop need? Do you remember working without version control at all? It's incredibly fast, until stuff gets overwritten or lost, of course. Joel and Eric maintain that writing a compiler is an important rite of passage for a programmer. There's an enormous class of programming problems where writing a lexer, parser, recursive descent, and parse trees will help you. Once you understand how easy it is to set up a state machine, you'll never try to use a regular expression inappropriately ever again. Eric also wrote a web browser. It's interesting to contrast the experience of writing a compiler, which is typically extremely strict and will fail to compile if a single character is out of place, versus writing a web browser, which accepts all kinds of malformed and downright incorrect HTML and JavaScript. Eric and Joel think this was categorically a huge mistake; I'm not so sure. On Postel's robustness principle: "be conservative in what you send, liberal in what you accept." This becomes a painful war of attrition at some level; everybody is vying to accept just a little bit liberally than the next guy, so isn't there an implied element of mutually assured destruction at the end? Eric: "Computer Science degrees do not teach programming; they teach how to learn". And he's OK with that. We all agree that it's hugely important to complement your computer science curriculum with either hobby projects, internships in the so-called "real world" -- or both! These extracurriculars will improve your chances of landing that first junior developer job. Congratulations to our new UserVoice community moderators -- Joel Coehoorn and Sean Massa. Do participate on the Stack Overflow UserVoice feedback site, we check it every day, and we read all the feedback we get! Our favorite Stack Overflow questions this week are: Eric: What is the single most effective thing you did to improve your programming skills...

This is the 35th episode of the StackOverflow podcast, where Joel and Jeff discuss the mysteries of server hardware, anomalous voting patterns, change fatigue, and whether or not Joel is the Martha Stewart of the software industry. A discussion of the bizarre world of server pricing, as discussed in the Best (or Worst) Geek Christmas Ever. It is sort of mysterious how a lot of the same hardware parts are is rebranded "server" parts, along with an instant 50%+ markup. Building your own servers up probably doesn't make any sense from a business perspective, but Joel and I both enjoy learning about this stuff -- and it is critical to our business. I doubt I would personally handle every server we ever use, but the first few, absolutely. I believe understanding the commodity hardware market is important for programmers. Joel notes that you really, really want to test your RAID failover scenarios before deploying your servers. That's one of the exact reasons I wanted to have the servers here, for me to play with RAID scenarios while the servers are up and running. After receiving a number of complaints, we now check for anomalous voting patterns on Stack Overflow. I guess it shouldn't be surprising that there were 4x as many anomalous upvote patterns as downvote patterns. Answers to older questions don't tend to get voted up as aggressively as rapid answers. There's an aspect of the Fastest Gun in the West to our system. Joel and I believe there are two audiences here; the daily users and the long tail. Sometimes a little (or a lot) of patience in order. We are now reverse engineering the JavaScript based Markdown editor we use on Stack Overflow. Believe me, we don't want to, but we have no choice. If you know JavaScript and/or GIT, we welcome your contribution! Joel had a great response to a forum post by a programmer thinking of leaving the industry, which I summarized as Programming: Love It or Leave It. Apparently Joel thinks of himself as the Martha Stewart of the software industry? Who am I to judge; listen to the audio yourself. If you're unhappy with your job as a programmer, it might simply be because your situation is not a good one. If you're in a bad situation, recognize that: either change your organization, or change your organization. Alternately, if you want to have a great 10 to 15 year career goal, why not start your own software company where programmers are able to work under great conditions, building awesome software with their peers? The Joel on Software discussion forums may soon require (nominal) paid registration, much like MetaFilter. This is something we discussed with Josh Millard, a MetaFilter moderator, on Podcast #22. Joel and I struggle with the definition of "change fatigue" as a career hazard for programmers. Isn't change the very basis of programming, and the reason most people enter the field? Rather than being a hazard, isn't the continual change a destination? Admittedly, it must be painful to be a specialist and have your knowledge obsoleted; Joel and I are both broad generalists, so it's easier for us. I discovered a great Stack Overflow post through Damien Katz: Arrays, What's the Point? Good Question. This is a fine example of a question that seems sort of ridiculous on the surface, but can provide amazingly insightful answers -- and a deeper understanding of programming. Jonathan Holland's accepted response has 190 upvotes! Our favorite Stack Overflow questions this week are: Jeff and Joel: Arrays, What's the point? This is exactly why we created Stack Overflow; fantastic result. Understanding data structures is as fundamental as it gets -- and so is questioning them. We answered one listener question on this podcast: Ian Varley: "A lot of programmers eventually become exhausted by the pace of change in our industry. How do you keep from getting change fatigue?" If you'd like to submit a question to be answered in our next episode, record an...

This is the 34th episode of the StackOverflow podcast, where Joel and Jeff discuss the following: Joel recommends a pink HDMI cable as a christmas gift for the special ladies in your life. This will go nicely with her pink ethernet cable, pink USB cable. We've placed an order for our dedicated servers and have found a dedicated host we tentatively like. We'll be exploring the "buy" axis of build vs. buy because it gives us greater power and control over our setup. This will also be my first time building a RAID 10 array, which is supposedly a far better solution than RAID 5, and I'm geeking out a bit over that. Joel remembers George Orwell's 1984 and The Thin Red Line. It's advisable for programmers to spend some time rotating through customer support so they can lean to share the customer's pain. Alternately, Joel has an even better idea: perform usability tests periodically, with your developers observing. Developers sometimes come up with great new features when exposed to actual users working with the software. The Fog Creek Team completed the Endless Setlist on hard in Rock Band 2. That's 84 songs in a row, mightily impressive, at least to me. I have mixed feelings about easter eggs; if you're going to put something cool and hidden in your product, why not make it a standard feature so most people will find it and benefit from it? There are two discussions of easter eggs on Stack Overflow: What Easter Eggs have you placed in code, and Is it a good idea to put Easter Eggs in applications? You could argue that a lot of modern large application features are effectively easter eggs because people can't find them! This is what motivated the move to the Ribbon UI in Office 2007. Nine out of ten feature requests for Office were already in the product. That's the ultimate easter egg, and not in a good way. The proposed Stack Overflow question bounty feature -- to help get those persistently unanswered questions some new attention -- has two gating clauses: first, you can only attach a small reputation bounty after 24 hours; second, the majority of the bounty will come from the asker's own reputation. You have to be willing to slice off a part of your own rep as a reward. We noticed that Jason Calacanis of Mahalo will be doing a Q&A site. The difference is that this site will pay answerers in real money, part of what Joel calls Jason's Econ 101 management style. The danger is that financial incentives can destroy intrinsic motivation. What's your management style? Command and Control, Econ 101, or Identity? There is a whole new generation of programmers growing up with code like we did. Here's hoping they're learning from our mistakes, so they can make all new mistakes and not repeat all the same dumb mistakes we made. It's almost like a system of software apprenticeship. Is code elegance in the eye of the beholder? Consider the two answers to Parameterizing a SQL IN Clause -- which one is more elegant, and why? Our favorite Stack Overflow questions this week are: Jeff: How old are you, and how old were you when you started coding? This is a great example of using of Stack Overflow to do cool stuff we didn't anticipate. It also broke our hot question algorithm pretty badly, because it had such a huge number of responses. Joel: What was your biggest CS eye opener? Joel points to K&R's observation that a[5] and 5[a] are the same thing. Bonus: Parameterizing a SQL IN Clause. Great example of my accepted answer not being accepted by the community. I just wanted to highlight the cleverness of Joel's solution, which I literally would never have thought of. I ended up using the community's solution, though. We answered the following listener questions on this podcast: Chris: "It's often said that the job of the software development manager's job is to insulate developers from customers. But I've found it's helpful for developers to have interacti...

This is the thirty-third episode of the StackOverflow podcast, where Joel and Jeff sit down with special guest Babak Ghahremanpour, the lead developer for FogBugz. I gifted the Fog Creek office with a set of cymbals for the Rock Band 2 drums, to complement the sweet Rock Band setup I bought for them earlier this year. And yes, I already got them a tambourine and cowbell. We're starting to seriously consider buying our own servers and renting rackspace for the Stack Overflow servers. It makes sense to us from both from a financial standpoint and from a performance standpoint. We're also considering some of the cloud services like Amazon EC2 and Windows Azure. We wonder why so much of the software that's bundled with hardware is so terrible. There's nothing scarier to me as a software developer than the DVD labelled "Install me!" provided with some bit of hardware that I just bought. Why is that? I tend to agree that one danger sign for a new programming job is the requirement to be on call. This is a bit more normal for sysadmin positions, but it's unusual (and arguably unhealthy) for programmers. Joel and I note that developing software predisposes you to "debug" real world processes that largely aren't worth the effort. Beware! We've probably mentioned this before, but whatever else you decide to do with your database, it is incredibly important that you get your database under version control. Does it make sense for every software developer to start their own company, which is what Paul Graham seems to advocate? It's certainly one of the few paths to becoming very wealthy, if that's your primary goal. If you just can't get enough Spolsky, Joel was featured on the Startup Success podcast with Bob Walsh and Patrick Foley. Our favorite Stack Overflow questions this week: Jeff: Why doesn't IE7 copy PRE CODE blocks to the clipboard correctly? This is sort of cheating because it's my own question, but it's a perfect example of using Stack Overflow to build Stack Overflow! We are the target audience, too. Joel: Dealbreakers for new programming jobs. Some great responses to this question; worth a read if you're looking for a new job. Babak: What real life bad habits has programming given you? We alluded to this question in the previous podcast; it's a classic. We answered the following listener questions on this podcast: Peter Bailey: "When you're designing a new application, how much code (triggers and stored procedures) do you put in the database?" Vincent Tan: "What are your top 3 costs in running a software business, and how do you reduce them?" If you'd like to submit a question to be answered in our next episode, record an audio file (90 seconds or less) and mail it to podcast@stackoverflow.com. You can record a question using nothing but a telephone and a web browser. We also have a dedicated phone number you can call to leave audio questions at 646-826-3879. The transcript wiki for this episode is available for public editing.

This is the 2^5 episode of the StackOverflow podcast, where Joel and Jeff discuss software piracy, dealing with public criticism, how to get people to answer your questions, and the ideal programmer office. A brief digression about the analog to digital television switch. Make sure you watch this brief instructional video so you're prepared. Joel gives an overview of Fog Creek's attitude towards software piracy and licensing. At its root, we believe that buying your software has to be easier than stealing it. Stop thinking of it as fighting an enemy and think of it as simply being better than the alternative. On advertising: if you can get people to talk about or willingly watch your ad, you've already won. This is why Wil Shipley's rant about the Mojave Experiment is counter-productive. I argue that Apple is making real headway in mainstream marketshare, largely because they've won the advertising war so decisively. Joel argues that they haven't. Which of us is right? One of our major performance optimizations for the "related questions" query is removing the top 10,000 most common English dictionary words (as determined by Google search) before submitting the query to the SQL Server 2008 full text engine. It's shocking how little is left of most posts once you remove the top 10k English dictionary words. This helps limit and narrow the returned results, which makes the query dramatically faster. Congratulations to Stack Overflow team member Geoff Dalgas; he's the proud father of a new baby boy as of last week! We plan to mail out weekly email summaries of other people's answers and comments to your posts -- much like FriendFeed -- for people who haven't visited Stack Overflow in a week or longer. Some topics are just fundamentally hard to understand, like the Monty Hall Problem, and lead to a lot of extra discussion. The two banned topics on the XKCD forums are 0.9999 = 1 and airplane on a treadmill for that reason. Joel "doesn't want to get too inspirational and stuff" but if you're not doing a few things where you're failing a little, you're probably not trying hard enough. I say if everyone likes you, you probably aren't doing anything interesting. Joel points to Guidelines on Writing a Philosophy Paper: "pretend that your reader is lazy, stupid, and mean." And the converse, as a reader, is to try to infer the most charitable possible interpretation of what you're reading. Not that anyone actually does this, in my experience. Can you tell the difference between people who honestly want to have a discussion about a topic, and people who are using that topic as a prop for their ego? Learning to make this distinction can ultimately save you a lot of time. My favorite Stack Overflow question this week is a classic, one of my early favorites: How to get attention for your old, unanswered questions. The response by icelava is absolutely on target -- if you want people to pay attention to your question, you have to pay attention to your question. Provide status and progress reports from your own continued effort to answer the question yourself. An excellent example of this is lassevik's tenacious return to his oddball problem which ultimately turned out to be a bug in the video driver, not his code! Joel's favorite Stack Overflow question this week is Must haves for a developer's office. Hard to disagree with the community on this one: a quality chair, two large monitors, whiteboards, and control of the lighting. If possible -- and this is usually a hard thing to get, so you have to really consider how realistic a negotiating position asking for your own private office is. That said, Joel and I continue to believe having a private office is the most ideal programming environment; it offers choice between open/noisy and closed/private. This also depends how much noise you can tolerate. In any case, demand your Programmer's Bill of Rights. As Joel says, "it's only money"....

This is the thirty-first episode of the StackOverflow podcast, where Joel and Jeff discuss.. stuff! Based on some comments from Podcast #30, we now know that "Learning about NP-Completeness from Jeff is like learning about irony from Alanis Morissette". It's funny because it's true! It was also noted that "[Jeff] certainly no mathematician." In my defense, I've always been very open about my lack of math skills. It's even item #3 in my Five Things We Didn't Know About You list. I noticed that someone posted the halting problem to GetACoder.com, so if nothing else good came of this, at least humor was delivered. How useful is math to most programmers? In other words, how often does a typical software developer use something they learned in, say, Calculus class or beyond? Joel cites the original Google PageRank algorithm paper (pdf) and Google's MapReduce (pdf) as good examples of math in practice for mainstream programming. I believe it's important for programmers to develop skills that aren't programming, necessarily, but that are complementary to programming, such as graphic design. (Or databases, or HTML/CSS, etcetera) Remember, good programmers write code; great programmers steal code. This applies triply to design. Joel and I both believe that status reports should be treated as a "public wall", never as weapons to determine how people get paid or promoted. The single best thing I've ever read on this is Poppendieck's Team Compensation (pdf) -- which I discovered through Joel's first collection of software writing. If I could, I'd print out a copy of this and staple it to the face of every person in the world who manages software developers. Yes, it really is that good. Go read it! One team at Fog Creek instituted a daily standup meeting for their project, which is a staple of most agile development approaches. In addition to the "Daily Kiwi" convention, Fog Creek also use a locally hosted instance of Laconica, an open source Twitter clone. It's certainly an interesting alternative to email. Joel believes most small to midsize software companies will deal with an economic downturn by (temporarily) deferring development of new versions of their products. For companies that have a lot of "extra" staff, the economy might be an excuse to get rid of the worst performing 10% of your employees. Joel justifies having a nice office space as 1) a recruiting tool 2) enabling higher programmer productivity and 3) the cost of a nice office space is a tiny number relative to all your other expenses running a company. I argue that companies which don't intuitively understand why nice office space is important to their employees who spend 8+ hours every day there.. well, those companies aren't smart enough to survive anyway. My favorite Stack Overflow question this week is Are Parameters really enough to prevent Sql injections? Joel and I have a long discussion about the importance of parameterized SQL, both for performance and for security (beware Little Bobby Tables!). But you should know that it's not 100% foolproof; it is possible (though rare) to have latent SQL injection exploits even when fully parameterized. Joel's favorite Stack Overflow question this week is How do you pull yourself out of a programming 'slump'? Joel knows tons of programmers who have burned out by age 50, and feels it is rare to find programmers who have written code for 20 to 30 years. Joel's article Fire and Motion and my article Moving The Block sort of cover this topic. Joel also recommends the book Death March as a reference book for what to avoid. We answered the following listener questions in this episode: Mike Akers: "How much time should programmers be spending in Photoshop?" David from the UK: "How do you handle status reports at Fog Creek?" Matthew Glidden: "How do you run a software company in lean economic times?" If you'd like to submit a question to be answered in our next episode, record an audio...